bpglob is executed through the rules in build-globs.ninja to determine
whether soong_build needs to rerun. That means when the arguments
accepted by bpglob change it will be called with the old arguments,
then soong_build will rerun and update build-globs.ninja with the new
arguments.
To avoid having to maintain backwards compatibility with old arguments
across the transition, a version argument is used to detect the
transition in order to stop parsing arguments, touch the output file
and exit immediately. The version number in
pathtools.BPGlobArgumentVersion should be manually incremented when
the bpglob argument format changes.
If the version argument is not passed then a version mismatch is assumed.
Bug: 159845846
Test: build/soong/tests/bootstrap_test.sh
Change-Id: Id6742c7acc51af8d0d224d51e47bddea78e6e363
- Make the parsed command line arguments accessible from other packages
- Separate writing the depfile from writing the Ninja file
- Extract a method to compute the name of the file list file for globs
Test: Presubmits.
Change-Id: I4d0daf0f3776a9336da305be0108ebf70a314776
This is accomplished by plumbing PrimaryBuilderInvocation in from
bootstrap.Args.
The confusing removal of "primaryBuilderExtraFlags" from documentation
generation is because it used to (erroneously) refer to the func defined
in command.go and its removal apparently doesn't break anything. I
suspect that this code path is dead within Android, but we'll need to
wait until May to remove it.
Test: Presubmits.
Change-Id: I8763579b82bd79d5ee0f233edfd48327282e401c
This makes it possible to invoke the primary builder more than once in a
single Ninja file.
This required adding arguments that were hard-wired in the build.ninja
rule to extraArgs and adding a "set of primary builder invocations"
member to bootstrap.Config . The only command line arguments that remain
hard-wired are those which are expected to be relevant to every
invocation of the primary builder.
As a welcome side effect, I was able to remove a number of random
environment variables that were so far special-cased ($SOONG_DELVE,
$SOONG_DELVE_PATH and $SOONG_OUTDIR). I was also able to move writing
the empty stub ninja glob file to command.go from the bowels of the
Blueprint machinery.
In theory, $TOP and $BUILDER could be removed, too, but this would
require hard-coding the value of $TOP into build.ninja and I don't know
what would break if I did that (it's okay to hard-wired $SOONG_DELVE and
$SOONG_DELVE_PATH because those are only used for debugging and
$SOONG_OUTDIR turned out to be superfluous)
Test: Presubmits.
Change-Id: Idbfd9976c4b270bc3e5a8926c8c760a8534596cf
Replace the individual matches and deps return values with a GlobResult
struct. Use the GlobResult to create the file list file in GlobWithDepFile
(used by bpglob) and in the glob singleton, as the generated files must
match exactly so that soong_build is not rerun.
Bug: 159845846
Test: glob_test.go
Change-Id: I2159cc9d85f388073198eac7456e5bf43e813096
Previously, a struct (anonymous or named) whose fields all matched the
predicate would not be sharded and would simply be reused. However,
that could break the maxNameSize limitation which could cause problems
for the caller.
This change makes sure that the supplied struct is only reused if it
does not exceed the maxNameSize, and otherwise is sharded.
Bug: 183777071
Test: m nothing
Change-Id: I8af272ec121077a43333e72b67cfd0e493c83362
This makes it possible to debug the primary builder without manually
adding the respective flags to the compilation command lines.
Test: "m nothing", "m nothing" with debugging, then "m nothing" again
Change-Id: I7d403af48528a94e5e0033d9a5c1fa9981100cfd
It's replaced with a flag in bootstrap.Args. It's a little bit ugly that
bootstrap.Main also gained a new Boolean flag in the process. The plan
is to delete that method in favor of directly calling RunBlueprint().
Test: "m nothing" (presubmits take a full workday these days)
Change-Id: Iae3421ae517a90570259b6278f18c3614bb1bb73
In addition to the command line interface, a new struct is created that
holds all the information Blueprint needs to do its job.
Test: Presubmits.
Change-Id: I0d6c924e09e069b3c1ae2000722efb52fd561e79
The end goal of this exercise is to remove all mutable global state so
that multiple Blueprint instances can be run in the same process.
This is done by plumbing the necessary data through the configuration.
Test: Presubmit.
Change-Id: I040a461ae17705258b882b3a44740a4dd5027a2c
ShellEscape(string) doesn't escape a string with spaces like "arg1
arg2". However, when we want to escape a string to use it as an
argument, then strings with spaces should be escaped.
For example,
"command " + ShellEscapeIncludingSpaces("a b")
becomes "command 'a b'" so that "command" will get "a b" as a single
argument.
Bug: 182092664
Test: Added tests to escape_test.go
Change-Id: I8f88c18bc4f9f7aacfe9e701b8f0876dd8b9a8c3
Some of these were introduced in aosp/1610784 and some others were
present since a good while.
Test: Manual: "m nothing"
Change-Id: I7d5cb56ca2d87cb439354fd1b298c3b8d97d665d
numericStringLess("1a", "11a") would strip the equal prefix "1" and
then compare the bytes "a" and "1", when it should have compared the
numbers 1 and 11. Fix it by handling the case where the last equal
byte was numeric and the first differing byte is numeric in one
string and non-numeric in the other.
numericStringLess("12", "101") would strip the equal prefix "1" and
then compare the numbers 2 and 01, when it should have compared the
numbers 12 and 101. Fix it by tracking the beginning of the sequence
of numeric bytes containing the differing byte.
Test: sort_test.go
Change-Id: I8d9252a64625ba6a3c75d09bb1429dcb1115e3e1
- Add the -N -l command line arguments to Microfactory
- Pass the the Delve arguments to the primary builder on the command line
Test: Manual.
Change-Id: I4034f2d48b3e40d9863529053715183dc3dce1f5
This is so that soong_build doesn't know the current directory, thereby
avoiding one way to leak information.
Test: Manual.
Change-Id: I976a8663b11d99c5382726487102df10043c5a61
The function now can traverse into a field whose type is slice of
struct. When reading field values from the returned indexes, Soong will
check if the next field is a slice of struct or not. If so, it will
recurse into all the values in the slice.
Bug: 181018147
Test: m nothing
Change-Id: Ib8a7b7911a0be37a6dc03079adeb906497e60875
Previously, an incantation like `bpfmt -o <$BAD_BP` would print to
stderr as expected, but then it would exit with 0. This is different
than the behavior of `bpfmt -o $BAD_BP`, which correctly exits with a
non-zero status code (in addition to printing the issues to stderr).
Test: m bpfmt; out/soong/host/linux-x86/bin/bpfmt -o <$BAD_BP; echo $?
Change-Id: I0a970cddf94fbd818525f9167d9113086b023c81
ReplaceExtension had an unexpected behaviour when the file did not have
an extension. In certain cases, the final path would be severely
trimmed: out/.intermediates/my_file would become out/.new_extension.
Explicitly handle the case by appending the new extension.
Test: Run checkbuild on Android Soong
Change-Id: Ie27a98845894cfaee5af5e2a02d44168c40ed821
This is an imported pull request from
https://github.com/google/blueprint/pull/345
GitOrigin-RevId: f9166c0e6151499b4b1a23b89b0bc133203a1116
Change-Id: I63f0798177545792440b8a84b04f1090590f1642
* aosp/upstream:
Fix detecting cycles in parallelVisit
Optimize HasTag function
Test: m checkbuild
Change-Id: Ia121f48bafa1461b8ea0ae6103d1b5bb503e6c16
Fix detecting cycles in parallelVisit when the first alphabetical
module is not part of the cycle, but depends on the cycle. Instead of
starting from the first alphabetical module, check every module in a
determinsitic order and return the first time a cycle is found.
Test: Test_parallelVisit
Change-Id: I03726f838ec42975251088ba75158103940115c2
The commits optimizes HasTag function by avoiding
the call to strings.Split that allocates a list of strings.
HasTag is called from the ExtendProperties family of functions.
So this new implementation would benefit any primary builder
that extensively uses property appending.
Add a small benchmark for HasTag function.
It shows that HasTag is speed up by a factor of 2-3 with the new
implementation.
Before:
goos: linux
goarch: amd64
BenchmarkHasTag/NoTag-56 20000000 118 ns/op
BenchmarkHasTag/EmptyTag-56 20000000 120 ns/op
BenchmarkHasTag/OtherTag-56 10000000 131 ns/op
BenchmarkHasTag/MatchingTag-56 10000000 177 ns/op
BenchmarkHasTag/ExtraValues-56 5000000 392 ns/op
BenchmarkHasTag/ExtraTags-56 10000000 183 ns/op
After:
goos: linux
goarch: amd64
BenchmarkHasTag/NoTag-56 200000000 11.5 ns/op
BenchmarkHasTag/EmptyTag-56 200000000 11.5 ns/op
BenchmarkHasTag/OtherTag-56 50000000 25.2 ns/op
BenchmarkHasTag/MatchingTag-56 20000000 61.4 ns/op
BenchmarkHasTag/ExtraValues-56 20000000 94.3 ns/op
BenchmarkHasTag/ExtraTags-56 20000000 79.1 ns/op
Signed-off-by: Francois Berder <francois.berder@arm.com>
Change-Id: Ib45498e9ad6aebeca2beddea63543da40c0b1a21
* aosp/upstream:
Optimize updateDependencies
Write build definitions directly to output writer
Memoize full names of variables, pools and rules
Optimize arg parsing in buildDef.WriteTo
Increase ninja file writer buffer
Use io.StringWriter in ninjaWriter
Test: m checkbuild
Change-Id: I8ff3ca4a95c351809b62af841e9bb77b2de7d1ed
Avoid reallocating module.forwardDeps and module.reverseDeps every
time through updateDependencies by resetting the slices without
reducing their capacity. Accumulate dependencies to visit directly
into module.forwardDeps. Use a loop instead of a map to check
for duplicates, average number of dependencies is measured to be
9.5, although there are a few outliers with up to 2108.
Reduces mean soong_build execution time on internal master from 87s
to 82.7s (5%).
Test: context_test.go
Change-Id: I58fcd5514e494bafa965443461851b21b7bce382
buildDef.WriteTo was calling valueList to convert all the build
parameter ninjaStrings into strings, which uses ValueWithEscaper
to build a strings.Builder. This results in building a string
only to immediately copy it into the output writer's buffer.
Instead, pass an io.StringWriter to ValueWithEscaper so it can
build the string directly into the output writer's buffer. This
requires converting ninjaWriterWithWrap into an io.StringWriter.
Test: ninja_writer_test.go
Change-Id: I02e1cf8259306267b9d2d0ebe8c81e13dd443725
Variables, pools and rules each computed their full names every time
they were referenced, which required string concatenations. Since
every one is guaranteed to be accessed at least twice, once when the
definition is written into the ninja file and once for each reference,
precompute the full name. For local variables that can be done
during initialization, but for global variables add a pass to
PrepareBuildActions to compute the name for each live variable using
the final package names.
Test: ninja_writer_test.go
Change-Id: I2264b05e0409e36651db2fb5d463c16c698d4d5e
