pathtools.Match was trimming the trailing slash from patterns. Make
it handle a trailing slash by first confirming that both the pattern
and name either both have or both do not have a trailing slash, and
then trimming it from both.
Bug: 111389216
Test: TestMatch in glob_test.go
Change-Id: I743e00c14d885de5b5a060aa9e2b22c81dc7e09d
Instead of sometimes re-running minibp/the primary builder during the
next phase, run bpglob earlier to check dependencies.
We've run into issues where the environment is slightly different
between bootstrapping phase and the main build phase. It's also a
problem because our primary builder (Soong) exports information used by
another tool (Kati) that runs in between the bootstrapping phases and
the main phase. When Soong would run in the main phase, it could get out
of sync, and would require the build to be run again.
To do this, add a "subninja" include a build-globs.ninja file to each
build.ninja file. The first time, this will be an empty file, but we'll
always run minibp / the primary builder anyway. When the builder runs,
in addition to writing a dependency file, write out the
build-globs.ninja file with the rules to run bpglob.
Since bpglob may need to be run very early, before it would normally be
built, build it with microfactory.
Change-Id: I89fcd849a8729e892f163d40060ab90b5d4dfa5d
This commit refines `compat` condition in `parseProperty()` so that a
module definition would either use the new syntax or the old syntax,
but not something in the between, such as
cc_library {
name: "bad_example",
srcs= ["bad.c"],
}
Test: lunch aosp_arm64-userdebug; make # runs unit test
Change-Id: If2d3e5d55edccc28d314d99b83b0f54e5c53ac35
WalkDeps was following every possible path through the dependency
tree, which can be enormous. Modify it to only call visit for
any particular (child, parent) pair once for each direct dependency
by not recursing into child if visitDown returns true but child
has already been visited.
Test: TestWalkDeps, TestWalkDepsDuplicates
Change-Id: Ieef28399bd10e744417cdeb661dfa04fbeb4ec60
Allow adding multiple dependencies on the same module. Adds a flag
to Context.walkDeps to determine whether dependencies should be
visited multiple times, and clarifies the behavior of
VisitDepsDepthFirst (continues to visit each module once, even if
there are multiple dependencies on it) and VisitDirectDeps (may
visit a module multiple times).
Test: TestWalkDepsDuplicates, TestVisit
Change-Id: I58afbe90490aca102d242d63e185386e1fe55d73
When someone used a multiline string:
cmd: "..." +
"...",
bpfmt used to print this out as:
cmd: "..." +
"...",
This change doesn't do the quote alignment like I see in some of our
user-created instances of this, but it does indent a single level:
cmd: "..." +
"...",
Test: unit tests
Test: Compared bpfmt results before and after across AOSP
Change-Id: I61bf790be9d08a187857b2725facf71e8b38e372
Misusing Rename was causing a nil pointer derefernece, return
an error if group was not found in s.modules.
Bug: 77922456
Change-Id: I7e7b20b1595d569ae07c755bd29d701e0e5dab78
Running with the data race detector enabled hits a 8192 active
goroutine limit. Reduce the maximum number of active goroutines
by limiting parallelVisit to 1000 goroutines. Also rewrite
cloneModule in terms of parallelVisit.
Change-Id: Icdd63648e8e010557b624e12592156490b40adb9
Patch and PatchList provide an API for making changes to substrings
of a Blueprint file by parsing the AST and using the token positions
to perform text replacements.
Test: modify_test.go
Change-Id: Ibb8993221982b54602ba5a05486198fab8d35a67
End() was previously only used to determine if a comment was within
a Node, so it used the expedient definition of the position of the
last token in the node. In the next patch it will be used for
capturing substrings of the Blueprint file, so make it point to
the character after the last token instead.
Also add tests for it.
Test: parser_test.go
Change-Id: Icaff3915b41e251ef9d0aad5615021bf37406aee
When passing "-c" to the Go compiler, any time this value changes, we'd
force all of the Go compiles to rebuild. This could trigger a
substantial portion of the tree to rebuild (anything that transitive
depends on a Go helper tool).
We're running into issues when moving output directories between
multiple GCE machines with different core counts, but are otherwise
identical. This could also hit users moving/mounting disks between
machines, though changes to other host tools can make an impact too.
On my 48-core machine, I get a ~15% benefit from going from -c 1 to -c
48, but also ~12% benefit from going from -c 1 to -c 8. So this will
still let us scale somewhat, but prevent rebuilds when transitioning
between machines that are more likely building Android.
In Go 1.10, runtime.GOROOT() will attempt to find the current location
of the go binaries, instead of using the GOROOT_FINAL that was baked
into the binaries. This means that GOROOT() will usually return an
absolute path.
We avoid putting absolute paths into the ninja file, since any change to
the paths would then cause all of the actions including it to rebuild.
Since we've got a decent number of build tools in Android using Go now,
this causes us to rebuild a decent portion of the tree.
Instead of passing the GOROOT around manually in a side channel, just
let the Go 1.10 detection do its thing, and always try to turn the
result into a relative path.
This makes it easy for users of Glob to detect whether the match is a
file or directory. Doing the check at this level means that the filelist
file used as a dependency will be updated if a directory is replaced
with a file of the same name, or vice versa.
Change-Id: I79ebba39327218bcdcf50b393498306119de9d6c
Performing the same glob twice, once with excludes=nil and once
with excludes=[]string{} would hit the same entry in the glob
cache (since the glob filename would be the same), but fail
the verifyGlob check because DeepEqual considers []string(nil)
and []string{} to be different. Use a manual array check
instead.
Test: glob_test.go
Change-Id: If0d4fe80163a871077b7276e1b4a3e888a4a4898
We're already going to load the .ninja_deps file, so we may as well use
it. These are the only two files in our build that don't.
Test: dump the .ninja_deps file, see the dependencies
Allow the primary builder to stop bootstrap between
ResolveDependencies and PrepareBuildActions so that it can
use the build graph to write customized documentation.
Change-Id: I67ee5e6e438a0c2c3ec844c4c453d223971e60a2
Move the docs to the main stage and make a phony rule to build
them. This will allow the primaray builder to choose not to
build them so that it can use its own rule instead.
Change-Id: Ib788f74717ccdcd85c2e5b6a581cc744d439341a
Separate the extraction of documentation-related info from writing
it out so that the primary builder can choose to write it out in
a customized format.
Change-Id: If2c91d6f13b8cea2e9e1cca75d8c2d684ad7f6df
Glob names were appending every excluded file, which could result
in too many / characters for ninja to handle. Use a hash if the
result contains too many / characters.
Change-Id: I88920f0596df49639a6c2b4f8af47ea8fdc1ee5a
Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
'build/blueprint:work^^^' 'build/blueprint:work'
Test: put a syntax error in a file and see that the
reported error reports the location of the violation
Change-Id: Iaeedb91ea8e816cb8e9ee954f21cd6c6bc4afa48
