There are a few cases that force all globs to be rerun at the beginning
of the build (changes to bpglob or dependencies, second build after a
clean build). The number of globs has gotten high enough that rerunning
them all can have significant overhead to start bpglob for each one.
Replace the per-glob bpglob invocations with sharded invocations using
1024 hash buckets.
Bug: 159845846
Test: glob_test.go
Test: m nothing && m nothing
Test: build/soong/bootstrap_test.sh
Change-Id: Ife1f7a03c8f6b25d1be01531425d8dc2c76d1ea0
- Make the parsed command line arguments accessible from other packages
- Separate writing the depfile from writing the Ninja file
- Extract a method to compute the name of the file list file for globs
Test: Presubmits.
Change-Id: I4d0daf0f3776a9336da305be0108ebf70a314776
Replace the individual matches and deps return values with a GlobResult
struct. Use the GlobResult to create the file list file in GlobWithDepFile
(used by bpglob) and in the glob singleton, as the generated files must
match exactly so that soong_build is not rerun.
Bug: 159845846
Test: glob_test.go
Change-Id: I2159cc9d85f388073198eac7456e5bf43e813096
Callers to glob methods may do in-place modifications on the returned
list of globs, return a copy instead of the cached value.
Test: m nothing && m nothing
Change-Id: Ic9140d1e1900e8724ba0a484f27786e5c15dea90
Allow the caller to specify whether symlinks should be followed
(the old behavior) or not.
Test: glob_test.go
Test: fs_test.go
Change-Id: I550dc91b8e6370fb32a9a1cbdcb2edade48bda46
Performing the same glob twice, once with excludes=nil and once
with excludes=[]string{} would hit the same entry in the glob
cache (since the glob filename would be the same), but fail
the verifyGlob check because DeepEqual considers []string(nil)
and []string{} to be different. Use a manual array check
instead.
Test: glob_test.go
Change-Id: If0d4fe80163a871077b7276e1b4a3e888a4a4898
Glob names were appending every excluded file, which could result
in too many / characters for ninja to handle. Use a hash if the
result contains too many / characters.
Change-Id: I88920f0596df49639a6c2b4f8af47ea8fdc1ee5a
Add globbing to filesystem mocking so that more code can be tested
against the mock. Also moves the filesystem mock to pathtools,
and renames pathtools.GlobWithExcludes to pathtools.Glob, replacing
the existing pathtools.Glob.
Test: blueprint tests
Change-Id: I722df8121bc870c4a861d7c249245c57dcb607be
Add globbing with dependency checking to blueprint. Calling
ModuleContext.GlobWithDeps or SingletonContext.GlobWithDeps will return
a list of files that match the globs, while also adding efficient
dependencies to rerun the primary builder if a file that matches the
glob is added or removed.
Also use the globbing support for optional_subdirs=, subdirs= and build=
lines in blueprints files. The globbing slightly changes the behavior
of subname= lines, it no longer falls back to looking for a file called
"Blueprints". Blueprint files that need to include a subdirectory with
a different name can use build= instead of subdir= to directly include
them. The Blueprints file is updated to reset subname="Blueprints" in
case we want to include subdirectories inside blueprint and the primary
builder has changed the subname.
Also adds a new test directory that contains a simple primary builder
tree to test regeneration for globbing, and runs the tests in travis.
Change-Id: I83ce525fd11e11579cc58ba5308d01ca8eea7bc6
