Avoid reallocating module.forwardDeps and module.reverseDeps every
time through updateDependencies by resetting the slices without
reducing their capacity. Accumulate dependencies to visit directly
into module.forwardDeps. Use a loop instead of a map to check
for duplicates, average number of dependencies is measured to be
9.5, although there are a few outliers with up to 2108.
Reduces mean soong_build execution time on internal master from 87s
to 82.7s (5%).
Test: context_test.go
Change-Id: I58fcd5514e494bafa965443461851b21b7bce382
buildDef.WriteTo was calling valueList to convert all the build
parameter ninjaStrings into strings, which uses ValueWithEscaper
to build a strings.Builder. This results in building a string
only to immediately copy it into the output writer's buffer.
Instead, pass an io.StringWriter to ValueWithEscaper so it can
build the string directly into the output writer's buffer. This
requires converting ninjaWriterWithWrap into an io.StringWriter.
Test: ninja_writer_test.go
Change-Id: I02e1cf8259306267b9d2d0ebe8c81e13dd443725
Variables, pools and rules each computed their full names every time
they were referenced, which required string concatenations. Since
every one is guaranteed to be accessed at least twice, once when the
definition is written into the ninja file and once for each reference,
precompute the full name. For local variables that can be done
during initialization, but for global variables add a pass to
PrepareBuildActions to compute the name for each live variable using
the final package names.
Test: ninja_writer_test.go
Change-Id: I2264b05e0409e36651db2fb5d463c16c698d4d5e
Arguments to build definitions were copied from the input map to an
map with the name and value expanded, then to a list of names for
sorting, and then written, which required iterating over a map three
times. Expand the name and value into a list of name value pairs,
and then do the rest of the operations on the list instead.
Test: ninja_writer_test.go
Change-Id: Id8ff644dafbaa3b4812747c60dc28cce22e21dbe
ninjaWriter repeatedly called io.WriteString() on its writer, which
does a type assertion every time. Replace its io.Writer with an
io.StringWriter and call WriteString on it directly.
Test: ninja_writer_test.go
Change-Id: Ie073d996a319190242bf6a00af07a13a60d078b5
Callers to glob methods may do in-place modifications on the returned
list of globs, return a copy instead of the cached value.
Test: m nothing && m nothing
Change-Id: Ic9140d1e1900e8724ba0a484f27786e5c15dea90
Glob was calling IsSymlink and IsDir on each visited directory entry,
which resulted in an lstat and then a stat call on each.
Instead, use lstat when not following symlinks and use stat when
following symlinks, then use the result to check if the entry is a
directory.
Test: glob_test.go
Change-Id: I83d769e2de64ce8221e952e5204d365aeaf47687
Values assigned to map properties were silently ignored instead of
reported as an error. Add a check when recursing into structs that
the value is a map.
Fixes: 177706602
Test: m nothing
Test: TestUnpackErrors
Change-Id: Ic56aeb1b9da6d5c86b6d98adae7bddb60c450404
Returning a fmt.Errorf error causes Blueprint to identify the error
as an "internal error", return an UnpackError instead.
Bug: 177706602
Test: TestUnpackErrors
Change-Id: I19fba134ad778d08d5a4d90a0335bdf8cbea6a20
Add more tests, give the tests names, run them as subtests, and add
benchmarks.
Test: unpack_test.go
Change-Id: Iff22538ce44ed503d5d088cfb55673448db998f1
bpmodify sorts touched list respecting numbers embedded in strings. For
example, "foo-2" comes before "foo-10".
Test: bpmodify_test.go
Change-Id: If2fe9bc871a463a47dd3c0b52982b34c9fde05f0
Co-authored-by: Jooyung Han <jooyung@google.com>
In the coordination between #316 and #318 the calls to mctx.pause()
were forgotten, causing dependencies returned by the Add*Dependency
calls to be in an undefined state. Add the missing calls to
mctx.pause().
Change-Id: I648ad269449777363801785059b13b866424d4b5
* Invalidate module group cache if deps are modified
PreSingletons run after the blueprints have been parsed and can run
VisitAllModules; however, this seeds the cache of sorted modules which
may change after mutators have run.
This causes recomputation of the cache if any deps have been
modified since the last time it was run.
Change-Id: I79fc822dd630f84790f309ba4e6024588a8fe28e
In particular the formatting of dependencies with variants which lacked
braces that caused them to float together with the dependency names.
Also add some context to the ReplaceDependenciesIf panic message.
Test: m
Change-Id: Ibd03690317ca53f3ecbdd95033404d89d849b4dd
This allows exiting bootstrap directly before writing ninja files.
This facilitates shorter runtime on integrations which do processing
which do not require ninja file output.
Test: Manually verified on Soong integration use case which involves
running bootstrap twice in a single program; stopping before ninja
output reduces runtime by ~20s, or ~11%.
This allows exiting bootstrap directly before writing ninja files.
This facilitates shorter runtime on integrations which do processing
which do not require ninja file output.
Test: Manually verified on Soong integration use case which involves
running bootstrap twice in a single program; stopping before ninja
output reduces runtime by ~20s, or ~11%.
Some anonymous nested properties are missing from property structs,
since setting the property to anonymous is to allow future filtering,
there is no issue if we cannot find the struct.
test: go bpdoc tests
test: m soong_docs
Providers are a new concept for Blueprint, based on providers in Bazel:
https://docs.bazel.build/versions/master/skylark/rules.html#providers
Providers aim to simplify the interaction between modules by replacing
type asserting to arbitrary interfaces with requesting optional data
objects from modules. This will also move Blueprint closer to supporting
incremental analysis by serializing the providers and only rerunning
the analysis phase on modules whose inputs have changed.
Change-Id: I39f5f78b372412a7dbf151ceccb3f917f6c874bf
The motivaion for this change is to allow writing code that uses the
newly added dependency module in the same mutator pass, for example to
add more dependencies. Like this:
for _, m := range ctx.AddVariationDependencies(nil, tag, deps...) {
if someModuleProperty(m); ok {
ctx.AddVariationDependencies(nil, tag, otherDep)
}
}
Note that there is no guarantee that the returned module has already
been processed by the current mutator.
The patch does not add runtime overhead on findng dependency modules,
as this has already been done previously.
Test: go test
Pass a channel to visitor functions called by parallelVisit that
allows them to pause the current visitor until a given visitor
has finished. This allows parallelVisit to work on a dependency
graph while it is being mutated.
Test: Test_parallelVisit
Change-Id: Id8b1542c22ac9914439310e31d992ae0d7318d69
Pull request #317 made the equality check in moduleMatchingVariant
more correct by comparing the variant maps instead of the names.
Using only the names effectively ignores any variation with an
empty name. Unfortuantely this broke a current user of
ReplaceDependenciesIf. Go back to the relaxed check for now.
Test: build on a branch with sdk modules
Change-Id: I11016c8df7bd91fd022d04fd3033756f54d7fa0b
AddFarVariationDependencies claims to take the first variant that
matches all the requested variations, but it did nothing of the sort.
It took the first variant that either matched or did not contain each
of the requested variations. A module with no variations was always
matched, and requesting variations that didn't apply to a module
still matched (for example, requesting an image variation for a
host module that was ignored by the image mutator).
Fix AddFarVariationDependencies by making subset actually check
for subsets.
Test: Test_findVariant
Change-Id: I10063fec342db2a1c0685a7db08e4a650d14bd4e
AddFarVariationDependencies takes the first matching variant. To
maintain sensible behavior on a module with aliases, the ordering
of aliases and module variants needs to be maintained so that
AddFarVariationDependencies can find an earlier matching alias
instead of a more specific variant.
Test: go test .
Change-Id: I78f4e96edd98159f3a62d94e240e5d652667bec4
Add tests for findVariant behavior that provides the matching
behaviors of AddVariationDependencies, AddFarVariationDependencies,
etc.
Test: Test_findVariant
Change-Id: I3494d57179c8b3d62f7d32e5a1b43c9b9672c2df
The primary builder may want to create variants of bootstrap
modules if they need to fit in to the primary builder's dependency
graph. Enable arbitrary variants of bootstrap modules by only
running the module's actions on the primary variant and then
copying the result to any other variants that exist.
Test: m checkbuild
Change-Id: I24b97771bb11faeacab4079ed8cf69aef59da140
