Before this cl, blueprint expressions were evaluated as they were
parsed. We want to add a feature to select statements where we can
bind the value of soome value from soong into a blueprint variable,
that then can be used like a regular variable in the .bp file. This
means that select statements need to hold whole unevalated expression
trees, and have the ability to evaluate them later on when the value
of the bound variable is known.
This cl doesn't implement the new select syntax, but it does split
blueprint's parsing and evaluating into two separate stages. We also
store expressions in selects and evaluate them when the select is
resolved.
I didn't do extensive performance evaluation, but a simple comparison
of the time of `touch Android.bp && m nothing` before/after this cl
showed a 1 second speedup. (That was probably just noise)
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I12f373719991afeb4aec76517153f32229d97ff2
This feature can be used to prune Android.bp files from analysis. This
was introduced in `T` to support co-existence of BA and Go apexes. With
the recent changes to apex prebuilt build rules, this pruning is no
longer necessary to support co-existence.
Pruning via PRODUCT_SOURCE_ROOT_DIRS is still supported.
Bug: 308188212
Test: m nothing
Change-Id: I1e1391665963b1ad7cb3837dc67500b69b0833af
This makes it faster than the write_files step that it's in parallel
with. It's a ~3 second improvement to soong's runtime on my computer
on aosp main.
I'm not using parallelVisit() this time around, because
VerifyProvidersWereUnchanged is run in parallel with other code
that also uses parallelVisit(), and parallelVisit() modifies
module.waitingCount, so they conflict with each other.
Bug: 335718784
Test: m nothing
Change-Id: I4c6a4c30e2ffc4606faad378f59e003a02826848
This reverts commit 2071d15a29.
Reason for revert: Droidmonitor triggered revert due to b/342483801.
Change-Id: Ib892a3b17a5fcefb40d316e9dce9d5432293ec25
This makes it faster than the write_files step that it's in parallel
with. It's a ~3 second improvement to soong's runtime on my computer
on aosp main.
Bug: 335718784
Test: m nothing
Change-Id: I9dc9bd9c4cb80e61f880531dbb661c971298607c
The input variationMap in applyTransitions may be nil, initialize
it if necessary.
Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: I091aa5a04732fdf18931b2534c4c479152c2a9e1
* changes:
Don't sort the results of TransitionMutator.Split
Apply outgoing transitions when adding dependencies
Delete transition variation when no matching variant was found
TransitionMutators can always return "" from OutgoingVariation in
order to implement the equivalent of CreateLocalVariations - a
variation that doesn't affect which variation of dependencies is
selected. The same need applies when creating the dependency after
the TransitionMutator. Call OutgoingVariation on the parent module
when adding new dependencies unless the variation was explicitly
requested.
Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: Iadb72a5a3f19579a6c1d34e3c7169ee9e2b53210
TransitionMutators sometimes apply to one variant of a module but
not to another. Deleting the variation when all variants were
untouched is insufficient if a post-mutator dependency is being added
on a variant that was untouched. Delete the variation whenever
no matching incoming variant was found.
Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: Ib073f6ec3090d09e4798b6f9ca3061ec5d58d722
ReplaceDependencies[If] currently replaces dependencies on a given
module with the current module. It expects to find a variant of the
given module that has the exact same variations as the current module.
That was sometimes handled via aliases, but TransitionMutators don't
support aliases, they use IncomingTransition to rewrite the variation
instead.
In all current usages of ReplaceDependencies[If], the given module
is also a direct dependency of the current module. Instead of looking
for the exact same variations, look for the variant that is a dependency
of the current module.
Bug: 319288033
Test: all soong tests pass
Flag: NONE
Change-Id: I3e33111322040b187f6e951554366ccdcaf1bc11
When adding a dependency on a module that has already had variants
created by a TransitionMutator, ignore any variation added by the
TransitionMutator to the current module if the TransitionMutator
didn't apply to the target module. This requires passing in the
list of variations requested by the AddVariationDependencies call
so that explicitly requested variations are not ignored.
This fixes TestAndroidTest_FixTestConfig when the override mutator
is converted to a TransitionMutator, as it can't add a dependency from
the "foo_test" module with an empty override variation to the
"foo" module that has no override variation.
Bug: 319288033
Test: TestAndroidTest_FixTestConfig
Flag: NONE
Change-Id: If0fca7adc28cd702f953885aef440a0805ac017c
Adding a dependency on a module that has already had a TransitionMutator
run on it may require adjusting the variation name based on the results
of IncomingTranstion. Store the variants that existed before the
TransitionMutator ran, find one that is a subset of the requested
variant, and call TranstionMutator.IncomingTransition to update the
value.
Bug: 319288033
Test: TestPostTransitionDeps
Change-Id: I690357f9792401a3edbc5ae9fdcb666495954fbc
* changes:
Store mutator in mutatorContext instead of name
Explicitly delete old logicModule entries from Context.moduleInfo
Keep logicModule for obsolete variants
Add Provider to transition contexts
Cache outgoing transitions
Use maps and slices packages
TransitionMutators will need access to the original *mutatorInfo,
store it in mutatorContext instead of the name, and update all
accesses to the name to go through the *mutatorInfo.
Bug: 319288033
Test: go test ./...
Change-Id: I4bb1a17f44e55b23dd70708c9a5fde2ae80ce154
Creating variants has an optimization to reuse the logicModule of the
original variant as the logicModule of the first new variant. Using
the same logicModule meant that updating the Context.moduleInfo map
from logicModules to *moduleInfo would overwrite the old entry for
the original variant with the new entry for the first variant.
TransitionMutators will need to keep the original logicModule for
later use, which will require disabling the optimization that reuses
it. When the first variant is added to the map it no longer overwrites
the original variant. Excplicitly track the original logicModule and
remove it from the map if necessary.
Bug: 319288033
Test: go test ./...
Change-Id: I05ffdf6d7ce60508f6d9e9657c21032f2c4f5d9c
When splitting a module into variants the original *moduleInfo was
left in place but with logicModule set to nil as a marker that the
variant was obsolete and any incoming dependencies needed to be resolved
onto one of the newly split variants. A change to allow TransitionMutator
IncomingTransition calls when adding dependencies later will require
keeping the obsolete *moduleInfo and logicModule around so that
IncomingTransition can be called on it. so use an explicit boolean
to mark the module as obsolete instead of clearing logicModule.
Bug: 319288033
Test: go test ./...
Change-Id: I5dc201f43442aa07ac1b858240c675bab3782b55
Allow TransitionMutator OutgoingTransition and IncomingTransition
methods to access Providers from the current and dependency module
respectively.
Bug: 319288033
Test: none
Change-Id: If16ee7980ee0b8f4abaf3c112738fca3f13ab61c
TransitionMutators were calling OutgoingTransition and IncomingTransition
twice, once to determine all the necessary variations for each module,
and then again when creating the variations to resolve dependencies.
Store the final variation needed for each dependency during the first
computation, and use it to resolve the dependency variation in the second
pass.
Bug: 319288033
Test: all soong tests
Change-Id: I2e42c0d69efbfcff915267c484c18554b857e0b6
filterPropertyStructFields should treat configurable properties as
leaf nodes and not recurse into them. Configurable properties can be
made arch_variant, at least while we transition to full use of
Configurable.
Bug: 323382414
Test: soong tests when changing enabled to be configurable
Change-Id: Ib22dd3797f69e912dab0e0c3c1030d6c1f04db7e
...and pass property name to ConfigurableEvaluator.
When trying to convert Enabled to a configurable property, I enountered
a bunch of different context types that all had different error methods.
OtherModulePropertyErrorf is an error method that can be implemented
by all contexts.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I38b2a545c0ee8d23281cd88bc397f79f39835bc4
ctx.ModuleErrorf for the main blueprint config object doesn't
report an error, it just returns it.
If go had something like C++'s `[[nodiscard]]` or rust's `#[must_use]`
that would be useful here, but the issue has been open since 2017:
https://github.com/golang/go/issues/20803
Test: Presubmits
Change-Id: I72709e6c5466d55f5c0f3fe51a25c29df0826271
It's not super useful because of Go's stupid restriction on not letting
you reflect private fields.
Test: m nothing
Change-Id: I1f73f06f5e64eaf4adce58a667b98b131aede53f
blueprint.variationMap.Equal was responsible for 11.5% of allocations
and 2.2% of allocated memory. reflect.DeepEquals is an expensive way
to compare a map. variationMap.subsetOf is already iterating through
the elements of the map to compare them, replace equal with a check that
the maps are the same length and then reuse subsetOf to check that the
have the same keys and values.
Test: SOONG_PROFILE_MEM=/tmp/mem.pprof m nothing
Change-Id: Ifb7cdf612e5455fd2f412488b7f94416c4e70c54
keyForPhonyCandidate was using sha256, which is a crypto hash and
unnecessarily expensive for this use case. hash/maphash would be
much faster because it implements WriteString and so doesn't cause
an extra allocation to copy to a byte slice for every write, but it
insists on randomizing the seed, which makes it unsuitable for writing
to the build.ninja file. Use hash/fnv instead, and use unsafe to
write strings to the hash to avoid the extra allocation.
Also replace the manually rolled parallelism with the existing
parallelVisit, which will reuse goroutines and limit the parallelism
to a useful value.
The hash could collide, and using a 64-bit hash makes that more
likely, so also check the full contents to make sure they are really
equal.
Cuts 1 second off Soong analysis time.
Test: SOONG_PROFILE_MEM=/tmp/mem.pprof m nothing
Change-Id: I4d1292cb158cfc5823a0f4d8b4aeac1d0b10230e
When setProvider() is called, hash the provider and store the hash in
the module. Then after the build is done, hash all the providers again
and compare the hashes. It's an error if they don't match.
Also add a flag to control it in case this check gets slow as we convert
more things to providers. However right now it's fast (unnoticable
in terms of whole seconds) so just have the flag always enabled.
Bug: 322069292
Test: m nothing
Change-Id: Ie4e806a6a9f20542ffcc7439eef376d3fb6a98ca
The description of TransitionMutators says that "the outgoing transition
should not take the properties of the dependency into account, only those
of the module that depends on it. For this reason, the dependency is not
even passed into it as an argument." However, OutgoingTransitionContext
was returing the dependency from ctx.Module(), not the parent. This
didn't matter for the only existing TransitionMutator, as it only used
the module to get a constant value.
Test: sanitize_test.go
Change-Id: I1ce5b3144787f57be4d50e95f0c923da9b2b079f
Force a resort of the module groups before running singletons
so that two singletons running in parallel don't cause a data
race when they trigger a resort in VisitAllModules.
Test: go test -race ./...
Change-Id: Iec041cec08c33c56787aadbde6a1b2b619815142
memoizeFullName was added to variables, rules and pools as an
optimization to prevent recomputing the full name repeatedly,
but the storage of variables, rules and pools are generally global
and not tied to the Context. When running multiple tests in
parallel there will be multiple Context objects all trying to
update the memoized names on the global variables, causing a data
race.
Package names were previously memoized via a pkgNames map stored
on the Context. Expand pkgNames to a nameTracker object that
contains maps for packages, variables, rules and pools, and replace
calls to fullName with calls through nameTracker.
Test: context_test.go
Change-Id: I15040b85a6d1dab9ab3cff44f227b22985acee18
Now that nothing calls *Context.*Provider directly, make the blueprint
methods return a nil any interface instead of the zero value that was
constructed via reflection. The type-safe wrappers will return a
zero value that can be constructed without any reflection or copying.
Bug: 316410648
Test: provider_test.go
Change-Id: I0abde5bacab9964a83f03c1644b51295a6c34d0b
Using generics for the providers API allows a type to be associated
with a ProviderKey, resulting in a type-safe API without that doesn't
require runtime type assertions by every caller.
Unfortunately, Go does not allow generic types in methods, only in
functions [1]. This prevents a type-safe API on ModuleContext, and
requires moving the API to be functions that take a ModuleContext as
a parameter.
[1] https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#no-parameterized-methods)
Bug: 316410648
Test: provider_test.go
Change-Id: Ide91de9f2a2a7d075b05e287c7cc86b395db0edb
