Commit graph

100 commits

Author SHA1 Message Date
Colin Cross
a6074ea049 Merge changes I4bb1a17f,I05ffdf6d,I5dc201f4,If16ee798,I2e42c0d6, ... into main
* changes:
  Store mutator in mutatorContext instead of name
  Explicitly delete old logicModule entries from Context.moduleInfo
  Keep logicModule for obsolete variants
  Add Provider to transition contexts
  Cache outgoing transitions
  Use maps and slices packages
2024-04-02 17:24:37 +00:00
Colin Cross
d5d55e5b26 Store mutator in mutatorContext instead of name
TransitionMutators will need access to the original *mutatorInfo,
store it in mutatorContext instead of the name, and update all
accesses to the name to go through the *mutatorInfo.

Bug: 319288033
Test: go test ./...
Change-Id: I4bb1a17f44e55b23dd70708c9a5fde2ae80ce154
2024-04-01 15:55:16 -07:00
Colin Cross
fab4866a68 Add Provider to transition contexts
Allow TransitionMutator OutgoingTransition and IncomingTransition
methods to access Providers from the current and dependency module
respectively.

Bug: 319288033
Test: none
Change-Id: If16ee7980ee0b8f4abaf3c112738fca3f13ab61c
2024-04-01 15:55:13 -07:00
Colin Cross
fca423d9e6 Cache outgoing transitions
TransitionMutators were calling OutgoingTransition and IncomingTransition
twice, once to determine all the necessary variations for each module,
and then again when creating the variations to resolve dependencies.
Store the final variation needed for each dependency during the first
computation, and use it to resolve the dependency variation in the second
pass.

Bug: 319288033
Test: all soong tests
Change-Id: I2e42c0d69efbfcff915267c484c18554b857e0b6
2024-04-01 15:54:42 -07:00
Cole Faust
4f968700d1 Add OtherModulePropertyErrorf
...and pass property name to ConfigurableEvaluator.

When trying to convert Enabled to a configurable property, I enountered
a bunch of different context types that all had different error methods.
OtherModulePropertyErrorf is an error method that can be implemented
by all contexts.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I38b2a545c0ee8d23281cd88bc397f79f39835bc4
2024-03-22 13:02:51 -07:00
Colin Cross
8514b5c26d Optimize returning the zero value from provider APIs
Now that nothing calls *Context.*Provider directly, make the blueprint
methods return a nil any interface instead of the zero value that was
constructed via reflection.  The type-safe wrappers will return a
zero value that can be constructed without any reflection or copying.

Bug: 316410648
Test: provider_test.go
Change-Id: I0abde5bacab9964a83f03c1644b51295a6c34d0b
2023-12-14 16:59:19 -08:00
Colin Cross
ed49204e85 Use generics for providers API
Using generics for the providers API allows a type to be associated
with a ProviderKey, resulting in a type-safe API without that doesn't
require runtime type assertions by every caller.

Unfortunately, Go does not allow generic types in methods, only in
functions [1].  This prevents a type-safe API on ModuleContext, and
requires moving the API to be functions that take a ModuleContext as
a parameter.

[1] https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#no-parameterized-methods)

Bug: 316410648
Test: provider_test.go
Change-Id: Ide91de9f2a2a7d075b05e287c7cc86b395db0edb
2023-12-14 16:59:16 -08:00
Dan Albert
629a6a3bc3 Plumb module tags through to ninja.
Bug: http://b/259130368
Test: built android, saw tags set in perfetto trace
Change-Id: I09618114832b143a10a2f84baccfaf3aa0fd5bd8
2023-06-22 17:15:33 +00:00
Liz Kammer
6af6061b12 Add a function to check missing deps
Test: go test Soong tests
Change-Id: If24e52b51a00bc45d6b6a2212d4e045bb36c598c
2023-05-18 09:52:46 -04:00
Liz Kammer
c9df6d7964 Add a stronger wording to don't use ModuleFromName
Test: n/a
Change-Id: I42de1e46a50a103fca0d6f60a0748f5cc107bac7
2023-04-04 11:56:55 -04:00
Sam Delmerico
26e44b7b78 apply gofmt
Change-Id: I2416e246b3d8485a6b7810b998cff2f45f6a0494
2023-02-21 15:11:20 -05:00
Lukacs T. Berki
eb641de659 Implement transition mutators.
These are more limited than bottom-up or top-down mutators but in
exchange have some pleasant properties:

- "variant not found" errors are impossible
- The logic is pleasantly split into multiple, mostly orthogonal
  parts
- Theoretically, if every mutator is refactored like this, they
  make it possible to partially cache the module graph
- Are quite close to a "configuration transition" in Bazel.

Bug: 231370928
Test: Presubmits.
Change-Id: Idcdb66b5ea75c0d2838f527aaa988df3b12553d8
2022-06-17 17:51:04 +02:00
Treehugger Robot
5208ee885c Merge "Update CreateModule to require a type name" 2022-04-27 16:00:35 +00:00
Liz Kammer
9967d62fad Update CreateModule to require a type name
This makes the LoadHook and TopDownMutatorContext interfaces for
CreateModule the same.

Test: CI
Change-Id: I30122099bdf109cda40f9d81a574dd6fa861f459
2022-04-26 09:15:13 -04:00
Lukacs T. Berki
83a5a308b1 Remove the concept of "early mutator".
It was unused and as such, maintenance cost for no benefit.

Test: Presubmits.
Change-Id: Ifc35c0f55647ef88ed0bfa44bcf709e51a904ea4
2022-04-19 18:12:03 +02:00
Treehugger Robot
310b0cec5d Merge "force LoadHookContext to specify module type" 2022-03-10 23:37:08 +00:00
Sam Delmerico
518966d09e force LoadHookContext to specify module type
Modules created by a LoadHookContext do not currently set a module type
when creating new modules, but it would make analysis of the
module-graph.json much easier if this module type information was available.

Bug: 174879786
Test: m json-module-graph && check that module which previously had a
    blank module type now have the field populated
Change-Id: Ie2fa4244113b6254e6678da9a663d883e2a48a41
2022-03-09 20:50:48 +00:00
jiajia tang
eb27645af9 Fix duplicate typo
Comments typo fix

Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Change-Id: I89bf5aa5a3a57cda4b1e74195894341f9089e11c
2022-03-08 19:50:09 +08:00
Usta Shrestha
ee7a5d7a16 Typos and missing parameter names use in doc comment
Test: N/A
Bug: N/A
Change-Id: I01331365925decef22502da02a23ed4ce610da98
2022-01-18 16:46:30 -05:00
Inseob Kim
1e01a4e7c2 Reset contexts before calling LoadHook
If more than one loadhooks exist, calls to CreateModule will make an
error because the contexts object isn't reset and the same modules are
added more than once. This fixes the bug by initializing contexts object
every time.

Bug: 213297238
Test: manual
Change-Id: I545c1592c3217b764968a8937c962e9d58a85291
2022-01-05 19:56:08 +09:00
Usta
5ff5fe3e66 Minor refactoring to remove a couple of warnings
Test: m nothing
Bug: n/a

Change-Id: I3d5cf50974e84725d1d49113dfb9b74d4f426fb8
2021-12-02 15:21:59 -05:00
Chris Parsons
030150d8f9 Add function to retrieve module by name.
This function should only be used in corner-cases where it is not
possible to retrieve the module by traversing a dependency graph; there
are no guarantees about which variant is returned, and the function will
eagerly panic if something is amiss (the name doesn't belong to a module
but to an alias instead).

This function is particularly useful for bp2build, which does not use
real variants when evaluating the blueprint graph.

Test: With soong change, USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I72b8335b642ed2d05e0a38e448cd380acc8d65b0
2021-07-21 16:16:40 -04:00
Treehugger Robot
69e2bbe2df Merge "Fix AddNinjaFileDeps in a LoadHook" 2021-05-21 23:02:18 +00:00
Martin Stjernholm
53bf14cda7 Add OtherModuleFarDependencyVariantExists.
This is the logical extension corresponding to
AddFarVariationDependencies in the same way
OtherModuleDependencyVariantExists corresponds to
AddVariationDependencies.

Test: m nothing
Bug: 188398129
Change-Id: I5517bfd6be5e2c58432c2902dfd1ca7668c76598
2021-05-20 16:30:02 +00:00
Colin Cross
13b5befc5c Fix AddNinjaFileDeps in a LoadHook
Propagate the ninja file deps from a LoadHook to the build.ninja.d
file.

Bug: 188547846
Test: next CL
Change-Id: If8176474b5094ee40d07df12f5da79a906ce7290
2021-05-19 10:30:58 -07:00
Colin Cross
2f7d3dd6fb Make AddVariationDependencies return list of nil for non-parallel mutators
According to the documentation, Add[Variation]Dependencies should be
returning a list the same length as the input list.  A non-parallel
mutator doesn't support pausing to wait for new dependencies to finish
the current mutator pass, so it can't return the new dependencies.  Make
it return a list of nil instead to match the contract.

Bug: 186539038
Test: TestAddVariationDependencies
Change-Id: I07d0ac1f3024dcb7c94a2518910a68b61cd731e2
2021-04-28 11:50:37 -07:00
colincross
cfa2e91dc4
Actually pause mutators when adding dependencies (#335)
In the coordination between #316 and #318 the calls to mctx.pause()
were forgotten, causing dependencies returned by the Add*Dependency
calls to be in an undefined state.  Add the missing calls to
mctx.pause().

Change-Id: I648ad269449777363801785059b13b866424d4b5
2020-12-14 11:23:23 -08:00
Martin Stjernholm
0f1637b19b
Improve formatting of some error messages. (#327)
In particular the formatting of dependencies with variants which lacked
braces that caused them to float together with the dependency names.

Also add some context to the ReplaceDependenciesIf panic message.

Test: m
Change-Id: Ibd03690317ca53f3ecbdd95033404d89d849b4dd
2020-11-16 12:15:36 -08:00
Colin Cross
2da8492b9d Add Providers to Blueprint
Providers are a new concept for Blueprint, based on providers in Bazel:
https://docs.bazel.build/versions/master/skylark/rules.html#providers

Providers aim to simplify the interaction between modules by replacing
type asserting to arbitrary interfaces with requesting optional data
objects from modules.  This will also move Blueprint closer to supporting
incremental analysis by serializing the providers and only rerunning
the analysis phase on modules whose inputs have changed.

Change-Id: I39f5f78b372412a7dbf151ceccb3f917f6c874bf
2020-09-22 18:20:18 -07:00
colincross
66fa73dd6e
Merge pull request #316 from skvadrik/add-dep-ret-mod
Return dependency modules from dependency-adding methods.
2020-09-22 10:59:46 -07:00
Ulya Trafimovich
9577bbc922 Return dependency modules from dependency-adding methods.
The motivaion for this change is to allow writing code that uses the
newly added dependency module in the same mutator pass, for example to
add more dependencies. Like this:

  for _, m := range ctx.AddVariationDependencies(nil, tag, deps...) {
      if someModuleProperty(m); ok {
          ctx.AddVariationDependencies(nil, tag, otherDep)
      }
  }

Note that there is no guarantee that the returned module has already
been processed by the current mutator.

The patch does not add runtime overhead on findng dependency modules,
as this has already been done previously.

Test: go test
2020-09-17 11:43:15 +01:00
Colin Cross
c4773d90a2 Support pausing parallelVisit
Pass a channel to visitor functions called by parallelVisit that
allows them to pause the current visitor until a given visitor
has finished.  This allows parallelVisit to work on a dependency
graph while it is being mutated.

Test: Test_parallelVisit
Change-Id: Id8b1542c22ac9914439310e31d992ae0d7318d69
2020-09-16 12:58:39 -07:00
Colin Cross
5df74a8e38 Maintain ordering between variants and aliases
AddFarVariationDependencies takes the first matching variant.  To
maintain sensible behavior on a module with aliases, the ordering
of aliases and module variants needs to be maintained so that
AddFarVariationDependencies can find an earlier matching alias
instead of a more specific variant.

Test: go test .
Change-Id: I78f4e96edd98159f3a62d94e240e5d652667bec4
2020-09-09 18:29:15 -07:00
Colin Cross
39644c0903 Add tests for findVariant
Add tests for findVariant behavior that provides the matching
behaviors of AddVariationDependencies, AddFarVariationDependencies,
etc.

Test: Test_findVariant
Change-Id: I3494d57179c8b3d62f7d32e5a1b43c9b9672c2df
2020-09-09 18:27:32 -07:00
Colin Cross
18926521dc Make VisitAllModuleVariants, PrimaryModule and FinalModule available to mutators
Move the VisitAllModuleVariants, PrimaryModule and FinalModule methods to
baseModuleContext so they can be used by mutators.

Test: m checkbuild
Change-Id: I1827ce2fc75f017460a7f6a53b1dab6a81109435
2020-09-09 13:25:54 -07:00
Colin Cross
279489c017 Add CreateAliasVariation
CreateAliasVariation creates a new variation for the current mutator
that is an alias to a variation that was created with CreateVarations
by the same mutator.  It is similar to AliasVariation, except that
AliasVariation creates an alias from the pre-mutated variant to one
of the new variations, while CreateAliasVariation creates an alias
from a new post-mutated variation to one of the new variations.

Bug: 164216768
Test: TestCreateAliasVariation
Change-Id: I8847b8d6fc1d3248abc910b2fe2399881f9bdaee
2020-08-14 11:15:58 -07:00
Colin Cross
edc41769fe Combine variant fields into variant struct
Combine variant, variations and dependencyVariations into a single
struct.

Test: blueprint tests
Change-Id: I76811bb2bdf76f1b4cffc6a7b9bc7362ecb5f7b9
2020-08-14 11:15:58 -07:00
Daniel Cardenas
5de3dc04e0 Android inclusive language fixit. Make comment more succinct, clear and remove use of term "sanity".
Bug: 162536543
Change-Id: I4b40b1637e142d2f2fd513f62357213a9832bbaa
2020-07-30 16:48:42 -07:00
Paul Duffin
8969cb6170 Add ReplaceDependeciesIf to allow for conditional replacement
Adds support for making the replacement of one dependency with another
conditional on the from and to module as well as the tag used.
2020-06-30 12:15:26 +01:00
Martin Stjernholm
2f212479e2 Add ctx.OtherModule(Reverse)DependencyVariantExists.
This allows guarding calls to AddVariationDependencies and
AddReverseDependency to register dependencies on optional modules.
2020-05-26 20:52:43 +01:00
Paul Duffin
2a2c58ef46 Support checking syntax of generated Blueprint files
Adds a CheckBlueprintSyntax(...) method to check the syntax of a
Blueprint file.

Changes processModuleDef and newModule from being method on *Context to
being standalone functions. That ensures that CheckBlueprintSyntax(...)
does not need to take a context and so there is no chance that it can
change its state.
2020-05-13 09:06:17 +01:00
Colin Cross
c03a50c49c Add EarlyModuleContext.BlueprintsFile
EarlyModuleContext already has ModuleDir, add BlueprintsFile too.

Test: m checkbuild
Change-Id: Ia9e0c724562e1461dfcd5e01f56ff8abbea2ac06
2020-01-14 13:00:01 -08:00
Colin Cross
9672d86a87 Add scoped module factories
Add RegisterScopedModuleType to LoadHookContext that registers
a module type factory that will be visible for the remainder of
the file.  Also add ModuleFactories that returns the globally
register module factories.

Test: all blueprint tests
Change-Id: If646a73befe3b8e45c4b0984531c6a39ddc8d066
2020-01-02 20:32:51 -08:00
Colin Cross
da70fd0b84 Move LoadHooks from Soong to Blueprint
Move LoadHooks from Blueprint and run them during ParseBlueprintsFiles.
This will allow them to add scoped module types that are visible to the
rest of the Blueprints file.  Requires passing the config object to
ParseBlueprintsFiles.

Test: all blueprint tests
Change-Id: Ia2a2c9a0223d5458bfd48bd22ebed0fdbd0156c6
2020-01-02 20:32:51 -08:00
Colin Cross
f7beb89df5 Add support for module variant aliases
Adding a dependency on a module with variants can be problematic
if the code adding the dependency is not aware of every mutator
that has created variants.  Add an AliasVariations to
BottomUpMutatorContext, which allows a mutator to alias the
original variant of a module to one of the new variants of the
module, which will allow future dependencies to be added using
the original list of variations.  The aliases are transient,
and only exist until the next mutator that calls CreateVariations
when visiting the module without also calling AliasVariations.

Test: TestAlises
Change-Id: Ieaa04b5a6bdcb5a1ff5114b1e03460de795d4479
2019-11-14 12:58:52 -08:00
Colin Cross
9403b5a790 Delay allocating variationMaps until populating them
Modules are created with a nil variantionMap, which avoids allocating
an empty map if the module will never be split.
Context.addVariationDependencies constructs a variationMap to
compare against the variationMap of each variant, but constructed
an empty variationMap if no variations were specified.  A nil
map is not the same as an empty map, so consistently use a nil
map and create it when populating the first entry.

Change-Id: I48b604659f9cdb23326b504a093cdfe5a3eb4f68
2019-11-14 12:58:52 -08:00
Colin Cross
6af7284d08 Make CreateModule return the newly created module
Allow mutators to modify properties of the newly created module
by returning it.

Change-Id: Ia1acc70b6196195d1bad58c88ab3def70b99e42c
2019-09-25 14:53:25 -07:00
Jiyong Park
1e2e56dc62 Add SetDefaultDependencyVariation
SetDefaultDependencyVariation sets the variation name that will be used
when a dangling dependency is found while a module is being split. A
dangling dependency can occur if a module is split to a variant that one
of its dependencies is not split into. When the default variation is not
set, such dangling dependency is a hard error. But with the new
function, the default variation can be set and subsequent calls to
CreateVariations and its variations on the same context uses the default
variation when necessary.

(If even the default variation does not exist for the dependent module,
it is an hard error)

Note that this is different from calling SetDependencyVariation("foo")
followed by CreateVariations("foo", "bar"). In that case, regardless of
whether a dependency of the current module has the variant 'bar' or not,
only the 'foo' variant is chosen.

With SetDefaultDependencyVariation("foo") followed by
CreateVariations("foo", "bar"), 'foo' variant is used only when the
'bar' variant of the current module depends on a module that does not
have 'bar' variant.

Bug: 138103882
Test: m
Change-Id: I4520ca87487994de024fdbacda3bef6636225f0d
2019-08-09 12:49:11 -07:00
Colin Cross
d8367ff746 Add MutatorName to mutator contexts
Allow querying the mutator name for debugging.

Bug: 136473661
Change-Id: I6a9f1f8cc4fd456d32cc2f2ef85a497c193d7e09
2019-07-02 10:08:55 -07:00
Colin Cross
322cc01803 Report creating modules in errors on created modules
Bug: 133172285
Change-Id: I0782c722f5877508691868f96dbf7dc0bed3f9f4
2019-05-20 15:09:48 -07:00