Commit graph

2995 commits

Author SHA1 Message Date
Yu Liu
82e444710d Write ninja file in parallel.
Bug: 335718784
Test: CI
Change-Id: I26f7babca349c654780711cfe0f0ece3faa5f436
2024-05-20 21:46:19 +00:00
Colin Cross
515e7991fc Merge "Handle nil variant in applyTransitions" into main 2024-05-09 22:15:46 +00:00
Colin Cross
eb077c3123 Handle nil variant in applyTransitions
The input variationMap in applyTransitions may be nil, initialize
it if necessary.

Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: I091aa5a04732fdf18931b2534c4c479152c2a9e1
2024-05-09 10:51:20 -07:00
Treehugger Robot
0bc93e1a92 Merge "Create embedcfg directory before writing the file" into main 2024-05-09 16:42:08 +00:00
Tomasz Wasilczyk
2fbe882aa8 Create embedcfg directory before writing the file
Bug: 302718225
Test: rm -rf out/ && m out/host/linux-x86/bin/go/soong-cc/pkg/android/soong/cc.a
Change-Id: I206f4e0f83fae6a8c6b69c648d23a8c6d1f51477
2024-05-09 07:27:58 -07:00
Colin Cross
5bc2b73593 Merge changes I648ef95f,Iadb72a5a,Ib073f6ec into main
* changes:
  Don't sort the results of TransitionMutator.Split
  Apply outgoing transitions when adding dependencies
  Delete transition variation when no matching variant was found
2024-05-08 23:43:56 +00:00
Colin Cross
c5b3c0ca9e Don't sort the results of TransitionMutator.Split
The ordering of variants can be significant  when adding
inter-variant dependencies, as a variant can only depend on earlier
variants.  Allow maintaining the existing variant ordering when
converting mutators to TransitionMutator by keeping the ordering
of TransitionMutator.Split.  Variations that were requested by
incoming dependencies that are not present in Split are still
sorted, as they have no inherent ordering.

Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: I648ef95f08a05f9a64ce97e6a39bae10ce88771a
2024-05-08 15:22:27 -07:00
Colin Cross
5ac6648829 Apply outgoing transitions when adding dependencies
TransitionMutators can always return "" from OutgoingVariation in
order to implement the equivalent of CreateLocalVariations - a
variation that doesn't affect which variation of dependencies is
selected.  The same need applies when creating the dependency after
the TransitionMutator.  Call OutgoingVariation on the parent module
when adding new dependencies unless the variation was explicitly
requested.

Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: Iadb72a5a3f19579a6c1d34e3c7169ee9e2b53210
2024-05-08 15:22:21 -07:00
Colin Cross
02aa24823e Delete transition variation when no matching variant was found
TransitionMutators sometimes apply to one variant of a module but
not to another.  Deleting the variation when all variants were
untouched is insufficient if a post-mutator dependency is being added
on a variant that was untouched.  Delete the variation whenever
no matching incoming variant was found.

Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: Ib073f6ec3090d09e4798b6f9ca3061ec5d58d722
2024-05-07 14:36:48 -07:00
Cole Faust
0a1533d786 Merge "Support generating docs for Configurable properties" into main 2024-05-06 23:28:57 +00:00
Cole Faust
b9ff002303 Support generating docs for Configurable properties
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6e8a03c5785dbc3a90f458155dc4b9cd6ce7700b
2024-05-06 15:14:17 -07:00
Colin Cross
38b56e136f Change semantics of ReplaceDependencies[If]
ReplaceDependencies[If] currently replaces dependencies on a given
module with the current module.  It expects to find a variant of the
given module that has the exact same variations as the current module.
That was sometimes handled via aliases, but TransitionMutators don't
support aliases, they use IncomingTransition to rewrite the variation
instead.

In all current usages of ReplaceDependencies[If], the given module
is also a direct dependency of the current module.  Instead of looking
for the exact same variations, look for the variant that is a dependency
of the current module.

Bug: 319288033
Test: all soong tests pass
Flag: NONE
Change-Id: I3e33111322040b187f6e951554366ccdcaf1bc11
2024-05-02 14:39:19 -07:00
Colin Cross
de024f6167 Ignore dependency variations for TransitionMutators that didn't apply
When adding a dependency on a module that has already had variants
created by a TransitionMutator, ignore any variation added by the
TransitionMutator to the current module if the TransitionMutator
didn't apply to the target module.  This requires passing in the
list of variations requested by the AddVariationDependencies call
so that explicitly requested variations are not ignored.

This fixes TestAndroidTest_FixTestConfig when the override mutator
is converted to a TransitionMutator, as it can't add a dependency from
the "foo_test" module with an empty override variation to the
"foo" module  that has no override variation.

Bug: 319288033
Test: TestAndroidTest_FixTestConfig
Flag: NONE
Change-Id: If0fca7adc28cd702f953885aef440a0805ac017c
2024-05-01 15:25:37 -07:00
Colin Cross
9f260255f2 Rename TransitionMutator top down mutator
TransitionMutator's top down mutator is suffixed with _deps, but
that name is often used by other mutators related to the
TransitionMutator, causing unexpected conflicts.  Use _propagate
instead.

Bug: 319288033
Test: all soong tests pass
Flag: NONE
Change-Id: Ib42c66288e47287cfda8252d435f86c7045bf4c0
2024-05-01 15:25:37 -07:00
Cole Faust
1c48101091 Improve error message when assigning select to nonconfigurable property
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ideac33cd286566e14e7ff1ff579930add48a94db
2024-04-30 13:58:17 -07:00
Treehugger Robot
ca5ffdd3ce Merge changes from topics "fix_selects_appending", "refactor_selects" into main
* changes:
  Refactor selects
  Fix bugs when appending selects
  Allow extending configurable propeties with non-configurable properties
2024-04-30 01:15:42 +00:00
Cole Faust
a52b058ccc Refactor selects
In order to do less cloning, refactor selects so that all the
soong-visibile structs are immutable to soong and can be reused.

Additionally, refactor how the inner linked list of selects is managed,
so that the append/prepend/replace logic is simpler.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Iba5d27405decc1b0596590c3e0555daeb044bf9e
2024-04-29 13:23:30 -07:00
Cole Faust
70b814a964 Fix bugs when appending selects
The biggest issue here is that Configurable objects needed to be
cloned before appended to one another, otherwise some Configurables
that were used in defaults could be edited after being applied to one
module and apply a different value to another module.

Also fix an issue where a select without a defined appendWrapper
always evaluated to nil.

I plan to make a followup refactor cl to make these things clearer,
but start with the bugfix.

Bug: 323382414
Test: m nothing --no-skip-soong-tests (see other cl in topic for tests)
Change-Id: Icf68d0ee1779c76bfb3d68db43b35d7e09bc0dd9
2024-04-25 15:38:56 -07:00
Cole Faust
4560bb086e Allow extending configurable propeties with non-configurable properties
Sometimes modules add arch-variant properties in load hooks, to disable
modules by default on certain platforms for example. When changing the
property to a Configurable property, these load hooks would also need
to be changed in order to have a matching type for
ExtendMatchingProperties.

Since this can be kindof a pain to address everywhere, for now,
special case the extension functions to promote non-configurable
properties to configurable ones. We can remove this later when
everything switches to configurable properties.

Bug: 323382414
Test: go tests
Change-Id: Iac96587dbd60ccdd6aa667dd69a71ad252abe589
2024-04-25 15:31:00 -07:00
Haamed Gheibi
c10b49b5e9 Use relative GOPATH for microfactory golang builds.
Bug: b/336890676
Change-Id: I15ff839986d6c4997d5abccaa2ae8a9703d6cfb6
2024-04-25 04:44:01 +00:00
Tomasz Wasilczyk
36b6322979 Merge "Add support for go:embed" into main 2024-04-24 12:33:37 +00:00
Cole Faust
185cb44bef Export ConfigurableCase and add constructors
Some soong code sets arch-variant properties in order to control a
module's default behavior. I'll make this continue to work, but long
term the arch-variant properties should be replaced with selects,
so expose an API for creating select statements in soong code.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6c65d6e112b6f826f1027777b6fdf36915d34b1d
2024-04-23 13:55:59 -07:00
Tomasz Wasilczyk
51b4b03520 Add support for go:embed
Bug: 302718225
Test: Build aosp/2798713
Change-Id: I8c09f1604aa1a702c639bba91cefd104f8f2f36d
2024-04-22 11:52:00 -07:00
Colin Cross
d7474dd743 Call TransitionMutator.IncomingTransition when adding dependencies later
Adding a dependency on a module that has already had a TransitionMutator
run on it may require adjusting the variation name based on the results
of IncomingTranstion.  Store the variants that existed before the
TransitionMutator ran, find one that is a subset of the requested
variant, and call TranstionMutator.IncomingTransition to update the
value.

Bug: 319288033
Test: TestPostTransitionDeps
Change-Id: I690357f9792401a3edbc5ae9fdcb666495954fbc
2024-04-16 15:18:29 -07:00
Colin Cross
6ed94b7f85 Merge "Move TransitionMutator to transition.go and add tests" into main 2024-04-16 20:14:53 +00:00
Cole Faust
28357db9d0 Support unpacking a variable to a configurable property
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I504e53c34305f5de0af8629bf21ac1ce94704732
2024-04-12 16:37:51 -07:00
Cole Faust
3311debbb3 Support multi-variable selects and typed selects
This adds support for selecting on multiple variables at once, so that
you can do AND/OR combindations of them. For example:

select((
    arch(),
    os(),
), {
    ("arm64", "linux"): ["libfoo64"],
    (default, "linux"): ["libfoo"],
    (default, "windows"): ["libfoowindows"],
    (default, default): ["libbar"],
})

It also allows for select conditions to be boolean-typed. You can
write literal true and false without quotes to select on them. Currently
we don't have any boolean-typed variables though, so a fake one was
added for testing.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ibe586e7b21865b8734027848cc421594cbd1d8cc
2024-04-12 16:33:01 -07:00
Colin Cross
95b3627f6f Move TransitionMutator to transition.go and add tests
Bug: 319288033
Test: transition_test.go
Change-Id: Ia6fd96a69f559e3356155954fd021ec8d7293908
2024-04-12 14:58:58 -07:00
Cole Faust
09fe90e407 Rename Evaluate() to Get() and add GetDefault()
Part of the design of property structs is that they were easy to access.
In keeping with that spirit, use a shorter and easier to spell name
for the getter, and add GetDefault() so that you don't need to pass
the result of Get() to one of the
proptools.StringDefault/BoolDefault/etc functions.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ib9a69dcf2ab56a758935a461f37fe46bc0e17e27
2024-04-04 14:46:15 -07:00
Cole Faust
2437d5edb9 Add android:replace_instead_of_append
If a property is a pointer to a bool/string, that property would be
replaced instead of appended to when calling ExtendProperties().

This is confusing behavior, and I don't want to give anyone any reason
to use a pointer to a configurable property, so add this relflection
tag to recreate the functionality.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6b9e4dfee1d2685e0c9dc6a646cf45724cc04756
2024-04-04 14:35:18 -07:00
Cole Faust
0173a2268b Rename default select branch to 'default' keyword
Previously I was using an underscore to denote the default branch
because I was thinking that I would allow variable bindings in the
select branches, and 'default' could be mistaken for the name of a
variable. But I think it's better to just introduce alternate syntax,
like `default @ my_var: "foo" + my_var,` to do the variable bindings,
so that we can have a clearer name for the default case.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ied762694e453855c03dd471898ebb52e97a5a671
2024-04-04 12:02:46 -07:00
Cole Faust
021cc8f5b8 Add support for unset select branches
Currently, with the arch/os mutator, you can override a property
using the default value for just a few arch types, for example:

cc_defaults {
    name: "my_defaults",
    target: {
        windows: {
            enabled: true,
        }
    }
}

cc_binary {
    name: "foo",
    enabled: false,
    defaults: ["my_defaults"],
}

You could make a select statment that acts like the above if it were
all in one module, but currently with select statements you can't make
a defaults module that can be generically applied to any other module
and have the same behavior as the above.

After this cl, the defaults module could look like:

cc_defaults {
    name: "my_defaults",
    enabled: select(variant("arch"), {
        "windows": true,
        _: unset,
    }),
}

Which would have the same behavior. Unset may also be useful for
setting the property under some configurations, but wanting to leave
the implementation-specific default value in others.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I3ea3277ea8b9a0ac5e613b4378945388b9df036a
2024-04-02 16:35:33 -07:00
Colin Cross
a6074ea049 Merge changes I4bb1a17f,I05ffdf6d,I5dc201f4,If16ee798,I2e42c0d6, ... into main
* changes:
  Store mutator in mutatorContext instead of name
  Explicitly delete old logicModule entries from Context.moduleInfo
  Keep logicModule for obsolete variants
  Add Provider to transition contexts
  Cache outgoing transitions
  Use maps and slices packages
2024-04-02 17:24:37 +00:00
Colin Cross
d5d55e5b26 Store mutator in mutatorContext instead of name
TransitionMutators will need access to the original *mutatorInfo,
store it in mutatorContext instead of the name, and update all
accesses to the name to go through the *mutatorInfo.

Bug: 319288033
Test: go test ./...
Change-Id: I4bb1a17f44e55b23dd70708c9a5fde2ae80ce154
2024-04-01 15:55:16 -07:00
Colin Cross
1d5e1a56fe Explicitly delete old logicModule entries from Context.moduleInfo
Creating variants has an optimization to reuse the logicModule of the
original variant as the logicModule of the first new variant.  Using
the same logicModule meant that updating the Context.moduleInfo map
from logicModules to *moduleInfo would overwrite the old entry for
the original variant with the new entry for the first variant.

TransitionMutators will need to keep the original logicModule for
later use, which will require disabling the optimization that reuses
it.  When the first variant is added to the map it no longer overwrites
the original variant.  Excplicitly track the original logicModule and
remove it from the map if necessary.

Bug: 319288033
Test: go test ./...
Change-Id: I05ffdf6d7ce60508f6d9e9657c21032f2c4f5d9c
2024-04-01 15:55:16 -07:00
Colin Cross
7b5888a4f6 Keep logicModule for obsolete variants
When splitting a module into variants the original *moduleInfo was
left in place but with logicModule set to nil as a marker that the
variant was obsolete and any incoming dependencies needed to be resolved
onto one of the newly split variants.  A change to allow TransitionMutator
IncomingTransition calls when adding dependencies later will require
keeping the obsolete *moduleInfo and logicModule around so that
IncomingTransition can be called on it. so use an explicit boolean
to mark the module as obsolete instead of clearing logicModule.

Bug: 319288033
Test: go test ./...
Change-Id: I5dc201f43442aa07ac1b858240c675bab3782b55
2024-04-01 15:55:15 -07:00
Colin Cross
fab4866a68 Add Provider to transition contexts
Allow TransitionMutator OutgoingTransition and IncomingTransition
methods to access Providers from the current and dependency module
respectively.

Bug: 319288033
Test: none
Change-Id: If16ee7980ee0b8f4abaf3c112738fca3f13ab61c
2024-04-01 15:55:13 -07:00
Colin Cross
fca423d9e6 Cache outgoing transitions
TransitionMutators were calling OutgoingTransition and IncomingTransition
twice, once to determine all the necessary variations for each module,
and then again when creating the variations to resolve dependencies.
Store the final variation needed for each dependency during the first
computation, and use it to resolve the dependency variation in the second
pass.

Bug: 319288033
Test: all soong tests
Change-Id: I2e42c0d69efbfcff915267c484c18554b857e0b6
2024-04-01 15:54:42 -07:00
Colin Cross
4cc5290aac Use maps and slices packages
Use slices.Clone, slices.Contains, maps.Clone and maps.Equal.

Test: go test ./...
Change-Id: I96596f157dec2558007da4917c998a64d0cc4d79
2024-04-01 15:54:22 -07:00
Cole Faust
46cf6764fd OR booleans together when appending configurables
This is to match how (non-pointer) booleans work in AppendProperties().

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I16791fc5ac684eedf8064a65953b8b68f18c37c1
2024-03-27 16:39:07 -07:00
Cole Faust
02790f32c9 Don't recurse into configurable properties in filterPropertyStructFields
filterPropertyStructFields should treat configurable properties as
leaf nodes and not recurse into them. Configurable properties can be
made arch_variant, at least while we transition to full use of
Configurable.

Bug: 323382414
Test: soong tests when changing enabled to be configurable
Change-Id: Ib22dd3797f69e912dab0e0c3c1030d6c1f04db7e
2024-03-26 16:46:48 -07:00
Cole Faust
4f968700d1 Add OtherModulePropertyErrorf
...and pass property name to ConfigurableEvaluator.

When trying to convert Enabled to a configurable property, I enountered
a bunch of different context types that all had different error methods.
OtherModulePropertyErrorf is an error method that can be implemented
by all contexts.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I38b2a545c0ee8d23281cd88bc397f79f39835bc4
2024-03-22 13:02:51 -07:00
Cole Faust
894b318528 Fix error reporting in MatchesIncludeTags
ctx.ModuleErrorf for the main blueprint config object doesn't
report an error, it just returns it.

If go had something like C++'s `[[nodiscard]]` or rust's `#[must_use]`
that would be useful here, but the issue has been open since 2017:
https://github.com/golang/go/issues/20803

Test: Presubmits
Change-Id: I72709e6c5466d55f5c0f3fe51a25c29df0826271
2024-03-21 16:16:49 -07:00
Cole Faust
d2f1141871 Get rid of some extra newlines in bpfmt
Bug: 323382414
Test: go tests, and I also ran both the old/new bpfmts on the whole aosp source tree, and the only difference between the results was one extra removed line in external/uwb
Change-Id: I4942c9247a66f1de5028de39caa5cd34b66093c3
2024-03-19 17:07:00 -07:00
Cole Faust
a962edf763 Add proptools.Slice
To make it easier to handle the result of
Configurable[[]string].Evaluate()

Bug: 329711542
Test: go tests
Change-Id: I7364170564b1049a33873424c6da4fc26aff305b
2024-03-14 14:28:52 -07:00
Cole Faust
6437d4e737 Select statements
Select statements are a new blueprint feature inspired by bazel's select
statements. They are essentially alternative syntax for soong config
variables that require less boilerplate. In addition, they support
making decisions based on a module's variant, which will eliminate
the need for manual property struct manipulation, such as the arch
mutator's arch: and target: properties.

In order to support decisions based on the variant, select statements
cannot be evaluated as soon as they're parsed. Instead, they must be
stored in the property struct unevaluated. This means that individual
properties need to change their type from say, string, to
Configurable[string]. Currently, only configurable strings, bools, and
string slices are supported, but more types can be added later.
The module implementation must call my_property.Evaluate(ctx) in order
to get the final, resolved value of the select statement.

Bug: 323382414
Test: go tests
Change-Id: I62f8721d7f0ac3d1df4a06d7eaa260a5aa7fcba3
2024-03-06 15:00:39 -08:00
LaMont Jones
34847c828d debugValue: handle Interfaces better. am: 89e90b4c60
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/2975295

Change-Id: Iba23845605d8cdeb400f65cc3da34e4e2239b1d6
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-02-26 20:35:19 +00:00
LaMont Jones
89e90b4c60 debugValue: handle Interfaces better.
Bug: none
Test: manual
Change-Id: Ic8ee668abf4387fc2f00e1f520efea360eebd66c
2024-02-23 15:04:04 -08:00
Joe Onorato
2e39a573ce Add maps to soong-debug-info.json am: 5920e5507e
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/2968297

Change-Id: I3506b8e2ab0928381d11d0ff14e07f207ac28a43
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2024-02-20 18:49:14 +00:00
Joe Onorato
5920e5507e Add maps to soong-debug-info.json
Test: m nothing
Change-Id: Iffa308e1b93a5f7e1cea7587bef3540d1eb37de5
2024-02-18 14:47:44 -08:00