Commit graph

3018 commits

Author SHA1 Message Date
Yu Liu
d5133cfc64 Use FNV instead of maphash as the hasher for soong.
This is to support recalculate hash in different proceses.

Bug: 335718784
Test: unit tests and CI
Change-Id: I08909fe0332a7adcfcc158698c5d8ba501116ba2
2024-05-29 22:03:29 +00:00
Yu Liu
b83d6420e2 Merge "Need to catch the errors when closing and flushing." into main 2024-05-24 22:41:19 +00:00
Yu Liu
01a4667ba2 Need to catch the errors when closing and flushing.
Bug: 335718784
Test: CI
Change-Id: I6ae984b36ed786d1ccaa67d5c05efd3d6b89504b
2024-05-24 21:37:32 +00:00
Treehugger Robot
177fd07327 Merge "Parallelize VerifyProvidersWereUnchanged" into main 2024-05-24 20:28:27 +00:00
Cole Faust
7b7b1db4a8 Parallelize VerifyProvidersWereUnchanged
This makes it faster than the write_files step that it's in parallel
with. It's a ~3 second improvement to soong's runtime on my computer
on aosp main.

I'm not using parallelVisit() this time around, because
VerifyProvidersWereUnchanged is run in parallel with other code
that also uses parallelVisit(), and parallelVisit() modifies
module.waitingCount, so they conflict with each other.

Bug: 335718784
Test: m nothing
Change-Id: I4c6a4c30e2ffc4606faad378f59e003a02826848
2024-05-24 12:10:06 -07:00
Treehugger Robot
0c529c480a Merge "Update go.mod file to go 1.22" into main 2024-05-24 18:57:09 +00:00
Yu Liu
da55eb2a45 Merge "Pass the parameters to goroutine to avoid relying on a go 1.22 fix." into main 2024-05-24 18:05:25 +00:00
Yu Liu
a99fc848a4 Pass the parameters to goroutine to avoid relying on a go 1.22 fix.
Bug: 335718784
Test: CI
Change-Id: Iae55e59a58101bc373666455ae4f2b6146c1abca
2024-05-24 16:45:50 +00:00
Priyanka Advani
6daa49ee97 Merge "Revert "Parallelize VerifyProvidersWereUnchanged"" into main 2024-05-24 00:05:46 +00:00
Priyanka Advani
533959eb27 Revert "Parallelize VerifyProvidersWereUnchanged"
This reverts commit 2071d15a29.

Reason for revert: Droidmonitor triggered revert due to b/342483801.

Change-Id: Ib892a3b17a5fcefb40d316e9dce9d5432293ec25
2024-05-23 23:55:27 +00:00
Cole Faust
a3a125aaf2 Update go.mod file to go 1.22
Bug: 330574836
Test: Presubmits
Change-Id: I871cabd7aa403b315d5efcf75f5eff6df492478f
2024-05-23 16:26:02 -07:00
Treehugger Robot
f887364cd2 Merge "Parallelize VerifyProvidersWereUnchanged" into main 2024-05-23 23:12:56 +00:00
Cole Faust
239c43a66c Merge "Revert "Bump go.mod file to go 1.22"" into main 2024-05-23 22:24:35 +00:00
Cole Faust
f764ab9f16 Revert "Bump go.mod file to go 1.22"
This reverts commit 65b6f3cb23.

Reason for revert: breaks build-tools build

Change-Id: Ib3dbaa57d871796024a0f88569f1ad933ddf4f76
2024-05-23 22:22:56 +00:00
Cole Faust
2071d15a29 Parallelize VerifyProvidersWereUnchanged
This makes it faster than the write_files step that it's in parallel
with. It's a ~3 second improvement to soong's runtime on my computer
on aosp main.

Bug: 335718784
Test: m nothing
Change-Id: I9dc9bd9c4cb80e61f880531dbb661c971298607c
2024-05-23 15:09:36 -07:00
Treehugger Robot
077df5875e Merge "Preserve type when promoting non-selects to selects" into main 2024-05-23 19:50:12 +00:00
Cole Faust
65b6f3cb23 Bump go.mod file to go 1.22
Blueprint bootstraps itself, go 1.22 will be used, but if you used
go build or go test manually, the version in the go.mod will be used.

Go 1.22 has a subtle change that we want to make sure we're always
using: https://go.dev/blog/loopvar-preview

Test: Presubmits
Change-Id: I192dbcba16f431458dfb758c505a63027cdb2533
2024-05-23 11:27:22 -07:00
Cole Faust
0673769aab Merge "Add proptools.PrintfIntoConfigurable" into main 2024-05-22 23:58:50 +00:00
Cole Faust
9ffb480975 Add proptools.PrintfIntoConfigurable
This will be used by soong config value variables to support their
usage on configurable properties.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I20b8ef29e81512548ecdc056533700a30fa7815a
2024-05-22 14:32:24 -07:00
Cole Faust
86a7abd927 Preserve type when promoting non-selects to selects
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I1ad7e891f5422bb3587ebac1f8bda7dd5792ae3b
2024-05-22 13:40:57 -07:00
Cole Faust
65e5724978 Merge "Fix error message when a select condition isn't handled" into main 2024-05-22 18:51:10 +00:00
Cole Faust
ad00dd50bf Return an Optional[T] from Configurable.Get()
Previously, Configurable.Get() copied the value in the property,
because it needed to return a pointer in order to indicate whether
the property was set or not. Now, it returns a ConfigurableOptional[T],
which is the same as the pointer, but it prevents users from altering
the pointed-to value, so we don't need to copy it.

There are still copies for slice properties, because those are also
pointers. In the future we may want to consider making an ImmutableList
type to use instead.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ic9ed5ba269d10158e3eac1fea272555c9fa5c0e8
2024-05-21 14:48:39 -07:00
Yu Liu
82e444710d Write ninja file in parallel.
Bug: 335718784
Test: CI
Change-Id: I26f7babca349c654780711cfe0f0ece3faa5f436
2024-05-20 21:46:19 +00:00
Cole Faust
fa652f4714 Fix error message when a select condition isn't handled
Preivously it attempted to print all conditions and their values,
but that doesn't tell you which one was wrong, and the formatting was
all weird due to trying to print complex go types.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: If9653796f5e9139c0b4a7843b441eb0409967b55
2024-05-09 15:50:50 -07:00
Colin Cross
515e7991fc Merge "Handle nil variant in applyTransitions" into main 2024-05-09 22:15:46 +00:00
Colin Cross
eb077c3123 Handle nil variant in applyTransitions
The input variationMap in applyTransitions may be nil, initialize
it if necessary.

Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: I091aa5a04732fdf18931b2534c4c479152c2a9e1
2024-05-09 10:51:20 -07:00
Treehugger Robot
0bc93e1a92 Merge "Create embedcfg directory before writing the file" into main 2024-05-09 16:42:08 +00:00
Tomasz Wasilczyk
2fbe882aa8 Create embedcfg directory before writing the file
Bug: 302718225
Test: rm -rf out/ && m out/host/linux-x86/bin/go/soong-cc/pkg/android/soong/cc.a
Change-Id: I206f4e0f83fae6a8c6b69c648d23a8c6d1f51477
2024-05-09 07:27:58 -07:00
Colin Cross
5bc2b73593 Merge changes I648ef95f,Iadb72a5a,Ib073f6ec into main
* changes:
  Don't sort the results of TransitionMutator.Split
  Apply outgoing transitions when adding dependencies
  Delete transition variation when no matching variant was found
2024-05-08 23:43:56 +00:00
Colin Cross
c5b3c0ca9e Don't sort the results of TransitionMutator.Split
The ordering of variants can be significant  when adding
inter-variant dependencies, as a variant can only depend on earlier
variants.  Allow maintaining the existing variant ordering when
converting mutators to TransitionMutator by keeping the ordering
of TransitionMutator.Split.  Variations that were requested by
incoming dependencies that are not present in Split are still
sorted, as they have no inherent ordering.

Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: I648ef95f08a05f9a64ce97e6a39bae10ce88771a
2024-05-08 15:22:27 -07:00
Colin Cross
5ac6648829 Apply outgoing transitions when adding dependencies
TransitionMutators can always return "" from OutgoingVariation in
order to implement the equivalent of CreateLocalVariations - a
variation that doesn't affect which variation of dependencies is
selected.  The same need applies when creating the dependency after
the TransitionMutator.  Call OutgoingVariation on the parent module
when adding new dependencies unless the variation was explicitly
requested.

Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: Iadb72a5a3f19579a6c1d34e3c7169ee9e2b53210
2024-05-08 15:22:21 -07:00
Colin Cross
02aa24823e Delete transition variation when no matching variant was found
TransitionMutators sometimes apply to one variant of a module but
not to another.  Deleting the variation when all variants were
untouched is insufficient if a post-mutator dependency is being added
on a variant that was untouched.  Delete the variation whenever
no matching incoming variant was found.

Bug: 319288033
Test: TestPostTransitionDeps
Flag: NONE
Change-Id: Ib073f6ec3090d09e4798b6f9ca3061ec5d58d722
2024-05-07 14:36:48 -07:00
Cole Faust
0a1533d786 Merge "Support generating docs for Configurable properties" into main 2024-05-06 23:28:57 +00:00
Cole Faust
b9ff002303 Support generating docs for Configurable properties
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6e8a03c5785dbc3a90f458155dc4b9cd6ce7700b
2024-05-06 15:14:17 -07:00
Colin Cross
38b56e136f Change semantics of ReplaceDependencies[If]
ReplaceDependencies[If] currently replaces dependencies on a given
module with the current module.  It expects to find a variant of the
given module that has the exact same variations as the current module.
That was sometimes handled via aliases, but TransitionMutators don't
support aliases, they use IncomingTransition to rewrite the variation
instead.

In all current usages of ReplaceDependencies[If], the given module
is also a direct dependency of the current module.  Instead of looking
for the exact same variations, look for the variant that is a dependency
of the current module.

Bug: 319288033
Test: all soong tests pass
Flag: NONE
Change-Id: I3e33111322040b187f6e951554366ccdcaf1bc11
2024-05-02 14:39:19 -07:00
Colin Cross
de024f6167 Ignore dependency variations for TransitionMutators that didn't apply
When adding a dependency on a module that has already had variants
created by a TransitionMutator, ignore any variation added by the
TransitionMutator to the current module if the TransitionMutator
didn't apply to the target module.  This requires passing in the
list of variations requested by the AddVariationDependencies call
so that explicitly requested variations are not ignored.

This fixes TestAndroidTest_FixTestConfig when the override mutator
is converted to a TransitionMutator, as it can't add a dependency from
the "foo_test" module with an empty override variation to the
"foo" module  that has no override variation.

Bug: 319288033
Test: TestAndroidTest_FixTestConfig
Flag: NONE
Change-Id: If0fca7adc28cd702f953885aef440a0805ac017c
2024-05-01 15:25:37 -07:00
Colin Cross
9f260255f2 Rename TransitionMutator top down mutator
TransitionMutator's top down mutator is suffixed with _deps, but
that name is often used by other mutators related to the
TransitionMutator, causing unexpected conflicts.  Use _propagate
instead.

Bug: 319288033
Test: all soong tests pass
Flag: NONE
Change-Id: Ib42c66288e47287cfda8252d435f86c7045bf4c0
2024-05-01 15:25:37 -07:00
Cole Faust
1c48101091 Improve error message when assigning select to nonconfigurable property
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ideac33cd286566e14e7ff1ff579930add48a94db
2024-04-30 13:58:17 -07:00
Treehugger Robot
ca5ffdd3ce Merge changes from topics "fix_selects_appending", "refactor_selects" into main
* changes:
  Refactor selects
  Fix bugs when appending selects
  Allow extending configurable propeties with non-configurable properties
2024-04-30 01:15:42 +00:00
Cole Faust
a52b058ccc Refactor selects
In order to do less cloning, refactor selects so that all the
soong-visibile structs are immutable to soong and can be reused.

Additionally, refactor how the inner linked list of selects is managed,
so that the append/prepend/replace logic is simpler.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Iba5d27405decc1b0596590c3e0555daeb044bf9e
2024-04-29 13:23:30 -07:00
Cole Faust
70b814a964 Fix bugs when appending selects
The biggest issue here is that Configurable objects needed to be
cloned before appended to one another, otherwise some Configurables
that were used in defaults could be edited after being applied to one
module and apply a different value to another module.

Also fix an issue where a select without a defined appendWrapper
always evaluated to nil.

I plan to make a followup refactor cl to make these things clearer,
but start with the bugfix.

Bug: 323382414
Test: m nothing --no-skip-soong-tests (see other cl in topic for tests)
Change-Id: Icf68d0ee1779c76bfb3d68db43b35d7e09bc0dd9
2024-04-25 15:38:56 -07:00
Cole Faust
4560bb086e Allow extending configurable propeties with non-configurable properties
Sometimes modules add arch-variant properties in load hooks, to disable
modules by default on certain platforms for example. When changing the
property to a Configurable property, these load hooks would also need
to be changed in order to have a matching type for
ExtendMatchingProperties.

Since this can be kindof a pain to address everywhere, for now,
special case the extension functions to promote non-configurable
properties to configurable ones. We can remove this later when
everything switches to configurable properties.

Bug: 323382414
Test: go tests
Change-Id: Iac96587dbd60ccdd6aa667dd69a71ad252abe589
2024-04-25 15:31:00 -07:00
Haamed Gheibi
c10b49b5e9 Use relative GOPATH for microfactory golang builds.
Bug: b/336890676
Change-Id: I15ff839986d6c4997d5abccaa2ae8a9703d6cfb6
2024-04-25 04:44:01 +00:00
Tomasz Wasilczyk
36b6322979 Merge "Add support for go:embed" into main 2024-04-24 12:33:37 +00:00
Cole Faust
185cb44bef Export ConfigurableCase and add constructors
Some soong code sets arch-variant properties in order to control a
module's default behavior. I'll make this continue to work, but long
term the arch-variant properties should be replaced with selects,
so expose an API for creating select statements in soong code.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6c65d6e112b6f826f1027777b6fdf36915d34b1d
2024-04-23 13:55:59 -07:00
Tomasz Wasilczyk
51b4b03520 Add support for go:embed
Bug: 302718225
Test: Build aosp/2798713
Change-Id: I8c09f1604aa1a702c639bba91cefd104f8f2f36d
2024-04-22 11:52:00 -07:00
Colin Cross
d7474dd743 Call TransitionMutator.IncomingTransition when adding dependencies later
Adding a dependency on a module that has already had a TransitionMutator
run on it may require adjusting the variation name based on the results
of IncomingTranstion.  Store the variants that existed before the
TransitionMutator ran, find one that is a subset of the requested
variant, and call TranstionMutator.IncomingTransition to update the
value.

Bug: 319288033
Test: TestPostTransitionDeps
Change-Id: I690357f9792401a3edbc5ae9fdcb666495954fbc
2024-04-16 15:18:29 -07:00
Colin Cross
6ed94b7f85 Merge "Move TransitionMutator to transition.go and add tests" into main 2024-04-16 20:14:53 +00:00
Cole Faust
28357db9d0 Support unpacking a variable to a configurable property
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I504e53c34305f5de0af8629bf21ac1ce94704732
2024-04-12 16:37:51 -07:00
Cole Faust
3311debbb3 Support multi-variable selects and typed selects
This adds support for selecting on multiple variables at once, so that
you can do AND/OR combindations of them. For example:

select((
    arch(),
    os(),
), {
    ("arm64", "linux"): ["libfoo64"],
    (default, "linux"): ["libfoo"],
    (default, "windows"): ["libfoowindows"],
    (default, default): ["libbar"],
})

It also allows for select conditions to be boolean-typed. You can
write literal true and false without quotes to select on them. Currently
we don't have any boolean-typed variables though, so a fake one was
added for testing.

Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ibe586e7b21865b8734027848cc421594cbd1d8cc
2024-04-12 16:33:01 -07:00