Some soong code sets arch-variant properties in order to control a
module's default behavior. I'll make this continue to work, but long
term the arch-variant properties should be replaced with selects,
so expose an API for creating select statements in soong code.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6c65d6e112b6f826f1027777b6fdf36915d34b1d
This adds support for selecting on multiple variables at once, so that
you can do AND/OR combindations of them. For example:
select((
arch(),
os(),
), {
("arm64", "linux"): ["libfoo64"],
(default, "linux"): ["libfoo"],
(default, "windows"): ["libfoowindows"],
(default, default): ["libbar"],
})
It also allows for select conditions to be boolean-typed. You can
write literal true and false without quotes to select on them. Currently
we don't have any boolean-typed variables though, so a fake one was
added for testing.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ibe586e7b21865b8734027848cc421594cbd1d8cc
Part of the design of property structs is that they were easy to access.
In keeping with that spirit, use a shorter and easier to spell name
for the getter, and add GetDefault() so that you don't need to pass
the result of Get() to one of the
proptools.StringDefault/BoolDefault/etc functions.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ib9a69dcf2ab56a758935a461f37fe46bc0e17e27
If a property is a pointer to a bool/string, that property would be
replaced instead of appended to when calling ExtendProperties().
This is confusing behavior, and I don't want to give anyone any reason
to use a pointer to a configurable property, so add this relflection
tag to recreate the functionality.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I6b9e4dfee1d2685e0c9dc6a646cf45724cc04756
Previously I was using an underscore to denote the default branch
because I was thinking that I would allow variable bindings in the
select branches, and 'default' could be mistaken for the name of a
variable. But I think it's better to just introduce alternate syntax,
like `default @ my_var: "foo" + my_var,` to do the variable bindings,
so that we can have a clearer name for the default case.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ied762694e453855c03dd471898ebb52e97a5a671
Currently, with the arch/os mutator, you can override a property
using the default value for just a few arch types, for example:
cc_defaults {
name: "my_defaults",
target: {
windows: {
enabled: true,
}
}
}
cc_binary {
name: "foo",
enabled: false,
defaults: ["my_defaults"],
}
You could make a select statment that acts like the above if it were
all in one module, but currently with select statements you can't make
a defaults module that can be generically applied to any other module
and have the same behavior as the above.
After this cl, the defaults module could look like:
cc_defaults {
name: "my_defaults",
enabled: select(variant("arch"), {
"windows": true,
_: unset,
}),
}
Which would have the same behavior. Unset may also be useful for
setting the property under some configurations, but wanting to leave
the implementation-specific default value in others.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I3ea3277ea8b9a0ac5e613b4378945388b9df036a
* changes:
Store mutator in mutatorContext instead of name
Explicitly delete old logicModule entries from Context.moduleInfo
Keep logicModule for obsolete variants
Add Provider to transition contexts
Cache outgoing transitions
Use maps and slices packages
This is to match how (non-pointer) booleans work in AppendProperties().
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I16791fc5ac684eedf8064a65953b8b68f18c37c1
filterPropertyStructFields should treat configurable properties as
leaf nodes and not recurse into them. Configurable properties can be
made arch_variant, at least while we transition to full use of
Configurable.
Bug: 323382414
Test: soong tests when changing enabled to be configurable
Change-Id: Ib22dd3797f69e912dab0e0c3c1030d6c1f04db7e
...and pass property name to ConfigurableEvaluator.
When trying to convert Enabled to a configurable property, I enountered
a bunch of different context types that all had different error methods.
OtherModulePropertyErrorf is an error method that can be implemented
by all contexts.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I38b2a545c0ee8d23281cd88bc397f79f39835bc4
To make it easier to handle the result of
Configurable[[]string].Evaluate()
Bug: 329711542
Test: go tests
Change-Id: I7364170564b1049a33873424c6da4fc26aff305b
Select statements are a new blueprint feature inspired by bazel's select
statements. They are essentially alternative syntax for soong config
variables that require less boilerplate. In addition, they support
making decisions based on a module's variant, which will eliminate
the need for manual property struct manipulation, such as the arch
mutator's arch: and target: properties.
In order to support decisions based on the variant, select statements
cannot be evaluated as soon as they're parsed. Instead, they must be
stored in the property struct unevaluated. This means that individual
properties need to change their type from say, string, to
Configurable[string]. Currently, only configurable strings, bools, and
string slices are supported, but more types can be added later.
The module implementation must call my_property.Evaluate(ctx) in order
to get the final, resolved value of the select statement.
Bug: 323382414
Test: go tests
Change-Id: I62f8721d7f0ac3d1df4a06d7eaa260a5aa7fcba3
The property squashing functions were performing 19% of all allocations
and allocating 3.43 GB, most of which was used to track the name of the
property, but the name of the property is only used to print errors.
Keep the elements of the name in a preallocated slice instead, and only
compute the name when an error occurs.
Calling reflect.Value.Interface() was also expensive, and only passed
to filter and order functions, none of which use the values. Modify
the signature of the filter and order functions to remove the interfaces
and the property name.
Test: extend_test.go
Change-Id: I517f89daf251bb43f7cfefa6f1e83951c0e271b7
maphash.Hash implements WriteString, which avoids an allocation in
order to convert the string to a byte slice. Using the concrete
type instead of the io.Writer interface also allows int64Array to
be allocated on the stack.
Test: SOONG_PROFILE_MEM=/tmp/mem.pprof m nothing
Change-Id: I5894f7399c2a232f5f67d7d0724a6115ba2c278f
When setProvider() is called, hash the provider and store the hash in
the module. Then after the build is done, hash all the providers again
and compare the hashes. It's an error if they don't match.
Also add a flag to control it in case this check gets slow as we convert
more things to providers. However right now it's fast (unnoticable
in terms of whole seconds) so just have the flag always enabled.
Bug: 322069292
Test: m nothing
Change-Id: Ie4e806a6a9f20542ffcc7439eef376d3fb6a98ca
When property a.b.c is not used, (also there is no a.* or a.b.* used)
"a", "a.b" and "a.b.c" are all in unusedNames.
removeUnnecessaryUnusedNames only keeps the last "a.b.c" as the
real unused name.
Test: TestNonExistentPropertyInSoongConfigModule, unpack_test.go and CI
Bug: 171232169
Change-Id: I861fa6933e558b07694ee5ff40ef549117d115ff
TestExternalShellEscaping and TestExternalShellEscapeIncludingSpaces
use "echo -n", which fails on darwin. These tests weren't running on
darwin because they were only run in Soong, which always limits to
only short tests. The test are now run in aosp-build-tools, which
doesn't limit to short tests.
Remove the unsupported -n argument from echo and trim the added newline
instead.
Test: TestExternalShellEscaping and TestExternalShellEscapeIncludingSpaces
Change-Id: I3d8ff1c0db0af386e1dc13cb6c2dabe561c1c89e
proptools.Clear(ptr) clears a property with its zero value.
Bug: 313806237
Test: m blueprint_tests
Change-Id: Ib78f9f88a9b0a8b04e1ab6c5e545b55ba4269e5d
Go 1.21 does a better job using the same empty allocation for empty
structs, allow cloned properties to point to the original when it
is an empty struct.
Bug: 309895579
Test: TestCloneProperties
Change-Id: I064f2316a8a8017a109968671ac305dbbe3246af
NinjaEscapeList is called on every input or output of a rule, and
most of the time does not escape anything. Optimize it by returning
the input slice when nothing was escaped. This avoids 1.336 GB of
allocations in my AOSP aosp_cf_x86_64_phone-userdebug build.
Test: TestNinjaEscapeList
Change-Id: I33b9e7b77b33d10401d1ec3546caa6794c567b16
This limits support to allow-listed property names to prevent
proliferation of map types requiring additional support to migrate.
Test: go test blueprint tests
Test: m nothing && diff build.ninja & Android-aosp_arm.mk -- no changes
Change-Id: Id12637462f19ac5de1b562f63507de989a51600d
Add a method that ninja and shell escapes each entry in a list
of strings as a single argument.
Test: escape_test.go
Change-Id: Iac05c64f1ae48cd6186f563790ea49b90618038d
Recurse into embedded anonymous structs and the BlueprintEmbed
workaround structs when looking for properties in
extendPropertiesRecursive.
Test: proptools/extend_test.go
Change-Id: I975651a64e5173747403629a09263562761f1495
This support enables specifying properties of the type "map" within a
Soong module, but explicitly does not allow them to be used within a bp
file.
This means that rather than specifying each arch/os/target within a
struct to support arch-variant properties/attributes, we can use a map.
This allows us to simplify the implementation of LabelAttribute,
StringListAttribute, and LabelListAttribute as the number of select
statements supported becoming large and hard results in a lot of
duplication.
Test: go test blueprint tests
Test: m nothing
Change-Id: I88cc5952a6bdb60a2344fa0737216f016086cea5
Previously, a struct (anonymous or named) whose fields all matched the
predicate would not be sharded and would simply be reused. However,
that could break the maxNameSize limitation which could cause problems
for the caller.
This change makes sure that the supplied struct is only reused if it
does not exceed the maxNameSize, and otherwise is sharded.
Bug: 183777071
Test: m nothing
Change-Id: I8af272ec121077a43333e72b67cfd0e493c83362
ShellEscape(string) doesn't escape a string with spaces like "arg1
arg2". However, when we want to escape a string to use it as an
argument, then strings with spaces should be escaped.
For example,
"command " + ShellEscapeIncludingSpaces("a b")
becomes "command 'a b'" so that "command" will get "a b" as a single
argument.
Bug: 182092664
Test: Added tests to escape_test.go
Change-Id: I8f88c18bc4f9f7aacfe9e701b8f0876dd8b9a8c3
The function now can traverse into a field whose type is slice of
struct. When reading field values from the returned indexes, Soong will
check if the next field is a slice of struct or not. If so, it will
recurse into all the values in the slice.
Bug: 181018147
Test: m nothing
Change-Id: Ib8a7b7911a0be37a6dc03079adeb906497e60875
The commits optimizes HasTag function by avoiding
the call to strings.Split that allocates a list of strings.
HasTag is called from the ExtendProperties family of functions.
So this new implementation would benefit any primary builder
that extensively uses property appending.
Add a small benchmark for HasTag function.
It shows that HasTag is speed up by a factor of 2-3 with the new
implementation.
Before:
goos: linux
goarch: amd64
BenchmarkHasTag/NoTag-56 20000000 118 ns/op
BenchmarkHasTag/EmptyTag-56 20000000 120 ns/op
BenchmarkHasTag/OtherTag-56 10000000 131 ns/op
BenchmarkHasTag/MatchingTag-56 10000000 177 ns/op
BenchmarkHasTag/ExtraValues-56 5000000 392 ns/op
BenchmarkHasTag/ExtraTags-56 10000000 183 ns/op
After:
goos: linux
goarch: amd64
BenchmarkHasTag/NoTag-56 200000000 11.5 ns/op
BenchmarkHasTag/EmptyTag-56 200000000 11.5 ns/op
BenchmarkHasTag/OtherTag-56 50000000 25.2 ns/op
BenchmarkHasTag/MatchingTag-56 20000000 61.4 ns/op
BenchmarkHasTag/ExtraValues-56 20000000 94.3 ns/op
BenchmarkHasTag/ExtraTags-56 20000000 79.1 ns/op
Signed-off-by: Francois Berder <francois.berder@arm.com>
Change-Id: Ib45498e9ad6aebeca2beddea63543da40c0b1a21
Values assigned to map properties were silently ignored instead of
reported as an error. Add a check when recursing into structs that
the value is a map.
Fixes: 177706602
Test: m nothing
Test: TestUnpackErrors
Change-Id: Ic56aeb1b9da6d5c86b6d98adae7bddb60c450404
Returning a fmt.Errorf error causes Blueprint to identify the error
as an "internal error", return an UnpackError instead.
Bug: 177706602
Test: TestUnpackErrors
Change-Id: I19fba134ad778d08d5a4d90a0335bdf8cbea6a20
Add more tests, give the tests names, run them as subtests, and add
benchmarks.
Test: unpack_test.go
Change-Id: Iff22538ce44ed503d5d088cfb55673448db998f1
Field "X86" has no lowercase runes and was being left uppercase.
Change the new PropertyNameForField rules to lowercase the name unless
it has any uppercase rune after the first rune (which is always
uppercase) and no lowercase runes.
Bug: 148865218
Test: proptools_test.go
Change-Id: Ifd1c10fc03f5ae1765d25b3f73dba8fd61c5c956
Soong config variables may propagate an uppercase name from Make.
Blueprint properties have traditionally been all lowercase, and
using an uppercase property struct field name resulted in a strange
Blueprint property name with the first rune lowercase and the
remaining runes uppercase.
Update the rules for proptools.PropertyNameForField to not lowercase
the first rune if the field name has mulitple runes and is not all
uppercase.
Fixes: 148865218
Test: proptools_test.go
Change-Id: I8de2f65ffb00e5a8ce0aea0caf09f5859315f6b8
The proptools functions took an inconsistent variety of
struct and *struct types. Some methods even took a struct
but returned a *struct. Make all the exported methods
take a *struct, with internal helpers for the ones that need
to take a struct.
Test: proptools tests
Change-Id: I60ce212606e96adcef66c531d57f69c39e1a1638
The filter tag is unused, replaced with FilterPropertyStruct to
generate a new type at runtime that only contains the filtered
fields.
Test: unpack_test.go
Change-Id: Id91cf99290832094d05426f3263279836f0fea73
FilterPropertyStructSharded was just sharding the top level
properties into groups of 10. For nested property structs
this can be insufficient - there could be a single top level
property with many properties below it.
Take a maximum name size, and track the size used by parent
structs to determine when sharding a nested struct is necessary.
Bug: 146234651
Test: filter_test.go
Change-Id: I5b5ed11ea27a0325b2fd6c2c3fb427ea1e2af0c2
override_* in Soong requires a module to override certain properties of
other module. In that case, values of a slice property (e.g. []string)
should be replaced by the same property value in the overriding module.
However, since proptools only supports Append and Prepend orders where
the original values are kept for slice properties, the behavior
couldn't be implemented. To support the use case, Replace order is
introduced, in which case slice property values are completely replaced.
For other types of properties, the Replace order behaves exactly the
same as the Append order.
Bug: 144338929
Test: m
Change-Id: Iae9feda035177fe6a22e6e8319c0fdaa9e08e85e
