Add a method that ninja and shell escapes each entry in a list
of strings as a single argument.
Test: escape_test.go
Change-Id: Iac05c64f1ae48cd6186f563790ea49b90618038d
Recurse into embedded anonymous structs and the BlueprintEmbed
workaround structs when looking for properties in
extendPropertiesRecursive.
Test: proptools/extend_test.go
Change-Id: I975651a64e5173747403629a09263562761f1495
This support enables specifying properties of the type "map" within a
Soong module, but explicitly does not allow them to be used within a bp
file.
This means that rather than specifying each arch/os/target within a
struct to support arch-variant properties/attributes, we can use a map.
This allows us to simplify the implementation of LabelAttribute,
StringListAttribute, and LabelListAttribute as the number of select
statements supported becoming large and hard results in a lot of
duplication.
Test: go test blueprint tests
Test: m nothing
Change-Id: I88cc5952a6bdb60a2344fa0737216f016086cea5
Previously, a struct (anonymous or named) whose fields all matched the
predicate would not be sharded and would simply be reused. However,
that could break the maxNameSize limitation which could cause problems
for the caller.
This change makes sure that the supplied struct is only reused if it
does not exceed the maxNameSize, and otherwise is sharded.
Bug: 183777071
Test: m nothing
Change-Id: I8af272ec121077a43333e72b67cfd0e493c83362
ShellEscape(string) doesn't escape a string with spaces like "arg1
arg2". However, when we want to escape a string to use it as an
argument, then strings with spaces should be escaped.
For example,
"command " + ShellEscapeIncludingSpaces("a b")
becomes "command 'a b'" so that "command" will get "a b" as a single
argument.
Bug: 182092664
Test: Added tests to escape_test.go
Change-Id: I8f88c18bc4f9f7aacfe9e701b8f0876dd8b9a8c3
The function now can traverse into a field whose type is slice of
struct. When reading field values from the returned indexes, Soong will
check if the next field is a slice of struct or not. If so, it will
recurse into all the values in the slice.
Bug: 181018147
Test: m nothing
Change-Id: Ib8a7b7911a0be37a6dc03079adeb906497e60875
The commits optimizes HasTag function by avoiding
the call to strings.Split that allocates a list of strings.
HasTag is called from the ExtendProperties family of functions.
So this new implementation would benefit any primary builder
that extensively uses property appending.
Add a small benchmark for HasTag function.
It shows that HasTag is speed up by a factor of 2-3 with the new
implementation.
Before:
goos: linux
goarch: amd64
BenchmarkHasTag/NoTag-56 20000000 118 ns/op
BenchmarkHasTag/EmptyTag-56 20000000 120 ns/op
BenchmarkHasTag/OtherTag-56 10000000 131 ns/op
BenchmarkHasTag/MatchingTag-56 10000000 177 ns/op
BenchmarkHasTag/ExtraValues-56 5000000 392 ns/op
BenchmarkHasTag/ExtraTags-56 10000000 183 ns/op
After:
goos: linux
goarch: amd64
BenchmarkHasTag/NoTag-56 200000000 11.5 ns/op
BenchmarkHasTag/EmptyTag-56 200000000 11.5 ns/op
BenchmarkHasTag/OtherTag-56 50000000 25.2 ns/op
BenchmarkHasTag/MatchingTag-56 20000000 61.4 ns/op
BenchmarkHasTag/ExtraValues-56 20000000 94.3 ns/op
BenchmarkHasTag/ExtraTags-56 20000000 79.1 ns/op
Signed-off-by: Francois Berder <francois.berder@arm.com>
Change-Id: Ib45498e9ad6aebeca2beddea63543da40c0b1a21
Values assigned to map properties were silently ignored instead of
reported as an error. Add a check when recursing into structs that
the value is a map.
Fixes: 177706602
Test: m nothing
Test: TestUnpackErrors
Change-Id: Ic56aeb1b9da6d5c86b6d98adae7bddb60c450404
Returning a fmt.Errorf error causes Blueprint to identify the error
as an "internal error", return an UnpackError instead.
Bug: 177706602
Test: TestUnpackErrors
Change-Id: I19fba134ad778d08d5a4d90a0335bdf8cbea6a20
Add more tests, give the tests names, run them as subtests, and add
benchmarks.
Test: unpack_test.go
Change-Id: Iff22538ce44ed503d5d088cfb55673448db998f1
Field "X86" has no lowercase runes and was being left uppercase.
Change the new PropertyNameForField rules to lowercase the name unless
it has any uppercase rune after the first rune (which is always
uppercase) and no lowercase runes.
Bug: 148865218
Test: proptools_test.go
Change-Id: Ifd1c10fc03f5ae1765d25b3f73dba8fd61c5c956
Soong config variables may propagate an uppercase name from Make.
Blueprint properties have traditionally been all lowercase, and
using an uppercase property struct field name resulted in a strange
Blueprint property name with the first rune lowercase and the
remaining runes uppercase.
Update the rules for proptools.PropertyNameForField to not lowercase
the first rune if the field name has mulitple runes and is not all
uppercase.
Fixes: 148865218
Test: proptools_test.go
Change-Id: I8de2f65ffb00e5a8ce0aea0caf09f5859315f6b8
The proptools functions took an inconsistent variety of
struct and *struct types. Some methods even took a struct
but returned a *struct. Make all the exported methods
take a *struct, with internal helpers for the ones that need
to take a struct.
Test: proptools tests
Change-Id: I60ce212606e96adcef66c531d57f69c39e1a1638
The filter tag is unused, replaced with FilterPropertyStruct to
generate a new type at runtime that only contains the filtered
fields.
Test: unpack_test.go
Change-Id: Id91cf99290832094d05426f3263279836f0fea73
FilterPropertyStructSharded was just sharding the top level
properties into groups of 10. For nested property structs
this can be insufficient - there could be a single top level
property with many properties below it.
Take a maximum name size, and track the size used by parent
structs to determine when sharding a nested struct is necessary.
Bug: 146234651
Test: filter_test.go
Change-Id: I5b5ed11ea27a0325b2fd6c2c3fb427ea1e2af0c2
override_* in Soong requires a module to override certain properties of
other module. In that case, values of a slice property (e.g. []string)
should be replaced by the same property value in the overriding module.
However, since proptools only supports Append and Prepend orders where
the original values are kept for slice properties, the behavior
couldn't be implemented. To support the use case, Replace order is
introduced, in which case slice property values are completely replaced.
For other types of properties, the Replace order behaves exactly the
same as the Append order.
Bug: 144338929
Test: m
Change-Id: Iae9feda035177fe6a22e6e8319c0fdaa9e08e85e
Move some code from Soong to support creating a property struct
at runtime by filtering fields out of another property struct.
Test: TestFilterPropertyStruct
Change-Id: Ic5ae390a885195bebad6f3ecb7c752c0582a60b1
Add proptools.Int and proptools.IntDefault that behave analogously
to proptools.String and proptools.StringDefault.
Change-Id: I41fd3417c973c9ff4a5aa6680546b4b893784745
When CopyProperties comes across private properties, it silently ignores
them. This is error-prone and can result in a bug very hard to debug if
the developer is unaware of the behavior.
Change-Id: Id6a0752e384ec7fed35728c1e87dbfa95fea84f2
Add a function that returns all of the indexes to properties in
a property struct that are tagged with `name:"value"`.
Test: proptools/tag_test.go
Change-Id: I00294934c1a0383c8b64ecaabc0e138682efb2e5
The type field cache was using an atomic pointer to an immutable
map, which required copying the entire map each time a cache
entry was added. Profiling showed that this was a significant
hot spot. Use sync.Map instead.
Change-Id: Ie7c779c5e9e2be1cd530747d74025dcfd206763a
Property structs can now contain slices of structs and other
non-strings as long as they are tagged with `blueprint:"mutated"`.
Test: clone_test.go
Test: unpack_test.go
Change-Id: Ib77348dc6e7314a24f17caba10040f7d3ac54e54
NinjaEscape and ShellEscape operated on lists, which led to
awkward NinjaEscape([]string{s})[0]. Replace NinjaEscape
and ShellEscape with NinjaEscapeList and ShellEscapeList,
and add new NinjaEscape and ShellEscape functions that
operate on a string.
Test: m checkbuild
Change-Id: I283d92cdddc8e0066a300015863a3eab66f77c23
Support int64 number instead of int to be more fixed to bit size so
that the underlying arch won't affect overflow cases. Besides,
refection: func (v Value) Int() int64 always cast to int64 no matter the
input is int, int16, int32. Currently we always treat "-" as negative
sign to bind to next value, and "+" as plus operator to add operands
together.
So we allow:
a = 5 + -4 + 5 or a = -4 + 5
But we don't allow:
a = +5 + 4 + -4 since we don't treat "+" as a positive sign, otherwise,
a = 5 + +5 would exist which looks pretty weird. In the future, we may
want fully support number calculator logic eg, "+"/"-" can be
positive/negative sign or operator, and "(" and ")" will be considered
to group expressions with a higher precedence.
int & uint properties within struct keeps unchanged, which is only
allowed when tagged with 'blueprint:mutated'. We only allow *int64
property instead of int64 property within struct since it does't make
sense to do prepending or appending to int64.
Change-Id: I565e046dbd268af3538aee148cd7300037e56523
Blueprint was using "replace" semantics when unpacking properties
into property structs, meaning if a module factory pre-set property
values they would be overwritten by whatever was in the Blueprint
file. This is different than what would happen if the same property
was updated using the Append*Properties functions in proptools, which
would use "append" semantics, which append strings and lists,
logically ORs booleans and replaces pointers to strings and booleans.
Replace unpack's semantics with append semantics for consistency.
Any previous users of pre-set properties can move to using a pointer
to a string or boolean if they want the old behavior.
Test: unpack_test.go
Test: extend_test.go
Change-Id: I02eebe80916e578938142f8e76889bd985223afc
Blueprint properties that end up as command line arguments need to be
both ninja and shell escaped. Provide helpers that primary builders can
use to appropriately escape them.
Change-Id: Ifd697d87edb1c6f0a910377835c391bbe8f95b42
Allow using ExtendMatchingProperties to extend pointer to a struct or an
interface containing a pointer to a struct using a struct, and
vice-versa.
Also fixes a pre-existing bug where extending a nested structure could
fail if there were multiple possible destnations and some of them did
not have a matching nested property.
Change-Id: I6e69d78eb6595ba7dd2603e3aa7dd8de3f292744
Allow primary builders to reduce allocations of empty structures by
allowing nil pointers to concrete struct types. Property readers will
not recurse into nil pointers, property writers will replace the nil
pointer with a pointer to the zero value of the pointer element type.
Allows a >50% primary builder time improvement with a trivial change in
Soong.
Change-Id: If6ad674bf7bf2a694c335378a074643a97d3c50b
proptools cloning and extending are a significant portion of the run
time for Soong. Optimize out calls to reflect.Type.Field(), which must
allocate a []int to store the index, by caching all the fields of each
type as it is seen, and by iterating over a slice of cached fields
instead of calling Field(i) for each one. Also avoid calling
reflect.Value.Interface() twice on the same Value.
Change-Id: I4e13fc85f30d8614a5586283e928c0a6d7f24809
ExtendProperties is the same as AppendProperties or PrependProperties,
but takes a function that determines whether each property should be
appended or prepended.
Change-Id: I26e400d56d75a88bab9c27c382ee5321bc623ee5
This makes *string values act like *bool values -- instead of appending
or prepending the contents of the string, the entire string is replaced.
The use case here is for overriding filenames, where appending doesn't
work.
Allow property structs to contain anonymous embedded structs and
interfaces. Properties in an anonymous embedded struct or interface are
treated as if they were properties in the embedding struct.
ZeroProperties was setting nested structs to their zero value, even if
they contained an interface that should be recursed into, not replaced
with nil.
When appending properties, it may be necessary to determine if two
property structs are the same "type". A simple Go type comparison is
not sufficient, as there may be interface{} values in the property
structs that contain different types. Add proptools.TypeEqual that
returns true if they have equal types and all embedded pointers to
structs and interfaces to pointers to structs have the same nilitude and
type.
The only append semantics for bool that result in a no-op when the zero
value is appended is to OR the two values together, but that is rarely
the desired semantics. Add support for *bool and *string as property
types, where appending a nil pointer is a no-op. For *bool, appending a
non-nil pointer replaces the destination with the value. For *string,
appending a non-nil pointer appends the value.
This also provides a more reliable replacement for
ModuleContext.ContainsProperty, as the build logic can tell that the
property was set, even if it was set by a mutator and not by the
blueprints file, by testing against nil.
[]string already provides these semantics for lists.
Setting a *bool or *string property from a blueprints file is the same
syntax as setting a bool or a string property.
Add tests for CloneProperties, CloneEmptyProperties and ZeroProperties
and fix detected bugs related to nil pointers to structs and interfaces
containing nil pointers to structs.
It is common for a mutator to append or prepend property structs
together. Add helper functions to append or prepend properties in property
structs. The append operation is defined as appending string and slices
of strings normally, OR-ing bool values, and recursing into embedded
structs, pointers to structs, and interfaces containing pointers to
structs. Appending or prepending the zero value of a property will
always be a no-op.
The primary builder will now generate a rule to call itself with
--docs=.bootstrap/docs/<name>.html to produce an automatically
generated documentation file.
The documentation generation process is:
- Call each factory once to get empty property structs associated
with the module type
- Use reflection to determine the names of the type of each property
struct
- Use the bootstrap_go_package modules from reading the Blueprints files
to find the source files for each Go package used to build the primary
builder
- Use the go/parser module to find the type declaration for each
property struct
- Extract comments for the property struct and each property declaration
- Format all the comments into HTML
Change-Id: Icae9307cc10549a30bfc14d6922824099de5a9b0
