This allows us to recreate soong config value variables in selects.
This adds a new "any" pattern to selects, which is the same as "default"
except that it doesn't match undefined variables, and it's (currently)
the only pattern that can accept a binding.
The syntax looks like:
```
select(soong_config_variable("my_namespace", "my_variable"), {
any @ my_binding: "foo" + my_binding,
default: "other value",
})
```
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I4feb4073172d8797dee5472f43f9c248a76c3f1f
Before this cl, blueprint expressions were evaluated as they were
parsed. We want to add a feature to select statements where we can
bind the value of soome value from soong into a blueprint variable,
that then can be used like a regular variable in the .bp file. This
means that select statements need to hold whole unevalated expression
trees, and have the ability to evaluate them later on when the value
of the bound variable is known.
This cl doesn't implement the new select syntax, but it does split
blueprint's parsing and evaluating into two separate stages. We also
store expressions in selects and evaluate them when the select is
resolved.
I didn't do extensive performance evaluation, but a simple comparison
of the time of `touch Android.bp && m nothing` before/after this cl
showed a 1 second speedup. (That was probably just noise)
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I12f373719991afeb4aec76517153f32229d97ff2
This adds support for selecting on multiple variables at once, so that
you can do AND/OR combindations of them. For example:
select((
arch(),
os(),
), {
("arm64", "linux"): ["libfoo64"],
(default, "linux"): ["libfoo"],
(default, "windows"): ["libfoowindows"],
(default, default): ["libbar"],
})
It also allows for select conditions to be boolean-typed. You can
write literal true and false without quotes to select on them. Currently
we don't have any boolean-typed variables though, so a fake one was
added for testing.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ibe586e7b21865b8734027848cc421594cbd1d8cc
Previously I was using an underscore to denote the default branch
because I was thinking that I would allow variable bindings in the
select branches, and 'default' could be mistaken for the name of a
variable. But I think it's better to just introduce alternate syntax,
like `default @ my_var: "foo" + my_var,` to do the variable bindings,
so that we can have a clearer name for the default case.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: Ied762694e453855c03dd471898ebb52e97a5a671
Currently, with the arch/os mutator, you can override a property
using the default value for just a few arch types, for example:
cc_defaults {
name: "my_defaults",
target: {
windows: {
enabled: true,
}
}
}
cc_binary {
name: "foo",
enabled: false,
defaults: ["my_defaults"],
}
You could make a select statment that acts like the above if it were
all in one module, but currently with select statements you can't make
a defaults module that can be generically applied to any other module
and have the same behavior as the above.
After this cl, the defaults module could look like:
cc_defaults {
name: "my_defaults",
enabled: select(variant("arch"), {
"windows": true,
_: unset,
}),
}
Which would have the same behavior. Unset may also be useful for
setting the property under some configurations, but wanting to leave
the implementation-specific default value in others.
Bug: 323382414
Test: m nothing --no-skip-soong-tests
Change-Id: I3ea3277ea8b9a0ac5e613b4378945388b9df036a
Bug: 323382414
Test: go tests, and I also ran both the old/new bpfmts on the whole aosp source tree, and the only difference between the results was one extra removed line in external/uwb
Change-Id: I4942c9247a66f1de5028de39caa5cd34b66093c3
Select statements are a new blueprint feature inspired by bazel's select
statements. They are essentially alternative syntax for soong config
variables that require less boilerplate. In addition, they support
making decisions based on a module's variant, which will eliminate
the need for manual property struct manipulation, such as the arch
mutator's arch: and target: properties.
In order to support decisions based on the variant, select statements
cannot be evaluated as soon as they're parsed. Instead, they must be
stored in the property struct unevaluated. This means that individual
properties need to change their type from say, string, to
Configurable[string]. Currently, only configurable strings, bools, and
string slices are supported, but more types can be added later.
The module implementation must call my_property.Evaluate(ctx) in order
to get the final, resolved value of the select statement.
Bug: 323382414
Test: go tests
Change-Id: I62f8721d7f0ac3d1df4a06d7eaa260a5aa7fcba3
When given as an input
```
array: [
"a",
// Unicorn
"b",
]
```
bpfmt with `-s` option was outputing
```
array: [
"a", // Unicorn
"b",
]
```
Which is not ideal because the comment was targetting
the second value and now it seems to be targetting the
first one
This patch preserve the difference in line number between
the value and the comment to give the same output when
```
array: [
"a",
// Unicorn
"b",
]
```
is given as input
Test: Manual tests + run bpfmt -w -s on packages/modules/Bluetooth
Change-Id: I2b58f20da463bea77c22a4e6978aa9beb4b4fcc8
Soong analyzes the entire source tree even though not every lunch target
needs to know about every module. For example, OEM sources can be
ignored for cuttlefish products. This functionality allows blueprint to
ignore a list of undesired directories.
Bug: 269457150
Change-Id: Icbbf8f3b66813ad639a7ebd27b1a3ec153cbf269
newParser was calling p.next(), which could trigger a scanner error
that results in a panic. Move the first p.next() into parse(p), which
correctly converts the paanic into a reportable error.
Bug: 254831383
Test: TestParserError
Change-Id: I2a427010379bb8dd5087550c7f159499cbb84066
Test: go run bpmodify.go -w -m=libcore-memory-metrics-tests -property=something,static_libs,deps,required,test_suites -replace-property=ahat:ahat_lib,general-tests:something -s ~/aosp-master-with-phones/libcore/metrictests/memory/host/Android.bp
go test -v
Change-Id: I005b6dd675beb205f544e89c729fe9191e6470c2
bpmodify can know move the contents of a property into another
property using moveProperty. After moving the contents, the original
property is deleted.
Bug: 226636335
Change-Id: Id68d11d59f00909b4c93aa78666d14f433f236fb
Test: manually ran on several Android.bp files in bug 226636335
-a works with only string values. When we need to add numbers, booleans,
or even structs, -add-literal can be used now.
bpmodify -m foo -property list-of-ints -add-literal 123
bpmodify -m foo -property list-of-structs \
-add-literal "{key: \"value\"}"
Bug: 146436251
Test: go test ./bpmodify
Change-Id: I91d23d7bf89491643aa595f5ebccd9410a9cbb09
Since a struct(parser.Map) occupies multiple lines, adding newlines
around brackets([]) looks better even the list has only a single value.
prop: [ {
name: "foo",
}],
vs
prop: [
{
name: "foo",
},
],
Bug: n/a
Test: go test ./parser
Change-Id: I1a574aa038a26235848b6c9b5b4f01a0ab2c8c00
This limits support to allow-listed property names to prevent
proliferation of map types requiring additional support to migrate.
Test: go test blueprint tests
Test: m nothing && diff build.ninja & Android-aosp_arm.mk -- no changes
Change-Id: Id12637462f19ac5de1b562f63507de989a51600d
numericStringLess("1a", "11a") would strip the equal prefix "1" and
then compare the bytes "a" and "1", when it should have compared the
numbers 1 and 11. Fix it by handling the case where the last equal
byte was numeric and the first differing byte is numeric in one
string and non-numeric in the other.
numericStringLess("12", "101") would strip the equal prefix "1" and
then compare the numbers 2 and 01, when it should have compared the
numbers 12 and 101. Fix it by tracking the beginning of the sequence
of numeric bytes containing the differing byte.
Test: sort_test.go
Change-Id: I8d9252a64625ba6a3c75d09bb1429dcb1115e3e1
bpmodify sorts touched list respecting numbers embedded in strings. For
example, "foo-2" comes before "foo-10".
Test: bpmodify_test.go
Change-Id: If2fe9bc871a463a47dd3c0b52982b34c9fde05f0
Co-authored-by: Jooyung Han <jooyung@google.com>
Parser.parseVariable method should always set the value of the variable
it creates. Failure to do so may end up in the following:
```
$ androidmk <(printf "FOO:=(X)\nFOO:=bar\n")
parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = %!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference) (%!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference)) false
```
The cause is that calling Parser.Parse to parse `FOO=abc` created
a Variable instance with nil value, causing panic on print attempt.
Test: m androidmk && androidmk <(printf "FOO:=(X)\nFOO:=bar\n")
(should print:
ERROR: parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = X = Not Evaluated (X = Not Evaluated) false)
Change-Id: I296d7984df6d8796e0075f9eb692b234f8c94f08
This commit refines `compat` condition in `parseProperty()` so that a
module definition would either use the new syntax or the old syntax,
but not something in the between, such as
cc_library {
name: "bad_example",
srcs= ["bad.c"],
}
Test: lunch aosp_arm64-userdebug; make # runs unit test
Change-Id: If2d3e5d55edccc28d314d99b83b0f54e5c53ac35
When someone used a multiline string:
cmd: "..." +
"...",
bpfmt used to print this out as:
cmd: "..." +
"...",
This change doesn't do the quote alignment like I see in some of our
user-created instances of this, but it does indent a single level:
cmd: "..." +
"...",
Test: unit tests
Test: Compared bpfmt results before and after across AOSP
Change-Id: I61bf790be9d08a187857b2725facf71e8b38e372
Patch and PatchList provide an API for making changes to substrings
of a Blueprint file by parsing the AST and using the token positions
to perform text replacements.
Test: modify_test.go
Change-Id: Ibb8993221982b54602ba5a05486198fab8d35a67
End() was previously only used to determine if a comment was within
a Node, so it used the expedient definition of the position of the
last token in the node. In the next patch it will be used for
capturing substrings of the Blueprint file, so make it point to
the character after the last token instead.
Also add tests for it.
Test: parser_test.go
Change-Id: Icaff3915b41e251ef9d0aad5615021bf37406aee
Support int64 number instead of int to be more fixed to bit size so
that the underlying arch won't affect overflow cases. Besides,
refection: func (v Value) Int() int64 always cast to int64 no matter the
input is int, int16, int32. Currently we always treat "-" as negative
sign to bind to next value, and "+" as plus operator to add operands
together.
So we allow:
a = 5 + -4 + 5 or a = -4 + 5
But we don't allow:
a = +5 + 4 + -4 since we don't treat "+" as a positive sign, otherwise,
a = 5 + +5 would exist which looks pretty weird. In the future, we may
want fully support number calculator logic eg, "+"/"-" can be
positive/negative sign or operator, and "(" and ")" will be considered
to group expressions with a higher precedence.
int & uint properties within struct keeps unchanged, which is only
allowed when tagged with 'blueprint:mutated'. We only allow *int64
property instead of int64 property within struct since it does't make
sense to do prepending or appending to int64.
Change-Id: I565e046dbd268af3538aee148cd7300037e56523
value.Type and value.Pos changed to methods, change the users that were
not caught by error checking to use Type() or Pos().
Change-Id: I295a658c007fa2de68c89fb85ee367fbea5ed1aa
There may be multiple skipped comments in a row, convert
p.skippedComments to a list and add a test.
Change-Id: I30dcff269bee56fd51ef9513dab7c7885c44b7d7
Determining which comments are contiguous is difficult once they have
been parsed into an out-of-band comment list, as any intervening nodes
are in a separate structure. Group the comments into CommentGroups
during the parsing stage instead.
Change-Id: I9444c58e75333b7521b58dbfbd36ff29d139b6e3
Pos is going to be part of the Node interface, rename the Pos member
of structs to be more specific.
Change-Id: Ibd31119863b96d38bf8dac216e026200a54bbe18
It wasn't adding anything useful, and it resulted in Name.Name to get to
the identifier. Replace Name Ident with Name string; NamePos
scanner.Position.
Change-Id: Idf9b18b31dd563a18f27c602c2d14298955af371
Refactor the blueprint parser Value object, which contained a Type enum
and members to hold every possible type, into an interface (now called
Expression). Rename the existing Expression object that represented a binary
operator Operator.
Also adds and fixes some new printer test cases with mulitline expressions.
Change-Id: Icf4a20f92c8c2a27f18df8ca515a9d7f282ff133
The Go race detector found a race condition in the parser, which
highlighted a few related bugs. A variable could be defined but
not referenced in a Blueprints file, then appended to in multiple
subdirs= Blueprints files. The race detector caught the multiple
writes to assignment.Referenced from the parsers for the subdirs
Blueprints files, but multiple appends would be much more serious.
To fix this, keep local and inherited variables separate in the
Scope object and export that info to the parser. Disallow
appending to non-local variables, which was already the intended
behavior. Only update the referenced boolean for local variables.
Together, this should prevent all writes to Assignment objects
from parsers other than the one that created them.
Also improves the error handling code and some error messages.
Change-Id: Idb4f7d2e61bbe28d90b93074764e64e60d1eba8f
Refactor parsing Blueprints and walking the Blueprints tree to
allow reuse for tasks that don't involve creating moduleInfo
structures.
Change-Id: I677857a3462999426c8306432074ea97fdcb86c8
Add String() to print out the Comment, Text() to print out the
Comment while stripping /*, //, and */, and EndLine() to return
the line number of the last line of the comment.
Change-Id: I076bc0990143acfc03c42a8cc569894fced8ee24
Running bpfmt or bpmodify on a Blueprints file that references
variables defined in a parent Blueprints file causes parse errors
on unknown variables. Modify the parser to parse in two modes,
parser.Parse and parser.ParseAndEval. The first parses without
attempting to evaluate variables and expressions, the second
performs a full evaluation.
Change-Id: Ic11948ea77c8e65379d7260591ada15db0e4f5b9
Trying to handle all the whitespace and newline printing inside
printToken got overly complicated, and resulted in a few bugs in
the layout around comments and indentation that were hard to fix.
Rewrite the whitespace and newline handling to be handled directly
by the object printers, using requestSpace() to ensure whitespace
is inserted and requestNewline() to ensure a newline is inserted.
Also fixes unnecessarily left aligning all comments that contain
indentation, and fixes accidentally unindenting comments that are
the last token in their indented block.
Change-Id: I18707802726107cf0b6ec7de9b542d0ec1d2c0dd
