Commit graph

98 commits

Author SHA1 Message Date
Cole Faust
74ac027962 Allow using include_top to filter results even when there is a constant prefix
Previously, when seeing a `include foo/$(VAR)/bar.mk`, the include
tops wouldn't apply to filter down the results any further.

Allowing include_tops is useful here to prevent cycles in the
load graph.

Fixes: 236025787
Test: go test
Change-Id: I3ea445f222ee2283a68d4aca5002df750204076f
2022-06-14 13:40:32 -07:00
Cole Faust
1323877b7e Support complicated variable references
Bug: 226974242
Test: go test
Change-Id: Iaec16f5c498e7c75c9ee5d53d3499efadfba16bc
2022-05-10 05:26:35 +00:00
Cole Faust
72374fc628 Flatten foreach calls that produce 2d lists
In make, the result of $(foreach $(x),$(y),$(foreach $(z),(w),a))
is a regular list, but in Starlark it's a list of lists. Flatten
the results of foreach expressions where each element is a list
so that they're regular lists of strings again.

Bug: 226974242
Test: go test
Change-Id: I3210d409aba0d807a5890e341ab1e0c0478f5930
2022-05-09 22:47:34 +00:00
Cole Faust
7940585cf0 Merge "Convert firstword/lastword to starlark functions instead of index operators" 2022-04-29 18:32:40 +00:00
Cole Faust
2b6a312662 Merge "Recognize some unsupported functions" 2022-04-29 00:41:15 +00:00
Treehugger Robot
f8c3412822 Merge "Convert $(words) to Starlark" 2022-04-28 22:18:07 +00:00
Cole Faust
16bb2d044d Merge "Allow seeing include $(x) when there is an include_top comment" 2022-04-28 20:49:17 +00:00
Treehugger Robot
e8f19f3dd7 Merge "Convert includes/inherits inside evals" 2022-04-28 20:43:29 +00:00
Cole Faust
1a3e5effa8 Merge "Allow comparing $(wildcard) results to non-empty values" 2022-04-28 19:54:32 +00:00
Cole Faust
5a13aaf112 Convert firstword/lastword to starlark functions instead of index operators
firstword/lastword need to be able to give an empty string when
the input is empty, which the indexing operator can't do.

Bug: 226974242
Test: go test
Change-Id: I44162a258297b32ec27e7acfb1fa06223391dcc6
2022-04-27 17:49:35 -07:00
Cole Faust
a59059f3a1 Merge "Ignore assignments to .KATI_READONLY" 2022-04-27 17:56:54 +00:00
Cole Faust
5d5fcc3092 Ignore assignments to .KATI_READONLY
Since the rbc results file is sorted, .KATI_READONLY
ends up trying to make a variable that hasn't been defined
yet readonly, which is an error.

It's not really worth supporting moving it down in the list,
because it wouldn't affect the variable being writable while
still in Starlark code.

Bug: 226974242
Test: go test
Change-Id: I9402f69be97e5c7cf010ad86f124422ea55fda7f
2022-04-26 18:02:05 -07:00
Cole Faust
9df1d73657 Allow seeing include $(x) when there is an include_top comment
Previously, seeing anything other than an interpolate variable
was not allowed.

Bug: 226974242
Test: go test
Change-Id: I48a060f9a3fd19cd67a114d2cb0756ab2be25ce1
2022-04-26 17:46:49 -07:00
Cole Faust
1e27586012 Recognize some unsupported functions
Some functions are called without the use of $(call),
since $(call) is not necessary when the function doesn't
take any arguments. mk2rbc thinks these function calls
are local variables, and converts them as such. This
leads to undefined variable errors when trying to load
(before even executing) the starlark file.

Hardcode a known function that should not be converted
to a local variable.

Bug: 226974242
Test: go test
Change-Id: I5567a34fcc282b181a7e78ac3d5cc9b40bd025a2
2022-04-26 14:53:25 -07:00
Cole Faust
00afd4f8b9 Ignore variable assignments that come after a rule definition
These private variables sometimes have references to make
variables like $< or $@. When converted to starlark, they
become invalid local variables like _< and _@, preventing
the file from being loaded even if it's never executed.

Bug: 226974242
Test: go test
Change-Id: Iafd4c6939731f3b7c051c9e41464134d5b672f23
2022-04-26 14:01:56 -07:00
Cole Faust
a99afdfc22 Allow comparing $(wildcard) results to non-empty values
Also remove the file existence functions because they
can just be wildcards instead.

Bug: 226974242
Test: go test
Change-Id: Icbf65c47af97a710580864e8b76e2697aba96dd8
2022-04-26 12:06:49 -07:00
Cole Faust
2005298b68 Convert includes/inherits inside evals
These occur in some vendor makefiles in
order to include files in a foreach loop.

Bug: 226974242
Test: go test
Change-Id: I59242eba9a2d7c659994556d9123b6481f4a8130
2022-04-26 10:44:08 -07:00
Cole Faust
94c4a9ac62 Convert $(words) to Starlark
Bug: 226974242
Test: go test
Change-Id: I8ab5943306586c6a993df470e7a2894dd7ff1619
2022-04-22 17:43:52 -07:00
Cole Faust
7cd255fa8d Merge "Move variable assignment handling to generation context" 2022-04-14 16:59:10 +00:00
Cole Faust
6c41b8a729 Add handle to enforce_product_packages_exist
Bug: 229132189
Test: go test
Change-Id: I76804354051f0866c8cad48c1189b36de4f73e23
2022-04-13 14:04:27 -07:00
Treehugger Robot
19da477d4d Merge "Add the handle to artifact path requirement functions" 2022-04-08 17:32:25 +00:00
Cole Faust
f06326648b Move variable assignment handling to generation context
This allows the parsing code to be cleaner, as it
doesn't have to care about variable assignments.

Bug: 228518745
Test: go test
Change-Id: I33425c2fb51acab4901bfa82a53d337b75210f8e
2022-04-07 15:40:12 -07:00
Cole Faust
62e0511202 Allow include $(sort $(wildcard */font.mk)) and variants
Include/inherit-product calls already allow asterisks, the
wildcard is not necessary. If we see an include using $(wildcard),
act as if it doesn't exist.

Bug: 218736658
Test: go test
Change-Id: Ib21007a2042fbfaa95c07571792983e14135a035
2022-04-06 16:37:54 -07:00
Cole Faust
95b95cb647 Convert $(sort) to Starlark
This is mostly so that we can not error out when
seeing sort inside a `include $(sort $(wildcard foo/*.mk))`.

Bug: 218736658
Test: go test
Change-Id: I827ecfc751a908227cb2dd9ed1e4b3f313791a16
2022-04-05 16:37:39 -07:00
Treehugger Robot
8210fbec1f Merge "Support converting simple $(eval) expressions" 2022-04-05 22:44:07 +00:00
Cole Faust
0e2b256846 Correct type of dir function
It should return a string, not a list.

Bug: 227245326
Test: go test
Change-Id: If0cb382dede52200c0595ae939a8064641216af1
2022-04-01 11:49:24 -07:00
Cole Faust
f035d405d8 Support converting simple $(eval) expressions
While supporting $(eval) in the general case is
impossible, as it would require emitting code at
runtime, it is possible to handle some special cases
that are common throughout the code base.

Specifically, an assignement expression (where the
left hand side is constant) can be converted without
needing to evaluate strings as code, as its whole
right hand side is treated as one string.

However, this eval with an assignemnt can only be
used as a statement, not an expression. So it requires
the eval to be either a top-level expression, or nested
within other expressions that can be converted to
statements such as $(foreach) or $(if).

Bug: 226974242
Test: go test
Change-Id: Ifc52ef9ab7d62a69251918fcde5463f80a98a542
2022-03-31 12:44:59 -07:00
Cole Faust
ea9db58de9 Add the handle to artifact path requirement functions
Bug: 188079133
Test: go test
Change-Id: I320f4f14fb47414cb14ebb404efab4bceb1fa1e4
2022-03-22 11:17:16 -07:00
Treehugger Robot
5beafc1900 Merge "Replace $(call my-dir) with a string literal" 2022-03-22 03:24:47 +00:00
Treehugger Robot
c9e38327dd Merge "Parse variable references with #s" 2022-03-22 03:15:47 +00:00
Cole Faust
f5adedce03 Replace $(call my-dir) with a string literal
This is so that we can set LOCAL_PATH to the result
of my-dir, as LOCAL_PATH can only be set to a string
literal of the exact value of LOCAL_PATH.

It's probably also the correct choice to start phasing
out LOCAL_PATH.

Bug: 214405650
Test: go test
Change-Id: Ia97d7fedf4ce62643921d90a176e65edd4e2fce6
2022-03-18 14:05:06 -07:00
Cole Faust
e309a91ca8 Parse variable references with #s
Inside a variable reference, a # does not start
a comment.

Fixes: 218742602
Test: go test
Change-Id: I16cf04c74a8aa30482fd9293175f893e4efb60f1
2022-03-17 17:15:12 -07:00
Cole Faust
421a192d05 Convert values to strings when assigning to a string variable
Currently, a string variable can easily be reassigned to
a different type. Make it so that the value it's being
reassigned to is converted to a string first.

Bug: 224601891
Test: go test
Change-Id: I82252cf9e106b5a3677458cf1df2e9d1dfefe0f6
2022-03-16 14:35:45 -07:00
Treehugger Robot
0ddc5724b0 Merge "Add type hints to mk2rbc" 2022-03-16 00:40:11 +00:00
Cole Faust
f92c9f2809 Add type hints to mk2rbc
Type hints have the format #RBC# type_hint MY_VAR list
and must be specified at the top of the Makefile. Setting
one will cause that variable to have that type for the
remainder of the Makefile. This can be used where mk2rbc's
type inference detects the wrong type and it must be
manually changed.

Bug: 224601891
Test: go test
Change-Id: I6db2c50056d0298227e1d2801a522adf8bbd1df8
2022-03-15 12:52:20 -07:00
Cole Faust
e2a37988ff Simplify and correct variable assignments
- Remove asgnMaybeAppend, it was only used to indicate
  that the asignment needs a setDefault. But really, all
  types of variable assignments need setDefault if they're
  self-referential.
- Remove local_append/local_set_default because there was
  no implementation for them written in product_config.rbc
  anyways.
- Correct productConfigVariable.emitDefined using the global
  variables instead of the product config variables.
- Emit setDefaults for all types of assignments if they're
  self referential.

Bug: 222737841
Test: go test
Change-Id: I06a0f90f16d5900049d473281e6d5ef5e93e67da
2022-03-10 15:00:57 -08:00
Cole Faust
7d49b1e70a Merge "Call rblf.setDefault() when appending to a variable without +=" 2022-03-08 22:58:58 +00:00
Cole Faust
816e080c4d Call rblf.setDefault() when appending to a variable without +=
Bug: 222737841
Test: go test
Change-Id: I10e9e994fb1979e2e06ad30bbe66a21657d1e3db
2022-03-08 22:58:42 +00:00
Cole Faust
3c4fc99593 Support variables with dashes in their names
Bug: 221946551
Test: go test
Change-Id: I085fc35159c4f3afe53868fbc731fcaeac3a69a8
2022-03-07 11:24:45 -08:00
Cole Faust
93f8d3987a Fix include paths that match one file being included unconditionally
In aosp/1962181 an "optimization" was added where if an include
path with a variable only matches one file, that file would be
called directly instead of going through the whole _entry
variable.

This doesn't work when there is an inherit-product-if-exists call
that matches one file, but that file isn't the one that is specified
at runtime. In reality, the inherit-product-if-exists call should
see that the file doesn't exist and not include anything.

Fixes: 222341691
Test: go test
Change-Id: If8789de0013726137683078355161eecdec009b1
2022-03-02 13:31:30 -08:00
Cole Faust
fc74246c98 Merge "Convert add-product-dex-preopt-module-config to Starlark" 2022-03-02 18:40:36 +00:00
Cole Faust
b67aa082aa Fix issue with referencing the loop variable in a foreach
Fixes: 222160672
Test: go test
Change-Id: I3f9238d4f1684cf4a2d24c4f7f49c832c3f72b97
2022-03-01 16:11:50 -08:00
Cole Faust
1cc088523f Convert add-product-dex-preopt-module-config to Starlark
Bug: 221877397
Test: go test
Change-Id: I79f5209bb42f2c689af96ded376c08522cbec24e
2022-02-28 11:12:08 -08:00
Treehugger Robot
190e536e64 Merge "Allow filter calls with a list as a pattern" 2022-02-11 00:46:58 +00:00
Cole Faust
9932f75151 Allow filter calls with a list as a pattern
This commit doesn't attempt to replace the filter
calls with anything more idomatic for now.

It also removes the case for filter-out, because
the actual function name is filter_out and wasn't
being used anyways. Even if I were to change it
to filter_out, that would produce buggy results:

ifneq (,$(filter $(TARGET_BUILD_VARIANT), userdebug eng))
endif
ifneq (,$(filter-out $(TARGET_BUILD_VARIANT), userdebug eng))
endif

Both of these would produce:

if g["TARGET_BUILD_VARIANT"] in ["userdebug", "eng"]:
  pass

Fixes: 218702402
Test: go test
Change-Id: I566079e5d3a364c42db14045aa1bab9d99eba05f
2022-02-09 14:15:47 -08:00
Cole Faust
f4e72cf642 Update warning message
As requested on aosp/1975486

Fixes: 217248902
Test: Presubmits
Change-Id: I14e65722d8ab96125827d34e8501b4a518a27b57
2022-02-08 13:10:26 -08:00
Cole Faust
9b6111aaed Remove --root, require the cwd to be the root
This allows us to use relative paths everywhere.
It also produces more accurate emulation of TOPDIR,
which should be an empty string.

Bug: 213508006
Test: go test
Change-Id: Ie4e357687486e84e9f4aad0f6776d8feb2b9fc63
2022-02-07 11:45:01 -08:00
Cole Faust
dd569aea07 Return starlarkNodes from the functions that parse them
Currently, mk2rbc is structured around having a global
"receiver" object that accepts all starlarkNodes. As soon
as they are parsed they are added to the receiver.

Returning the parsed nodes to the calling function is more
flexible, as it allows the calling function to restructure
them as necessary. This is the first step to supporting
complicated statements involving $(eval), such as
`$(foreach v,$(MY_LIST),$(eval MY_LIST_2 += $(v)))`

Test: go test
Change-Id: Ia194123cf090d2b9559a25b975ccbc5749357cc0
2022-02-02 14:59:59 -08:00
Cole Faust
7940c6a5b5 Remove RBC hints from generated starlark
The hints no longer need to be there in the generated code.

Fixes: 217269465
Test: go test
Change-Id: If2049780a874b42eb9df9351dda4f29c85482470
2022-02-02 11:53:19 -08:00
Cole Faust
f8a4bb6d7f Merge "Allow variable-prefixed include statements in mk2rbc" 2022-02-01 18:44:28 +00:00