Commit graph

109 commits

Author SHA1 Message Date
Cole Faust
8e15f69709 Remove ?= assignements to product variables
In make, all product variables are assigned to an empty string before
including the product config makefiles. In starlark, we don't do that
just so the memory usage is smaller. This means that in make, using
?= to assign a product config variable has no effect. Replicate this
behavior in starlark by not emitting any code for ?= assignments of
product variables.

Fixes: 304324003
Test: go test
Change-Id: Id31531506ac9e372a1bea92ce9ae8e17ae0ca70c
2023-10-09 12:26:21 -07:00
Cole Faust
b0b2457842 Handle include statements with trailing whitespace
Fixes: 303832950
Test: go test
Change-Id: I919556ee43a60f59dbd7c040025f01abb53da2a8
2023-10-06 11:57:29 -07:00
Cole Faust
8ff3c63376 Revert "mk2rbc: temporarily bump matching paths limit to 155. "
This reverts commit 30d48351be.

Reason for revert: The underlying issue was fixed in ag/23603641

Change-Id: Ibe2749c3bb7412132de6d96d178f54cd8d1d0666
2023-06-08 22:53:16 +00:00
Jingwen Chen
30d48351be mk2rbc: temporarily bump matching paths limit to 155.
It was 150, but an internal change bumped up the usage to 153. Temporarily increase it to fix build break.

Bug: b/284854738

Change-Id: I23478f4c5333c5ef1ad344aadafe5803685c5d92
Test: abtd 
Test: treehugger
2023-05-29 16:01:34 +00:00
Cole Faust
c85d08f63b Make mk2rbc output paths relative to android root
Previously, mk2rbc would output absolute paths in the load statements.
This caused issues with the new module resolution logic in rbcrun,
and is not something we'd want to check in anyways.

Bug: 280685526
Test: go test
Change-Id: Iae6e3f6be9cecc8e19b6e1fd33c5575ff6864654
2023-05-09 15:00:58 -07:00
Cole Faust
7366042870 Show correct line numbers for errors in an $(eval)
Bug: 264488559
Test: go test
Change-Id: I95ce0345912a2af351be0ecc4595bdc4a2047c0c
2023-01-05 11:22:46 -08:00
Cole Faust
5a3449f11c mk2rbc support for clear_var_list
Bug: 262303006
Test: Manually (Added usage to aosp_arm64 and ran rbc_dashboard.py on it)
Change-Id: I3cba8f1d69ba4f42b4bc125f0a1131c1d33f7efb
2022-12-12 19:02:42 -08:00
Cole Faust
2dee63d9a8 Add to-lower/to-upper
Bug: 262303006
Test: go test
Change-Id: I80af8ace90556e58676adac4cb90d5d62c5b7b96
2022-12-12 18:18:15 -08:00
Cole Faust
d2daabfa02 Add support for and/or in mk2rbc
Bug: 262303006
Test: go test
Change-Id: I04130648084775c0828e95cd18a82e348c3f09eb
2022-12-12 18:17:37 -08:00
Colin Cross
d079e0b270 Reformat build/soong for go 1.19
Test: none
Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-08-17 10:43:13 -07:00
Cole Faust
74ac027962 Allow using include_top to filter results even when there is a constant prefix
Previously, when seeing a `include foo/$(VAR)/bar.mk`, the include
tops wouldn't apply to filter down the results any further.

Allowing include_tops is useful here to prevent cycles in the
load graph.

Fixes: 236025787
Test: go test
Change-Id: I3ea445f222ee2283a68d4aca5002df750204076f
2022-06-14 13:40:32 -07:00
Cole Faust
1323877b7e Support complicated variable references
Bug: 226974242
Test: go test
Change-Id: Iaec16f5c498e7c75c9ee5d53d3499efadfba16bc
2022-05-10 05:26:35 +00:00
Cole Faust
72374fc628 Flatten foreach calls that produce 2d lists
In make, the result of $(foreach $(x),$(y),$(foreach $(z),(w),a))
is a regular list, but in Starlark it's a list of lists. Flatten
the results of foreach expressions where each element is a list
so that they're regular lists of strings again.

Bug: 226974242
Test: go test
Change-Id: I3210d409aba0d807a5890e341ab1e0c0478f5930
2022-05-09 22:47:34 +00:00
Cole Faust
7940585cf0 Merge "Convert firstword/lastword to starlark functions instead of index operators" 2022-04-29 18:32:40 +00:00
Cole Faust
2b6a312662 Merge "Recognize some unsupported functions" 2022-04-29 00:41:15 +00:00
Treehugger Robot
f8c3412822 Merge "Convert $(words) to Starlark" 2022-04-28 22:18:07 +00:00
Cole Faust
16bb2d044d Merge "Allow seeing include $(x) when there is an include_top comment" 2022-04-28 20:49:17 +00:00
Treehugger Robot
e8f19f3dd7 Merge "Convert includes/inherits inside evals" 2022-04-28 20:43:29 +00:00
Cole Faust
1a3e5effa8 Merge "Allow comparing $(wildcard) results to non-empty values" 2022-04-28 19:54:32 +00:00
Cole Faust
5a13aaf112 Convert firstword/lastword to starlark functions instead of index operators
firstword/lastword need to be able to give an empty string when
the input is empty, which the indexing operator can't do.

Bug: 226974242
Test: go test
Change-Id: I44162a258297b32ec27e7acfb1fa06223391dcc6
2022-04-27 17:49:35 -07:00
Cole Faust
a59059f3a1 Merge "Ignore assignments to .KATI_READONLY" 2022-04-27 17:56:54 +00:00
Cole Faust
5d5fcc3092 Ignore assignments to .KATI_READONLY
Since the rbc results file is sorted, .KATI_READONLY
ends up trying to make a variable that hasn't been defined
yet readonly, which is an error.

It's not really worth supporting moving it down in the list,
because it wouldn't affect the variable being writable while
still in Starlark code.

Bug: 226974242
Test: go test
Change-Id: I9402f69be97e5c7cf010ad86f124422ea55fda7f
2022-04-26 18:02:05 -07:00
Cole Faust
9df1d73657 Allow seeing include $(x) when there is an include_top comment
Previously, seeing anything other than an interpolate variable
was not allowed.

Bug: 226974242
Test: go test
Change-Id: I48a060f9a3fd19cd67a114d2cb0756ab2be25ce1
2022-04-26 17:46:49 -07:00
Cole Faust
1e27586012 Recognize some unsupported functions
Some functions are called without the use of $(call),
since $(call) is not necessary when the function doesn't
take any arguments. mk2rbc thinks these function calls
are local variables, and converts them as such. This
leads to undefined variable errors when trying to load
(before even executing) the starlark file.

Hardcode a known function that should not be converted
to a local variable.

Bug: 226974242
Test: go test
Change-Id: I5567a34fcc282b181a7e78ac3d5cc9b40bd025a2
2022-04-26 14:53:25 -07:00
Cole Faust
00afd4f8b9 Ignore variable assignments that come after a rule definition
These private variables sometimes have references to make
variables like $< or $@. When converted to starlark, they
become invalid local variables like _< and _@, preventing
the file from being loaded even if it's never executed.

Bug: 226974242
Test: go test
Change-Id: Iafd4c6939731f3b7c051c9e41464134d5b672f23
2022-04-26 14:01:56 -07:00
Cole Faust
a99afdfc22 Allow comparing $(wildcard) results to non-empty values
Also remove the file existence functions because they
can just be wildcards instead.

Bug: 226974242
Test: go test
Change-Id: Icbf65c47af97a710580864e8b76e2697aba96dd8
2022-04-26 12:06:49 -07:00
Cole Faust
2005298b68 Convert includes/inherits inside evals
These occur in some vendor makefiles in
order to include files in a foreach loop.

Bug: 226974242
Test: go test
Change-Id: I59242eba9a2d7c659994556d9123b6481f4a8130
2022-04-26 10:44:08 -07:00
Cole Faust
94c4a9ac62 Convert $(words) to Starlark
Bug: 226974242
Test: go test
Change-Id: I8ab5943306586c6a993df470e7a2894dd7ff1619
2022-04-22 17:43:52 -07:00
Cole Faust
7cd255fa8d Merge "Move variable assignment handling to generation context" 2022-04-14 16:59:10 +00:00
Cole Faust
6c41b8a729 Add handle to enforce_product_packages_exist
Bug: 229132189
Test: go test
Change-Id: I76804354051f0866c8cad48c1189b36de4f73e23
2022-04-13 14:04:27 -07:00
Treehugger Robot
19da477d4d Merge "Add the handle to artifact path requirement functions" 2022-04-08 17:32:25 +00:00
Cole Faust
f06326648b Move variable assignment handling to generation context
This allows the parsing code to be cleaner, as it
doesn't have to care about variable assignments.

Bug: 228518745
Test: go test
Change-Id: I33425c2fb51acab4901bfa82a53d337b75210f8e
2022-04-07 15:40:12 -07:00
Cole Faust
62e0511202 Allow include $(sort $(wildcard */font.mk)) and variants
Include/inherit-product calls already allow asterisks, the
wildcard is not necessary. If we see an include using $(wildcard),
act as if it doesn't exist.

Bug: 218736658
Test: go test
Change-Id: Ib21007a2042fbfaa95c07571792983e14135a035
2022-04-06 16:37:54 -07:00
Cole Faust
95b95cb647 Convert $(sort) to Starlark
This is mostly so that we can not error out when
seeing sort inside a `include $(sort $(wildcard foo/*.mk))`.

Bug: 218736658
Test: go test
Change-Id: I827ecfc751a908227cb2dd9ed1e4b3f313791a16
2022-04-05 16:37:39 -07:00
Treehugger Robot
8210fbec1f Merge "Support converting simple $(eval) expressions" 2022-04-05 22:44:07 +00:00
Cole Faust
0e2b256846 Correct type of dir function
It should return a string, not a list.

Bug: 227245326
Test: go test
Change-Id: If0cb382dede52200c0595ae939a8064641216af1
2022-04-01 11:49:24 -07:00
Cole Faust
f035d405d8 Support converting simple $(eval) expressions
While supporting $(eval) in the general case is
impossible, as it would require emitting code at
runtime, it is possible to handle some special cases
that are common throughout the code base.

Specifically, an assignement expression (where the
left hand side is constant) can be converted without
needing to evaluate strings as code, as its whole
right hand side is treated as one string.

However, this eval with an assignemnt can only be
used as a statement, not an expression. So it requires
the eval to be either a top-level expression, or nested
within other expressions that can be converted to
statements such as $(foreach) or $(if).

Bug: 226974242
Test: go test
Change-Id: Ifc52ef9ab7d62a69251918fcde5463f80a98a542
2022-03-31 12:44:59 -07:00
Cole Faust
ea9db58de9 Add the handle to artifact path requirement functions
Bug: 188079133
Test: go test
Change-Id: I320f4f14fb47414cb14ebb404efab4bceb1fa1e4
2022-03-22 11:17:16 -07:00
Treehugger Robot
5beafc1900 Merge "Replace $(call my-dir) with a string literal" 2022-03-22 03:24:47 +00:00
Cole Faust
f5adedce03 Replace $(call my-dir) with a string literal
This is so that we can set LOCAL_PATH to the result
of my-dir, as LOCAL_PATH can only be set to a string
literal of the exact value of LOCAL_PATH.

It's probably also the correct choice to start phasing
out LOCAL_PATH.

Bug: 214405650
Test: go test
Change-Id: Ia97d7fedf4ce62643921d90a176e65edd4e2fce6
2022-03-18 14:05:06 -07:00
Cole Faust
421a192d05 Convert values to strings when assigning to a string variable
Currently, a string variable can easily be reassigned to
a different type. Make it so that the value it's being
reassigned to is converted to a string first.

Bug: 224601891
Test: go test
Change-Id: I82252cf9e106b5a3677458cf1df2e9d1dfefe0f6
2022-03-16 14:35:45 -07:00
Treehugger Robot
0ddc5724b0 Merge "Add type hints to mk2rbc" 2022-03-16 00:40:11 +00:00
Cole Faust
f92c9f2809 Add type hints to mk2rbc
Type hints have the format #RBC# type_hint MY_VAR list
and must be specified at the top of the Makefile. Setting
one will cause that variable to have that type for the
remainder of the Makefile. This can be used where mk2rbc's
type inference detects the wrong type and it must be
manually changed.

Bug: 224601891
Test: go test
Change-Id: I6db2c50056d0298227e1d2801a522adf8bbd1df8
2022-03-15 12:52:20 -07:00
Cole Faust
e2a37988ff Simplify and correct variable assignments
- Remove asgnMaybeAppend, it was only used to indicate
  that the asignment needs a setDefault. But really, all
  types of variable assignments need setDefault if they're
  self-referential.
- Remove local_append/local_set_default because there was
  no implementation for them written in product_config.rbc
  anyways.
- Correct productConfigVariable.emitDefined using the global
  variables instead of the product config variables.
- Emit setDefaults for all types of assignments if they're
  self referential.

Bug: 222737841
Test: go test
Change-Id: I06a0f90f16d5900049d473281e6d5ef5e93e67da
2022-03-10 15:00:57 -08:00
Cole Faust
3c4fc99593 Support variables with dashes in their names
Bug: 221946551
Test: go test
Change-Id: I085fc35159c4f3afe53868fbc731fcaeac3a69a8
2022-03-07 11:24:45 -08:00
Cole Faust
93f8d3987a Fix include paths that match one file being included unconditionally
In aosp/1962181 an "optimization" was added where if an include
path with a variable only matches one file, that file would be
called directly instead of going through the whole _entry
variable.

This doesn't work when there is an inherit-product-if-exists call
that matches one file, but that file isn't the one that is specified
at runtime. In reality, the inherit-product-if-exists call should
see that the file doesn't exist and not include anything.

Fixes: 222341691
Test: go test
Change-Id: If8789de0013726137683078355161eecdec009b1
2022-03-02 13:31:30 -08:00
Cole Faust
fc74246c98 Merge "Convert add-product-dex-preopt-module-config to Starlark" 2022-03-02 18:40:36 +00:00
Treehugger Robot
6028b556ec Merge "Inline board configuration call into printvars call" 2022-03-02 03:31:25 +00:00
Cole Faust
a060466be6 Inline board configuration call into printvars call
This is so that:
 - It's consistent with product configuration
 - The return type of board/product config's main functions
   and the arguments to printvars can be changed without
   also making a change in the soong repository. (what I'm about
   to do)
 - Removes obsolete TODO

Bug: 221312856
Test: Presubmits
Change-Id: I593f180881eab9521012df202af8f0233bc0b6ae
2022-02-28 11:56:06 -08:00
Cole Faust
1cc088523f Convert add-product-dex-preopt-module-config to Starlark
Bug: 221877397
Test: go test
Change-Id: I79f5209bb42f2c689af96ded376c08522cbec24e
2022-02-28 11:12:08 -08:00