Commit graph

150 commits

Author SHA1 Message Date
Cole Faust
8e15f69709 Remove ?= assignements to product variables
In make, all product variables are assigned to an empty string before
including the product config makefiles. In starlark, we don't do that
just so the memory usage is smaller. This means that in make, using
?= to assign a product config variable has no effect. Replicate this
behavior in starlark by not emitting any code for ?= assignments of
product variables.

Fixes: 304324003
Test: go test
Change-Id: Id31531506ac9e372a1bea92ce9ae8e17ae0ca70c
2023-10-09 12:26:21 -07:00
Cole Faust
b0b2457842 Handle include statements with trailing whitespace
Fixes: 303832950
Test: go test
Change-Id: I919556ee43a60f59dbd7c040025f01abb53da2a8
2023-10-06 11:57:29 -07:00
Cole Faust
8ff3c63376 Revert "mk2rbc: temporarily bump matching paths limit to 155. "
This reverts commit 30d48351be.

Reason for revert: The underlying issue was fixed in ag/23603641

Change-Id: Ibe2749c3bb7412132de6d96d178f54cd8d1d0666
2023-06-08 22:53:16 +00:00
Jingwen Chen
30d48351be mk2rbc: temporarily bump matching paths limit to 155.
It was 150, but an internal change bumped up the usage to 153. Temporarily increase it to fix build break.

Bug: b/284854738

Change-Id: I23478f4c5333c5ef1ad344aadafe5803685c5d92
Test: abtd 
Test: treehugger
2023-05-29 16:01:34 +00:00
Cole Faust
c85d08f63b Make mk2rbc output paths relative to android root
Previously, mk2rbc would output absolute paths in the load statements.
This caused issues with the new module resolution logic in rbcrun,
and is not something we'd want to check in anyways.

Bug: 280685526
Test: go test
Change-Id: Iae6e3f6be9cecc8e19b6e1fd33c5575ff6864654
2023-05-09 15:00:58 -07:00
Cole Faust
c7335de0dd Don't type variables as booleans in mk2rbc
Our boolean support is not fully fleshed out enough to make variables
real booleans. Keep them strings for now to resolve CI failures.

Bug: 275865081
Test: ./build/bazel/ci/rbc_dashboard.py --quick with aosp/2526609
Change-Id: I636bd9f39b50a47ecf92aecd2bf8ea41eac4d604
2023-04-12 12:13:28 -07:00
Dan Willemsen
aa2b7cc21d Rename mk2rbc/cmd to make go install work properly
The standard go tooling assumes that a package's binary name should be
the name of the containing folder. Before this change, `go install` or
`go build` (without `-o`) names this binary `cmd`.

Test: treehugger
Change-Id: I45a4e97f4a0d59744d644e30636250ceefe3bd1c
2023-03-15 22:51:01 -04:00
Alfred Piccioni
2d0349f133 Merge "Removing references to arm7 emulator targets." 2023-01-24 09:55:13 +00:00
Cole Faust
7366042870 Show correct line numbers for errors in an $(eval)
Bug: 264488559
Test: go test
Change-Id: I95ce0345912a2af351be0ecc4595bdc4a2047c0c
2023-01-05 11:22:46 -08:00
Cole Faust
5a3449f11c mk2rbc support for clear_var_list
Bug: 262303006
Test: Manually (Added usage to aosp_arm64 and ran rbc_dashboard.py on it)
Change-Id: I3cba8f1d69ba4f42b4bc125f0a1131c1d33f7efb
2022-12-12 19:02:42 -08:00
Cole Faust
2dee63d9a8 Add to-lower/to-upper
Bug: 262303006
Test: go test
Change-Id: I80af8ace90556e58676adac4cb90d5d62c5b7b96
2022-12-12 18:18:15 -08:00
Cole Faust
d2daabfa02 Add support for and/or in mk2rbc
Bug: 262303006
Test: go test
Change-Id: I04130648084775c0828e95cd18a82e348c3f09eb
2022-12-12 18:17:37 -08:00
Alfred Piccioni
b42eaef006 Removing references to arm7 emulator targets.
Test: Makefiles no longer exist.
Bug: 258197384
Change-Id: Id7ba8957dee12259379c4852aed9cc7640a28a9a
2022-12-05 14:00:56 +01:00
Colin Cross
d079e0b270 Reformat build/soong for go 1.19
Test: none
Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-08-17 10:43:13 -07:00
Cole Faust
74ac027962 Allow using include_top to filter results even when there is a constant prefix
Previously, when seeing a `include foo/$(VAR)/bar.mk`, the include
tops wouldn't apply to filter down the results any further.

Allowing include_tops is useful here to prevent cycles in the
load graph.

Fixes: 236025787
Test: go test
Change-Id: I3ea445f222ee2283a68d4aca5002df750204076f
2022-06-14 13:40:32 -07:00
Cole Faust
1323877b7e Support complicated variable references
Bug: 226974242
Test: go test
Change-Id: Iaec16f5c498e7c75c9ee5d53d3499efadfba16bc
2022-05-10 05:26:35 +00:00
Cole Faust
72374fc628 Flatten foreach calls that produce 2d lists
In make, the result of $(foreach $(x),$(y),$(foreach $(z),(w),a))
is a regular list, but in Starlark it's a list of lists. Flatten
the results of foreach expressions where each element is a list
so that they're regular lists of strings again.

Bug: 226974242
Test: go test
Change-Id: I3210d409aba0d807a5890e341ab1e0c0478f5930
2022-05-09 22:47:34 +00:00
Cole Faust
7940585cf0 Merge "Convert firstword/lastword to starlark functions instead of index operators" 2022-04-29 18:32:40 +00:00
Cole Faust
01624a9c33 Merge "Show load cycles when running mk2rbc" 2022-04-29 00:44:12 +00:00
Cole Faust
2b6a312662 Merge "Recognize some unsupported functions" 2022-04-29 00:41:15 +00:00
Treehugger Robot
f8c3412822 Merge "Convert $(words) to Starlark" 2022-04-28 22:18:07 +00:00
Cole Faust
16bb2d044d Merge "Allow seeing include $(x) when there is an include_top comment" 2022-04-28 20:49:17 +00:00
Treehugger Robot
e8f19f3dd7 Merge "Convert includes/inherits inside evals" 2022-04-28 20:43:29 +00:00
Cole Faust
1a3e5effa8 Merge "Allow comparing $(wildcard) results to non-empty values" 2022-04-28 19:54:32 +00:00
Cole Faust
5a13aaf112 Convert firstword/lastword to starlark functions instead of index operators
firstword/lastword need to be able to give an empty string when
the input is empty, which the indexing operator can't do.

Bug: 226974242
Test: go test
Change-Id: I44162a258297b32ec27e7acfb1fa06223391dcc6
2022-04-27 17:49:35 -07:00
Cole Faust
da9770eb67 Show load cycles when running mk2rbc
Previously, if there was a cycle of load statements,
mk2rbc would exit with status code 1 but not give
any message indicating what went wrong.

Bug: 226974242
Test: Manually
Change-Id: I7d30cad08bd3abcd761b5f28a63a2cc66b45e805
2022-04-27 15:13:47 -07:00
Cole Faust
a59059f3a1 Merge "Ignore assignments to .KATI_READONLY" 2022-04-27 17:56:54 +00:00
Cole Faust
5d5fcc3092 Ignore assignments to .KATI_READONLY
Since the rbc results file is sorted, .KATI_READONLY
ends up trying to make a variable that hasn't been defined
yet readonly, which is an error.

It's not really worth supporting moving it down in the list,
because it wouldn't affect the variable being writable while
still in Starlark code.

Bug: 226974242
Test: go test
Change-Id: I9402f69be97e5c7cf010ad86f124422ea55fda7f
2022-04-26 18:02:05 -07:00
Cole Faust
9df1d73657 Allow seeing include $(x) when there is an include_top comment
Previously, seeing anything other than an interpolate variable
was not allowed.

Bug: 226974242
Test: go test
Change-Id: I48a060f9a3fd19cd67a114d2cb0756ab2be25ce1
2022-04-26 17:46:49 -07:00
Cole Faust
1e27586012 Recognize some unsupported functions
Some functions are called without the use of $(call),
since $(call) is not necessary when the function doesn't
take any arguments. mk2rbc thinks these function calls
are local variables, and converts them as such. This
leads to undefined variable errors when trying to load
(before even executing) the starlark file.

Hardcode a known function that should not be converted
to a local variable.

Bug: 226974242
Test: go test
Change-Id: I5567a34fcc282b181a7e78ac3d5cc9b40bd025a2
2022-04-26 14:53:25 -07:00
Cole Faust
00afd4f8b9 Ignore variable assignments that come after a rule definition
These private variables sometimes have references to make
variables like $< or $@. When converted to starlark, they
become invalid local variables like _< and _@, preventing
the file from being loaded even if it's never executed.

Bug: 226974242
Test: go test
Change-Id: Iafd4c6939731f3b7c051c9e41464134d5b672f23
2022-04-26 14:01:56 -07:00
Cole Faust
a99afdfc22 Allow comparing $(wildcard) results to non-empty values
Also remove the file existence functions because they
can just be wildcards instead.

Bug: 226974242
Test: go test
Change-Id: Icbf65c47af97a710580864e8b76e2697aba96dd8
2022-04-26 12:06:49 -07:00
Cole Faust
2005298b68 Convert includes/inherits inside evals
These occur in some vendor makefiles in
order to include files in a foreach loop.

Bug: 226974242
Test: go test
Change-Id: I59242eba9a2d7c659994556d9123b6481f4a8130
2022-04-26 10:44:08 -07:00
Cole Faust
94c4a9ac62 Convert $(words) to Starlark
Bug: 226974242
Test: go test
Change-Id: I8ab5943306586c6a993df470e7a2894dd7ff1619
2022-04-22 17:43:52 -07:00
Cole Faust
7cd255fa8d Merge "Move variable assignment handling to generation context" 2022-04-14 16:59:10 +00:00
Cole Faust
6c41b8a729 Add handle to enforce_product_packages_exist
Bug: 229132189
Test: go test
Change-Id: I76804354051f0866c8cad48c1189b36de4f73e23
2022-04-13 14:04:27 -07:00
Treehugger Robot
19da477d4d Merge "Add the handle to artifact path requirement functions" 2022-04-08 17:32:25 +00:00
Cole Faust
f06326648b Move variable assignment handling to generation context
This allows the parsing code to be cleaner, as it
doesn't have to care about variable assignments.

Bug: 228518745
Test: go test
Change-Id: I33425c2fb51acab4901bfa82a53d337b75210f8e
2022-04-07 15:40:12 -07:00
Cole Faust
62e0511202 Allow include $(sort $(wildcard */font.mk)) and variants
Include/inherit-product calls already allow asterisks, the
wildcard is not necessary. If we see an include using $(wildcard),
act as if it doesn't exist.

Bug: 218736658
Test: go test
Change-Id: Ib21007a2042fbfaa95c07571792983e14135a035
2022-04-06 16:37:54 -07:00
Cole Faust
95b95cb647 Convert $(sort) to Starlark
This is mostly so that we can not error out when
seeing sort inside a `include $(sort $(wildcard foo/*.mk))`.

Bug: 218736658
Test: go test
Change-Id: I827ecfc751a908227cb2dd9ed1e4b3f313791a16
2022-04-05 16:37:39 -07:00
Treehugger Robot
8210fbec1f Merge "Support converting simple $(eval) expressions" 2022-04-05 22:44:07 +00:00
Cole Faust
0e2b256846 Correct type of dir function
It should return a string, not a list.

Bug: 227245326
Test: go test
Change-Id: If0cb382dede52200c0595ae939a8064641216af1
2022-04-01 11:49:24 -07:00
Cole Faust
f035d405d8 Support converting simple $(eval) expressions
While supporting $(eval) in the general case is
impossible, as it would require emitting code at
runtime, it is possible to handle some special cases
that are common throughout the code base.

Specifically, an assignement expression (where the
left hand side is constant) can be converted without
needing to evaluate strings as code, as its whole
right hand side is treated as one string.

However, this eval with an assignemnt can only be
used as a statement, not an expression. So it requires
the eval to be either a top-level expression, or nested
within other expressions that can be converted to
statements such as $(foreach) or $(if).

Bug: 226974242
Test: go test
Change-Id: Ifc52ef9ab7d62a69251918fcde5463f80a98a542
2022-03-31 12:44:59 -07:00
Cole Faust
ea9db58de9 Add the handle to artifact path requirement functions
Bug: 188079133
Test: go test
Change-Id: I320f4f14fb47414cb14ebb404efab4bceb1fa1e4
2022-03-22 11:17:16 -07:00
Treehugger Robot
5beafc1900 Merge "Replace $(call my-dir) with a string literal" 2022-03-22 03:24:47 +00:00
Treehugger Robot
c9e38327dd Merge "Parse variable references with #s" 2022-03-22 03:15:47 +00:00
Cole Faust
f5adedce03 Replace $(call my-dir) with a string literal
This is so that we can set LOCAL_PATH to the result
of my-dir, as LOCAL_PATH can only be set to a string
literal of the exact value of LOCAL_PATH.

It's probably also the correct choice to start phasing
out LOCAL_PATH.

Bug: 214405650
Test: go test
Change-Id: Ia97d7fedf4ce62643921d90a176e65edd4e2fce6
2022-03-18 14:05:06 -07:00
Cole Faust
e309a91ca8 Parse variable references with #s
Inside a variable reference, a # does not start
a comment.

Fixes: 218742602
Test: go test
Change-Id: I16cf04c74a8aa30482fd9293175f893e4efb60f1
2022-03-17 17:15:12 -07:00
Cole Faust
421a192d05 Convert values to strings when assigning to a string variable
Currently, a string variable can easily be reassigned to
a different type. Make it so that the value it's being
reassigned to is converted to a string first.

Bug: 224601891
Test: go test
Change-Id: I82252cf9e106b5a3677458cf1df2e9d1dfefe0f6
2022-03-16 14:35:45 -07:00
Treehugger Robot
0ddc5724b0 Merge "Add type hints to mk2rbc" 2022-03-16 00:40:11 +00:00