Commit graph

52 commits

Author SHA1 Message Date
Cole Faust
2b5b3f3be8 Make words() work on a list of non-strings
words() attempts to join all the elements of a list
and then resplit them to more closely match make.
But sometimes, like when calling words() on a product
variable, not all of the elements are strings. In
that case, just return the list unchanged.

Bug: 267407943
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I738d0c86c8935f446807cc79623f796e8cae3c01
2023-02-07 12:28:47 -08:00
Treehugger Robot
3d3c7ee1d9 Merge "Add test for bazel server shutdown/restart" 2023-01-12 21:48:07 +00:00
Cole Faust
f07dcc48e8 Fix printing variables that have inherit references in them
Inherit references (@inherit in make, 1-tuples in starlark)
usually appear in list product variables, where they're
evaluated before printing. But sometimes they can be copied
into a regular global variable, in which case they won't
be evaluated before printing. The 1-tuples were failing to
print, so adapt them to their make representation before
printing.

Fixes: 264554449
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I5ac6eb996c25ee5e77aa26ed5c4b6b6cc31819d1
2023-01-05 12:06:26 -08:00
Cole Faust
44de6d8f7a Make clear_var_list set nonexistant variables to empty strings
Make's clear-var-list causes the variables to exist as empty strings.
Mimic that functionality in starlark for the variable diff and ?=.

Bug: 262303006
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I5f9c8cf342d875b1022c5c74906e59fa68fcd6c9
2022-12-13 13:02:10 -08:00
Cole Faust
8397c8efee Add rbc implementation of clear_var_list
Bug: 262303006
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: Ib5d7d37a7d7fa40ec990d9506fe0bef7da5b5bcd
2022-12-13 10:17:16 -08:00
Cole Faust
f2b0c734a9 Add test for bazel server shutdown/restart
We recently had an issue where the bazel server
couldn't be shutdown/restart because it was being
run in a docker container with a PID 1 that wasn't
capable of reaping zombie processes.

Bug: 234449134
Test: abtd run of this test because it's not run on presubmit
Change-Id: I1e179902364ea84fd55003389f3045de2e998991
2022-12-02 10:48:24 -08:00
Jingwen Chen
6db4d4f435 Call lunch before using b.
lunch is required to set up PATH to access build/bazel/bin/b as b.

Test: run the test
Bug: 255711288
Fixes: 255711288
Change-Id: Ie5bad8ffdacc3d59114f70d7f533486ebd12f778
2022-10-27 12:52:17 +00:00
Jingwen Chen
83eeebb4df envsetup: add bmod to get Bazel label of a Soong module.
This benefits interactive workflows of folks who are used to typing just
module names on the command line, and lowers the cognitive load to find
the package name, and can be used to check if a module is converted.

Can be used as :

$ bmod libaapt2
//frameworks/base/tools/aapt2:libaapt2
$ b build $(bmod aapt2)

If the mod is not in the bp2build metrics file, it returns:

$ bmod nonexistent
nonexistent is not converted to Bazel.

Test: build/make/core/tests/b_tests.sh
Change-Id: I245665a393621a47598e6743c0fedcd87ac33631
2022-10-05 04:11:22 +00:00
Sam Delmerico
7017ac77fd Merge "remove zsh from roboleaf_tests.sh" 2022-09-28 19:40:39 +00:00
Sam Delmerico
43444ec2fe remove zsh from roboleaf_tests.sh
zsh command was not found on the build server. Remove it from this test.

Bug: 249559295
Test: build/make/tests/roboleaf_tests.sh
Change-Id: I7a79b183d9fd40829ab03bc54ce899efaa974a60
2022-09-28 15:34:34 -04:00
Sam Delmerico
d0c8cf02b6 re-enable b run commands in test suite
Zsh has some incompatibilties that we've been trying to work around in
the b command. Hopefully this prevents future issues.

Bug: 242759256
Test: build/make/tests/roboleaf_tests.sh
Change-Id: I5e887932e943112377d38e77a77a0f294b43d9b6
2022-09-28 09:52:22 -04:00
Sam Delmerico
4f6ca1a546 re-enable b test suite
Bug: 244771601
Test: build/make/tests/roboleaf_tests.sh
Change-Id: I648960b336acd31709aafe858fcbc5be9dc8397a
2022-09-28 09:46:21 -04:00
Sam Delmerico
3e5fcd8a61 temporarily disable roboleaf_tests.sh
When I enabled the envsetup_tests target on CI, this script was failing.
Rather than break the build again by enabling CI on a potentially
failing script, let's just disable this for now, enable the CI target,
and then this script can be enabled and tested in presubmits before
submitting.

Bug: 244771601
Test: build/make/tests/roboleaf_tests.sh
Change-Id: I7c3469b49258e37c0ff9b49dd8805031216a7570
2022-09-21 10:56:49 -04:00
Sam Delmerico
4ee8508854 add test suite for b command
Ensuring that the b command works properly cross-terminal has been a
little tricky, so some test commands would help in testing changes.

Test: build/make/tests/roboleaf_tests.sh
Change-Id: Iecff678fbba6e81c211aadd3665839c1c21acc8c
2022-09-09 17:29:35 -04:00
Sam Delmerico
4f4d069efa move lunch specific tests to separate file
This is in preparation to add some more tests to this file.

Test: build/make/tests/envsetup_tests.sh
Change-Id: Ic32115d2a6073f3265b84846f489f3acb8bfad88
2022-09-09 17:27:22 -04:00
Cole Faust
6f6060a358 Add function for flattening 2d lists
Bug: 226974242
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: Iae977b34ca0ac156ec795c54f4951c769fa994dd
2022-05-05 11:41:59 -07:00
Cole Faust
c93109dc36 Add implementations for firstword/lastword
Bug: 226974242
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: Ibb992f42a59212bae48acd55647b2d0872c2f69e
2022-04-28 15:56:33 -07:00
Treehugger Robot
9ec1a7aa72 Merge "Correct abspath implementation" 2022-04-20 01:32:16 +00:00
Cole Faust
426c7441b0 Correct abspath implementation
realpath doesn't return a path if the file doesn't exist,
but $(abspath) in make does.

Bug: 229132189
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: Ief7f634024cc52a9e8c5e478666b15512512f0d8
2022-04-19 14:46:09 -07:00
Cole Faust
d370a3f7bd Sort inherited products correctly
We pass the filename without the extension to
rblf.inherit(). Removing the extension changes the
sort order when one file's name is a prefix of another:

```
>>> sorted(["base", "base-secondary"])
['base', 'base-secondary']
>>> sorted(["base.mk", "base-secondary.mk"])
['base-secondary.mk', 'base.mk']
```

Correct the sort order so that global variables get
their correct ordering.

Bug: 229132189
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I22367eb49b33956b71ac1b966fe78c1308b94257
2022-04-18 17:18:08 -07:00
Cole Faust
1c08360ca8 Evaluate intermediate products properly
When an intermediate product is evaluated, it needs to
act as if it's the only product being evaulated. However,
currently, if it inherited a makefile that was also being
inherited by the overall top level product via a different
path, it would not get the values from that makefile.

Copy the configs dictionary before evaluating each product
that needs artifact path requirements, and create seperate
postfix orders for all of them that don't contain any products
that they don't inherit from.

Bug: 221312707
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I235ad78d587a2e315ba446b5e126d8f6d0fbbea7
2022-04-12 15:25:14 -07:00
Cole Faust
2491ad52c1 Add mksort implementation
Bug: 218736658
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I0069e7059453c16d299cc650f56c320d905f1958
2022-04-05 16:36:27 -07:00
Treehugger Robot
52233be25c Merge "Fix single value variable inheritance order" 2022-04-05 21:34:26 +00:00
Treehugger Robot
ea8eedde00 Merge "Revert "Fix inheritance order"" 2022-04-04 21:07:26 +00:00
Treehugger Robot
bd78ed0726 Merge "Remove regex functionality from rbcrun" 2022-04-04 20:36:35 +00:00
Cole Faust
32e413a294 Fix single value variable inheritance order
List variables needed to be percolated in the order of inherit() calls.
children.keys() was in that order, due to starlark dictionaries being
iterable in the order of insertion, but the previous cl broke that
behavior by sorting them. Instead, only sort the children for
single value variables.

Fixes: 226206409
Fixes: 228044099
Test: ./out/rbcrun ./build/make/tests/run.rbc and testing aosp_arm64
Change-Id: I5b91514e87b158b615e4d4ec7868fccb0248379b
2022-04-04 11:55:02 -07:00
Cole Faust
56d7c008bc Revert "Fix inheritance order"
This reverts commit 670c587c09.

Reason for revert: Breaks RBC product config for more products than it fixes: b/228044099

Change-Id: I614663311f67b3631a0cee427453c550fc426c8b
2022-04-04 17:28:28 +00:00
Cole Faust
670c587c09 Fix inheritance order
The children to inherit from were not sorted. This
wasn't a problem for list variables, because all those
inheritances get merged together based on the @inherit/
values in the list variables. But for single value variables
it made a difference.

Bug: 226206409
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: Ib56bbb91a79fe8c7cb780c253f5bd8d6c0e87765
2022-03-31 17:30:35 -07:00
Cole Faust
62878a2cef Remove regex functionality from rbcrun
As a first step to making .rbc files compatible with bazel,
remove regex support since bazel doesn't have it.

Fixes: 227384703
Test: ./out/rbcrun ./build/make/tests/run.rbc
Change-Id: I8b946c20cc42897a47a5516a167732f4e16b6158
2022-03-30 12:00:15 -07:00
Cole Faust
31fce2f404 Fix single value inheritance
Single-value product variables resolve to the first
instance of that variable in prefix order. What we
have right now is taking the value from the current
makefile if it's set, but if it's not, we take it
from the last child, not the first.

Fixes: 226206409
Test: Manually
Change-Id: Id23f7c269ff9a352bf0b67cb57156b72449f978e
2022-03-23 15:28:48 -07:00
Cole Faust
fdff6b1e9c Allow multiple patterns in rblf_wildcard
This is to match the functionality of make's $(wildcard)

Bug: 201700692
Test: go test
Change-Id: Ib20c4a4aa9642a2d76b5da5c7accd60e6b91c8fc
2021-12-08 17:52:40 -08:00
Cole Faust
f1f49bb910 Pass input variables to product config
Passing variables via a makefile instead of
rblf_cli / rblf_env allows us to give them correct
types while converting the makefile to starlark,
as opposed to the variables always being strings
when given via rblf_cli / rblf_env.

This also allows us to remove some hand-converted
starlark code.

Bug: 201700692
Test: ./out/soong/rbcrun ./build/make/tests/run.rbc
Change-Id: I58c4f20b29171c14e5ae759beb26a849426f6961
2021-12-02 20:47:31 -08:00
Cole Faust
3be5b72ad6 Correctly copy the board configuration input variables
dict(**h[0]) was only copying the top level dictionary
object, but not any other dictionaries/lists inside of
it. This was causing us to not find all the changes to
variables the board configuration was making, and so
some variables were not included in the output.

Bug: 201700692
Test: ./build/bazel/ci/rbc_regression_test.sh mainline_system_x86_64-userdebug
Test: ./out/soong/rbcrun ./build/make/tests/run.rbc
Change-Id: I4467eead7e597f6f49119a8c4832126f39d646e7
2021-12-01 11:06:23 -08:00
Sasha Smundak
c20dd972a9 Runtime reporting of the conversion errors.
Bug: 204062171
Test: rbcrun build/make/tests/conversion_error.rbc (displays a message, stops)
  rbcrun RBC_MK2RBC_CONTINUE=t build/make/tests/conversion_error.rbc  (displays a message and continues)
Change-Id: Idf0fe6bbb08b1eca1f392d81950eed4a214f3289
2021-11-18 11:27:55 -08:00
Sasha Smundak
91fc734100 Runtime support for dist-for-goals.
Convert dist-for-goals.

Bug: 198496782
Test: rbcrun build/make/tests/run.rbc
Change-Id: I46ef9e897143aecf8bd28ad8569d48e74530b0e5
2021-11-18 11:20:34 -08:00
Cole Faust
0cc94d3275 Add notdir implementation
mk2rbc would already translate notdir calls,
but since there was no implementation it would
fail at runtime.

Bug: 201700692
Test: m rbcrun; rbcrun build/make/tests/run.rbc
Change-Id: Ie0f4e7b65448e612fa56f87b0bc138648cc0ad58
2021-11-17 10:50:14 -08:00
Cole Faust
70a886c7ce Add soong_config_get
Having a function to get soong config variables
makes it easier to convert to starlark.

Bug: 201700692
Test: m RBC_PRODUCT_CONFIG=1 RBC_BOARD_CONFIG=1 nothing and check output files
Change-Id: I8627555df10d3f66f6154c00be9f0565076f1f94
2021-11-09 17:09:04 -08:00
Sasha Smundak
3370ad5b70 Add board_platform_in/board_platform_is runtime functions.
Bug: 190051051
Test: treehugger
Change-Id: I27269fc0f0f7063635fbb1b1645919b604afb762
2021-10-25 10:41:56 -07:00
Sasha Smundak
dc154164db Better Soong namespace support in the RBC runtime.
Includes:
* Defining namespace again should have no effect
* Allow appending to a variable defined in a namespace
* Consistent naming
* Print namespace variable assignments even if they are empty

Bug: 200297238
Test: rbcrun make/build/tests/run.rbc
Change-Id: I64aa22c4498ae89e4bc3a077d6206ad37d5c7c38
2021-10-25 10:41:46 -07:00
Sasha Smundak
b2220c2990 Make global config depend on version defaults.
Starlark-based product configuration can now share version settings
with makefile product config (mk2rbc converts version_defaults.mk into
version_defaults.rbc which is consumed by runtime initialization).

Bug: 198995713
Test: rbcrun build/make/tests/run.rbc
Change-Id: I1d3ddfed3b15d346b3e10714a195a9f0a3a55a56
2021-09-17 11:00:29 -07:00
Sasha Smundak
f00e35ecae If filter/filter_out pattern is a list, remove empty elements from it.
Also, fix __mk2regex bug: the returned pattern should end with $.
Bug: 193540681
Test: internal
Change-Id: Ia56856826c6b05ccf857ae5ab7a70609bf4f1e1d

Change-Id: Ie2a9bf62ee48824ce7cd5fe9e9ec2f55311473e3
2021-08-26 13:43:03 -07:00
Sasha Smundak
6797bfaac1 Fix copy_files implementation
It has to copy all the files into the same directory, stripping the
relative path from the source file.

Bug: 193540681
Test: rbcrun build/make/tests/run.rbc
Change-Id: I2d61d966ad94d73131c7ad6016ce319f8d2ae509
2021-08-19 13:56:09 -07:00
Sasha Smundak
6b795dc6a5 Add find_files builtin, use it to fix find_and_copy implementation
The macro find-and-copy finds all the files in the given source tree that
match the given filename patten and create <source>:<dest> pair with the
same relative path in the destination tree.

Bug: 193540681
Test: rbcrun build/make/tests/run.rbc
Change-Id: Ic4315ce2fab7a7791ab55dd9eed039205a1c721a
2021-08-19 13:54:53 -07:00
Sasha Smundak
ed1f09cf4a Support product-copy-files-by-pattern macro
Bug: 193566316
Test: rbcrun build/make/tests/run.rbc
Change-Id: Idf462d7f58e8d4a6e8b3a1506306f9eb67130dc8
2021-08-18 09:28:04 -07:00
Sasha Smundak
a93e3d9985 Implement copy_files function
Bug: 193540681
Test: treehugger
Change-Id: Iffc64551e733747a9a29c14b1f674b3a533f44ba
2021-07-29 11:42:29 -07:00
Sasha Smundak
c106138baf Implement add_soong_namespace and add_soong_config_var_value functions
Bug: 193540681
Test: rbcrun build/make/tests/run.rbc
Change-Id: I129136e83d2d00ef5b64d3aab07b98719198dcfe
2021-07-29 11:24:35 -07:00
Sasha Smundak
3b25eb1c98 Add mkpatsubst
Bug: 181797530
Test: rbcrun build/make/tests/run.rbc
Change-Id: Id15daaf6e3ed68982e50cdc710563095ffcf57ba
2021-07-13 09:21:07 -07:00
Sasha Smundak
9afdb1c92a Add mksubst
Bug: 181797530
Test: rbcrun build/make/tests/run.rbc
Change-Id: I196c69259b830b6466430f6c933afb49fa787948
2021-07-12 13:36:13 -07:00
Sasha Smundak
be4ebca4ba Add mkstrip
Bug: 181797530
Test: rbcrun build/make/tests/run.rbc
Change-Id: I6b7ade400273cf5a4fd561322e8e9936bc9d8df6
2021-07-09 10:44:22 -07:00
Sasha Smundak
5c8b09b2ef Product configuration in Starlark support files.
Bug: 170637441
Test: rbcrun build/make/tests/run.rbc
Change-Id: Ie8f0c3dea66a287f2b0a3b588c446bfbb2c9a978
2021-03-18 10:52:05 -07:00