Commit graph

255 commits

Author SHA1 Message Date
Cole Faust
74639be8a7 Merge "Sandbox LTP genrules" into main 2023-11-03 20:16:00 +00:00
Cole Faust
9cae593d97 Sandbox LTP genrules
gen_ltp_config used to read files from the source tree,
but this cl refactors it to package all the files it
needs into the tool itself, so it's more hermetic and
all the genrules don't need to explicitly list the files
the tool needs.

go/roboleaf-busy-beavers-sandboxing

Bug: 307824623
Test: diff'd the files produced by the ltp_config_* genrules before/after this change
Change-Id: Ia72084965dcb8659394068b7c6877adb1c882dc2
2023-11-02 22:23:01 -07:00
Treehugger Robot
66135a79f4 Merge "Sandbox test apex genrules" into main 2023-11-03 02:11:36 +00:00
Cole Faust
06e73446de Sandbox test apex genrules
With the other cls in this topic, the modules build with sandboxing.
They still fail the genrule_sandobox_test.py because their builds are
non-deterministic though. I wasn't able to find the cause of the
non-determinism, so relying on presubmits to make sure nothing breaks.

Bug: 307824623
Test: m com.android.apex.test.bar_stripped com.android.apex.test.baz_stripped com.android.apex.test.foo_stripped com.android.apex.test.pony_stripped com.android.apex.test.sharedlibs_generated com.android.apex.test.sharedlibs_secondary_generated with sandboxing enabled
Change-Id: I4006732bf1ba08e846dee2e18d89dbf45f2cd7ba
2023-11-02 18:08:20 -07:00
Cole Faust
f45290a496 Remove UpdatableSystemFontTest modules from sandboxing denylist
These modules actually work fine with sandboxing, they were just added
to the list because they have non-deterministic outputs, which caused
the genrule sandboxing script to think that sandboxing affected their
outputs.

Bug: 307824623
Test: ./build/soong/tests/genrule_sandbox_test.py UpdatableSystemFontTest_NotoColorEmojiV0.sig UpdatableSystemFontTest_NotoColorEmojiV0.ttf UpdatableSystemFontTest_NotoColorEmojiVPlus1.sig UpdatableSystemFontTest_NotoColorEmojiVPlus1.ttf UpdatableSystemFontTest_NotoColorEmojiVPlus2.sig UpdatableSystemFontTest_NotoColorEmojiVPlus2.ttf with aosp/2816385
Change-Id: I07b1df779c8d47ad875a4fc2d3af5e46945cff83
2023-11-02 17:15:23 -07:00
Cole Faust
0605687dd6 Remove AudioFocusControlProtoStub_cc/h from denylist
Test: m AudioFocusControlProtoStub_cc AudioFocusControlProtoStub_h
Change-Id: I7743f1d7d05caa9c14c7257fb117d07d02b7edaf
2023-10-31 13:07:40 -07:00
Cole Faust
6100a84dd1 Merge "Remove certificate genrules from denylist" into main 2023-10-26 23:18:40 +00:00
Cole Faust
d86a9094a2 Remove bluetooth genrules from denylist
Bug: 307824623
Test: m all of them
Change-Id: I7d0af72c7ed1fd190854c1953dc93808e3cb3958
2023-10-25 17:33:56 -07:00
Cole Faust
831f2bfbe5 Remove certificate genrules from denylist
The denylist was built by running `genrule_sandbox_test.py` on all
modules in the tree. `genrule_sandbox_test.py` checks that the sandboxed
genrules build, but also that they get the same results as the
unsandboxed version.

In this case, the genrule actually builds with sandboxing just fine,
but they have non-deterministic results, which caused
`genrule_sandbox_test.py` to think that they didn't work with
sandboxing.

Test: m com.android.apex.apkrollback.test.pem com.android.apex.apkrollback.test.pubkey com.android.apex.cts.shim.debug.pem com.android.apex.cts.shim.debug.pubkey com.android.apex.cts.shim.pem com.android.apex.cts.shim.pubkey com.android.apex.cts.shim.v2_no_pb com.android.apex.cts.shim.v2_signed_bob com.android.apex.cts.shim.v2_signed_bob_rot com.android.apex.cts.shim.v2_signed_bob_rot_rollback com.android.apex.cts.shim.v2_unsigned_apk_container com.android.apex.cts.shim.v3_signed_bob com.android.apex.cts.shim.v3_signed_bob_rot com.android.apex.cts.shim_not_pre_installed.pem com.android.apex.cts.shim_not_pre_installed.pubkey com.android.apex.rotation.key.bob.pem com.android.apex.rotation.key.bob.pk8 com.android.apex.rotation.key.bob.rot com.android.apex.rotation.key.bob.rot.rollback com.android.apex.rotation.key.bob.x509.pem com.android.overlaytest.overlaid.pem com.android.overlaytest.overlaid.pubkey com.android.overlaytest.overlay.pem com.android.overlaytest.overlay.pubkey
Change-Id: I950767449025163d8c71bb5a7b2e2f15a1ce4a84
2023-10-25 15:10:38 -07:00
Spandan Das
db6a189a41 Turn on sandboxing for external/perfetto
Contingent on aosp/2788322

Test: GENRULE_SANDBOXING=true m libperfetto_client_experimental # bit
identical

Change-Id: I885f420850a165f042b94685e7cf1215d5620716
2023-10-18 18:04:11 +00:00
Cole Faust
3af559c9ce Denylist CompilationTestCases_package-dex-usage
I'm not sure how this was missed earlier.

Bug: 290816499
Test: build/soong/tests/genrule_sandbox_test.py -t sdk_phone_x86_64 all with aosp/2666142
Change-Id: I8d21a34e3b13ac568fa6153a31c43ba3e4d516bd
2023-10-12 09:54:53 -07:00
Cole Faust
dc3c1454fa Sort genrule allowlists
To avoid merge conflicts.

Bug: 290816499
Test: Presubmits
Change-Id: I8f8ba5112ae87c21dd95513ee1cf0d3d16bccf3f
2023-10-11 12:37:25 -07:00
Cole Faust
10b31340a6 Denylist genrules that fail to build with sandboxing
These were added with the android 14 release.

Bug: 290816499
Test: run genrule_sandbox_test.py with a local change to check all genrules in the tree
Change-Id: Icf3627c245638ab3f73b83e24ef04c916d7ab58b
2023-10-06 12:01:02 -07:00
Cole Faust
8a25345b7d Denylist genrules that fail to build with sandboxing
Bug: 290816499
Test: run genrule_sandbox_test.py with a local change to check all genrules in the tree
Change-Id: Id18b801c2306dd59b5b593b004b513b578ce3705
2023-10-06 05:12:07 +00:00
Christopher Parsons
c331812a59 Merge "Have ConvertWBp2build use Bp2buildMutatorContext" into main 2023-09-20 17:56:24 +00:00
Chris Parsons
637458d326 Have ConvertWBp2build use Bp2buildMutatorContext
This no-op refactoring facilitates some upcoming functional changes for
"bp2build allowlist v2". The work requires that the bp2build conversion
mutator be changed from a TopDown mutator to a BottomUp mutator.
Refactoring all bp2build-related methods so that they use Bp2buildMutatorContext
makes it easier to make this functional change without touching tens of
files and multiple projects.

Bug: 285631638
Test: m bp2build
Change-Id: I3d1ef3064146e959c6f0dc315350fc9764bf2bd2
2023-09-20 14:49:35 +00:00
Liz Kammer
9e2a5a7d6d Don't panic for unhandled product vars
Instead, we return an error. This allows us to access some product
variable information earlier when it will not be used as an attribute
without panicing

Test: m nothing
Change-Id: Id094b2b9e1364a8d174d99b3824fa149fb235b3e
2023-09-19 15:58:22 -04:00
Romain Jobredeaux
9973ace1c7 Bp2build support for app R8-related fields
Bug: 293304784
Test: CI
Change-Id: I0aefa57f968cb41e50d7c1d29b2d8a1480655355
2023-08-31 01:35:44 -04:00
Sam Delmerico
f877563735 update docs for genrule tool_files prop
According to go/roboleaf-busy-beavers-sandboxing, this field should just take in any data files needed by the tool. The existing description makes it sound like this property should only contain a single file that will be used as the tool itself.

Change-Id: I3ef3b8ceb52f7a7e6de9e0a897d5cc05c9c2d336
2023-08-14 23:54:55 +00:00
Cole Faust
f0d4d4f9f2 Bp2build genrule commands as configurable properties
So that users can use soong config variables / product variables
to adjust a genrule's command.

Bug: 295910468
Test: m nothing
Change-Id: I9fedf8d5d52e515c3fdb913411ce1b3fecb7ba81
2023-08-14 11:58:20 -07:00
Spandan Das
a99348dca4 cc Bp2build support for genrules that generate .proto file
If `srcs` contains a gensrcs/genrule module, the current bp2build module
will put it in the catch-all `srcs` attribute. This is reserved for .cpp
sources, so if the genrule produces a .proto/.aidl/... file, this will
fail.

This handles genrules that produce .proto files. To implement this, this
creates an additional partition that detects if the other module is a
genrule/gensrc that produces .proto files. If true, it will append it to
the proto partition.

This CL does not handle
- genrule that produce .c/.aidl/.yacc/.... files. They will continue to
  be partitioned into the catch-all partition
- java modules

Test: unit tests
Test: TH
Bug: 293205700
Change-Id: Ib720fdf3a053ff5dd256e6faa632e3fa7776966d
2023-08-03 18:56:05 +00:00
Liz Kammer
0db0e34c68 Gen a header library when genrules export includes
Embedding multiple includes from a genrule may be difficult to read,
instead we generate a header library that contains the headers and all
include dirs that Soong generates.

Test: go test bp2build tests
Change-Id: I590c74c133f015f27cccf5a2fd916153ad9c125e
2023-07-24 13:57:17 -04:00
Cole Faust
d95158ef1a Merge "Denylist genrules that fail to build with sandboxing" into main 2023-07-21 18:20:06 +00:00
Liz Kammer
d07cb48c6f Merge "Make genrule export_include_dirs more consistent" into main 2023-07-21 13:41:23 +00:00
Cole Faust
edbabc6075 Denylist genrules that fail to build with sandboxing
Bug: 290816499
Test: run genrule_sandobx_test.py with a local change to check all genrules in the tree
Change-Id: I258fe11640c71d532ef48ed88270dec72bd69814
2023-07-20 15:39:04 -07:00
Liz Kammer
d38c87c035 Make genrule export_include_dirs more consistent
Previously, genrule export_include_dirs always added ModuleDir to
exported include dirs when export_include_dirs is set but not when
export_include_dirs is not set. Now when export_include_dirs is set, we
also export the directory without the additional ModuleDir subdir.

Test: genrule go tests
Test: set export_include_dirs and test
Change-Id: I46e860b2c20c1a96bddd14367d7fa737d901994d
2023-07-17 09:58:50 -04:00
Justin Yun
4da4ccc203 Add write_if_changed property to genrule
When "write_if_changed: true" is set, it will call restat for ninja.
With this option the output file will be copied only if it is changed.

Bug: 290130959
Test: ninja rule include "--write-if-changed"
Change-Id: I8bd77b43b22eb0115e0bdc73718b2d6997d92652
2023-07-17 20:35:13 +09:00
Igor Foox
ae2e23a4ba Merge "Remove openwrt modules from genrule denylist." into main 2023-07-13 13:29:13 +00:00
Liz Kammer
796921d211 Ensure data is passed to all generation actions
Test: go test genrule tests
Test: build/soong/tests/genrule_sandbox_test.py \
      framework-cppstream-protos framework-javastream-protos
Change-Id: I16fa939280c7ff0f230a0a8cf18af5e55f16eb03
2023-07-11 08:27:51 -04:00
Igor Foox
24739676c7 Remove openwrt modules from genrule denylist.
With the other change in this topic,
`OpenwrtControlServerProto_h` and
`OpenwrtControlServerProto_cc` both successfully build with sandboxing
on.

Test: build/soong/tests/genrule_sandbox_test.py --show-diff OpenwrtControlServerProto_h OpenwrtControlServerProto_cc
Test: GENRULE_SANDBOXING=true m OpenwrtControlServerProto_h OpenwrtControlServerProto_cc
Test: GENRULE_SANDBOXING=true m droid
Change-Id: Id56824ed935e1d16d2333e5a1aeac248cdfcaeb6
2023-07-10 23:15:43 -04:00
Igor Foox
04ff5f90ae Update genrule denylist to make m droid build.
Test: lunch aosp_cf_x86_phone && GENRULE_SANDBOXING=true m droid
Change-Id: I9092df9a407ffe9a7f1c12f18fb0ce1c5fa027a1
2023-07-10 14:58:51 -04:00
Jason Macnak
4ae9c14557 Merge "Remove FrontendStub_[cc|h] from SandboxingDenyModuleList" into main 2023-07-07 19:16:14 +00:00
Cole Faust
32ce0049d0 Remove cts renderscript modules from sandboxing denylist
Test: build/soong/tests/genrule_sandbox_test.py CtsRsBlasTestCases-rscript CtsRenderscriptTestCases-rscript
Change-Id: I4d1d883c387f7cc4b62784c98b0b7f5d938d0a83
2023-06-23 16:43:57 -07:00
Jason Macnak
5146d553ea Remove FrontendStub_[cc|h] from SandboxingDenyModuleList
Bug: b/288170603
Test: GENRULE_SANDBOXING=true m FrontendStub_cc
Test: GENRULE_SANDBOXING=true m FrontendStub_h
Test: build/soong/tests/genrule_sandbox_test.py FrontendStub_cc
Test: build/soong/tests/genrule_sandbox_test.py FrontendStub_h
Change-Id: I863d65ce921a98b91e1aa4c5eebfcf94178f633d
2023-06-20 23:52:47 +00:00
Christopher Parsons
bf1b88dd35 Merge "Add -gen suffix to genrule targets with conflicts" 2023-06-20 16:32:18 +00:00
Chris Parsons
b7950a93df Add -gen suffix to genrule targets with conflicts
Fixes: 224580460
Test: m nothing
Change-Id: I3af3eb0fb7f65216087482eb620484229ac7d674
2023-06-16 17:41:42 +00:00
Jason Macnak
be29dc3d4f Remove WmediumdServerProto_[cc|h] from SandboxingDenyModuleList
Bug: b/287470077
Test: GENRULE_SANDBOXING=true m WmediumdServerProto_cc
Test: GENRULE_SANDBOXING=true m WmediumdServerProto_h
Test: build/soong/tests/genrule_sandbox_test.py WmediumdServerProto_cc
Test: build/soong/tests/genrule_sandbox_test.py WmediumdServerProto_h
Change-Id: I38079eb7b4bfb60f1b4c3adf6c774f207d7ec55e
2023-06-15 10:58:55 -07:00
David Duarte
cb991cf1ae Merge "Remove pandora-python-gen-src from SandboxingDenyModuleList" 2023-06-14 16:51:56 +00:00
Yu Liu
accbd555a7 Merge "Fix a race problem in sandboxing genrule logic." 2023-06-14 04:38:55 +00:00
David Duarte
ff833a95e8 Remove pandora-python-gen-src from SandboxingDenyModuleList
Test: Manual diff, see linked commit
Change-Id: Icec1edd6a4420377dd79fc736bea77910daad1b9
2023-06-14 00:08:41 +00:00
Yu Liu
e7f7cbfad2 Fix a race problem in sandboxing genrule logic.
Bug: 286978650
Test: locally build
Change-Id: I4fab03f74f5634666d22f7ed6e18ee69f780d6ca
2023-06-13 19:04:24 +00:00
David Duarte
795f7fd531 Remove pandora_experimental-python-gen-src from SandboxingDenyModuleList
Test: build/soong/tests/genrule_sandbox_test.py pandora_experimental-python-gen-src
Change-Id: Ica01cd2f582702825d2f7dcc1cd4b02704e0540a
2023-06-13 17:18:36 +00:00
Liz Kammer
8bd92429e5 bp2build data property -> data attr in gensrcs
Test: go test bp2build tests
Change-Id: I2192cf9846a5783385397337d40cca85a1ef73e7
2023-06-09 15:37:45 -04:00
Liz Kammer
81fec18a5e Add property data to gensrcs for additional inputs
This enables sandboxing for inputs that are necessary but do not need to
have a source file generated.

Test: GENRULE_SANDBOXING=true m framework-javastream-protos \
      framework-cppstream-protos
Change-Id: Id5ca1dab5799c25fa96b564a7d2008c2e7b5382b
2023-06-09 15:29:32 -04:00
Yu Liu
45d6af544a Refactor the genrule allowlists code to support internal allowlists
Bug: 281067655
Test: build locally
Change-Id: Iedac64c3d59fc48f1b5a1b461c53d00e92b653b5
2023-05-31 21:40:47 +00:00
Yu Liu
6a7940cc99 Support sandboxing genrule
Bug: 281067655
Test: local build
Change-Id: I123b5e51eaf1476d1215e5208331b6540d45fd98
2023-05-22 15:02:36 -07:00
Spandan Das
39b6cc5336 Ignore test apexes from bp2build generated tags
Soong does not enforce apex_available on the contents of test apex. To
prevent special-casing test apexes in the apex validation aspect in
Bazel, drop the test apexes from the tags altogether.

( The core problem I am trying to solve is making sure that stub
libraries in Bazel have a single apex available. apex validation happens
to be a nice side benefit)

Bug: 277651159
Test: go test ./bp2build
Change-Id: Ibb3cfedb5c0f2cda0464bf3758c70b67cb5885d1
2023-04-27 23:24:49 +00:00
Sam Delmerico
eddd3c0ba4 add APEX transitive dependency validation
Bug: 218419109
Test: b build //packages/modules/adb/apex:com.android.adbd
Change-Id: I93b0c99d6521e419e52c63271646448f6d708c22
2022-12-13 14:32:19 -05:00
Yu Liu
d6201013eb Make cc_genrule.srcs configurable.
Also disallow arch variant of genrule.out.

This is to be consistent with bazel where we are migrating to.

Bug: 254114674
Test: Manual
Change-Id: I685a2e64102b7bb68128b39931f0bc85878bc6de
2022-10-19 12:35:26 -07:00
Vinh Tran
370e08c1af Add more Soong unit test coverage for gensrcs
When gensrcs's srcs prop is set with filegroup from external package and especially when gensrcs generates cpp headers, it's not trivial to know what the output paths and their correponding include paths look like.

In mixed build, there are a few bugs in gensrcs's include paths for cpp headers (b/248555356 and b/248554581) that we'll eventually need to fix. These tests serve as an documentation of the existing behavior when gensrcs generate cpp headers.

Test: go test
Bug: 248555356
Change-Id: I10168dd4229be8f110a31955d214ef792c8050de
2022-09-25 20:21:17 -04:00