Commit graph

62 commits

Author SHA1 Message Date
Sam Delmerico
75dbca2eae Revert "Revert "mixed builds correctly reference stubs libs""
This reverts commit 1db4348734.

Changes from original:
  - extracted function to add the current version to stub
    versions for bp2build.
  - added libc++ to mixed builds denylist

Reason for revert: re-uploading with fix in topic

Change-Id: Ifa0ed456bf8cb4a7f861d6826263adfedb4fdd9c
2023-04-25 00:25:20 +00:00
Usta (Tsering) Shrestha
1db4348734 Revert "mixed builds correctly reference stubs libs"
This reverts commit ca438e6b72.

Reason for revert: ci post-submit failures in aosp-master-bazel of targets mixed-droid-clean and mixed-droid-incremental:
FAILED: ninja: 'out/target/product/generic_arm64/obj/SHARED_LIBRARIES/libc_intermediates/libc.so.toc', needed by 'out/target/product/generic_arm64/obj/EXECUTABLES/updater_intermediates/LINKED/updater', missing and no known rule to make it
12:21:27 ninja failed with: exit status 1

Change-Id: I081b499d23f2568cdf6227c4e3b0278164086b69
2023-04-19 14:02:52 +00:00
Sam Delmerico
ca438e6b72 mixed builds correctly reference stubs libs
Bug: 270408757
Test: go test
Test: m NeuralNetworksTest_shared_partial --bazel-mode-staging
  && verify that Ninja commands link libneuralnetworks via stubs
Change-Id: I19bf5a829cea7fd00c0f82511ad2bddbc40fae4f
2023-04-14 12:42:23 -04:00
Trevor Radcliffe
6b9fb9ab25 Merge "Supporting changes for cc_binary LTO transitions" 2023-03-22 20:27:02 +00:00
Trevor Radcliffe
dc210faca4 Supporting changes for cc_binary LTO transitions
Bug: 261733821
Test: Unit tests
Change-Id: I4218a93ce20d28b425f216d2328446c9b2ad9088
2023-03-20 18:25:08 +00:00
Cole Faust
8181f4b916 Replace json_encode with json.encode
Now that the json module is available in the cquery starlark execution
enviornment.

Bug: 245315355
Test: m nothing
Change-Id: Ie7c314e2f4e2ef4257bd84abc1add01ddb668f0b
2023-03-19 14:24:23 -07:00
Jingwen Chen
2d37b641c5 Populate apexBundle#filesInfo using bazel info.
This CL adds a few things:

1) Populate the filesInfo struct with cquery'd information from an
apex's ApexMkInfo provider. This filesInfo is then used in
apex/androidmk.go to generate Make modules (soong_cc_rust_prebuilt.mk),
which are then used in packaging to generate zip files of symbols in $PRODUCT_OUT.
2) Make a list of dicts of primitives JSON-encodable.
3) Tests.

Bug: 271423316
Bug: 271423062
Test: presubmits
Change-Id: Iaa34f51044de310510e580d9cf1fe60bbef801c1
2023-03-16 18:09:27 +00:00
Sam Delmerico
4ed95e263f mixed build targets need transitive tidy files
Previously in mixed builds, only the tidy files for the boundary module
were built, whereas all of its transitive dependencies' tidy files were
not being built. Instead we should export the list of transitive tidy
files for a module so that we can run clang-tidy for the boundary module
as well as its dependencies.

Bug: 195029134
Test: WITH_TIDY=1 DISABLE_ARTIFACT_PATH_REQUIREMENTS=true mss tidy-packages-modules-NeuralNetworks --bazel-mode-dev
Change-Id: I463646d2ae1fc4aa075a54c264e1c34571c3fd5c
2023-02-07 16:50:30 -05:00
Sam Delmerico
78aca80710 Merge "add androidmk cc-related variables to androidmk" 2023-01-27 21:04:29 +00:00
Jingwen Chen
29743c8423 Read ApexMkInfo for modules to be installed.
This piggybacks onto the ApexInfo cquery handler, so we're issuing a
single bazel query call that reads two providers in the starlark expr.

Also rename requiredDeps to makeModulesToInstall to differentiate it from
APEX's required/provided libs in the apex manifest.

Test: unit test
Test: mkdiff
Fixes: 263123189
Change-Id: Ib7e43f1586f29864eee8627dba3631bfaff27afa
2023-01-27 04:02:45 +00:00
Sam Delmerico
4e115cc90d add androidmk cc-related variables to androidmk
The adbd_test androidmk definition is missing some cc-related variables
for cc_test in mixed builds. These variables should be populated from
information from Bazel.

Bug: 265758350
Change-Id: I59d017e2eb2f139188ba3383c457cc0055372b61
2023-01-25 15:14:03 -05:00
Jingwen Chen
94098e880e apex mixed builds: use the .capex when required.
The .capex is the compressed apex. Depending on various conditions, the
.capex should be installed, and thus the
apex.go#ProcessBazelQueryResponse should set the .capex as the main apex
output file.

Fixes: 264873511
Test: new unit test with install rules
Test: m com.android.adbd --bazel-mode-staging gives:

$ ls -lah
out/target/product/generic_arm64/system/apex/com.android.adbd.capex
-rw-r----- 1 jingwen primarygroup 3.6M Jan 10 14:44 out/target/product/generic_arm64/system/apex/com.android.adbd.capex

Change-Id: Ic3143cc6999f111428c6adb76760be8ddc4b6344
2023-01-10 14:50:42 +00:00
Jingwen Chen
3952a9039e mixed builds: add cc_test bazel handler, and add adbd_test to staging allowlist.
This will allow the test_suites_{arm64, x86_64} CI targets on
git_master-bazel to build adbd_test with bazel in `m`.

This is currently implemented similar to the cc_binary handler to
hydrate the output path field.

The tradefed config is still currently handled by Soong in
testBinary#install, which is called from both ProcessBazelQueryResponse
and GenerateAndroidBuildActions. We'll move the tradefed config path to
a mixed builds handler when it's ready on the cc_test bazel rule side.

Tested with `atest` with cuttlefish. Since `atest` doesn't have a way to pass
--bazel-mode-staging to its internal call to Soong, so I manually tested
this by adding adbt_test to the prod allowlist, and confirmed that the
installed adbd_test is the same binary as the one in bazel-out.

Test: $ showcommands adbd_test | grep bazel-out | tail -n1

```
/bin/bash -c "rm -f
out/target/product/vsoc_x86_64/data/nativetest64/adbd_test/adbd_test &&
cp -f -d
out/bazel/output/execroot/__main__/bazel-out/android_x86_64_silvermont-opt-ST-59d4124ca041/bin/packages/modules/adb/adbd_test
out/target/product/vsoc_x86_64/data/nativetest64/adbd_test/adbd_test"
```

Test: Add adbd_test to prod allowlist and run 'lunch aosp_cf_x86_64_phone-userdebug; atest adbd_test', all tests pass.
Bug: 258426425
Bug: 261692372
Change-Id: I6be329f39d607af8daa1dc772b23e8dd0247418c
2022-12-15 08:01:44 +00:00
Sam Delmerico
eddd3c0ba4 add APEX transitive dependency validation
Bug: 218419109
Test: b build //packages/modules/adb/apex:com.android.adbd
Change-Id: I93b0c99d6521e419e52c63271646448f6d708c22
2022-12-13 14:32:19 -05:00
Yu Liu
24214aacc2 Support abi check in mixed build.
Bug: 253498204
Test: Manual tested.
Change-Id: I451a6109a2a7efa24bafc860b7b757e23fde09e1
2022-12-01 10:59:06 -08:00
Cole Faust
97d1527dd2 Improve mixed builds error messages
When a target does not have a certain provider we need,
currently we just fail at the dictionary lookup. Change
it so that we fail with a message that tells us what
target had the issue.

Bug: 260131489
Test: Presubmits
Change-Id: I99fd018835c14eba0d4db54ba2769b2da1b278f6
2022-11-22 14:38:24 -08:00
Wei Li
78c07de1dc Add support of <APEX>-installed-files.txt in mixed build.
Bug: 239084750
Test: CIs
Change-Id: Ie3823d1a620c3924111e71aa97df727565261c98
2022-11-11 22:46:33 +00:00
Liz Kammer
dbb9fe8e92 Merge "Increase testing of request_type" 2022-11-10 13:30:58 +00:00
Treehugger Robot
d5fd30d94a Merge "Add reference to bundle_file in request_type_test" 2022-11-10 01:04:12 +00:00
Treehugger Robot
01dd619766 Merge changes from topic "clang-tidy-mixed-builds"
* changes:
  add tidy files to mixed builds
  add --norun_validations to bazel build action
2022-11-10 01:00:15 +00:00
Liz Kammer
1b7ed9bcbc Increase testing of request_type
Test: go test soong tests
Change-Id: Iacb773a91a4cc097320f06ea1d1669c5923f8076
2022-11-09 11:22:11 -05:00
Sam Delmerico
9a9f652bdd add tidy files to mixed builds
Bug: 195029134
Test: m --bazel-mode-dev
Change-Id: I6fa370334c6ff2b6ae4313a7fcbaa34b16b0fa92
2022-11-09 10:12:00 -05:00
Liz Kammer
08626ec9d7 Add reference to bundle_file in request_type_test
Test: go test soong tests
Change-Id: I60e8930e4565a9b076e1f487300e1e2e1d3e66d0
2022-11-09 09:03:52 -05:00
Liz Kammer
303978dcd2 Connect base.zip to mixed builds
Test: go test soong tests
Test: mixed_droid
Change-Id: Iaf1fbd1e7cde63892b01a9c9f4cd29ae69e2d6e3
2022-11-09 08:16:54 -05:00
Jingwen Chen
1ec7785ec5 bazel apex: Add mixed builds support for <module>_using.xml file.
This file contains the java APIs parsed by the 'dexdeps' tool within the gen_java_usedby_apex.sh.

Bug: 239084755
Fixes: 239084755
Test: presubmits
Change-Id: Ia271783a6be3ea3a343481306cde1aaba2166e88
2022-11-09 06:30:03 +00:00
Wei Li
cc73a05670 Add support of <APEX>_backing.txt in mixed build.
Bug: 239081456
Test: CIs
Change-Id: I8a8a5e606d5b1bc19294fc92cd469e7aa01cdb43
2022-11-08 12:00:42 -08:00
Jingwen Chen
0c9a276ded bazel apex: Add mixed builds support for <module>_using.txt file.
Test: presubmits
Change-Id: I7203fcccb09f6c93e702550ffa47a4029c3351dd
2022-11-07 23:16:12 +00:00
Liz Kammer
be6a712767 Rename ApexCqueryInfo to ApexInfo
cquery is redundant given this is in the cquery package.

Test: go test soong tests
Change-Id: Ib38794b712f548129eec964b8fd85bf8401a5f86
2022-11-04 16:24:37 -04:00
Wei Li
32dcdf9038 Support new attributes added in ApexInfo in mixed build.
Bug: 243748589
Test: CI
Change-Id: I6c9869464fe91ebf62811d92cf4403f2c7c9e8a9
2022-10-28 14:06:57 -07:00
Sasha Smundak
33407b86c1 Refactor parsing JSON-encoded cquery results.
Test: treehugger
Change-Id: I8f1157ad4c3f0e563c9722bb6a1f0fc03e795700
2022-10-14 15:02:56 -07:00
Sasha Smundak
edd16668c6 Add CcUnstrippedInfo provider and use it in mixed builds
The build uses unstripped binary/shared library to extract function
signatures, so for each each target of this kind Bazel should return
its unstripped version, too.

Fixes: 220164721
Test: treehugger

Change-Id: Id5f6143340519bf2ae98791a9e981d1306bb08d1
2022-10-12 17:36:47 -07:00
Chris Parsons
86dc2c268a Support main repository prefixing in cquery
This makes mixed builds resilient to a recent backward-incompatbile
Bazel feature which explicitly adds the main repository (`@`) prefix to
all labels which are stringified by `str(target.label)` in Starlark.

This implementation is compatible with both current Bazel and
Bazel@HEAD.

After a new Bazel release to AOSP, we can clean up this code a little to
only support Bazel with this feature active (and that we need not hedge
on either Bazel behavior).

Bug: 248106697
Test: `m --bazel-mode nothing` with both current Bazel and a new Bazel
Change-Id: Id53c8505bb9080d5073c844de7f1ee57ceceae46
2022-09-28 19:02:51 +00:00
Jason Wu
07bb005b99 Rollforward w/fix of "Use json encoding for cquery ParseResult"
Previous commit: commit fcc53f992b.

Test: 1. use `TARGET_BUILD_VARIANT=userdebug TARGET_PRODUCT=aosp_arm64 build/bazel/ci/mixed_droid.sh` to verify no error occurred.
2. revise the test cases under request_type_test. 3. pass the test cases

Bug: 242587802
Change-Id: I7f70e62f7b6f42fa5f211772888040a0d07a710b
2022-09-21 18:54:14 +00:00
Christopher Parsons
3e1833f9e5 Revert "Use json encoding for cquery ParseResult"
This reverts commit fcc53f992b.

Reason for revert: Broke mixed_droid

Change-Id: I0f7b864a93624a6be94f8faf55cd9e5175de7f14
2022-09-20 21:59:29 +00:00
Jason
fcc53f992b Use json encoding for cquery ParseResult
also using a generic json_encode as an approximation for `json.encode`

Test: 1. use m --bazel-mode-dev nothing to verify no error occured. 2.
revise the test cases under request_type_test. 3. pass the test cases

Bug:242587802

Change-Id: I1288ecca1afd3e32f6473bcabae4ee2cb5838007
2022-09-19 21:54:23 +00:00
Usta Shrestha
79fccefc59 Use list comprehensions in Starlark
also using a generic json_encode as an approximation for `json.encode`

Test: USE_BAZEL_ANALYSIS=1 m libc
Bug: b/242587802
Change-Id: Ib83aeda3f99cc3966548b0cd47e72669422ed72d
2022-09-08 17:12:42 +00:00
Chris Parsons
9ae019cc76 Retrieve shared lib output from custom provider
Use of the native CcSharedLibraryInfo results in using a superfluous
symlink for shared libraries in mixed builds. The new provider
propagates an unwrapped output file (real file, not a symlink).

Test: With related rules changes, manually verified that installed
mixed-build libc was not a symlink. (cf_x86_64, inspected system.img
with `m droid`)

Change-Id: I2a59b348fbfc2a381c229d66c0098e87674a9ddc
2022-08-18 12:25:47 -04:00
Sasha Smundak
fe9a5b833d Implement mixed builds for apex modules, take 2.
* Add ApexCqueryInfo to obtain apex artifacts used by the makefile
  generator and downstream modules
* Refactor code common to GenerateAndroidBuildActions and ProcessBazelQueryResponse
* Implement android.MixedBuildBuildable for modules
* Enable mixed build for apex modules with payload_type:"image"

The first take 6a2b7c40b was setting compressed APEX suffix incorrectly, and
was reverted in 8a3c91494.

Fixes: 239925080 239695521 232085015
Test: treehugger
Change-Id: I1720f8db3c7cc773183d25a815d9b7eeaf7c73ad
2022-07-28 14:43:50 -07:00
Martin Stjernholm
8a3c914942 Revert "Implement mixed builds for apex modules."
Revert submission 2147387-mixed_builds

Reason for revert: Breaks zipapex build on git_master-art-host
Bug: 239925080
Reverted Changes:
I9f1e03e5e:Implement mixed builds for apex modules.
I13d9a0270:Add container and bundle encription key pairs to t...

Change-Id: I0eaad4dd81cac16d634f4c9cec18137a38e30d47
2022-07-26 09:35:39 +00:00
Sasha Smundak
6a2b7c40b3 Implement mixed builds for apex modules.
* Add ApexCqueryInfo to obtain apex artifacts used by the makefile
  generator and downstream modules
* Refactor code common to GenerateAndroidBuildActions and ProcessBazelQueryResponse
* Implement android.MixedBuildBuildable for modules
* Enable mixed build for apex modules with payload_type:"image"

Bug: 232085015
Test: treehugger
Change-Id: I9f1e03e5e7a5b5dde35a5db10c253069543ac973
2022-07-18 13:34:36 -07:00
Chris Parsons
cf553a6575 Use builtin CcSharedLibraryInfo for cquery
Test: mixed_droid.sh CI script
Change-Id: I1571ede5f3320344f69c136843eb662044ba9c5f
2022-05-31 16:47:12 -04:00
Liz Kammer
8c46bf96f5 Update cqueries for bazel rules dir rename
Test: USE_BAZEL_ANALYSIS=1 m updater
Change-Id: I1356828a00037c8ec42108438a2975d2ab44d1d7
2022-02-10 10:26:21 -05:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
c3b97c3568 libclang_rt_prebuilt_library_shared mixed builds
Also fix nil deref in setting $LOCAL_SOONG_UNSTRIPPED_BINARY

Bug: 201802518
Test: request_type_test.go
Test: prebuilt_test.go:TestPrebuiltLibrarySharedWithBazel
Test: mixed_{libc,droid}.sh
Change-Id: I22afb56c4b42d3412c2b2e1f079f1bcf8f3129a7
2022-01-20 12:03:18 -05:00
Liz Kammer
eb2d6d1ffd Propagate headers from bazel to mixed builds
Bug: 208503274
Test: mixed_libc.sh
Change-Id: I0be57f2a22f48be3a919208db4034d2bd03c18c0
2021-12-07 10:25:11 -05:00
Chris Parsons
9eae3cb80e cquery: obtain objects from custom provider
This prevents use of CcInfo to obtain object information, as doing so
would mean propagating linker inputs transitively up the graph (not a
feature that Soong supports)

Test: mixed_libc.sh
Change-Id: I5e5cbfb607b866bb57491dbc0693f79b71707492
2021-09-23 11:01:52 -04:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
62585e8436 Expose TocFile via CcInfo
To reflect what's provided in SharedLibraryInfoProvider

Move the logic for identifying the toc file into Starlark
instead of the Soong Go code.

Adapt TestGetCcInfoParseResults to expect the toc as well.

Test: build/bazel/cquery/request_type_test.go:TestGetCcInfoParseResults
Test: cc/library_test.go:TestCcLibrarySharedWithBazel
Test: build/bazel/ci/mixed_{libc,droid}.sh
Change-Id: I40ad47158cb98b14ca03c21e351988818cb01770
2021-09-16 20:55:10 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
4fbc62196d Clean up StarlarkFunctionBody for getCcInfoType
Instead of repeating, pull repetition into vars

Test: build/bazel/ci/mixed_{libc,droid}.sh
Change-Id: If3c8387ac0aeb6364e7e4bc58a4e3a39c3c708d5
2021-09-16 20:54:57 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
a05a255eac Add infrastructure to support PythonBinary
Add a new request type with its own StarLark function
Hook it up via GetPythonBinary
Add to MockBazelContext a LabelToPythonBinary
Add a test for the new request type

Test: request_type_test.go:TestGetPythonBinaryParseResults
Change-Id: I05f6506adfbbdac8b3f40475509ed02ab8e844e5
2021-08-12 18:48:15 +00:00
Chris Parsons
94a0bba5a9 Support cc_library_shared for mixed builds
Authors: cparsons, eakammer, jingwen

This CL also contains .toc file integration between Bazel and Make.

Fixes: b/190524879

Test: build/bazel/ci/mixed_droid.sh

Co-authored-by: Christopher Parsons <cparsons@google.com>
Co-authored-by: Liz Kammer <eakammer@google.com>
Co-authored-by: Jingwen Chen <jingwen@google.com>

Change-Id: If484042a58cb9f0db6d30a460f415f5684b4cbf6
2021-06-15 12:40:28 +00:00
Chris Parsons
bcc9b88d49 Support mixedbuilds for static variant cc_library
Test: mixed_droid CI
Change-Id: I3126b528b375462d78e8146b5432d1e5cec20829
2021-06-03 16:52:46 -04:00