Commit graph

128 commits

Author SHA1 Message Date
Treehugger Robot
94d107020a Merge "Fix CI failures, comment out dcla cases temporarily." 2023-03-20 21:35:10 +00:00
Wei Li
4a77c55b81 Fix CI failures, comment out dcla cases temporarily.
Bug: 274473434
Test: CIs
Change-Id: Ia1b8860ac46dad6b9670880613b6559561e37dbf
2023-03-20 18:07:40 +00:00
Wei Li
898918f7c8 Build partition images and SBOM seperately in case the build of SBOM introduces extra dependencies that included in partition images.
Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Test: CIs
Change-Id: I99bd2357344f9a3d5827cf052b69e3e8545241c0
2023-03-16 22:52:35 +00:00
Wei Li
44ac6dad27 Merge "Revert "Revert "Revert "Revert "Add integration test for testing generat...""" 2023-03-16 18:20:48 +00:00
Wei Li
73032c8fa3 Revert "Revert "Revert "Revert "Add integration test for testing generat...""
This reverts commit 1e75eccb18.

Reason for revert: The SBOM generation change has been submitted in aosp/2485540, so its integration test in this change should be ok to submit now.

Change-Id: Idb8c84a871b2a5633bed998eb7483fc91d30e97a
2023-03-15 18:31:38 +00:00
Kevin Han
a2e8ae1771 Merge "Revert "Revert "Revert "Add integration test for testing generat..."" 2023-03-13 21:03:23 +00:00
Kevin Han
1e75eccb18 Revert "Revert "Revert "Add integration test for testing generat..."
Revert submission 2481062-sbom

Reason for revert: b/273332655

Reverted changes: /q/submissionid:2481062-sbom

Change-Id: I9297f585ab426d0b5586a7a77e983e8bafb19f20
2023-03-13 20:57:32 +00:00
Wei Li
601fc0137e Merge "Revert "Revert "Add integration test for testing generated SBOM with .im...""" 2023-03-13 17:43:59 +00:00
Wei Li
cba64e4cf1 Revert "Revert "Add integration test for testing generated SBOM with .im...""
This reverts commit 68101fad40.

Reason for revert: fixed the build error and try to commit again.

Change-Id: Iec8afa95a59dc1c7ce9b48911a85667f2956fbce
2023-03-10 16:21:31 +00:00
Android Culprit Assistant
68101fad40 Revert "Add integration test for testing generated SBOM with .im..."
This revert was created by Android Culprit Assistant. The culprit was identified in the following culprit search session (http://go/aca-get/5b65d203-d364-4ade-aabb-1330fe45236a).

Change-Id: I10231bc624a15a2ba477712b3a5950f5fc9113e8
2023-03-10 06:18:01 +00:00
Spandan Das
68bcbb588a Add an integration test for API export from another bazel package
This test ensures that API export works ok if the api file exists in a
different directory (precisely, package) than the *_api_contribution
target.

Test: tests/run_integration_tests.sh
Change-Id: I1ff171b93773b514a9a081f962606f4c28abe42e
2023-03-10 02:32:18 +00:00
Wei Li
eecee5a695 Add integration test for testing generated SBOM with .img files.
Bug: 266726655
Test: CIs
Change-Id: Iacaaae50125c7f310828c79b7881ac1d83a1f9ed
2023-03-08 13:47:51 -08:00
Yu Liu
525bae7ac4 Merge "Run the dcla comparison test for both pure soong build and mixed build." 2023-03-08 06:20:58 +00:00
Yu Liu
fa3a152de7 Run the dcla comparison test for both pure soong build and mixed build.
Bug: 268337606
Test: Manually run the script.
Change-Id: I7afaf46de2a4575b36be0cf348de7a1b7f90719f
2023-03-02 15:43:36 -08:00
Christopher Parsons
7720f5704c Merge "Add a persistent bazel server between builds" 2023-03-02 14:22:34 +00:00
Christopher Parsons
647b8518ec Merge "Avoid deleting bp2build directory" 2023-03-02 14:22:13 +00:00
Chris Parsons
9402ca881c Add a persistent bazel server between builds
This feature is toggled on with USE_PERSISTENT_BAZEL, which is off by
default. Those that opt-in will have a bazel server running between
builds (with a 3hr default TTL) which will greatly improve analysis on
subsequent builds. (As Bazel maintains a cache of analysis results).

Bug: 266983462
Test: Manual `m nothing` runs (timing with and without the feature)
Test: New integration test
Test: Presubmits
Change-Id: I3af4948baa0c490e9b87c48ffdbe9f67732586c7
2023-03-01 23:30:19 -05:00
Chris Parsons
520e88b740 Avoid deleting bp2build directory
With this CL, we avoid deleting the bp2build directory and regenerating
all BUILD files. Instead, we regenerate BUILD files which have changes
on the filesystem, and delete old BUILD files which should no longer
exist.

This improves incremental Bazel server performance by about ~5 seconds.
Previously, Bazel would have cache misses for regenerated packages, and
would thus need to analyze them even if no meaningful changes took
place.

For ease of implementation, we avoid removing stale (empty) directories
from the bp2build workspace, but this should have no effect on the
build. (These will be removed on next clean)

Test: New integration tests
Test: Manual benchmarking in conjunction with persistent bazel server
Change-Id: I3e489ff403be34040122876012329060a2506366
2023-03-01 23:15:30 -05:00
Yu Liu
e0c4d35cc5 Add a test to compare the DCLA libs across the apexes.
Bug: 268337606
Test: manual run it.
Change-Id: I48525ed4724736594080af757800f0e224852abd
2023-02-25 14:02:36 -08:00
Usta (Tsering) Shrestha
9393264eff fix flakey test
since bp2build runs concurrently, errors can occur in any order

Bug: 265845583
Change-Id: Iea6d26bc73112dc89c963545b3e0fc7e1b7fa349
Test: run test localy multiple times
2023-02-13 20:25:12 +00:00
Usta (Tsering) Shrestha
c4c07b12b6 reduce forest generation to be incremental
Previously, symlink forest generation involved removing the entire
symlink forest and recreating it from scratch. With this change,
a) symlinks which need not change are untouched,
b) symlinks pointing to the wrong location are fixed, and
c) symlinks which should no longer exist are removed.

On AOSP on my local machine, this reduces the symlink forest generation
step from 2.5s to 1.1s clean, and 0.6s when a single file is added to
a source directory.

Bug: 257528847
Test: m bp2build, touch `fakefile` under the forest, remove a file
from the source tree, rerun m bp2build. Manually verify the new forest
does not retain the link to the deleted source file, and that fakefile
no longer exists in the forest.

Change-Id: I481371ae487e9419af6a3a4370c552578b07d650
2023-02-07 06:23:43 +00:00
Cole Faust
56080ddc08 Merge "Precompile python sources" 2023-01-30 20:02:27 +00:00
Cole Faust
5c503d1c43 Precompile python sources
This signifigantly improves the startup time of soong-built
python binaries. For example, running
`m apexer && time out/host/linux-x86/bin/apexer` gives
0.734s before this cl, and 0.094s after.

Fixes: 259718110
Test: Presubmits
Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
2023-01-27 15:43:38 -08:00
Wei Li
80f1ef15cc Change grep pattern so it can report number of CppCompile actions correctly without being impacted by setting -march on command line twice.
Bug: 266930443
Test: build/soong/tests/apex_cc_module_arch_variant_tests.sh
Change-Id: Ifd74e8cfc86dd94fce632beac049cc81430bfc46
2023-01-27 22:52:20 +00:00
Usta Shrestha
071f6c29a3 for consistency symlink_tree failure = os.Exit(1)
Brings down time taken to fail from 18s to 13s (on aosp branch)

Test: temporarily coded random failure
Bug: N/A
Change-Id: Ib694eec977293f4dd7054e779d1b82b8cace93f6
2023-01-13 14:07:10 -05:00
Treehugger Robot
0ddd60fc1d Merge "Reference deapexer directly in bazel-bin" 2023-01-12 01:01:24 +00:00
Spandan Das
df01030ab5 Merge changes I78639a2d,Iddb0aa1a
* changes:
  Create a smoke test for api_bp2build workspace
  Add a new CreateSoongInjectionDirFiles function
2023-01-11 22:37:27 +00:00
Chris Parsons
41eff2e4b6 Reference deapexer directly in bazel-bin
This keeps this test compatible with a Bazel release change which
affects which files are present in `bazel cquery --output=files` for
py_binary targets. Since the executable will be present in a predictable
location under bazel-bin, we can hardcode that, instead.

We could have alternatively used --output_groups=python_zip_file for
'hermetic' python, but the bp2build-built deapexer target is missing
some dependencies on protobuf.

Bug: 265176532
Test: Manual runs of apex_comparison_tests.sh with the new Bazel release
Change-Id: Ib7021e7e81d1a7bf5ff4d222e2a7947a06d16791
2023-01-11 17:30:55 -05:00
Spandan Das
255648c6d5 Create a smoke test for api_bp2build workspace
This test builds an "empty" filegroup. Running it can be useful to
verify that the api_bp2build workspace has been setup correctly.

Test: TH
Change-Id: I78639a2dcdeb659b2d6fdf6abc85d85c4fb697bf
2023-01-11 03:21:18 +00:00
Wei Li
ad2ac54d65 Add integration test for testing the arch variant passed to cc modules included in APEXES.
Resend aosp/2350557, with minor change on line 65 of apex_cc_module_arch_variant_tests.sh.
The pattern was:
  '.*android_target.*: ([0-9]+)$'
and is changed to:
  '.*opt-ST.*: ([0-9]+)$'
since the Bazel output directory name has been changed.

Bug: 258422889
Test: build/soong/tests/run_integration_tests.sh
Change-Id: Ib114bc0084d8ea2cd232d6478cc81a039b138bc4
2023-01-10 21:38:46 +00:00
Wei Li
2822aaa198 Merge "Revert "Add integration test for testing the arch variant passed to cc modules included in APEXES."" 2023-01-09 17:39:37 +00:00
Usta (Tsering) Shrestha
9955d076f6 Merge "cosmetic: auto-scan test_ functions" 2023-01-09 15:36:40 +00:00
Jingwen Chen
60ba3892df Revert "Add integration test for testing the arch variant passed to cc modules included in APEXES."
This reverts commit 6c588296a2.

Reason for revert: b/264827343

Change-Id: Iba5a4924349a90349a1e56181efb2d368444dfac
2023-01-09 05:32:27 +00:00
Wei Li
fcfc9d3130 Merge "Add integration test for testing the arch variant passed to cc modules included in APEXES." 2023-01-07 06:35:09 +00:00
Wei Li
6c588296a2 Add integration test for testing the arch variant passed to cc modules included in APEXES.
Bug: 258422889
Test: build/soong/tests/run_integration_tests.sh
Change-Id: I0852454e1d18ccd706f5270c0c87f9546cede5bf
2023-01-06 22:40:52 +00:00
Usta Shrestha
a4e0a0f7f6 cosmetic: auto-scan test_ functions
bug: NA
test: manually ran build/soong/run_integration_tests.sh and verified all tests ran
Change-Id: I1a0a2e227888efc2e7fd7f718b8e534e6dbdbcf1
2023-01-06 14:26:56 +00:00
Cole Faust
b85d1a15cc Bp2build support for multiple product configs
Create a
build/bazel/product_config/generated/products/<product_name>/BUILD
file that contains the platform definitions needed for
a particular product. Currently we just create it for the
current lunch target, but the idea is that eventually when
all product config is in starlark, all the products will
have their platform definitions in the tree at once.

Bug: 249685973
Test: Presubmits
Change-Id: I08c82ff28dcf62f09d3b1d2e3186a6b961e12f6e
2023-01-05 12:49:14 -08:00
Joe Onorato
fa5fc26ce2 Move common functions to script_utils.sh
Test: m
Bug: 260003429
Change-Id: Iac401eeaf97c904ea751583bf80091d50526f695
2023-01-03 14:50:36 -08:00
Treehugger Robot
a6d4e15c3a Merge "Allowlist build files in external/python/absl-py" 2022-12-15 20:23:35 +00:00
Sam Delmerico
eddd3c0ba4 add APEX transitive dependency validation
Bug: 218419109
Test: b build //packages/modules/adb/apex:com.android.adbd
Change-Id: I93b0c99d6521e419e52c63271646448f6d708c22
2022-12-13 14:32:19 -05:00
Usta (Tsering) Shrestha
b545f49f06 Merge "scan and run test_ methods" 2022-12-13 15:46:08 +00:00
Cole Faust
de12be3fb1 Allowlist build files in external/python/absl-py
Needed because we import absl-py in our bazel.workspace file.
It only worked before in b, not mixed builds, because b was
run from the real android tree instead of out/soong/workspace.

Bug: 234449134
Bug: 249685973
Test: Presubmits
Change-Id: Ie6985582c55a4a95338fb3a8ff7b13ddb522c391
2022-12-12 12:27:31 -08:00
Vinh Tran
cc1a017240 Change out path to android_target-opt in bazel integration tests
ag/c/platform/build/bazel/+/20584767/15/common.bazelrc#116 defaulted our build to opt compilation mode which affects bazel's output path. This change fixes the tests to reflect that.

Test: ./tests/apex_comparison_tests.sh
Test: ./tests/bp2build_bazel_test.sh
Bug: 253540178
Change-Id: Idc42a2c3173800db0d6155ef3d74e33c15d220d0
Merged-In: Idc42a2c3173800db0d6155ef3d74e33c15d220d0
2022-12-12 12:20:05 -05:00
Usta Shrestha
572ecec696 scan and run test_ methods
Bug: N/A
Test: run bootstrap_test.sh
Change-Id: I3f810277763144d4a6a2f9c1d08bed3a29b7598e
2022-12-09 01:41:49 -05:00
Usta Shrestha
4e17a2e503 cosmetic: getting rid of SC2155 warnings
https://github.com/koalaman/shellcheck/wiki/SC2155

Bug: N/A
Test: bootstrap_test.sh
Change-Id: I3ce7ab2c7b2cf827e32c3cde0dfbcc932fc48039
2022-12-08 01:21:08 -05:00
Wei Li
94bf3158fd Add soong integration test for Bazel-built apexes of 64bit only products.
Bug: 253223400
Test: build/soong/tests/apex_comparison_tests.sh module_arm64only
Change-Id: I3127aa2b8699f0af16c05036cab94a3054ccc5a1
2022-12-05 20:53:25 -08:00
Cole Faust
762c2def20 Revert "Allowlist build files in external/python/absl-py"
Revert submission 2308141-colefaust_run_b_from_out_folder

Reason for revert: Breaks CI due to more instances of the bazel shutdown issue
Reverted Changes:
I51c28e232:Allowlist build files in external/python/absl-py
I9e6530d95:Cd into the bazel workspace instead of using --pac...

Change-Id: Ice47ef2baaef9976546f9562dc7dd9b19b383d59
2022-11-30 05:16:30 +00:00
Cole Faust
93dc19d39d Allowlist build files in external/python/absl-py
Needed because we import absl-py in our bazel.workspace file.
It only worked before in b, not mixed builds, because b was
run from the real android tree instead of out/soong/workspace.

Bug: 234449134
Bug: 249685973
Test: Presubmits
Change-Id: I51c28e23260bed7ca7d4968fd823f52f0d26a0ab
2022-11-28 12:00:00 -08:00
Colin Cross
662d61430f Make PackageVarContext implement PathGlobContext
Make PackageVarContext implement PathGlobContext by implementing
GlobWithDeps.  This will allow calls to ExistentPathForSource
inside a VariableFunc to use optimized glob dependencies instead of
falling back to AddNinjaFileDeps, which is resulting in extra
dependencies from soong_build on top level directories, triggering
extra Soong regenerations.

Remove the fallback path in ExistentPathForSource by making it take
a PathGlobContext, which is now a superset of PathContext.

Rewrite TestNinjaDeps to not rely on the unoptimized glob dependencies
in VariableFuncs and instead call ctx.Config().AddNinjaFileDeps
directly.

Bug: 257079828
Test: test_create_global_include_directory
Change-Id: I48cf189157d78b9252d339dbc9baeb27e4694807
2022-11-04 09:04:02 -07:00
Chris Parsons
b6e9690776 Add groundwork to easily all-enable bazel-mode
This will make it easy to change a "false" boolean to "true" to push
`--bazel-mode` by default for all users.

Users may disable bazel-by-default with BUILD_BROKEN_DISABLE_BAZEL, and
bazel is disabled by default on Darwin hosts (due to lack of test
coverage).

Bug: 254628592
Bug: 254084584
Test: Presubmits
Test: Presubmits for aosp/2263623
Change-Id: Iec712119e06ca8ec93028207d88277f879184cc2
2022-11-01 09:53:46 -04:00