Commit graph

227 commits

Author SHA1 Message Date
Spandan Das
15a1d29233 Pin TARGET_PRODUCT to aosp_arm in integration tests
aosp/3019827 does not handle the case where integration_tests might be
run from an unlunched-terminal. In this scenario, the ninja file will be
generated at out/soong/build.ninja.

To support running integration tests with/without lunch as well for
consistency of tests across different setups (users/CI etc.), pin the
TARGET_PRODUCT to a default value (aosp_arm for now)

Test: build/soong/tests/run_integration_tests.sh without running lunch
Change-Id: I1197923867643986ed3736cfc284632aa82dca13
2024-04-01 21:37:13 +00:00
Spandan Das
08e90c2718 Fixes for run_integration_tests.sh
This script runs some useful integration tests, but is currently broken
at ToT. This CL enables us to rerun this script successfully.

Details
1. Remove the scripts related to bp2build/b/mixed_builds. These modes
   have been/are on the verge of deletion from soong
2. Remove the comparisision scripts for soong and bazel built artifcats,
   for the same reason
3. Add a TARGET_RELEASE to androidmk_test.sh
4. Add a TARGET_PRODUCT to o/s/build.ninja, since the generated .ninja files
   now have a product-specific suffix.
5. Remove a test that checks that globs are not generated in the first
   clean run. We always generate the glob file now (aosp/2893286)

Test: build/soong/tests/run_integration_tests.sh
Change-Id: I39f6706ab2a81a7b1b2e90d43195bc3e2c7a5c08
2024-03-29 19:00:20 +00:00
Cole Faust
a57c8c29bb Improve genrule sandboxing script
Do the sandboxed and non-sandboxed builds in two separate directories.
This allows us to keep the directories around so you can compare diffs
afterwards, and allows us to run the builds in parallel. It also means
that analysis isn't rerun twice every time you run the script.

Bug: 307824623
Test: Using it for the past few genrules I've fixed
Change-Id: Ib3be394f233b383c1bba41d31ada6c9af94e755b
2023-11-15 11:44:12 -08:00
Cole Faust
a00cc919dd Merge "Revert "Revert "Add GENRULE_SANDBOXING=false to non-sandboxed bu..."" into main 2023-10-19 23:10:55 +00:00
Cole Faust
33d7169928 Revert "Revert "Add GENRULE_SANDBOXING=false to non-sandboxed bu..."
Revert submission 2794133-revert-2782270-genrule_sandboxing_by_default-JMQJWWORHR

Reason for revert: Reapplying now that ag/25093969 and ag/25107100 are in

Reverted changes: /q/submissionid:2794133-revert-2782270-genrule_sandboxing_by_default-JMQJWWORHR

Change-Id: Ibb7dd6a0f925d043ab8df2f79cbcb9d30c3be458
2023-10-18 22:50:26 +00:00
Mateusz Cicheński
3106df7887 Merge "Revert "Add GENRULE_SANDBOXING=false to non-sandboxed builds"" into main 2023-10-17 21:04:56 +00:00
Mateusz Cicheński
f119ceb1b2 Revert "Add GENRULE_SANDBOXING=false to non-sandboxed builds"
Revert submission 2782270-genrule_sandboxing_by_default

Reason for revert: Potential culprit for b/305851039 and many other build breakages - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted.

Reverted changes: /q/submissionid:2782270-genrule_sandboxing_by_default

Change-Id: I7b47a92e1b315add65c8018500d9ffbd58f8f1dd
2023-10-17 17:35:52 +00:00
Cole Faust
ab3ed73157 Merge "Add GENRULE_SANDBOXING=false to non-sandboxed builds" into main 2023-10-17 00:53:09 +00:00
Treehugger Robot
36674db0b3 Merge "Use make_mock_top function" into main 2023-10-11 22:10:48 +00:00
Cole Faust
b36e79c72c Add GENRULE_SANDBOXING=false to non-sandboxed builds
Because sandboxing is becoming the default.

Bug: 290816499
Test: Presubmits
Change-Id: Icd8cb70f0275bafc580c7123a9d38991ac58adbf
2023-10-11 14:52:08 -07:00
MarkDacek
055a1207ad Add b_args_test.
Bug: 303269101
Test: run the test
Change-Id: I2c5e72d25dab601ef7354fb9ad624ce035e64812
2023-10-11 17:27:29 +00:00
Mark Dacek
b2441c52a4 Revert "Revert "Modify symlink_forest to rerun when soong_build has changed.""
This reverts commit aa5cc2cd6a.

Reason for revert: Fixed the bug in the original draft.
There was an error in how we compared the file's MTime to the
file's contents.

Bug: 300288299
Test: presubmits
Test: build/soong/tests/run_integration_tests.sh

Change-Id: I9e53432c0842c0b9fc13fe20d30ce9af37640c7f
2023-10-05 17:11:50 +00:00
Kousik Kumar
a902250962 Merge "Update symlinks in output directory when TOP dir changes" into main 2023-10-05 13:29:20 +00:00
Kousik Kumar
7983cc2876 Use make_mock_top function
Followup cleanup CL to address the comments from https://android-review.git.corp.google.com/c/platform/build/soong/+/2757787

Bug: b/300498226
Change-Id: If07e1fb21e97bd4b7550d29eaf7736b5894d797f
2023-10-05 13:06:09 +00:00
Kousik Kumar
ca390b2f00 Update symlinks in output directory when TOP dir changes
When TOP dir changes and we reuse the same output between the old and new checkouts, we need to rewrite all the symlinks used by Bazel and other tools in the out-directory to point to the new top directory. Otherwise, if the old source dir is deleted, the build will fail.

I used the OUT_DIR/soong/soong.environment.available file to find out the previous PWD.

Tested:
1. Create source dir 1, run build, create source dir 2, remote source dir 1, reuse out dir and rerun build => build succeeded with this change.
2. m libc after moving build TOP. Only the analysis phase was rerun, the actual build was not rerun.

Bug: b/300498226

Change-Id: I196625baa1f4efe7a4734accfa1f0be7c98a7920
2023-10-04 21:07:36 +00:00
Treehugger Robot
4cbd49810c Merge changes Ib2667798,I9a1fa32b into main
* changes:
  Symlink to rules_testing in soong integration test.
  Remove AconfigJavaHostTest from the allowlist.
2023-10-04 00:22:57 +00:00
Yu Liu
d61dcef2ff Symlink to rules_testing in soong integration test.
Bug: None
Test: CI
Change-Id: Ib2667798ba4ae330ff46978b37c9c50a5a0cd477
2023-10-03 14:49:50 -07:00
Mark Dacek
4ac292ffd8 Merge "Revert "Modify symlink_forest to rerun when soong_build has changed."" into main 2023-10-03 18:39:46 +00:00
Mark Dacek
aa5cc2cd6a Revert "Modify symlink_forest to rerun when soong_build has changed."
This reverts commit 23a4120c57.

Reason for revert: broke soong_integration

Change-Id: I4d51841756675b3745244d23e13aefda0916614b
2023-10-02 23:40:33 +00:00
Wei Li
0af40657ff Merge "Change sbom_test.sh to take command line arguments for target product, release and build variant, which is for a corresponding change in CI." into main 2023-10-02 18:14:33 +00:00
Wei Li
3b3173d5db Change sbom_test.sh to take command line arguments for target product, release and build variant, which is for a corresponding change in CI.
Bug: 276778945
Test: build/soong/tests/sbom_test.sh TARGET_PRODUCT=aosp_cf_x86_64_phone TARGET_RELEASE=trunk_staging TARGET_BUILD_VARIANT=userdebug
Change-Id: I2d9e271e4ae351e3626657aa2ac2a1a967c15fc8
2023-09-29 16:52:11 -07:00
MarkDacek
23a4120c57 Modify symlink_forest to rerun when soong_build has changed.
Also remove existing symlink_forest_version functionality.

This remedies issues pertaining to b/300122962 - symlinks not
clearing after a revert

Timing wise - this doesn't present a performance regression
on a clean build. When soong_build changes, it's considerably longer
but no different from the time when symlink_forest_version is changed.

Bug: 300288299
Test: build/soong/tests/symlink_forest_rerun_test.sh
Change-Id: I0e95aac315dfea7ea3b8bb9a3eb0c6408300bd3b
2023-09-29 22:53:07 +00:00
Sebastian Pickl
3d2b08fe0d Merge "Revert "Change symlink_forest to use relative symlinks."" into main 2023-09-12 18:53:47 +00:00
Sebastian Pickl
90355f79bf Revert "Change symlink_forest to use relative symlinks."
Revert submission 2673616-relativesymlinks-fix

Reason for revert: this breaks tests verified with go/abtd https://android-build.googleplex.com/builds/abtd/run/L36000000963001181

Bug: 300122962

Reverted changes: /q/submissionid:2673616-relativesymlinks-fix

Change-Id: I5a97c4fbe4df5727c0604a07137093d0f00c7776
2023-09-12 18:51:39 +00:00
Mark Dacek
4eabebec69 Merge "Change symlink_forest to use relative symlinks." into main 2023-09-12 15:47:42 +00:00
MarkDacek
b98b3a429f Change symlink_forest to use relative symlinks.
Also add script to remove the Bazel output base.
This will assist with supporting movable checkouts alongside
mixed builds.

Bug: 259191764
Test: m && (move topic and prepare_moved_top.sh) && m
Test: m && prepare_moved_top.sh && m
Test: build/soong/tests/relative_symlinks_test.sh

Change-Id: I0f53da8d99f752fad496cf3ac61b01f001b7296d
2023-09-12 04:02:23 +00:00
Wei Li
e7eb8f7445 Remove the diff resolved in aosp/q/topic:"install_symlink"
Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Change-Id: I065826d59f4bd4b4abc20aa5010d362b30fb6b99
2023-09-07 01:27:37 +00:00
Cole Faust
999881376b Merge "Delete ApiBp2build soong_build mode" into main 2023-09-06 18:04:25 +00:00
Chris Parsons
73f411b72c Delete ApiBp2build soong_build mode
This feature is obsolete.

This makes a large number of codepaths "dead code" (such as
module-specific implementations of ApiBp2build functionality). These
will be deleted in a followup CL.

Bug: 284029211
Test: Presubmits
Change-Id: Ib53b99f1fe8c24380d219caf44e9bb3b96724fa0
2023-09-05 17:09:51 -07:00
Wei Li
6c550d45d2 Remove all ignored files in SBOM diffs, which have been fixed in the following CLs:
aosp/2721721
aosp/2715849
aosp/2721815
aosp/2726742

Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Change-Id: I17d5be4dc59c08be43a59c6bc0ea212dac6cd13c
2023-09-01 17:26:22 +00:00
Wei Li
313c8166e1 Ignore the newly added symllink /system/bin/hwservicemanager so it doesn't cause CI failure in sbom_integration.
The symlink will be added to SBOM later.

Bug:
Test: build/soong/tests/sbom_test.sh
Change-Id: I170e554e8b805736c60b8d26533bd0ab41e7575a
2023-08-31 16:02:48 -07:00
Vinh Tran
d007cead3f Add bazelbuild-rules_rust and tinyjson to soong_integration_tests
aosp/2709668 registers Bazel repos using these directories so we need to symlink them for soong integration tests to pass.

Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: I8ee7e5c702bd0bfa24eb3778371f497a67bf815a
2023-08-21 13:35:08 -04:00
Vinh Tran
4f79359d80 Add bazelbuild-rules_cc to soong_integration_tests
This fixes breakage currently on CI.

Bug: 296628826
Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: Ibfcd2bc5d2b107879281075ab994f1e9a8f5da0e
2023-08-21 13:30:40 -04:00
Wei Li
9ff64e6aff Add new diffs found in vendor partition in sbom_test.
Bug: 295606843
Test: build/soong/tests/sbom_test.sh
Change-Id: I50af9df5ea91ca113c272139530b0c468d41f5c8
2023-08-14 13:31:44 -07:00
Wei Li
0634397cd8 Remove diffs of symlinks in root directory which are included in product SBOMs now.
Also explicitely ignore some diffs that were ignored because of "-I /bin".

Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Change-Id: I8a19fdaa89c9f6b4a11c107df98bca647323db26
2023-08-10 18:18:21 -07:00
Wei Li
dbedb0c8a4 Remove diffs of flags files that included in product SBOM.
Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Change-Id: I65acea2e854bc983b6e12ab247afcb702c410988
2023-08-08 17:36:30 -07:00
usta
36249c7b1e Add bazelbuild-rules_python to soong_integration_tests
Test: `atest_bazel_standalone_output_paths_contain_product_name`
Bug: 294616978
Change-Id: I40d00ee2453998122910a3ac87c681aa0b9df19c
2023-08-04 22:17:03 -04:00
Cole Faust
7997b3bb1c Merge "Make the bazel sandwich ignore targets that don't exist" into main 2023-08-03 16:52:41 +00:00
Wei Li
b82c236cbb Merge "Add integration test for verifying package verification code in SBOM." into main 2023-08-02 19:10:52 +00:00
Wei Li
8ea2ffc825 Add integration test for verifying package verification code in SBOM.
The test case calculates package verification code of package product according to the SPDX spec and compare it to the one in SBOM file which should have the same SHA1 hash value. This helps verify the python logic of generating it in SBOM files.

Bug: 293304694
Test: build/soong/tests/sbom_test.sh
Change-Id: I37c96d90a1990fbeb786f1bd4e8dc87102e0f0cd
2023-08-02 19:10:23 +00:00
Cole Faust
16d10942c5 Make the bazel sandwich ignore targets that don't exist
So that things can be allowlisted that aren't in all branches, or
that aren't present in tests.

Then remove the example partition from the soong integration tests
so that we don't need to pull all of the tools needed to build
partitions into that test.

Bug: 265127181
Fixes: 294220392
Test: ./build/soong/tests/run_integration_tests.sh and m bazel_sandwich
Change-Id: I66fb1ad1c2eee815bfb9700595a26e74e54c6214
2023-08-02 11:45:43 -07:00
Wei Li
9d1ff41f3d Remove sbom_test from soong integration test.
Bug: 294129889
Test: CIs
Change-Id: Ic2925b214bd1e9299b7fa39f7ad0540bca282b53
2023-08-01 16:40:15 -07:00
Treehugger Robot
e4d58523b6 Merge "Ignore some new files added to partition images in sbom_test.sh." into main 2023-07-21 14:34:44 +00:00
Romain Jobredeaux
ca4706b474 Symlink to rules_java in soong integration test.
Bug: 292023656
Test: CI + ABTD
Change-Id: I3ac87154c2d77cb18c7ba60f638b5787cef57606
2023-07-20 09:41:29 -04:00
Cole Faust
b51a05844c Use rbc_dashboard.py instead of rbc_regression_test.sh
rbc_dashboard.py is an alternative implementation of
rbc_regression_test.sh. These tests both run soong 2 times with
starlark product config enabled/disabled and diff the resulting ninja
files. However, the shell script test runs the soong invications in
series and the python one runs them in parallel, meaning the python
one is faster. On my computer the shell one takes ~7 minutes and the
python one takes ~4 minutes.

Bug: 291548757
Test: Presubmits
Change-Id: I20fd6ca09f47dd0547f2681fbd78ba70991493b0
2023-07-17 23:20:57 +00:00
Wei Li
08246b92a6 Ignore some new files added to partition images in sbom_test.sh.
Test: build/soong/tests/sbom_test.sh
Change-Id: I883ba741a47120f56f9f4d0594caebe78ce5242d
2023-07-17 11:33:33 -07:00
Treehugger Robot
b35cc91ba0 Merge "Cleanup genrule_sandbox_test.py" into main 2023-07-14 19:50:23 +00:00
Wei Li
9a8df6669b Some new libraries are added to system partition and are not captured in
product SBOM of cuttlefish devices, ignore these diffs for now in the
SBOM integration test to fix the CI breakage.

Bug: 291073898
Test: build/soong/tests/sbom_test.sh
Change-Id: Idf5569f67642784ee422847301c59dddc8ff0907
2023-07-13 10:29:55 -07:00
Cole Faust
bbe2cc6191 Cleanup genrule_sandbox_test.py
In preparation for adding the ability to check all genrules in the tree.

Bug: 290816499
Test: ./build/soong/tests/genrule_sandbox_test.py extensions_db.pb
Change-Id: I6454be7e88a876a2ab1105c04f5ecda03ecd7771
2023-07-12 18:01:32 -07:00
Wei Li
59df0ee442 Add integration test for m build SBOM of APEXs/APKs.
Bug: 266726655
Test: build/soong/tests/sbom_test.sh
Change-Id: Id8002042f607f8e158f561ca47e0a84a8b8927e2
2023-07-10 15:50:38 -07:00