Commit graph

215 commits

Author SHA1 Message Date
Mark Dacek
b2441c52a4 Revert "Revert "Modify symlink_forest to rerun when soong_build has changed.""
This reverts commit aa5cc2cd6a.

Reason for revert: Fixed the bug in the original draft.
There was an error in how we compared the file's MTime to the
file's contents.

Bug: 300288299
Test: presubmits
Test: build/soong/tests/run_integration_tests.sh

Change-Id: I9e53432c0842c0b9fc13fe20d30ce9af37640c7f
2023-10-05 17:11:50 +00:00
Kousik Kumar
a902250962 Merge "Update symlinks in output directory when TOP dir changes" into main 2023-10-05 13:29:20 +00:00
Kousik Kumar
ca390b2f00 Update symlinks in output directory when TOP dir changes
When TOP dir changes and we reuse the same output between the old and new checkouts, we need to rewrite all the symlinks used by Bazel and other tools in the out-directory to point to the new top directory. Otherwise, if the old source dir is deleted, the build will fail.

I used the OUT_DIR/soong/soong.environment.available file to find out the previous PWD.

Tested:
1. Create source dir 1, run build, create source dir 2, remote source dir 1, reuse out dir and rerun build => build succeeded with this change.
2. m libc after moving build TOP. Only the analysis phase was rerun, the actual build was not rerun.

Bug: b/300498226

Change-Id: I196625baa1f4efe7a4734accfa1f0be7c98a7920
2023-10-04 21:07:36 +00:00
Treehugger Robot
4cbd49810c Merge changes Ib2667798,I9a1fa32b into main
* changes:
  Symlink to rules_testing in soong integration test.
  Remove AconfigJavaHostTest from the allowlist.
2023-10-04 00:22:57 +00:00
Yu Liu
d61dcef2ff Symlink to rules_testing in soong integration test.
Bug: None
Test: CI
Change-Id: Ib2667798ba4ae330ff46978b37c9c50a5a0cd477
2023-10-03 14:49:50 -07:00
Mark Dacek
4ac292ffd8 Merge "Revert "Modify symlink_forest to rerun when soong_build has changed."" into main 2023-10-03 18:39:46 +00:00
Mark Dacek
aa5cc2cd6a Revert "Modify symlink_forest to rerun when soong_build has changed."
This reverts commit 23a4120c57.

Reason for revert: broke soong_integration

Change-Id: I4d51841756675b3745244d23e13aefda0916614b
2023-10-02 23:40:33 +00:00
Wei Li
0af40657ff Merge "Change sbom_test.sh to take command line arguments for target product, release and build variant, which is for a corresponding change in CI." into main 2023-10-02 18:14:33 +00:00
Wei Li
3b3173d5db Change sbom_test.sh to take command line arguments for target product, release and build variant, which is for a corresponding change in CI.
Bug: 276778945
Test: build/soong/tests/sbom_test.sh TARGET_PRODUCT=aosp_cf_x86_64_phone TARGET_RELEASE=trunk_staging TARGET_BUILD_VARIANT=userdebug
Change-Id: I2d9e271e4ae351e3626657aa2ac2a1a967c15fc8
2023-09-29 16:52:11 -07:00
MarkDacek
23a4120c57 Modify symlink_forest to rerun when soong_build has changed.
Also remove existing symlink_forest_version functionality.

This remedies issues pertaining to b/300122962 - symlinks not
clearing after a revert

Timing wise - this doesn't present a performance regression
on a clean build. When soong_build changes, it's considerably longer
but no different from the time when symlink_forest_version is changed.

Bug: 300288299
Test: build/soong/tests/symlink_forest_rerun_test.sh
Change-Id: I0e95aac315dfea7ea3b8bb9a3eb0c6408300bd3b
2023-09-29 22:53:07 +00:00
Sebastian Pickl
3d2b08fe0d Merge "Revert "Change symlink_forest to use relative symlinks."" into main 2023-09-12 18:53:47 +00:00
Sebastian Pickl
90355f79bf Revert "Change symlink_forest to use relative symlinks."
Revert submission 2673616-relativesymlinks-fix

Reason for revert: this breaks tests verified with go/abtd https://android-build.googleplex.com/builds/abtd/run/L36000000963001181

Bug: 300122962

Reverted changes: /q/submissionid:2673616-relativesymlinks-fix

Change-Id: I5a97c4fbe4df5727c0604a07137093d0f00c7776
2023-09-12 18:51:39 +00:00
Mark Dacek
4eabebec69 Merge "Change symlink_forest to use relative symlinks." into main 2023-09-12 15:47:42 +00:00
MarkDacek
b98b3a429f Change symlink_forest to use relative symlinks.
Also add script to remove the Bazel output base.
This will assist with supporting movable checkouts alongside
mixed builds.

Bug: 259191764
Test: m && (move topic and prepare_moved_top.sh) && m
Test: m && prepare_moved_top.sh && m
Test: build/soong/tests/relative_symlinks_test.sh

Change-Id: I0f53da8d99f752fad496cf3ac61b01f001b7296d
2023-09-12 04:02:23 +00:00
Wei Li
e7eb8f7445 Remove the diff resolved in aosp/q/topic:"install_symlink"
Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Change-Id: I065826d59f4bd4b4abc20aa5010d362b30fb6b99
2023-09-07 01:27:37 +00:00
Cole Faust
999881376b Merge "Delete ApiBp2build soong_build mode" into main 2023-09-06 18:04:25 +00:00
Chris Parsons
73f411b72c Delete ApiBp2build soong_build mode
This feature is obsolete.

This makes a large number of codepaths "dead code" (such as
module-specific implementations of ApiBp2build functionality). These
will be deleted in a followup CL.

Bug: 284029211
Test: Presubmits
Change-Id: Ib53b99f1fe8c24380d219caf44e9bb3b96724fa0
2023-09-05 17:09:51 -07:00
Wei Li
6c550d45d2 Remove all ignored files in SBOM diffs, which have been fixed in the following CLs:
aosp/2721721
aosp/2715849
aosp/2721815
aosp/2726742

Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Change-Id: I17d5be4dc59c08be43a59c6bc0ea212dac6cd13c
2023-09-01 17:26:22 +00:00
Wei Li
313c8166e1 Ignore the newly added symllink /system/bin/hwservicemanager so it doesn't cause CI failure in sbom_integration.
The symlink will be added to SBOM later.

Bug:
Test: build/soong/tests/sbom_test.sh
Change-Id: I170e554e8b805736c60b8d26533bd0ab41e7575a
2023-08-31 16:02:48 -07:00
Vinh Tran
d007cead3f Add bazelbuild-rules_rust and tinyjson to soong_integration_tests
aosp/2709668 registers Bazel repos using these directories so we need to symlink them for soong integration tests to pass.

Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: I8ee7e5c702bd0bfa24eb3778371f497a67bf815a
2023-08-21 13:35:08 -04:00
Vinh Tran
4f79359d80 Add bazelbuild-rules_cc to soong_integration_tests
This fixes breakage currently on CI.

Bug: 296628826
Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: Ibfcd2bc5d2b107879281075ab994f1e9a8f5da0e
2023-08-21 13:30:40 -04:00
Wei Li
9ff64e6aff Add new diffs found in vendor partition in sbom_test.
Bug: 295606843
Test: build/soong/tests/sbom_test.sh
Change-Id: I50af9df5ea91ca113c272139530b0c468d41f5c8
2023-08-14 13:31:44 -07:00
Wei Li
0634397cd8 Remove diffs of symlinks in root directory which are included in product SBOMs now.
Also explicitely ignore some diffs that were ignored because of "-I /bin".

Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Change-Id: I8a19fdaa89c9f6b4a11c107df98bca647323db26
2023-08-10 18:18:21 -07:00
Wei Li
dbedb0c8a4 Remove diffs of flags files that included in product SBOM.
Bug: 272358980
Test: build/soong/tests/sbom_test.sh
Change-Id: I65acea2e854bc983b6e12ab247afcb702c410988
2023-08-08 17:36:30 -07:00
usta
36249c7b1e Add bazelbuild-rules_python to soong_integration_tests
Test: `atest_bazel_standalone_output_paths_contain_product_name`
Bug: 294616978
Change-Id: I40d00ee2453998122910a3ac87c681aa0b9df19c
2023-08-04 22:17:03 -04:00
Cole Faust
7997b3bb1c Merge "Make the bazel sandwich ignore targets that don't exist" into main 2023-08-03 16:52:41 +00:00
Wei Li
b82c236cbb Merge "Add integration test for verifying package verification code in SBOM." into main 2023-08-02 19:10:52 +00:00
Wei Li
8ea2ffc825 Add integration test for verifying package verification code in SBOM.
The test case calculates package verification code of package product according to the SPDX spec and compare it to the one in SBOM file which should have the same SHA1 hash value. This helps verify the python logic of generating it in SBOM files.

Bug: 293304694
Test: build/soong/tests/sbom_test.sh
Change-Id: I37c96d90a1990fbeb786f1bd4e8dc87102e0f0cd
2023-08-02 19:10:23 +00:00
Cole Faust
16d10942c5 Make the bazel sandwich ignore targets that don't exist
So that things can be allowlisted that aren't in all branches, or
that aren't present in tests.

Then remove the example partition from the soong integration tests
so that we don't need to pull all of the tools needed to build
partitions into that test.

Bug: 265127181
Fixes: 294220392
Test: ./build/soong/tests/run_integration_tests.sh and m bazel_sandwich
Change-Id: I66fb1ad1c2eee815bfb9700595a26e74e54c6214
2023-08-02 11:45:43 -07:00
Wei Li
9d1ff41f3d Remove sbom_test from soong integration test.
Bug: 294129889
Test: CIs
Change-Id: Ic2925b214bd1e9299b7fa39f7ad0540bca282b53
2023-08-01 16:40:15 -07:00
Treehugger Robot
e4d58523b6 Merge "Ignore some new files added to partition images in sbom_test.sh." into main 2023-07-21 14:34:44 +00:00
Romain Jobredeaux
ca4706b474 Symlink to rules_java in soong integration test.
Bug: 292023656
Test: CI + ABTD
Change-Id: I3ac87154c2d77cb18c7ba60f638b5787cef57606
2023-07-20 09:41:29 -04:00
Cole Faust
b51a05844c Use rbc_dashboard.py instead of rbc_regression_test.sh
rbc_dashboard.py is an alternative implementation of
rbc_regression_test.sh. These tests both run soong 2 times with
starlark product config enabled/disabled and diff the resulting ninja
files. However, the shell script test runs the soong invications in
series and the python one runs them in parallel, meaning the python
one is faster. On my computer the shell one takes ~7 minutes and the
python one takes ~4 minutes.

Bug: 291548757
Test: Presubmits
Change-Id: I20fd6ca09f47dd0547f2681fbd78ba70991493b0
2023-07-17 23:20:57 +00:00
Wei Li
08246b92a6 Ignore some new files added to partition images in sbom_test.sh.
Test: build/soong/tests/sbom_test.sh
Change-Id: I883ba741a47120f56f9f4d0594caebe78ce5242d
2023-07-17 11:33:33 -07:00
Treehugger Robot
b35cc91ba0 Merge "Cleanup genrule_sandbox_test.py" into main 2023-07-14 19:50:23 +00:00
Wei Li
9a8df6669b Some new libraries are added to system partition and are not captured in
product SBOM of cuttlefish devices, ignore these diffs for now in the
SBOM integration test to fix the CI breakage.

Bug: 291073898
Test: build/soong/tests/sbom_test.sh
Change-Id: Idf5569f67642784ee422847301c59dddc8ff0907
2023-07-13 10:29:55 -07:00
Cole Faust
bbe2cc6191 Cleanup genrule_sandbox_test.py
In preparation for adding the ability to check all genrules in the tree.

Bug: 290816499
Test: ./build/soong/tests/genrule_sandbox_test.py extensions_db.pb
Change-Id: I6454be7e88a876a2ab1105c04f5ecda03ecd7771
2023-07-12 18:01:32 -07:00
Wei Li
59df0ee442 Add integration test for m build SBOM of APEXs/APKs.
Bug: 266726655
Test: build/soong/tests/sbom_test.sh
Change-Id: Id8002042f607f8e158f561ca47e0a84a8b8927e2
2023-07-10 15:50:38 -07:00
Cole Faust
837c3b9a55 Revert "Commend out dcla_apex_comparison_test.sh"
This reverts commit a56663f695.

Reason for revert: Original breaking change was reverted

Change-Id: I8e0c97f0b511f686f5efdb8a9e8c9f862fc41010
2023-06-30 22:53:10 +00:00
Cole Faust
a56663f695 Commend out dcla_apex_comparison_test.sh
Is currently failing on ci, uncomment when fix is submitted.

Bug: 289141798
Test: Presubmits
Change-Id: Ic7b220f917cb0b9f0914d8e9575081004610cf63
2023-06-29 12:36:00 -07:00
Treehugger Robot
3da9587eda Merge changes from topic "revert-2637031-revert-2626204-rules_go_repo-OSFOZKENFO-KSSEOFEKLG"
* changes:
  Revert^2 "Symlink external/bazelbuild-rules_go in mock tests"
  Revert^2 "Always use the BUILD files from bazelbuild-rules_go"
2023-06-26 08:01:19 +00:00
Spandan Das
f7373e62dc Revert^2 "Symlink external/bazelbuild-rules_go in mock tests"
5479e0f202

Change-Id: Idc4baa3d8afd1841ee3620e2f2e29a30c85a5473
2023-06-26 03:44:04 +00:00
Matthias Männich
5479e0f202 Revert "Symlink external/bazelbuild-rules_go in mock tests"
Revert submission 2626204-rules_go_repo

Reason for revert: breaks the aosp_kernel-build-tools (https://ci.android.com/builds/branches/aosp_kernel-build-tools/grid)
Reverted changes: /q/submissionid:2626204-rules_go_repo

Change-Id: I1c816bdb739e08794da4affb6fca3b3a052ed355
2023-06-23 07:45:26 +00:00
Wei Li
cd9948f66e Cleanup SBOM test script for later adding tests of generating SBOM of
unbundled APKs/APEXs.

Bug: 266726655
Test: build/soong/tests/sbom_test.sh
Change-Id: I2bb3e8405dad716837d3004d4c6fd54ae80527c1
2023-06-22 17:52:31 -07:00
Spandan Das
60133b3070 Symlink external/bazelbuild-rules_go in mock tests
Test: TH
Change-Id: I067bda143252c66bfac63699d51ca3b80f5a5f63
2023-06-21 00:48:05 +00:00
Jason Macnak
394f1b7b86 Fix local var error, warn if no outputs, handle "}generate{"
1. UnboundLocalError: cannot access local variable 'output' where
   it is not associated with a value

2. `build/soong/tests/genrule_sandbox_test.py WmediumdServerProto_cc`
   would try to build the entire world due to 3)

3. WmediumdServerProto_cc 's module action desc had '}generate{'
   without a space which would cause 2)

Bug: b/287470077
Test: build/soong/tests/genrule_sandbox_test.py WmediumdServerProto_cc
Change-Id: Ie9d5db44195a6f700cc876ec889bd01c11e66cff
2023-06-15 10:59:20 -07:00
Mark Dacek
ff0f42a3ac Merge "Add integration test for ensuring that stale metrics files are deleted." 2023-06-14 19:14:31 +00:00
MarkDacek
5040c36649 Add integration test for ensuring that stale metrics files are deleted.
Bug: 286083726
Test: run the script
Change-Id: I2dd382b3a0ee1b226f4931fed3bc4b5df0a3528c
2023-06-14 15:05:36 +00:00
Trevor Radcliffe
ded095ce45 Bp2build Sanitizer Blocklist
Bug: 286894426
Test: Unit tests
Change-Id: I382e028410a185a5017dba1fc47c83ad5b2432cf
2023-06-13 20:46:27 +00:00
Treehugger Robot
01f8784246 Merge "Read BUILD files in bp2build" 2023-06-09 22:42:39 +00:00