Commit graph

49 commits

Author SHA1 Message Date
MarkDacek
b98b3a429f Change symlink_forest to use relative symlinks.
Also add script to remove the Bazel output base.
This will assist with supporting movable checkouts alongside
mixed builds.

Bug: 259191764
Test: m && (move topic and prepare_moved_top.sh) && m
Test: m && prepare_moved_top.sh && m
Test: build/soong/tests/relative_symlinks_test.sh

Change-Id: I0f53da8d99f752fad496cf3ac61b01f001b7296d
2023-09-12 04:02:23 +00:00
Vinh Tran
d007cead3f Add bazelbuild-rules_rust and tinyjson to soong_integration_tests
aosp/2709668 registers Bazel repos using these directories so we need to symlink them for soong integration tests to pass.

Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: I8ee7e5c702bd0bfa24eb3778371f497a67bf815a
2023-08-21 13:35:08 -04:00
Vinh Tran
4f79359d80 Add bazelbuild-rules_cc to soong_integration_tests
This fixes breakage currently on CI.

Bug: 296628826
Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: Ibfcd2bc5d2b107879281075ab994f1e9a8f5da0e
2023-08-21 13:30:40 -04:00
usta
36249c7b1e Add bazelbuild-rules_python to soong_integration_tests
Test: `atest_bazel_standalone_output_paths_contain_product_name`
Bug: 294616978
Change-Id: I40d00ee2453998122910a3ac87c681aa0b9df19c
2023-08-04 22:17:03 -04:00
Cole Faust
16d10942c5 Make the bazel sandwich ignore targets that don't exist
So that things can be allowlisted that aren't in all branches, or
that aren't present in tests.

Then remove the example partition from the soong integration tests
so that we don't need to pull all of the tools needed to build
partitions into that test.

Bug: 265127181
Fixes: 294220392
Test: ./build/soong/tests/run_integration_tests.sh and m bazel_sandwich
Change-Id: I66fb1ad1c2eee815bfb9700595a26e74e54c6214
2023-08-02 11:45:43 -07:00
Romain Jobredeaux
ca4706b474 Symlink to rules_java in soong integration test.
Bug: 292023656
Test: CI + ABTD
Change-Id: I3ac87154c2d77cb18c7ba60f638b5787cef57606
2023-07-20 09:41:29 -04:00
Spandan Das
f7373e62dc Revert^2 "Symlink external/bazelbuild-rules_go in mock tests"
5479e0f202

Change-Id: Idc4baa3d8afd1841ee3620e2f2e29a30c85a5473
2023-06-26 03:44:04 +00:00
Matthias Männich
5479e0f202 Revert "Symlink external/bazelbuild-rules_go in mock tests"
Revert submission 2626204-rules_go_repo

Reason for revert: breaks the aosp_kernel-build-tools (https://ci.android.com/builds/branches/aosp_kernel-build-tools/grid)
Reverted changes: /q/submissionid:2626204-rules_go_repo

Change-Id: I1c816bdb739e08794da4affb6fca3b3a052ed355
2023-06-23 07:45:26 +00:00
Spandan Das
60133b3070 Symlink external/bazelbuild-rules_go in mock tests
Test: TH
Change-Id: I067bda143252c66bfac63699d51ca3b80f5a5f63
2023-06-21 00:48:05 +00:00
Trevor Radcliffe
ded095ce45 Bp2build Sanitizer Blocklist
Bug: 286894426
Test: Unit tests
Change-Id: I382e028410a185a5017dba1fc47c83ad5b2432cf
2023-06-13 20:46:27 +00:00
Jihoon Kang
f530696762 Add missing dependency in create_mock_soong
This change adds missing dependency of Soong modules as symlink
directory in create_mock_soong so that a correct ninja graph can be
created when invoking androidmk_test.sh.

Test: build/soong/test/run_integration_tests.sh
Bug: 283905797
Change-Id: I7dff0094fdc23b7c438c917dbc00a5143a4dc5ae
2023-05-23 06:57:38 +00:00
Ibrahim Kanouche
7c1de51d9e Add spdx-tools and licenseclassifier to soong mocks
Change-Id: I8d394fe1f0633f433b76f66929294e6537e1f809
2023-03-31 17:19:29 +00:00
Trevor Radcliffe
391a25d7fa Supporting changes for CFI toolchain features
Mostly exporting variables to Bazel, but also allowlisting a BUILD
file.

Bug: 251217226
Test: Unit tests
Change-Id: Id87015a3cd5d970700c4058ec989bb0c14c36bcb
2023-03-27 18:07:40 +00:00
usta
94e89a4091 cosmetic: log test completion
Test: run_intergration_tests.sh
Bug: NA
Change-Id: Iaacc708d05f0bfdc5fe6a0c60834efcb9b4eb98b
2023-03-21 17:01:52 +00:00
Cole Faust
5c503d1c43 Precompile python sources
This signifigantly improves the startup time of soong-built
python binaries. For example, running
`m apexer && time out/host/linux-x86/bin/apexer` gives
0.734s before this cl, and 0.094s after.

Fixes: 259718110
Test: Presubmits
Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
2023-01-27 15:43:38 -08:00
Cole Faust
b85d1a15cc Bp2build support for multiple product configs
Create a
build/bazel/product_config/generated/products/<product_name>/BUILD
file that contains the platform definitions needed for
a particular product. Currently we just create it for the
current lunch target, but the idea is that eventually when
all product config is in starlark, all the products will
have their platform definitions in the tree at once.

Bug: 249685973
Test: Presubmits
Change-Id: I08c82ff28dcf62f09d3b1d2e3186a6b961e12f6e
2023-01-05 12:49:14 -08:00
Joe Onorato
fa5fc26ce2 Move common functions to script_utils.sh
Test: m
Bug: 260003429
Change-Id: Iac401eeaf97c904ea751583bf80091d50526f695
2023-01-03 14:50:36 -08:00
Sam Delmerico
eddd3c0ba4 add APEX transitive dependency validation
Bug: 218419109
Test: b build //packages/modules/adb/apex:com.android.adbd
Change-Id: I93b0c99d6521e419e52c63271646448f6d708c22
2022-12-13 14:32:19 -05:00
Usta Shrestha
572ecec696 scan and run test_ methods
Bug: N/A
Test: run bootstrap_test.sh
Change-Id: I3f810277763144d4a6a2f9c1d08bed3a29b7598e
2022-12-09 01:41:49 -05:00
Chris Parsons
b6e9690776 Add groundwork to easily all-enable bazel-mode
This will make it easy to change a "false" boolean to "true" to push
`--bazel-mode` by default for all users.

Users may disable bazel-by-default with BUILD_BROKEN_DISABLE_BAZEL, and
bazel is disabled by default on Darwin hosts (due to lack of test
coverage).

Bug: 254628592
Bug: 254084584
Test: Presubmits
Test: Presubmits for aosp/2263623
Change-Id: Iec712119e06ca8ec93028207d88277f879184cc2
2022-11-01 09:53:46 -04:00
Jason Wu
1cb6739c01 Merge "Rollforward "Use proto output instead of jsonproto output for aquery""" 2022-10-28 14:47:52 +00:00
Jason Wu
118fd2b443 Rollforward "Use proto output instead of jsonproto output for aquery""
Test: use m --bazel-mode-dev and mixed_droid.sh
Bug: 233798334

This reverts commit d27e28c2a7.

Change-Id: Ifd37ac449979330529dd25c92a5eca2f5a80827e
2022-10-27 18:41:50 +00:00
Jason Wu
ab17626ba6 Merge "Revert "Use proto output instead of jsonproto output for aquery"" 2022-10-27 05:24:40 +00:00
Jason Wu
d27e28c2a7 Revert "Use proto output instead of jsonproto output for aquery"
This reverts commit 6fe8721e12.

Reason for revert: Broke aosp_kernel-build-tools/linux

Change-Id: I502ec4c275cbc56e1024a42fadf68b308750939d
2022-10-27 05:18:40 +00:00
Jason Wu
b1bd4bf716 Merge "Use proto output instead of jsonproto output for aquery" 2022-10-27 03:06:12 +00:00
Jason Wu
6fe8721e12 Use proto output instead of jsonproto output for aquery
Test: use m --bazel-mode-dev and mixed_droid.sh
Bug: 233798334
Change-Id: Ib945359d3d05e6f51afe70432ec380e4e584175b
2022-10-26 17:07:48 -04:00
Joe Onorato
ba29f380d3 Move bazel from being a shell function to being a script at build/bazel/bin/bazel
Bug: 254713799
Test: b build //packages/modules/adb/apex:com.android.adbd
Change-Id: I6a0a89438a46d8252c3e0c24319038b0cf2dbd59
2022-10-24 06:38:11 -07:00
Sasha Smundak
8bea267caf Implement bp2build for the package module
* We are interested mostly in the conversion the `default_applicable_licenses`
  attribute, as `default_visibility` cannot be handled until every module's
  `visibility` is handled.
* Several referenced license modules had to be manually enabled for
  conversion, and likewise a few trivial Android.bp containing only
  package and license modules.
* As Bazel allows only a single `package` rule, the package rules in a
  manually converted BUILD.bazel files were removed (in
  external/protobuf and prebuilts/clang/host/linux-x86 trees).
* The converted package rule is emitted before the `load` statements per
  Bazel documentation.

Bug: 190817312
Test: treehugger
Change-Id: If8bf6fee1580177de3bb402923615bcd48923ed2
2022-09-14 13:49:27 -07:00
Romain Jobredeaux
0dfe8d3f7e Allowlist Kotlin-related libraries.
Change-Id: Iaeea811d9ae71b6e57ca79246e0a1691d65ff6f2
Bug: 186462641
Test: Newly allowlisted libs fall under CI
2022-09-08 16:21:42 -04:00
Sam Delmerico
105a70418f Revert "add bp2build unit tests for aidl_interface"
This reverts commit 1fa2672be9.

Reason for revert: breaks tradefed branch

Change-Id: I69bba531774031890baece1796df24d5534b93a9
2022-08-29 22:24:10 +00:00
Sam Delmerico
1fa2672be9 add bp2build unit tests for aidl_interface
Bug: 229251008
Test: go test ./bp2build -run TestAidlInterface
Change-Id: Ie7214d5578bc21e348410ca91ea42d7a3190eb3d
2022-08-25 14:47:41 -04:00
Sam Delmerico
970b96dd03 set USE_RBE=false in Soong integration tests
RBE is already disabled for this test on CI. Locally, the mock checkout
created for the Soong integration tests does not set up RBE properly, so
we should just disable it explicitly so that the integration tests can
be run locally.

Test: build/soong/tests/run_integration_tests.sh
Change-Id: I38875a02b549a9d8c8368d6287c4516518d20e34
2022-08-04 14:09:37 -04:00
Yifan Hong
82799f2b7f Add build/bazel_common_rules to mock bazel directories.
Test: build/soong/tests/run_integration_tests.sh
Change-Id: I647d122b5fc530b5a26bd69c141d3fdc9cca7424
2022-06-28 16:25:56 -07:00
Usta Shrestha
2c9a5e3ddd Shellcheck warnings addressed
SC2236: use [[ -n "$var" ]] instead of [[ ! -z "$var" ]]
SC2035: use -- end of options, to accomodate filename w\ hyphens
SC2155: `local -r var=..` for readonly variable shellcheck ignores return value masking
SC2045: prefer glob to iteration over `ls`

Bug: N/A
Test: ran ./run_intergration_tests.sh
Change-Id: Iaf313857ecc417e855afd70c7455841b54b1c363
2022-06-09 15:47:05 -04:00
Jingwen Chen
d4b1dc8b2a bp2build: add support for planting unresolved symlinks in the symlink forest.
In the isDir check, there's a possibility that the Stat check fails
because the path is an unresolved symlink. Verify it with Lstat (which
would succeed, since it doesn't follow links), and treat it like a file
if Lstat succeeds.

Test: new integration test
Fixes: 232370097
Change-Id: I9807ca363a5dbdc20639b489b54627bd2cc1ca60
2022-05-17 12:17:53 +00:00
Lukacs T. Berki
e3487c8848 Add a test for correctness of C++ compilation.
This required the following:

- Adding Platform_base_sdk_extension_version to default soong.variables
- Teaching the symlink tree creation code to understand symlinks
- Making finder.go follow symlinks when requested

Adding yet another knob is unfortunate, but I can't allow that
unconditionally because the Android code base contains a number of
symlinks giving rise to infinite directory trees because they point back
to their parent and this seemed preferable to adding complicated logic
like "follow symlink but if only its fully resolved version does not
point under the source tree".

I could be convinced about the latter, though.

Test: Presubmits.
Change-Id: I453f6b7e5334771f5832c700db00f9d24ed1d82f
2022-05-04 09:12:01 +02:00
Cole Faust
d9932ad053 Always build mk2rbc/rbcrun
Currently, mk2rbc/rbcrun aren't built in certain phases
of the build, which causes the Starlark product configuration
to fail when using get_build_var if a regular build hasn't
been run before.

Bug: 226677850
Test: Manually
Change-Id: If85608eef63be25fddc4d916c82247c13eb0195f
2022-03-29 14:55:17 -07:00
Liz Kammer
0940b89937 Handle arch/cpu variants in mixed builds
Test: mixed_libc.sh
Change-Id: Ib5084060199b8e1a5955bd51bc3fd793bb83a1eb
2022-03-21 11:13:49 -04:00
Colin Cross
34d60c9281 Use --skip-config --soong-only instead of --skip-make
--skip-make is deprecated because of its odd behavior around the
out/.soong.kati_enabled file.  Replace it with --skip-config --soong-only
instead, which will unambigiously skip running kati.

Bug: 204136549
Test: build/soong/tests/run_integration_tests.sh
Test: build/soong/scripts/build-mainline-modules.sh
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I629ee7542687aaafd68eea50346b0cf53eafe1ef
2021-10-29 16:34:55 -07:00
Lukacs T. Berki
a1b9372ef7 Make running soong_build in alternate modes nicer.
This includes the JSON graph generator and bp2build.

Before:
GENERATE_BAZEL_FILES=1 m nothing
GENERATE_JSON_MODULE_GRAPH=1 m nothing

Now:
m json-module-graph
m bp2build

They can now also be combined with other targets or each other.

The longer-term goal is to run "m queryview" and "m soong_docs" using
the same infrastructure. There are two alternate approaches:

1. Call soong_build from within the main Ninja invocation. This requires
two sequential soong_build invocations and is thus slower.
2. Do everything requested in the same soong_build invocation. This
would be faster, but one AFAIU can't tell Ninja that multiple possible
actions can build the same output so that doesn't work.

(1) is somewhat more desirable because soong_docs seems to be built
from build/make/core/main.mk ; I assume that that can be worked around
although I haven't checked where the output of "m soong_docs" goes.

Test: Presubmits.
Change-Id: If5ba36490d9f3f60733e6d6be9286eb2b67c3ff5
2021-09-06 16:03:51 +02:00
Jingwen Chen
916322541a Allow //external/bazel-skylib to use existing BUILD files
Bug: 190605804
Test: bp2build-incremental
Change-Id: Ida765465532c0e98d035eef4c703d13fbf554538
2021-08-11 12:19:17 +00:00
Dan Willemsen
4591b6496d Upgrade to golang protobuf api v2
Major operations:
 * Fix the go_package entries to be consistent, as the new tool
   complains when it doesn't have a '/'.
 * Regenerate with the new protoc-gen-go tool.
 * github.com/golang/protobuf -> google.golang.org/protobuf
 * proto.[Un]MarshalText -> prototext.[Un]Marshal

Change-Id: Ie1147bd2457fafb66ba555461b3bf14f0561a25d
2021-07-23 11:12:05 -07:00
Spandan Das
0506361a60 Add errorHints to stdout when read-only file system errors are detected
The source tree will eventually be made ReadOnly, and recipes that write
directly to the source tree will fail. Use a pattern-match approach on
the results of stdout/stderr to provide hints to the user in such a
scenario.

If multiple patterns are found in raw output, print error hint
corresponding to first pattern match. first pattern match is chosen
since the failing function will be at the top of the stack, and hence
will be logged first

Test: Wrote a unit test to assert errorhint is added to output.
Wrote an integration test that writes to a file in the source tree
1. When source_tree is RO, the recipe fails and an error hint is printed
to stdout
2. When source tree is RW, the recipe succeeds and no error hint is
printed

Bug: 174726238
Change-Id: Id67b48f8094cdf8a571c239ae469d60464a1e89c
2021-06-29 00:20:21 +00:00
Liz Kammer
09f947d67e Dump bazel product config in Soong
Bug: 187862880
Test: build/bazel/ci/bp2build.sh
Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: I24b09baad973e25bec4476e1ea4a7692b72b7d20
2021-05-19 17:38:56 -04:00
Rupert Shuttleworth
2a4fc3ecdc Generate BUILD files for every directory that has an Android.bp file.
Test: Added an integration test
Test: bazel build --package_path=out/soong/workspace //bionic/...

Change-Id: Ie34bd23ab3c5428e6c9c9919e5fb6fcb4e709adc
2021-04-21 07:15:50 -04:00
Lukacs T. Berki
7a51907346 Colorize the name of the test to be run.
Test: Presubmits.
Change-Id: Ie1335f9fb91c199874e37d74d0853e8f441cc294
2021-04-15 18:18:45 +02:00
Lukacs T. Berki
686965baeb Speed up Soong integration tests.
This is done by running a build before the first test case, tarring up
$TOP (including out/) then untarring it for every test case instead of
rebuilding soong_build each time.

The speedup is approximately 2x.

Also add some cute ANSI escape sequences and delineation of individual
test cases.

Test: The aforementioned integration tests.
Change-Id: I6a98660cfe6a40bcaa978e8d7544cdffad86a7fa
2021-04-15 08:47:39 +02:00
Lukacs T. Berki
b448c34bfd Make integration tests clean up after themselves.
Well, this was embarrassing.

Test: These tests.
Change-Id: Id25f9f1f4bd2d2d01d49d11a5af8eb50dd51fe6e
2021-04-14 16:45:22 +02:00
Lukacs T. Berki
3b730c4bc8 Organize Soong's integration tests:
- Move them into a new directory
- Split tests by theme and add a library file
- Add a shell script that runs them all

Test: Manually ran build/soong/run_integration_tests.sh .
Change-Id: I758b91d679f41aee47d15472cc02547ce89f6386
2021-04-08 13:21:13 +02:00