Commit graph

105 commits

Author SHA1 Message Date
Cole Faust
5c503d1c43 Precompile python sources
This signifigantly improves the startup time of soong-built
python binaries. For example, running
`m apexer && time out/host/linux-x86/bin/apexer` gives
0.734s before this cl, and 0.094s after.

Fixes: 259718110
Test: Presubmits
Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
2023-01-27 15:43:38 -08:00
Usta Shrestha
071f6c29a3 for consistency symlink_tree failure = os.Exit(1)
Brings down time taken to fail from 18s to 13s (on aosp branch)

Test: temporarily coded random failure
Bug: N/A
Change-Id: Ib694eec977293f4dd7054e779d1b82b8cace93f6
2023-01-13 14:07:10 -05:00
Treehugger Robot
0ddd60fc1d Merge "Reference deapexer directly in bazel-bin" 2023-01-12 01:01:24 +00:00
Spandan Das
df01030ab5 Merge changes I78639a2d,Iddb0aa1a
* changes:
  Create a smoke test for api_bp2build workspace
  Add a new CreateSoongInjectionDirFiles function
2023-01-11 22:37:27 +00:00
Chris Parsons
41eff2e4b6 Reference deapexer directly in bazel-bin
This keeps this test compatible with a Bazel release change which
affects which files are present in `bazel cquery --output=files` for
py_binary targets. Since the executable will be present in a predictable
location under bazel-bin, we can hardcode that, instead.

We could have alternatively used --output_groups=python_zip_file for
'hermetic' python, but the bp2build-built deapexer target is missing
some dependencies on protobuf.

Bug: 265176532
Test: Manual runs of apex_comparison_tests.sh with the new Bazel release
Change-Id: Ib7021e7e81d1a7bf5ff4d222e2a7947a06d16791
2023-01-11 17:30:55 -05:00
Spandan Das
255648c6d5 Create a smoke test for api_bp2build workspace
This test builds an "empty" filegroup. Running it can be useful to
verify that the api_bp2build workspace has been setup correctly.

Test: TH
Change-Id: I78639a2dcdeb659b2d6fdf6abc85d85c4fb697bf
2023-01-11 03:21:18 +00:00
Wei Li
ad2ac54d65 Add integration test for testing the arch variant passed to cc modules included in APEXES.
Resend aosp/2350557, with minor change on line 65 of apex_cc_module_arch_variant_tests.sh.
The pattern was:
  '.*android_target.*: ([0-9]+)$'
and is changed to:
  '.*opt-ST.*: ([0-9]+)$'
since the Bazel output directory name has been changed.

Bug: 258422889
Test: build/soong/tests/run_integration_tests.sh
Change-Id: Ib114bc0084d8ea2cd232d6478cc81a039b138bc4
2023-01-10 21:38:46 +00:00
Wei Li
2822aaa198 Merge "Revert "Add integration test for testing the arch variant passed to cc modules included in APEXES."" 2023-01-09 17:39:37 +00:00
Usta (Tsering) Shrestha
9955d076f6 Merge "cosmetic: auto-scan test_ functions" 2023-01-09 15:36:40 +00:00
Jingwen Chen
60ba3892df Revert "Add integration test for testing the arch variant passed to cc modules included in APEXES."
This reverts commit 6c588296a2.

Reason for revert: b/264827343

Change-Id: Iba5a4924349a90349a1e56181efb2d368444dfac
2023-01-09 05:32:27 +00:00
Wei Li
fcfc9d3130 Merge "Add integration test for testing the arch variant passed to cc modules included in APEXES." 2023-01-07 06:35:09 +00:00
Wei Li
6c588296a2 Add integration test for testing the arch variant passed to cc modules included in APEXES.
Bug: 258422889
Test: build/soong/tests/run_integration_tests.sh
Change-Id: I0852454e1d18ccd706f5270c0c87f9546cede5bf
2023-01-06 22:40:52 +00:00
Usta Shrestha
a4e0a0f7f6 cosmetic: auto-scan test_ functions
bug: NA
test: manually ran build/soong/run_integration_tests.sh and verified all tests ran
Change-Id: I1a0a2e227888efc2e7fd7f718b8e534e6dbdbcf1
2023-01-06 14:26:56 +00:00
Cole Faust
b85d1a15cc Bp2build support for multiple product configs
Create a
build/bazel/product_config/generated/products/<product_name>/BUILD
file that contains the platform definitions needed for
a particular product. Currently we just create it for the
current lunch target, but the idea is that eventually when
all product config is in starlark, all the products will
have their platform definitions in the tree at once.

Bug: 249685973
Test: Presubmits
Change-Id: I08c82ff28dcf62f09d3b1d2e3186a6b961e12f6e
2023-01-05 12:49:14 -08:00
Joe Onorato
fa5fc26ce2 Move common functions to script_utils.sh
Test: m
Bug: 260003429
Change-Id: Iac401eeaf97c904ea751583bf80091d50526f695
2023-01-03 14:50:36 -08:00
Treehugger Robot
a6d4e15c3a Merge "Allowlist build files in external/python/absl-py" 2022-12-15 20:23:35 +00:00
Sam Delmerico
eddd3c0ba4 add APEX transitive dependency validation
Bug: 218419109
Test: b build //packages/modules/adb/apex:com.android.adbd
Change-Id: I93b0c99d6521e419e52c63271646448f6d708c22
2022-12-13 14:32:19 -05:00
Usta (Tsering) Shrestha
b545f49f06 Merge "scan and run test_ methods" 2022-12-13 15:46:08 +00:00
Cole Faust
de12be3fb1 Allowlist build files in external/python/absl-py
Needed because we import absl-py in our bazel.workspace file.
It only worked before in b, not mixed builds, because b was
run from the real android tree instead of out/soong/workspace.

Bug: 234449134
Bug: 249685973
Test: Presubmits
Change-Id: Ie6985582c55a4a95338fb3a8ff7b13ddb522c391
2022-12-12 12:27:31 -08:00
Vinh Tran
cc1a017240 Change out path to android_target-opt in bazel integration tests
ag/c/platform/build/bazel/+/20584767/15/common.bazelrc#116 defaulted our build to opt compilation mode which affects bazel's output path. This change fixes the tests to reflect that.

Test: ./tests/apex_comparison_tests.sh
Test: ./tests/bp2build_bazel_test.sh
Bug: 253540178
Change-Id: Idc42a2c3173800db0d6155ef3d74e33c15d220d0
Merged-In: Idc42a2c3173800db0d6155ef3d74e33c15d220d0
2022-12-12 12:20:05 -05:00
Usta Shrestha
572ecec696 scan and run test_ methods
Bug: N/A
Test: run bootstrap_test.sh
Change-Id: I3f810277763144d4a6a2f9c1d08bed3a29b7598e
2022-12-09 01:41:49 -05:00
Usta Shrestha
4e17a2e503 cosmetic: getting rid of SC2155 warnings
https://github.com/koalaman/shellcheck/wiki/SC2155

Bug: N/A
Test: bootstrap_test.sh
Change-Id: I3ce7ab2c7b2cf827e32c3cde0dfbcc932fc48039
2022-12-08 01:21:08 -05:00
Wei Li
94bf3158fd Add soong integration test for Bazel-built apexes of 64bit only products.
Bug: 253223400
Test: build/soong/tests/apex_comparison_tests.sh module_arm64only
Change-Id: I3127aa2b8699f0af16c05036cab94a3054ccc5a1
2022-12-05 20:53:25 -08:00
Cole Faust
762c2def20 Revert "Allowlist build files in external/python/absl-py"
Revert submission 2308141-colefaust_run_b_from_out_folder

Reason for revert: Breaks CI due to more instances of the bazel shutdown issue
Reverted Changes:
I51c28e232:Allowlist build files in external/python/absl-py
I9e6530d95:Cd into the bazel workspace instead of using --pac...

Change-Id: Ice47ef2baaef9976546f9562dc7dd9b19b383d59
2022-11-30 05:16:30 +00:00
Cole Faust
93dc19d39d Allowlist build files in external/python/absl-py
Needed because we import absl-py in our bazel.workspace file.
It only worked before in b, not mixed builds, because b was
run from the real android tree instead of out/soong/workspace.

Bug: 234449134
Bug: 249685973
Test: Presubmits
Change-Id: I51c28e23260bed7ca7d4968fd823f52f0d26a0ab
2022-11-28 12:00:00 -08:00
Colin Cross
662d61430f Make PackageVarContext implement PathGlobContext
Make PackageVarContext implement PathGlobContext by implementing
GlobWithDeps.  This will allow calls to ExistentPathForSource
inside a VariableFunc to use optimized glob dependencies instead of
falling back to AddNinjaFileDeps, which is resulting in extra
dependencies from soong_build on top level directories, triggering
extra Soong regenerations.

Remove the fallback path in ExistentPathForSource by making it take
a PathGlobContext, which is now a superset of PathContext.

Rewrite TestNinjaDeps to not rely on the unoptimized glob dependencies
in VariableFuncs and instead call ctx.Config().AddNinjaFileDeps
directly.

Bug: 257079828
Test: test_create_global_include_directory
Change-Id: I48cf189157d78b9252d339dbc9baeb27e4694807
2022-11-04 09:04:02 -07:00
Chris Parsons
b6e9690776 Add groundwork to easily all-enable bazel-mode
This will make it easy to change a "false" boolean to "true" to push
`--bazel-mode` by default for all users.

Users may disable bazel-by-default with BUILD_BROKEN_DISABLE_BAZEL, and
bazel is disabled by default on Darwin hosts (due to lack of test
coverage).

Bug: 254628592
Bug: 254084584
Test: Presubmits
Test: Presubmits for aosp/2263623
Change-Id: Iec712119e06ca8ec93028207d88277f879184cc2
2022-11-01 09:53:46 -04:00
Jason Wu
1cb6739c01 Merge "Rollforward "Use proto output instead of jsonproto output for aquery""" 2022-10-28 14:47:52 +00:00
Jason Wu
118fd2b443 Rollforward "Use proto output instead of jsonproto output for aquery""
Test: use m --bazel-mode-dev and mixed_droid.sh
Bug: 233798334

This reverts commit d27e28c2a7.

Change-Id: Ifd37ac449979330529dd25c92a5eca2f5a80827e
2022-10-27 18:41:50 +00:00
Lukacs T. Berki
c541cd27fa Create Bazel symlink forest in a separate process.
This helps with incrementality a lot: the symlink forest must depend on
almost every directory in the source tree so that if a new file is added
or removed from *anywhere*, it is regenerated.

Previously, we couldn't do this without invoking bp2build, which is
quite wasteful because bp2build takes way more time than the symlink
forest creation, even though we do the latter in a very suboptimal way
at the moment.

This means that if a source file is added or removed (which does not
affect globs), we don't pay the cost of bp2build anymore.

Also refactored symlink_forest.go on the side. Too much state was being
passed around in arguments.

This change reimplements aosp/2263423 ; the semantics of not touching an
output file is the exact same as order-only inputs and the latter is a
bit fewer lines of code.

Test: Presubmits.
Change-Id: I565c580df8a01bacf175d56747c3f50743d4a4d4
2022-10-27 08:08:45 +00:00
Jason Wu
d27e28c2a7 Revert "Use proto output instead of jsonproto output for aquery"
This reverts commit 6fe8721e12.

Reason for revert: Broke aosp_kernel-build-tools/linux

Change-Id: I502ec4c275cbc56e1024a42fadf68b308750939d
2022-10-27 05:18:40 +00:00
Jason Wu
6fe8721e12 Use proto output instead of jsonproto output for aquery
Test: use m --bazel-mode-dev and mixed_droid.sh
Bug: 233798334
Change-Id: Ib945359d3d05e6f51afe70432ec380e4e584175b
2022-10-26 17:07:48 -04:00
Joe Onorato
ba29f380d3 Move bazel from being a shell function to being a script at build/bazel/bin/bazel
Bug: 254713799
Test: b build //packages/modules/adb/apex:com.android.adbd
Change-Id: I6a0a89438a46d8252c3e0c24319038b0cf2dbd59
2022-10-24 06:38:11 -07:00
Jingwen Chen
7e11eb8609 Add integration tests for --config=bp2build with $OUT_DIR not named 'out'.
Bug: 251436403
Test: presubmits
Change-Id: Ife12777267ef06ce0efbeb90b303a61cd6ef0cfd
2022-10-13 14:05:27 +00:00
Sam Delmerico
ab558d92f0 Merge "Revert "Revert "run bp2build in mixed_mode_test.sh""" 2022-10-04 13:33:50 +00:00
Sam Delmerico
73d6bcc7bb Revert "Revert "run bp2build in mixed_mode_test.sh""
This reverts commit cb93d29aa9.

Reason for revert: retry this CL with additional changes in topic

Change-Id: Ibbb8926caef05c2177c06cb1c00795d407215f4a
2022-09-30 18:26:03 +00:00
Sam Delmerico
0d08b9ba35 document workaround test_bp2build_generates_all_buildfiles
test_bp2build_generates_all_buildfiles uses the same directory for
--output_base and --sandbox_tmpfs_path. This is required on the build
servers, but causes issues locally. This commit adds a more helpful
error message when this test fails.

Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: Ib29a7edf456f8ae39bd429ebb367c8115e02c284
2022-09-22 11:43:21 -04:00
Sasha Smundak
8bea267caf Implement bp2build for the package module
* We are interested mostly in the conversion the `default_applicable_licenses`
  attribute, as `default_visibility` cannot be handled until every module's
  `visibility` is handled.
* Several referenced license modules had to be manually enabled for
  conversion, and likewise a few trivial Android.bp containing only
  package and license modules.
* As Bazel allows only a single `package` rule, the package rules in a
  manually converted BUILD.bazel files were removed (in
  external/protobuf and prebuilts/clang/host/linux-x86 trees).
* The converted package rule is emitted before the `load` statements per
  Bazel documentation.

Bug: 190817312
Test: treehugger
Change-Id: If8bf6fee1580177de3bb402923615bcd48923ed2
2022-09-14 13:49:27 -07:00
Romain Jobredeaux
a2595a93fe Merge "Allowlist Kotlin-related libraries." 2022-09-13 13:28:39 +00:00
Romain Jobredeaux
0dfe8d3f7e Allowlist Kotlin-related libraries.
Change-Id: Iaeea811d9ae71b6e57ca79246e0a1691d65ff6f2
Bug: 186462641
Test: Newly allowlisted libs fall under CI
2022-09-08 16:21:42 -04:00
Jingwen Chen
60d8840f65 Update tests to use --config=android.
Bug: 244766775
Test: presubmits
Change-Id: Iaaabd275356c24b3e3713ea1047c134271c4561e
2022-09-08 15:54:32 +00:00
Treehugger Robot
f837a93337 Merge "Revert "add bp2build unit tests for aidl_interface"" 2022-08-30 17:19:13 +00:00
Treehugger Robot
b28effac58 Merge "improve error message when soong test fails early" 2022-08-30 08:04:48 +00:00
Sam Delmerico
105a70418f Revert "add bp2build unit tests for aidl_interface"
This reverts commit 1fa2672be9.

Reason for revert: breaks tradefed branch

Change-Id: I69bba531774031890baece1796df24d5534b93a9
2022-08-29 22:24:10 +00:00
Sam Delmerico
b6d1b03991 improve error message when soong test fails early
apex_comparison_test.sh defines a helper function to clean up state on
exit, but in error conditions the call_bazel function is not always
defined.

Test: added an exit 1 statement below trap
Change-Id: I73b0c56c0a4a2d9ed1d4c0bf38b6ac0ac985e716
2022-08-26 16:24:04 -04:00
Sam Delmerico
1fa2672be9 add bp2build unit tests for aidl_interface
Bug: 229251008
Test: go test ./bp2build -run TestAidlInterface
Change-Id: Ie7214d5578bc21e348410ca91ea42d7a3190eb3d
2022-08-25 14:47:41 -04:00
Sam Delmerico
970b96dd03 set USE_RBE=false in Soong integration tests
RBE is already disabled for this test on CI. Locally, the mock checkout
created for the Soong integration tests does not set up RBE properly, so
we should just disable it explicitly so that the integration tests can
be run locally.

Test: build/soong/tests/run_integration_tests.sh
Change-Id: I38875a02b549a9d8c8368d6287c4516518d20e34
2022-08-04 14:09:37 -04:00
Yifan Hong
82799f2b7f Add build/bazel_common_rules to mock bazel directories.
Test: build/soong/tests/run_integration_tests.sh
Change-Id: I647d122b5fc530b5a26bd69c141d3fdc9cca7424
2022-06-28 16:25:56 -07:00
Jingwen Chen
26f0b214ba Rewrite apex_diff_test as regular soong integration tests.
apex_diff_test targets were the last remaining dependents of make_injection. Rewriting them allows us to delete make_injection.

Test: CI
Change-Id: I928d8fe136d709b9f819c00c3127af66e720a7db
2022-06-20 06:42:37 +00:00
Usta Shrestha
2c9a5e3ddd Shellcheck warnings addressed
SC2236: use [[ -n "$var" ]] instead of [[ ! -z "$var" ]]
SC2035: use -- end of options, to accomodate filename w\ hyphens
SC2155: `local -r var=..` for readonly variable shellcheck ignores return value masking
SC2045: prefer glob to iteration over `ls`

Bug: N/A
Test: ran ./run_intergration_tests.sh
Change-Id: Iaf313857ecc417e855afd70c7455841b54b1c363
2022-06-09 15:47:05 -04:00