This fixes current breakage on aosp-master-bazel on bp2build-incremental. aosp/2401793 addes android.frameworks.stats-V2-ndk to libtestvendoratoms but android.frameworks.stats aidl_interface is not allowlisted yet.
However, `//frameworks/proto_logging/stats/stats_log_api_gen:libtestvendoratoms` is still not building because of another issue in genrule + proto.
```
ERROR: out/soong/workspace/frameworks/proto_logging/stats/stats_log_api_gen/BUILD.bazel:87:8: Executing genrule //frameworks/proto_logging/stats/stats_log_api_gen:test_vendor_atoms.cpp failed: (Segmentation fault): bash failed: error executing command (from target //frameworks/proto_logging/stats/stats_log_api_gen:test_vendor_atoms.cpp) /bin/bash -c ... (remaining 1 argument skipped)i
```
The fix to this is similar to aosp/2401794 and is fixed in aosp/2449265
Test: presubmit
Bug: 270131691
Change-Id: I8bcc336d843ee4f0de44f377c828a9a4c959b0ec
Bug: 240424572
Test: Manual tests:
1. m --dev-mode-staging com.android.adbd com.android.media.swcodec.
2. verify the DCLA libs from the two apexes have the same size and
sha1sum, and also match the libs in bazel-out.
3. empty the DCLA libs list in allowlist.go and repeat step 1
4. repeat step 2 and verify the opposite result
5. build git_master: mainline_modules_bundles-userdebug in ABTD
with the cl, then follow go/build-sideload-dcla-locally to
download the adbd and swcodec aab files, run the DCLA trimming
workflow locally, and verify the symlinks in the two trimmed
apexes are identical and also match the lib path in the DCLA
apex that was created by the workflow.
Change-Id: Ib2f8a29126a54829c0e10eba17b256a79930fd70
Previously, the symlink optimization for APEXes assumed that the target
of the symlinks are in the system partition. The assumption however
doesn't hold always because the file that was added to the APEX might be
with system_ext_specific: true or vendor: true.
Bug: 265598720
Test: m nothing
Change-Id: Ieb9a6769320c0ec697a88c0cae977e7d65288362
Creation of build statements is largely parallelizable because each
action is independent apart from updates/reads to
depsetHashToArtifactPathsCache. Locally resulted in build statements
taking ~.45 seconds on staging mode to ~.02 seconds
Test: CI
Change-Id: Iab00c8394a9eab17353f71230885ff0870e17f24
Only public, system, test, module_lib and system_server are
converted in order to generate api_fingerprint.txt in Bazel.
Test: java_sdk_library_conversion_test.go and TH
Bug: 266973526
Change-Id: I67a00806165e5afad3876b6cd5cdbc6b0dd65d8b
This reverts commit 1fbf7f21a4.
Reason for revert: broke docker image, pinning is incomplete it seems
Change-Id: I7102cb233a4fe5ce0f5a1ead98cfc661c3d82ce5
This is a more flexible version of ApexAvailableTags
that can be used for the aidl modules.
Bug: 260694842
Test: go test
Change-Id: Iddb15e737c19e95977f74e8c627d6044d8875746
Test: Manually verified artifact timestamps are unchanged on a second
invocation of `m nothing` (with bp changes)
Test: Presubmits
Change-Id: I6bdf106a464597a07ac290cb575425d700721ffe
This changes bp2build codegen, symlink forest generation, and
soong_build so that they do not rewrite output files if the contents are
unchanged.
Bug: 266983462
Test: m droid
Test: canonical_perf.sh benchmarking
Test: Manually verified that rerunning analysis did not regenerate
out/soong/workspace/prebuilts/sdk/BUILD.bazel unless contents changed
Change-Id: I5ec227df7a32b53c7fa0d741fb1403a51931024b
Previously in mixed builds, only the tidy files for the boundary module
were built, whereas all of its transitive dependencies' tidy files were
not being built. Instead we should export the list of transitive tidy
files for a module so that we can run clang-tidy for the boundary module
as well as its dependencies.
Bug: 195029134
Test: WITH_TIDY=1 DISABLE_ARTIFACT_PATH_REQUIREMENTS=true mss tidy-packages-modules-NeuralNetworks --bazel-mode-dev
Change-Id: I463646d2ae1fc4aa075a54c264e1c34571c3fd5c
Currently, droidstubs module create java_api_contribution module, but
when a java_sdk_library module dynamically creates a droidstubs module,
java_api_contribution is not created as the hook inside droidstubs get
lost. Therefore, call hook inside sdk_library after creating the
droidstubs module.
Test: m
Change-Id: I68bf1d796f6f9a6f81011ae35e4991b6ed1421ea
Perform a single pass over pathComponents in both validatePath and
validateSafePath instead.
Test: presubmits
Change-Id: I1499183e345ca180900c54a18f9dc8bd534f2588
Previously, Bazel was invoked during mixed builds with a stripped down
environment. This CL adds an allowlist of environment variables that are
passed to Bazel.
Test: WITH_TIDY=1 DISABLE_ARTIFACT_PATH_REQUIREMENTS=true mss tidy-packages-modules-NeuralNetworks --bazel-mode-dev
Change-Id: I23147bec59f6522953cf623e7bcaa0f1f99a75a3
This also fixes determinism of these files (by ensuring that bazel
requests and their configurations are sorted, as these are used in the
soong_injection file output)
Bug: 266983462
Test: Manually verified soong_injection files are the same among
multiple runs
Test: Unit test
Test: m nothing
Change-Id: I1568930549cff0bc5676825434cc448d16ebdd4e
Allowlisting MusicKotlin gives us CI coverage on android apps with kotlin code.
jsr305's auto-generated BUILD file is working for us for now.
QuickSearchBox has been migrated to Kotlin and now has an unsupported dependency.
Change-Id: I9d9e6abf49875b0f5981fbcefdbd70a1bff62a83
Still fall back to the hardcoded list for what to put in LOCAL_SHARED_LIBRARIES,
because I couldn't figure out a good way to get those modules at the moment.
Bug: 259718110
Test: Presubmits
Change-Id: Ia992ffea8819d8c67280b08bc45505f77af09de0
Instead of using a sentinel file that was created under bazel_workspace
prune them out. The sentinel file was being created under bazel_workspace,
which would be deleted and thus rendered useless from the perspective of
not retriggering a build.
Test: `touch Android.bp` then `NINJA_ARGS="-d explain" m --bazel-mode-staging adbd_test`
Bug: 265155778
Change-Id: If5d8f0ea7f3b8828fda0646faafd1a621f0cb27c
This piggybacks onto the ApexInfo cquery handler, so we're issuing a
single bazel query call that reads two providers in the starlark expr.
Also rename requiredDeps to makeModulesToInstall to differentiate it from
APEX's required/provided libs in the apex manifest.
Test: unit test
Test: mkdiff
Fixes: 263123189
Change-Id: Ib7e43f1586f29864eee8627dba3631bfaff27afa
This API surface is provided by platform and mainline modules to other
mainline modules. Rename it to module-lib API surface to align it with
the terminology in go/android-api-types
Test: go test ./bp2build
Test: go test ./cc
Change-Id: Ieb9f3214e66366fc2ceb6f47e2d6623acb537827
variable.go:
add a new product variable TrimmedApex which is controlled by
environment variable PRODUCT_TRIMMED_APEX
config.go:
all config to check if trimmed build is turned on or not
apex.go:
add overridable apex module property:
use_DCLA: <DCLA apex module>
"use_DCLA" allows user to just use the lib list in DCLA to determine
which native shared libraries to trim. This property is no-op if
TrimApexEnabled is set to false.
builder.go
1, if it is to produce a trimmed variant, fix the default version code
by making the last variant digit to 2. this is only useful in local
build. in production build, there is apex manifest injection by
coastguard that determines the actual version code.
2, create a new apex rule called: "TrimmedApexRule", this rule invokes
an apexer wrapper called apexer_with_trim_preprocessing which will
properly trim the target libs.
BUG: b/259381334
TEST: adbd banchan build for both static and trimmed variant
Change-Id: I38a91cdef86a3c9185d00610ab3dd7acd07fd90a
The adbd_test androidmk definition is missing some cc-related variables
for cc_test in mixed builds. These variables should be populated from
information from Bazel.
Bug: 265758350
Change-Id: I59d017e2eb2f139188ba3383c457cc0055372b61
* Remove `Bp2buildCcLibraryStaticOnlyList` it is never set
* Remove the rest of the sutff related to it
* Streamline MixedBuildsEnabled
* Rename BazelContext.BazelAllowlisted to BazelContext.NameFilter
to reflect its action
* Rename bazelContext to mixedBuildBazelContext
* Fix minor warnings from Go static analyzer
Test: treehugger
Change-Id: Ie245c992f94bbfd5c7c23dd06917026200a28263
This isn't benchmarked or anything, but should give
minor performance improvements.
Test: Verified ninja files didn't change on aosp_arm64-userdebug
Change-Id: I030991f91c326669b7b7e73b8171da4b2a68cbb4
This was always set to "." after aosp/1652613 and
aosp/1796650. While doing some rudimentary performance
profiling of soong, it turned out that SourcePath.String()
was taking 600+MB of memory joining the "." to the rest
of the path. Remove it for performance gains.
Bug: 262629589
Test: Verfied ninja files didn't change with this cl on aosp_arm64-userdebug
Change-Id: Id6b552ab8defe37e0c2b58e813fb1dbae427ae96
Previously, the soong_config_module_type (and related module types)
would make a special call to the module factory of the module type
being customized to get the properties from which it created the
conditional properties. That caused problems when trying to customize
a singleton module. See the bug for more details.
This change avoids that special call by deferring the creation of the
conditional properties struct until just after the
soong_config_module_type's factory function calls the customized module
factory to create the module and properties. The properties are then
used to create the conditional properties struct avoiding the extra
factory call.
The conditional properties struct is only created once per module type
that soong_config_module_type (and related module types) creates.
Bug: 264876909
Test: m nothing
Change-Id: I55dc71d2553cb59d921a96c6458d0bc877512bbb
Singleton module types have some special behavior that can cause
problems when trying to extend them with soong_config_module_type so
this adds some tests of how they work together.
Currently, the tests show that they do not work together, a follow up
change will correct that.
Bug: 264876909
Test: m nothing
Change-Id: I250ee4e18b3c04d1c91808f55722da74b813a570
Create a
build/bazel/product_config/generated/products/<product_name>/BUILD
file that contains the platform definitions needed for
a particular product. Currently we just create it for the
current lunch target, but the idea is that eventually when
all product config is in starlark, all the products will
have their platform definitions in the tree at once.
Bug: 249685973
Test: Presubmits
Change-Id: I08c82ff28dcf62f09d3b1d2e3186a6b961e12f6e
Bug: 262738679
Test: BAZEL_METRICS_DIR=dir1 m nothing
BAZEL_METRICS_DIR=dir2 m nothing
# Make sure that the second does not regenerate the ninja file.
Change-Id: If30f13eee5d27d902c894af04af38078f437b6df
These instances could use WriteFileRule instead of
making their own shell code to write a file.
Test: Presubmits
Change-Id: I9c809b2164a68b4ce1c22fbbd0d7497240110b39
This creates a new API surface for the APIs provided by ART to compile
other API domains (e.g. this contains LambdaMetaFactory). The scope of
this API surface is restricted to compile-time. Unlike the other API
surfaces (e.g. public) whose APIs will exist on device in the form of
implementation libraries, these classes are not guaranteed to exist on
device.
This also updates the naming convention map used in Multi-tree API
export
Bug: 261244752
Test: TH
Change-Id: I164714d68618a3c289bac1a695958e36fdbbda5d
- The converter runs for api_bp2build, not bp2build workspace
- Since droidstubs is an internal module created by java_sdk_library,
the conversion encompasses the latter as well
- Since droidstubs do not have an api_surface attribute, this conversion
uses naming convention to infer the api_surface represented by the api
file e.g. *stubs.source -> publicapi, *stubs.source.system -> systemapi)
- Also adds an SdkIntraCore enum to represent the API surface provided
by one core module to another
There is also ongoing work to check in java_api_contribution modules in
Soong. Once we have that, we can update this converter to operate on
that module type instead
Test: go test ./bp2build
Change-Id: Ia85828e04c738d9ffcc524856d7c3034ee29bbf9
The comments in aosp/2336916 are also addressed here
Test: TH and TestCcLibraryVariantPrependPropOrder
Change-Id: If1472658fcd4b5544dec2e2691049a180520c84e
linux_musl is an alternative host OS, return "linux" from
getConfigString when it is being used.
Bug: 259266326
Test: build/bazel/ci/mixed_libc.sh
Test: USE_HOST_MUSL=true build/bazel/ci/mixed_libc.sh
Change-Id: I47b8fdd93b36345f82dd4e8455edb4c579f805e9
A dependency added in aosp/2353522 resulted in bazel-mode-dev to start
failing. This CL updates the allowlist to support the new dependency.
Test: build/bazel/ci/mixed_libc.sh
Change-Id: I03ac035517a66ff0b21543e8b94d71c9d7ddb461
This reverts commit 8ec823cba1.
Reason for revert: DroidMonitor: Potential culprit for Bug b/262965953 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted.
Change-Id: I236cc36981d8b30527ca286632727f8ca267e969
* `android.Context` holds `android.Config`, so provide an accessor to it
and avoid passing them separately to a lot of functions.
* factor common code in `readBazelPaths`/`getExistingBazelRelatedFiles`
into `readFileLines`
* refactor check-error-and-quit into `maybeQuit`
* use switch instead of if-elseif-... where appropriate
* rearrange the code in `runApiBp2build`, `runSymlinkForestCreation`
Test: treehugger
Change-Id: I639929c21ec4999cdfd737c07136f32df2d488bc
Define a structure android.CmdArgs and keep all soong_build command line
arguments used to build a configuration in it. Some of them related to BazelMode
were previously kept in Blueprint's bootstrap.Args where they do not belong.
This simplifies the code and lays the gound work for future simplifying changes.
Test: treehugger
Change-Id: I995871f4e6b0e95cc9df65696ad2020df221d8c8
So that they the allowlists can be read from build_mainline_modules_bundles.sh.
Bug: 262302253
Test: m out/soong/bazel_prod_mixed_builds_enabled_list.txt out/soong/bazel_staging_mixed_builds_enabled_list.txt && cat out/soong/bazel_prod_mixed_builds_enabled_list.txt && echo && cat out/soong/bazel_staging_mixed_builds_enabled_list.txt
Change-Id: Id5f89ebb651a52dc5368681a20d6183703e05bd4
This will allow the test_suites_{arm64, x86_64} CI targets on
git_master-bazel to build adbd_test with bazel in `m`.
This is currently implemented similar to the cc_binary handler to
hydrate the output path field.
The tradefed config is still currently handled by Soong in
testBinary#install, which is called from both ProcessBazelQueryResponse
and GenerateAndroidBuildActions. We'll move the tradefed config path to
a mixed builds handler when it's ready on the cc_test bazel rule side.
Tested with `atest` with cuttlefish. Since `atest` doesn't have a way to pass
--bazel-mode-staging to its internal call to Soong, so I manually tested
this by adding adbt_test to the prod allowlist, and confirmed that the
installed adbd_test is the same binary as the one in bazel-out.
Test: $ showcommands adbd_test | grep bazel-out | tail -n1
```
/bin/bash -c "rm -f
out/target/product/vsoc_x86_64/data/nativetest64/adbd_test/adbd_test &&
cp -f -d
out/bazel/output/execroot/__main__/bazel-out/android_x86_64_silvermont-opt-ST-59d4124ca041/bin/packages/modules/adb/adbd_test
out/target/product/vsoc_x86_64/data/nativetest64/adbd_test/adbd_test"
```
Test: Add adbd_test to prod allowlist and run 'lunch aosp_cf_x86_64_phone-userdebug; atest adbd_test', all tests pass.
Bug: 258426425
Bug: 261692372
Change-Id: I6be329f39d607af8daa1dc772b23e8dd0247418c
The output libcore-library-notices gen_notice module is embedded in
core-module-lib-stubs, which is a transtiive dependency of effectively
every javac invocation targeting the device. The output files listed
in the notice file can contain a path that includes the device name,
for example "target/product/generic_arm64/system/framework/core-libart.jar".
This causes the stubs contents to be different for every product,
which drastically reduces the cache hit rate of RBE. Strip the
"out/target/product/generic_arm64" prefix from paths in the notice
file.
Bug: 262620891
Test: unzip -p out/soong/.intermediates/libcore/art.module.public.api.stubs.module_lib/android_common/turbine-combined/art.module.public.api.stubs.module_lib.jar NOTICES/libcore-NOTICES.txt
Change-Id: I705ed1252ef2a595dae53873993adefa1ea85662
Write long bazel commands returned by aquery to a shell script
before executing them to work around command line length limits.
Bug: 261536423
Test: USE_HOST_MUSL=true build/bazel/ci/mixed_libc.sh
Change-Id: Id98a07800df82d1538e6b16b733488a1b9943bf3
Add AndroidMkEmitAssignList to emit a line to assign the items from
the given list of string arrays.
Test: treehugger
Change-Id: Id5acbef38ea4e91349bd2461f226db352d4b8123
Needed because we import absl-py in our bazel.workspace file.
It only worked before in b, not mixed builds, because b was
run from the real android tree instead of out/soong/workspace.
Bug: 234449134
Bug: 249685973
Test: Presubmits
Change-Id: Ie6985582c55a4a95338fb3a8ff7b13ddb522c391
Previously, the memberDepsMutator was responsible for tracking the
membership of modules in an sdk/module_exports (or corresponding
snapshot). That was only used to determine whether a module was in a
versioned sdk snapshot so that it could be treated differently by the
build. As there are no versioned sdk snapshots these can now be
removed.
This change removes the memberDepsMutator, along with SdkAware and
related types that are no longer used.
Bug: 260237150
Test: m nothing
Change-Id: I2668679a42d5b3f33f9e06d5b59804dfdae5f6da
That method was provided to allow a java_sdk_library_import that was
part of a versioned sdk snapshot to create the name of its component
modules consistent with the way in which modules were named within the
sdk snapshot. As versioned sdk snapshots have all been removed this is
no longer necessary.
Bug: 260237150
Test: m nothing
Change-Id: Iee35abd6d210d9a33dfd8ef2f9c77a53329fda24
Previously, the sdk snapshot code used SdkAware instead of Module to
ensure that all its members had implemented SdkAware. However, it never
used any of the methods provided by SdkAware, or if it did it no longer
does. So, this change replaces usages of SdkAware with Module in
preparation for deleting it completely.
Bug: 260237150
Test: m nothing
Change-Id: Ia89e02394f27b2da776f0cf0f0bc86835a03433a
Previously, as all versioned sdk snapshots have been removed from
Android.bp files this method would always return false. This change
effectively replaces all calls to it with false, and then optimizes
away any unused code.
Bug: 260237150
Test: m nothing
Change-Id: I1b717ee8345e807bd888451f6e7e3c3a0d391ee2
Some tests need to add custom tradefed options, but still want to
keep most of the soong autogenerated tradefed xml file.
Expose a test_options: { tradefed_options: [...] } property that
will allow tests to add more options to the autogenerated xml file.
Fixes: 184895128
Test: go test, and verified that the ninja files did not change for aosp_arm64
Change-Id: I75f7eb002c8325ce7cdc76e12e76e16195320620
Add external/musl to the bp2build allowlists. Disable libc_musl_sysroot,
which fails due to a $(locations) reference to a glob that is
unsupported in bazel.
Bug: 259266326
Test: //external/musl:libc_musl builds
Change-Id: Icf0ea2f89854ba3605096dcaa95c1c829af34dc3
platform-robolectric-4.5.1-prebuilt was removed by
https://android-review.googlesource.com/c/platform/prebuilts/misc/+/2314082,
and soong's special rules for platform-robolectric-4.5.1-prebuilt
is not needed anymore.
Test: m
Signed-off-by: utzcoz <utzcoz@gmail.com>
Change-Id: I953ec775505ccd0924f5a3f7fc29bdc7e573d335
Singleton modules are not used in this workspace (they do not implement
`ConvertWithApiBp2build`). However, we still need to add them since
there is a check in `SingletonModuleFactoryAdaptor` that depends on the
singletons being registered. We do this for bp2build as well (see
`RegisterForBazelConversion`)
This was probably always an issue in api_bp2build, but became
incompatible as a side-effect of the refactoring in aosp/2287719
that removed ctx.Register() from the newContext function.
Test: m api_bp2build --skip-soong-tests
Change-Id: I9ba1df2cc1e8ec916635e5280c66f8df68d7e79b
Users can use this feature by
1. Setting PRODUCT_INCLUDE_TAGS += <val> in their product mk files
2. Set
```
blueprint_packge_includes {
match_al: ["<val>"],
}
other_module_type {name: foo}
other_module_type {name: bar}
```
bar and foo will be included if and only if <val> is set
Test: Unit tests in blueprint
Test: TH
Change-Id: I32eed4e3b5ac47fb565c62d13d8881fa984c86f4
Previously, the fixture test infrastructure was hard coded to
initialize itself in preparation for invoking the PrepareBuildActions()
method. That meant it could not be used for testing the bp2build and
apiBp2build modes which required different initialization and called a
different method.
This change extracts that behavior into a FixtureTestRunner and adds an
implementation that allows it to test the above two modes. It then uses
that to implement the runBp2BuildTestCaseWithSetup method.
The TestPrebuiltLibraryAdditionalAttrs was the only test which broke as
it supplied an invalid path to the MockFS (it does not support using a
trailing / to represent an empty build directory). So, it was modified
to add a couple of files inside those directories instead.
Test: m nothing
Change-Id: I6798a4f761160af8d1bfed81d46de9628bda3eb9
The first line of each section will be
```
include $(CLEAR_VARS) # <module type>
...
```
Bug: 257037252
Test: treehugger
Change-Id: Iba586155b682fe4e1e5817d8397eda8f9d9c8789
Also, fix compiler warnings about deprecated ioutil.WriteFile/ReadFile
and about redundant variables
Test: treehugger
Change-Id: Iaf8f89b0e991dae8c5da403d4a1ec3a270f1ae29
