Starting from Android V, vendor seapp_contexts files can't assign
coredomain to vendor apps, as it's Treble violation. This build broken
variable is to suppress the enforcement for devices launching with U or
prior.
Bug: 280547417
Test: set BUILD_BROKEN_VENDOR_SEAPP_USES_COREDOMAIN := true and build
Change-Id: Ic4b5309f0d9bab9b93e88988d1a5a942b2de220a
glob pattern could vary, depending on device configuration, but it uses
the same glob output path for now. It makes unnecessary soong
reevaluation due to the previous result is always overwritten whenever
the target is changed.
So make it product specific to avoid this.
And also let bootstrap.ninja use 'new' ninja file for globbing
Bug: 294378814
Bug: 294058160
Test: lunch a && m nothing && lunch b && m nothing && lunch a && m
nothing and then check if there is no glob work. (a and b has different
glob pattern due to RRO configuration or something)
Test: check if bootstreap.ninja include 'new' ninja file for globbing
Test: declare `cc_library { ... srcs: ["*.c"] }` && \
touch a.c && m nothing && touch b.c && m nothing && \
rm b.c && m nothing && mv a.c c.c && m nothing
and then check if m includes 'regenerate glob..' log.
Change-Id: I9337ae3b33cc2c0689f9731bbae5caae67612e3e
This reverts commit d93c67f64c.
Reason for revert: USE_RBE has again been made the default now, so we should no longer need this code.
Change-Id: I4163c61eed90163e763c29e07dd6edfc9c41b9b9
glob pattern could vary, depending on device configuration, but it uses
the same glob output path for now. It makes unnecessary soong
reevaluation due to the previous result is always overwritten whenever
the target is changed.
So make it product specific to avoid this.
Bug: 294058160
Test: lunch a && m nothing && lunch b && m nothing && lunch a && m
nothing and then check if there is no glob work. (a and b has different
glob pattern due to RRO configuration or something)
Change-Id: I7b6b7a326de681b046a55dbdb34d812cace510e6
Some versions of Fedora include patches to unzip to enable zipbomb
detection that incorrectly handle zip64 and data descriptors and fail
on large zip files produced by soong_zip. Disable zipbomb detection
inside the build.
Reported upstream in https://bugzilla.redhat.com/show_bug.cgi?id=2227130.
Bug: 286885495
Test: builds
Change-Id: I8e4438720bbb17a073ff3b5535f01c2827485838
Enabling the RBE variable can cause issues around pool parallelism when
we set the env variable, but don' run the build with RBE due to auth issues.
So, I am explicitly unsetting the variables in those cases.
Bug: b/292224253
Tested: https://b.corp.google.com/issues/292224253#comment26
Change-Id: I19718b4ee6c058ba1b11d3df260421bbf8c9567e
A heuristics is implemented in the build/bazel/scripts/bp2build_progress/bp2build_progress.py script to determine the reason for why a module is not converted.
Test: b build //build/bazel/scripts/bp2build_progress:bp2build_progress
Change-Id: I9d4eaf309dbf26bbb6de18e1af0d9cdc8fe09e94
even with USE_RBE=true.
These are alternate modes of soong_build that don't use rbe/reclient.
This lets us to remove USE_RBE=false from the bp2build run.
Test: presubmits
Change-Id: I439f9cf7e92ec85ca56baec5f62a83ee49b510d4
Often the slow commands (errorprone happens to be particularly bad)
print a lot, so this should make it easier to find the error without
lots of scrolling.
This doesn't attempt to parse the output and re-display errors, so
if a command prints a thousand warnings with one error in the middle,
it'll still be hard to find the error.
Bug: 277114612
Test: cd build/soong/ui/terminal ; go test
Change-Id: I6c8285fc2c6e4fc345de57b2c15bc5e7d46b1d1f
When there's no stubby and we are asked to use google prod creds, that's invalid configuration, so don't use RBE
Bug: b/283828386
Change-Id: I1564b4f70e46fb90c87a0432c46616caa1614aac
Use Go's generics for DepSets so they don't require a type-specific
wrapper and reflection.
Test: depsets_test.go
Change-Id: I22ba0b7d680d37d2cd05230b0f560d166c4dd20b
reproxy only requires LOAS credentials for authentication. Append the
-nocheck_ssh option to gcertstatus so we don't check ssh credentials.
Test: RBE successfully started after running gcertdestroy --nodestroy_loas2
Bug: b/287297140
Change-Id: I413931b10d8e8d9ae2f8acd7ebfe37d8b3bba455
The tags item in the trace data is arbitrary metadata added to `build`
steps by Soong (and my Kati via `.KATI_TAGS` target-specific
variables). Include this in the perfetto trace so we can analyze it.
Bug: http://b/259130368
Test: End to end test tracing cp time in dist targets
Change-Id: I85d33f579dc40dbae616b24cd4cb150d86262470
This is for integration with BES for bazel test metrics.
Bug: 287102416
Test: b build libcore:all --invocation_id=`uuidgen`
Test: build/bazel/scripts/analyze_build (to verify it's set)
Change-Id: I541b2d65bfa85718fc916582e0540384ae3810d9
This also changes the expectation of ConvertWithBp2build. Each
implementation must either create one or more Bazel target modules, or
mark the module as unconvertible (with a specific reason).
Manually verified no runtime hit in AOSP
In AOSP, the metrics file size increases from 252K to 1.6M
This changes some effective module counts in bp2build metrics:
- Removes "package" modules from the module count list in
metrics, as these will not be converted like regular modules.
- Counts Handcrafted modules as being "unconverted", as bp2build is not
responsible for them.
Bug: 285631638
Test: Verified generated BUILD.bazel files are bit-for-bit identical
with this change
Test: Manually verified one case of each implemented reasonType
Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
From the previous work on renaming build.ninja, it was found that there
are extra dependencies from build.ninja which can be varied by
TARGET_PRODUCT : which is soong.environment.used.<tag>. This change
renames soong.environment.used to have target product between 'used' and
'<tag>' if available.
Bug: 277029044
Test: Test confirmed that build.ninja is not being re-generated
Change-Id: I987b6bd1a8b4f06dac52537e4178d8556251d254
Test: Tested by following steps
1.m nothing: field is empty
2.USE_RBE=false m nothing: field log cc_wrapper and rbe_wrapper
3.USE_RBE=false m nothing: field is empty
Bug: 281922291
Change-Id: I1bbb324752b9a2dea1ff2c9df5817559d4cec3a6
In-order to automatically detect which type of authentication to use for
RBE, we need to be able to run these binaries in bootstrap. Hence
allowlisting them.
Bug: 283828386
Change-Id: I1e0e021acc8283ec3e66c96f6676c6095bf0892b
In non-incremental build, there is no ninja_log. For this case, use
HINT_FROM_SOONG as an alternative solution.
Bug: 273947040
Test: 1.m after removing out/.ninja_log
2.check if non-incremental CI build uses HINT_FROM_SOOONG
3.check if incremental CI build uses NINJA_LOG
4.check if there is no regression in CUJ
Change-Id: I00cd216df096cb2288eeab233729acefb0d1b73c
This allows USE_BAZEL_VERSION to be set for m builds, which will use
Bazelisk for any Bazel invocations during those builds.
This should be used only for manual debugging, typically to either test
new Bazel features, verify Bazel compatibility with Android, or culprit
find new Bazel breakages.
Test: Manually run builds with USE_BAZEL_VERSION, toggled off and on to
ensure the build was rerun. Tested with a broken commit, a working
commit, and 'last_green' special term
Change-Id: I8b475dca5c8d4bd849ee3724a8c3aca9b631bcb8
This reverts commit 6324647bcc.
Reason for revert: This change negatively impacts users running builds on gLinux laptops. Reverting to limit impact and reland with appropriate fix.
Bug: b/284985772
Change-Id: I04a3d0107bb3ea680ba6641240620736a2ef5f17
Prodaccess is apparently deprecated now.
Test: Build with/without credentials and check that it works
Change-Id: Iaadaebe84fe9531b2d7d5f0d265751462bb827d8
This code is no longer needed since we are now setting USE_RBE to true
for 100% of users (except two users, who I'll reach out to over email).
So fetching the configuration is no longer needed since sometimes this
can result in RBE config flipping between on and off (depending on
whether you have gcert or not).
Tested: Ran the build with latest sync and ensured RBE was used for the
build.
Bug: b/283828386
Change-Id: I3cda78607b46a0e161b7841560dc587a44d91f27
Current build.ninja does not contain any product name, while other ninja
files (such as combined ninja) do. This change adds product name to the
build.ninja so it can be separated over multiple lunch targets
Bug: 277029044
Test: build succeeded and checked if out/soong/build.ninja has been
renamed
Change-Id: I16dc71f829fd76f01b98da0d509a8e0ef6f62fa9
`rm -rf out` used to fail because bazel would not set write permission on some files. Now it's been fixed and thus there is little reason to prefer `m clean` (also users are accustomed to `rm -rf out`)
Bug: NA
Test: verified `m clean` works;interestingly soong_metrics file is created at the end
Change-Id: I000d16508613045811fc7792e5798f7c150dcc05
BUILD_NUMBER/HOSTNAME is passed by a file, not env variable.
To avoid kati recreation, unset BUILD_NUMBER/HOSTNAME after
writing it into a file
Bug: 278060169
Test: BUILD_NUMBER=1 m && BUILD_NUMBER=2 m (check there is no kati
re-run)
Change-Id: I5a31461dcf1e4b0634974bcb48a0d7482d42852a
* Extract BUILD_NUMBER, BUILD_HOSTNAME to file to avoid kati change
* Handle FILE_NAME_TAG_PLACEHOLDER string in dist in build/make/packaging/distdir.mk
Test: check if kati isn't invoked even though BUILD_NUMBER, BUILD_HOSTNAME
is changed
Test: m && m, and check if the second m is no-op
Bug: 278060169
Change-Id: I65eefc6bb86d4076098a1bf8b317b4cb88201499
As-is: Even with NINJA_LOG option, ninja weight list is copied from
ninja log
To-be: uses usesninjalogasweightlist to uses ninja log file directly
If weight list is large(more than 100,000 lines), reading the whole list
spends a few seconds(1-3s). It is ignorable in full build, but it might
be considerable in small build or m nothing scenario.
If weight list comes from ninja log, we can just directly read ninja log
data instead of writing weight list from ninja log outside, and read the
weight list. It doesn't spend additional time because ninja log is
loaded by default.
Bug: 271527305
Test: build with NINJA_LOG
Change-Id: Id7a4cca95898ce439d0a682f9bfd954b462f1849
This reverts commit f5c872f36b.
Revert reason: The original commit seems to have a performance regression for a number of benchmarked performance CUJs.
Bug: 283143307
Test: Benchmarking results. See attached bug.
Change-Id: Ib9463c8d30a5ba61640993424696a84e2e03040a
1) Use attribute name "default_package_metadata" instead of
"default_applicable_licenses" in packages to better describe its use.
2) Add a filegroup "default_metadata_file" in packages to search for
METADATA file in each package.
3) Include "default_metadata_file" in each package's
"default_package_metadata" attribute.
Bug: 275472038
Test: CIs
Change-Id: I645c013c39e3190fd96c4a549d39a331aced16bd
This is for use in tracking bazel exit codes from b invocations.
Bug: 279754118
Test: b build libcore:all (with prints to verify)
Change-Id: If36b5b73ff93cd15647e5c1c1f676e95137210ff
The issue where RBE was taking a long time has been fixed, so this
message isn't that important anymore, and it was overriding the last
ephemeral status message that soong output.
Test: Presubmits
Change-Id: I019f351a3307176ec7b4786d4387fde9061ba08b
This is a necessary step for running b on CI with metrics collection.
Bug: 280491043
Test: b build libcore:all
Change-Id: I37d621c38e3f891dd63e5977befa6ce6604487ea
Currently, there is little verification around allowlisted modules
actually being mixed-built. This flag would allow us to verify
that a module allowlisted is mixed-built for at least one variant.
Bug: 278910100
Test: m nothing --bazel-mode-staging --ensure-allowlist-integrity
Test: m nothing --bazel-mode-staging --ensure-allowlist-integrity
--bazel-force-enabled-modules=com.google.android.neuralnetworks (This
fails, as expected)
Test: build/soong/test/mixed_mode_test.sh
Change-Id: Icd5976f4f44f1a8caca1e5247d986642f7995f97
In most case, turning on HINT_FROM_SOONG improves build performance even
though a model is not so accurate. At least, it doesn't cause
performance regression (very likely)
Bug: 275458825
Test: m
Change-Id: Ie4ff69e68a93f6f0db3d060980657ae623816b2b
The cache directory stores files that should persist between builds.
Currently it will contain reproxy.creds, which caches the authentication
mechanism and token used to make subsequent builds faster.
Test: m nothing build takes 7s
Change-Id: Iaaa20b92630405918e5e91e09eca6c798b8fc872
Context
- from-text android.jar files are built using Metalava, and these can be
utilized in `decodeSdkDep` so that any modules that depends on APIs
can be compiled using from-text android.jars
- This change removes dependency on source java files when compiling
stub android.jar files
Implementation
- Modify java_api_library module to create system modules using the
generated android.jar
- Replace modules in decodeSdkDep to link against java_api_library
modules
- Add --build-from-text-stub flag to hide the feature behind a flag
Test: m --build-from-text-stub
Bug: 271154441
Change-Id: I104df595edc65c0006820d5ae5b15f1fb167e190
Soong analyzes the entire source tree even though not every lunch target
needs to know about every module. For example, OEM sources can be
ignored for cuttlefish products. This functionality allows blueprint to
ignore a list of undesired directories.
Bug: 269457150
Change-Id: I1eec5d7b6a268cae4c633d8d89ed485598ebca45
If this option is set, Soong generates ninja weight list including
modules in HugeModulesList in allowlists.go
Test: m --ninja_weight_source=soong
Bug: 273282046
Change-Id: Id92b7f9f9e8152c1c46ae071c5821a479cf47bce
1. extract .ninja_log as const
2. log an error during reading .ninja_log
Test: m nothing
Bug: 271527305
Change-Id: I395dd8419620bfa9fad3af23c96e5a22ca44e2fb
It has 3 options for now.
* ninja_log: uses ninja log file for data source
* evenly_distributed: pass empty list for ninja to consider every work as the same
* not_used: do not use critical path logic in ninja
In addition, I added the option in the metrics to track
Bug: 271527305
Test: m --ninja_weight_source=ninja_log|empty|not_used
Change-Id: Ib4c812c20606a34b17d3f0edb71057b477c4f90e
