The build currently waits for bootstrap to return an error before
failing. This means if the user does not have LOAS credentials, they
will need to wait for boostrap to attempt to start reproxy (~10s) and
the gcert timeout time (5-10s), which will take ~20s to return an error and fail the build.
Test: - Ran m with no gcert and build failed in 2s
- Ran RBE_use_google_prod_creds=false RBE_use_external_auth_token=false RBE_use_application_default_credentials=true m with no gcert and reproxy started
Bug: b/251425073
Change-Id: Id7fe3d6095f083643aaf0ab8f32a34b7961c61e8
The generated Bazel workspace will only contain api specific targets.
This is feasible since these targets do not have any cross dependencies
with the targets in the bp2build workspace
The advantages of a new mode are
1. Does not pollute bp2build workspace with api targets
2. Does not block api targets with the current allowlist conversion
mechansims in bp2build
(In the future we might want to combine these two workspaces)
A Soong module type will generate a Bazel target if it implements
ApiProvider interface
Test: m apigen
Test: m nothing
Change-Id: I69c57ca6539f932e0ad554ce84a87fb7936fdba0
Revert submission 2164342-use-jdk17
Reason for revert: DroidMonitor-triggered revert due to breakage b/250426490
Reverted Changes:
Ib1a24e7ea:Update path for libjli.so
I3375bd4e4:Use the toolchain from JDK 17
I9183e5511:Use the toolchain from JDK 17
Change-Id: I28fc32290dd337b7d44bb98bc52956ee84f01aaa
Bazel at HEAD has a bug which causes Bazel to crash when HOME is set to
a relative path. We circumvent this bug by specifying an absolute path.
Also added some documentation on the significance of setting this HOME
value.
Bug: 248104019
Test: Presubmits
Test: Verified `m --bazel-mode-dev nothing` continues to pass with Bazel@HEAD
Change-Id: Id05c8dc322c8cca04f3d1caf44f5b04ffa96fd3b
This verification was present to ensure that users previously using
USE_BAZEL_ANALYSIS=1 realized the change in usage. But this is now dead
code; USE_BAZEL_ANALYSIS=1 will be a complete no-op after this CL.
Test: USE_BAZEL_ANALYSIS=1 m nothing (and verified this behaves as m
nothing)
Change-Id: I08f75a562c2fb3ce262646fb187f19f3d5857e40
This environment variable has been out of use for some time. It used to
control Bazel-as-executor, which has been removed from Bazel for several
releases.
Bug: 243077098
Test: Treehugger
Change-Id: I4edfb5f3574c369a16547ea06ba780293c37ecb3
This also shaves off a couple seconds from the build.
Test: `time m nothing` 2 times with and without the change
Bug: 243699430
Change-Id: I5cc5cba2b98c845ddd6ba7c3b31ad678451afe41
This allows "bazel mixed builds prod mode", in additional to reworking
the mechanism in which mixed builds dev mode is enabled.
As a followup, CI scripts will be migrated to use the new flags, as
USE_BAZEL_ANALYSIS=1 is deprecated.
Test: Manually ran --bazel-mode with an allowlist verifying that the
module alone was enabled
Test: Manually verified --bazel-mode and --bazel-mode-dev cause a build
failure
Change-Id: If0d34360e60452f428b05828f4ec7596b7cb619a
Per aosp/2183398, out/soong/soong.variables provide some useful information for debugging purpose.
This change enables copying out/soong/soong.variables to out/dist/soong_ui/soong/soong.variables in dist mode, similar to build.ninja.gz in the same directory.
Test confirmed that the content of the out/soong/soong.variables is
equal to that of out/dist/soong_ui/soong/soong.variables.
Test: m dist $target &&
diff out/soong/soong.variables out/dist/soong_ui/soong/soong.variables
Change-Id: I1bfa53a8e116c49fda2cc9a7252f4da0c59ce5b5
Currently, running lunch writes multiple lines including less
informative ones.
This change removes unnecessary variables and enables the user to focus
on more useful information when running the lunch command.
Bug: 204196816
Test: m
Change-Id: Ifbbe01abf03a1701158881febe5982489f96dc65
Make sure the staging directory doesn't have stale file after
installclean.
Test: m installclean && ls $ANDROID_PRODUCT_OUT
Change-Id: Idcd7a883a6cba4b7eb1eb99751f5d03cba365a12
Export it so that we have one source of truth,
which can be used by the called tools.
Test: Use the exported value in ART build.
Change-Id: Iaa45b1780ffae11e09790e193ca2bd01e2b732a8
It is useless to run this binary when gcert creds aren't available, so
adding a check for gcert creds before calling this binary. Also modified
metrics to capture missing gcert.
Bug: b/237938234
Change-Id: I94b5aa2d53c4e5bb1a5fdc913a33ef5bae965d32
This is to generally warn Google devs about lack of LOAS creds.
Given the switch to RBE, lack of LOAS creds will start resulting in build
failures, so adding a warning message ahead of time to all builds. Will
convert this to a build failure after a week.
We need LOAS creds to be able to even fetch the CDPush config file to
determine whether to use RBE in a build a not, which is why I'm making
this a broad error message across builds.
Test: ran "m nothing" build with / without gcert and with / without
"stubby" in $PATH and confirmed appropriate error message.
Bug: b/235985591
Change-Id: I15cbaf372b0a7b79f868a06c7d5ede19b49ae687
This log is not super useful to be honest, so avoid printing it to
stdout of the build.
Bug: b/236709761
Change-Id: Ic96130a285160b386df3afc269dd5ef97c7965bf
RBE logs directory is currently messy:
1. We use RBE_output_dir variable to specify where rbe_metrics.txt /
rbe_metrics.pb file should go to.
2. We use proxy_log_dir to specify where *.rpl / *.rpi (detailed per
action info log file) should go to.
3. We use RBE_log_dir to specify where reproxy.* / bootstrap.* log files
should go to.
Ideally, all RBE related logs should go to one single directory. In this
CL, I'm creating a temporary log directory under out/soong/.temp/rbe/
where all RBE related log files per build would go to. The log dir
prefix is also being set to the same prefix as the socket address file.
Test: Ran a sample build with `m libc` and ensured that logs are getting
cleared across rebuilds and that `rbe_metrics.pb` file is properly
generated and being copied to the right location (i.e., from
out/soong/.temp/rbe/<rand> to out/ dir)
Bug: b/233382420
Change-Id: I46bd38d50419cb9e54e8202d24222979e47ff5ca
This reverts commit b018b83664.
Reason for revert: Builds will start failing now since the gcert change is about to go in.
Change-Id: I81d36ce6bf4f7a9bc120b34d411f2cb4ec64e5eb
This required the following:
- Adding Platform_base_sdk_extension_version to default soong.variables
- Teaching the symlink tree creation code to understand symlinks
- Making finder.go follow symlinks when requested
Adding yet another knob is unfortunate, but I can't allow that
unconditionally because the Android code base contains a number of
symlinks giving rise to infinite directory trees because they point back
to their parent and this seemed preferable to adding complicated logic
like "follow symlink but if only its fully resolved version does not
point under the source tree".
I could be convinced about the latter, though.
Test: Presubmits.
Change-Id: I453f6b7e5334771f5832c700db00f9d24ed1d82f
Now that we are ready to rollout gcert, adding a warning that builds
will fail if gcert doesn't exist. Doing this only for builds with
ANDROID_BUILD_ENVIRONMENT_CONFIG=googler.
Test: Ran a build without gcert and USE_RBE=true and it printed the
warning.
Bug: b/229743321
Change-Id: I616a4a534ab277873ac7e19baecb78620f07854c
These can take a minute or more of build time, and currently hold up
other processing. We could move to a parallel compression scheme to
speed them up, but it's simpler just to background them and continue the
build, waiting to make sure they've finished before exiting.
Testing this on git_master/flame-userdebug:
This brings a local no-op `m nothing dist` build from 2m1s to 1m39s,
even though most of that time is still spent waiting on these (since
there is very little else happening in a `nothing` build when Soong and
Kati don't need to run).
Running `touch art/Android.mk; m nothing dist` (so the above, but with
Kati) goes from 3m39s to 2m2s. We spent 9 seconds at the end waiting for
these to finish.
CI cases that almost always run Kati plus some number of other actions
are likely to completely hide the time taken to dist these files.
Bug: 229932999
Test: Check $DIST_DIR, files still exist
Change-Id: I1fb78d7c4d5103d72b5d71d9277dea0d452f8968
If we didn't need to run soong_build during the current run, we still
try to load the soong metrics. But in the case of `dist`, that's in a
directory that is not guaranteed to persist between runs.
Make loading the soong metrics optional if the file does not exist. Also
fixes a variable shadowing issue that meant we never passed it into
ctx.Metrics.
Test: treehugger
Change-Id: Ic836282f4d13e91daa0e7241ad7c488de3293d8b
This allows us to track how much time is spent in Clang.
Test: TOOLCHAIN_RUSAGE_OUTPUT=/tmp/rusage.txt m
Change-Id: Ib2961904f363bc59bd9d928bb055a96740cb9f17
Tracks the time to call the expconfigfetcher and whether it updated the
config file or not.
Test:
Ran a build with a "regular" user and verified the metrics indicated
NO_CONFIG and no config file was created.
Ran a build with an "experiment" user and verified the metrics indicated
CONFIG and a googler.json config file was created.
Bug: http://b/227186589
Change-Id: I030e9497c369459ae98c0c4f7765aef6de4c601f
Currently, mk2rbc/rbcrun aren't built in certain phases
of the build, which causes the Starlark product configuration
to fail when using get_build_var if a regular build hasn't
been run before.
Bug: 226677850
Test: Manually
Change-Id: If85608eef63be25fddc4d916c82247c13eb0195f
Metrics are added to a materialized protobuf soong_build_metrics.pb
Recorded events have scoped names and recorded runtimes.
For example, mixedbuilds includes the following events:
mixedbuilds.prepare
mixedbuilds.analyze.write_files
mixedbuild (this is a total of all mixedbuild events)
Test: Manually verified materialized protos for mixedbuild, bp2build,
and legacy build.
Change-Id: I2e72d4502c0373bee0279bb1320b2e4dd586129c
