Commit graph

762 commits

Author SHA1 Message Date
Steven Moreland
28d35a1f1e Revert "ninja error on missing files"
This reverts commit 10ea6214a1.

Reason for revert: b/254136231

Change-Id: I68884b0540557b9abc9e6661c1ff5dbeb7e9816d
2022-10-18 00:13:59 +00:00
Steven Moreland
10ea6214a1 ninja error on missing files
Bug: 253517226
Bug: 253694455
Change-Id: I24da473198958a0e82a5a6e10c3f3180bf5dcc41
Test: N/A
2022-10-17 21:28:53 +00:00
andusyu
838086f57a Fail the build before calling bootstrap if LOAS credentials are missing.
The build currently waits for bootstrap to return an error before
failing. This means if the user does not have LOAS credentials, they
will need to wait for boostrap to attempt to start reproxy (~10s) and
the gcert timeout time (5-10s), which will take ~20s to return an error and fail the build.

Test: - Ran m with no gcert and build failed in 2s
      - Ran RBE_use_google_prod_creds=false RBE_use_external_auth_token=false RBE_use_application_default_credentials=true m with no gcert and reproxy started

Bug: b/251425073
Change-Id: Id7fe3d6095f083643aaf0ab8f32a34b7961c61e8
2022-10-13 23:19:50 +00:00
MarkDacek
0d5bca51b6 Add bazel paths to ninja deps.
Test: m nothing
Test: m nothing --bazel-mode-dev
Change-Id: I120e62ac3f2aa066aedaecf2a5d16c79f7097c76
2022-10-12 16:30:20 +00:00
Sorin Basca
538911b686 Merge "Revert^2 "Use the toolchain from JDK 17"" 2022-10-07 05:10:30 +00:00
Sorin Basca
7e094b33b7 Revert^2 "Use the toolchain from JDK 17"
This reverts commit be3a561aac.

Reason for revert: Reland switch to JDK 17
Bug: 233029164
Test: TH

Change-Id: Ifa69e3fd50b853153a9c23756269ff12e72d7e9f
2022-10-05 08:23:17 +00:00
Spandan Das
ee08eb3c81 Merge "Create a new mode in soong_ui to generate API only BUILD files" 2022-10-05 01:37:28 +00:00
Spandan Das
5af0bd3e48 Create a new mode in soong_ui to generate API only BUILD files
The generated Bazel workspace will only contain api specific targets.
This is feasible since these targets do not have any cross dependencies
with the targets in the bp2build workspace

The advantages of a new mode are
1. Does not pollute bp2build workspace with api targets
2. Does not block api targets with the current allowlist conversion
   mechansims in bp2build
(In the future we might want to combine these two workspaces)

A Soong module type will generate a Bazel target if it implements
ApiProvider interface

Test: m apigen
Test: m nothing

Change-Id: I69c57ca6539f932e0ad554ce84a87fb7936fdba0
2022-10-04 17:32:56 +00:00
Treehugger Robot
02c1b12b1f Merge "Remove build failure with USE_BAZEL_ANALYSIS usage" 2022-10-04 15:20:21 +00:00
Ken Sun
44022c867d Merge "Revert "Use the toolchain from JDK 17"" 2022-10-03 03:17:13 +00:00
Ken Sun
be3a561aac Revert "Use the toolchain from JDK 17"
Revert submission 2164342-use-jdk17

Reason for revert: DroidMonitor-triggered revert due to breakage b/250426490

Reverted Changes:
Ib1a24e7ea:Update path for libjli.so
I3375bd4e4:Use the toolchain from JDK 17
I9183e5511:Use the toolchain from JDK 17

Change-Id: I28fc32290dd337b7d44bb98bc52956ee84f01aaa
2022-10-03 03:12:41 +00:00
Sorin Basca
1ea00ebb3e Merge "Use the toolchain from JDK 17" 2022-10-02 19:36:17 +00:00
Sorin Basca
8eac0005c6 Use the toolchain from JDK 17
Bug: 233029164
Test: m
Change-Id: I3375bd4e4186c0049490ee3ddc0f85552d25b579
2022-09-29 11:34:52 +01:00
Chris Parsons
a9bef14e8d Set bazel's HOME to an absolute path.
Bazel at HEAD has a bug which causes Bazel to crash when HOME is set to
a relative path. We circumvent this bug by specifying an absolute path.

Also added some documentation on the significance of setting this HOME
value.

Bug: 248104019
Test: Presubmits
Test: Verified `m --bazel-mode-dev nothing` continues to pass with Bazel@HEAD
Change-Id: Id05c8dc322c8cca04f3d1caf44f5b04ffa96fd3b
2022-09-28 21:12:12 +00:00
usta
6fffdd5fa8 cosmetic: mark the determination of ui build steps
Test: m libc
Bug: N/A
Change-Id: I0e7ba9cfccfe1ecf005b55a5912d1b748c294926
2022-09-26 18:58:42 +00:00
Chris Parsons
e4f148999b Remove build failure with USE_BAZEL_ANALYSIS usage
This verification was present to ensure that users previously using
USE_BAZEL_ANALYSIS=1 realized the change in usage. But this is now dead
code; USE_BAZEL_ANALYSIS=1 will be a complete no-op after this CL.

Test: USE_BAZEL_ANALYSIS=1 m nothing  (and verified this behaves as m
nothing)

Change-Id: I08f75a562c2fb3ce262646fb187f19f3d5857e40
2022-09-13 16:19:54 -04:00
Usta (Tsering) Shrestha
9ba17e8702 Merge "Drop bootstrap.ninja.d" 2022-09-07 22:47:41 +00:00
Chris Parsons
1bb58da17c Remove USE_BAZEL_ANALYSIS
Use --bazel-mode-dev instead.

Test: Treehugger
Bug: 244309479
Change-Id: I570d02651eea92375cd96efbeb8d986a9495e1ed
2022-09-07 14:58:03 -04:00
Treehugger Robot
2701ecd502 Merge "Remove USE_BAZEL and its dead code" 2022-09-01 17:32:00 +00:00
Chris Parsons
19ab9a455f Remove USE_BAZEL and its dead code
This environment variable has been out of use for some time. It used to
control Bazel-as-executor, which has been removed from Bazel for several
releases.

Bug: 243077098
Test: Treehugger
Change-Id: I4edfb5f3574c369a16547ea06ba780293c37ecb3
2022-09-01 11:33:02 -04:00
Usta (Tsering) Shrestha
f768231db1 Merge "cosmetic changes" 2022-08-30 20:13:15 +00:00
usta
5bb4a5db9a Drop bootstrap.ninja.d
This also shaves off a couple seconds from the build.

Test: `time m nothing` 2 times with and without the change
Bug: 243699430
Change-Id: I5cc5cba2b98c845ddd6ba7c3b31ad678451afe41
2022-08-29 22:23:10 +00:00
usta
fb67fd121f cosmetic changes
Test: m nothing
Bug: N/A
Change-Id: Ic3d70339c7b98bf0733dc0e946f3683fe94f5c5b
2022-08-29 19:53:45 +00:00
Chris Parsons
ef615e5841 Add --bazel-mode and --bazel-mode-dev
This allows "bazel mixed builds prod mode", in additional to reworking
the mechanism in which mixed builds dev mode is enabled.

As a followup, CI scripts will be migrated to use the new flags, as
USE_BAZEL_ANALYSIS=1 is deprecated.

Test: Manually ran --bazel-mode with an allowlist verifying that the
module alone was enabled
Test: Manually verified --bazel-mode and --bazel-mode-dev cause a build
failure

Change-Id: If0d34360e60452f428b05828f4ec7596b7cb619a
2022-08-29 10:56:04 -04:00
Jihoon Kang
9f4f8a3cc7 Copy soong variables to dist
Per aosp/2183398, out/soong/soong.variables provide some useful information for debugging purpose.
This change enables copying out/soong/soong.variables to out/dist/soong_ui/soong/soong.variables in dist mode, similar to build.ninja.gz in the same directory.
Test confirmed that the content of the out/soong/soong.variables is
equal to that of out/dist/soong_ui/soong/soong.variables.

Test: m dist $target &&
	diff out/soong/soong.variables out/dist/soong_ui/soong/soong.variables
Change-Id: I1bfa53a8e116c49fda2cc9a7252f4da0c59ce5b5
2022-08-23 21:37:48 +00:00
Jihoon Kang
f171dac47e Merge "Remove unnecessary lunch output" 2022-08-22 21:44:25 +00:00
Usta Shrestha
8dc8b0a065 SOONG_NINJA_ARGS to pass ninja args to soong_ui
Cf soong_build, allow soong_ui to support additional
ninja arguments

Test: NINJA_ARGS='-d explain' nothing
Bug: N/A
Change-Id: I42827fb817bed432251e9b0b94b15a4f6f78cb03
2022-08-19 20:41:56 +00:00
Jihoon Kang
3690485a16 Remove unnecessary lunch output
Currently, running lunch writes multiple lines including less
informative ones.
This change removes unnecessary variables and enables the user to focus
on more useful information when running the lunch command.

Bug: 204196816
Test: m
Change-Id: Ifbbe01abf03a1701158881febe5982489f96dc65
2022-08-17 21:16:44 +00:00
Colin Cross
d079e0b270 Reformat build/soong for go 1.19
Test: none
Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-08-17 10:43:13 -07:00
Treehugger Robot
c2cb917659 Merge "Move bp2build progress proto to soong." 2022-08-15 16:43:35 +00:00
Liz Kammer
15088e45a9 Move bp2build progress proto to soong.
This allows us to reuse existing infra to export to internal.

Test: b build //build/bazel/scripts/bp2build-progress:bp2build-progress
Change-Id: If4888f8f59d56ddf4e98bdb8456ca69ff7f78a49
2022-08-12 16:38:38 -04:00
Usta Shrestha
675564daa8 Cosmetic: More helpful error message
Test: various invocations of ./build/soong/soong_ui.bash
Bug: N/A
Change-Id: I7be5b84330bc7525b5633314a111e5e1fa9fc844
2022-08-12 13:43:48 -04:00
Usta Shrestha
9b3724ad74 bootstrap glob file list in one place
Test: m nothing
Bug: N/A
Change-Id: I2aa4132726b1491db767ccbe695bd98cda08ff50
2022-08-09 18:07:36 -04:00
Usta Shrestha
db46a9bbb8 Cosmetic Changes plus log errors
Test: N/A
Bug: N/A
Change-Id: I39e622a93270c922bdbea27ca79632a606431a2c
2022-08-02 20:24:57 +00:00
Yi-Yo Chiang
da44795563 Remove $PRODUCT_OUT/vendor_kernel_ramdisk on installclean
Make sure the staging directory doesn't have stale file after
installclean.

Test: m installclean && ls $ANDROID_PRODUCT_OUT
Change-Id: Idcd7a883a6cba4b7eb1eb99751f5d03cba365a12
2022-07-21 03:52:02 +00:00
Sasha Smundak
2cec8df3a8 Revert "Suppress all progress messages from Ninja if ANDROID_QUIET_BUILD is set."
Reverts commit 827aead340
The change being reverted is the result of misunderstanding of how the
status system works. It has to be reverted because it would suppress
_all_ error messages from Make. Achieving what this change purports is
is more involved and requires changing the code to separate progress message
stream from application output stream.

Test: run failing build with ANDROID_QUIET_BUILD=tree and observe errors being output
Change-Id: If9148a7fa773ae32fb0870a448e9470560e53900
2022-07-19 08:02:42 -07:00
David Srbecky
2285cfff3d Export default RBE_platform
Export it so that we have one source of truth,
which can be used by the called tools.

Test: Use the exported value in ART build.
Change-Id: Iaa45b1780ffae11e09790e193ca2bd01e2b732a8
2022-07-12 22:59:06 +01:00
Kousik Kumar
c75e1297a3 Run expconfigfetcher only when user has proper auth
It is useless to run this binary when gcert creds aren't available, so
adding a check for gcert creds before calling this binary. Also modified
metrics to capture missing gcert.

Bug: b/237938234
Change-Id: I94b5aa2d53c4e5bb1a5fdc913a33ef5bae965d32
2022-07-07 02:20:51 +00:00
Kousik Kumar
7bc781949a Add a broad warning about missing LOAS creds
This is to generally warn Google devs about lack of LOAS creds.

Given the switch to RBE, lack of LOAS creds will start resulting in build
failures, so adding a warning message ahead of time to all builds. Will
convert this to a build failure after a week.
We need LOAS creds to be able to even fetch the CDPush config file to
determine whether to use RBE in a build a not, which is why I'm making
this a broad error message across builds.

Test: ran "m nothing" build with / without gcert and with / without
"stubby" in $PATH and confirmed appropriate error message.
Bug: b/235985591

Change-Id: I15cbaf372b0a7b79f868a06c7d5ede19b49ae687
2022-06-29 17:48:52 +00:00
Kousik Kumar
595fb1c87b Change experiments config fetch log to be a verbose log
This log is not super useful to be honest, so avoid printing it to
stdout of the build.

Bug: b/236709761
Change-Id: Ic96130a285160b386df3afc269dd5ef97c7965bf
2022-06-24 16:49:55 +00:00
Dan Willemsen
e2cdecf7ba Strip ANSI codes for messages on simple terminals
We had been doing this for actions, but not messages.

Test: m nothing && TERM=dumb m nothing
Bug: 235414673
Change-Id: I5c6c009490abe1d02a4ae2272bd1831bd7ca9f7f
2022-06-09 03:40:14 +00:00
Kousik Kumar
4c180ad7de Cleanup RBE logs directory
RBE logs directory is currently messy:
1. We use RBE_output_dir variable to specify where rbe_metrics.txt /
   rbe_metrics.pb file should go to.
2. We use proxy_log_dir to specify where *.rpl / *.rpi (detailed per
   action info log file) should go to.
3. We use RBE_log_dir to specify where reproxy.* / bootstrap.* log files
   should go to.

Ideally, all RBE related logs should go to one single directory. In this
CL, I'm creating a temporary log directory under out/soong/.temp/rbe/
where all RBE related log files per build would go to. The log dir
prefix is also being set to the same prefix as the socket address file.

Test: Ran a sample build with `m libc` and ensured that logs are getting
cleared across rebuilds and that `rbe_metrics.pb` file is properly
generated and being copied to the right location (i.e., from
out/soong/.temp/rbe/<rand> to out/ dir)

Bug: b/233382420
Change-Id: I46bd38d50419cb9e54e8202d24222979e47ff5ca
2022-06-08 11:50:33 -04:00
Treehugger Robot
7d5cdb69ed Merge "Revert "Add a warning message about missing gcert at the end of the build"" 2022-06-08 04:45:23 +00:00
Colin Cross
d2ee3a6afa Merge "Remove references to *_HOST_DALVIK_*" 2022-05-12 15:44:20 +00:00
MarkDacek
5b08fe1e9e Add Go proto for metrics. The current output is different from head.
Test: N/A

Change-Id: I3230bcec0136e4859118edee2e5e36ff502b2d0e
2022-05-11 21:55:53 +00:00
Lukacs T. Berki
2388f64e76 Add comments for path_interposer.
Test: Presubmits.
Change-Id: I22c08f282019050da1198cce1f92f5d825ee649f
2022-05-06 12:42:05 +02:00
Kousik Kumar
2c8115521a Revert "Add a warning message about missing gcert at the end of the build"
This reverts commit b018b83664.

Reason for revert: Builds will start failing now since the gcert change is about to go in.

Change-Id: I81d36ce6bf4f7a9bc120b34d411f2cb4ec64e5eb
2022-05-04 20:06:00 +00:00
Lukács T. Berki
28ba9c472f Merge "Add a test for correctness of C++ compilation." 2022-05-04 12:06:38 +00:00
Lukacs T. Berki
e3487c8848 Add a test for correctness of C++ compilation.
This required the following:

- Adding Platform_base_sdk_extension_version to default soong.variables
- Teaching the symlink tree creation code to understand symlinks
- Making finder.go follow symlinks when requested

Adding yet another knob is unfortunate, but I can't allow that
unconditionally because the Android code base contains a number of
symlinks giving rise to infinite directory trees because they point back
to their parent and this seemed preferable to adding complicated logic
like "follow symlink but if only its fully resolved version does not
point under the source tree".

I could be convinced about the latter, though.

Test: Presubmits.
Change-Id: I453f6b7e5334771f5832c700db00f9d24ed1d82f
2022-05-04 09:12:01 +02:00
MarkDacek
ff851b83b6 Log information for Mixed Builds modules.
Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw
Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0

Change-Id: I88780c7cc52a189a72216c5e2e499c96574b3731
2022-05-03 18:11:32 +00:00