Commit graph

260 commits

Author SHA1 Message Date
Joe Onorato
e5ed34746e Add new soongdbg command and a big json file full of soong debugging info.
In order to use soongdbg, you must run analysis with GENERATE_SOONG_DEBUG=true set
in the environment.

Test: GENERATE_SOONG_DEBUG=true m nothing ; soongdbg ...
Change-Id: If43676fe2784f05cd87c0ecb4a46ab676b91023f
2024-02-03 14:44:11 -08:00
Colin Cross
8d411ff8f8 Remove bp2build and bazel from soong_ui
Bug: 315353489
Test: m blueprint_tests
Change-Id: I9df31b18caaae24e3cf2994e56bb90b50523f11e
2023-12-07 17:14:27 -08:00
Sorin Basca
0760c89882 Support experimental building with OpenJDK 21
Use EXPERIMENTAL_USE_OPENJDK21_TOOLCHAIN=true to build with OpenJDK 21
while still targeting java language version 17.

Bug: 313924276
Test: m EXPERIMENTAL_USE_OPENJDK21_TOOLCHAIN=true
Change-Id: Idc892bb7519e597f1e280ca0765c1a281bb29955
2023-11-29 19:13:55 +00:00
Jihoon Kang
2a929ad250 Revert^2 "Default from-text stub generation in build"
This change defaults Java stubs to be generated from API text files
during build. Using the `--build-from-source-stubs` flag, users can
toggle between the feature.

This change relands aosp/2617457.

Test: m nothing && verify ninja path exists between android_stubs_current and android_stubs_current.from-text, and does not exist between android_stubs_current.from-source, m nothing --build-from-source-stub && verify the opposite
Bug: 301522358
Change-Id: Ic129dafc3231e71470af982374739930947b69cb
2023-11-08 04:25:52 +00:00
LaMont Jones
9a9128610b Readd support for PRODUCT_RELEASE_CONFIG_MAPS
This allows product config to determine what build flags are present in
a product's build, as well as their value.

As product config moves to a declarative specification, this can be
handled with less unnecessary work.

Bug: b/302593603, b/308849337, b/309477343
Test: manual
Change-Id: Ic0551990864bba6d03b6bf49fcfd4ae2523d9c6d
2023-11-07 18:39:40 +00:00
Colin Cross
216ed6c918 Merge "Revert "Add support for PRODUCT_RELEASE_CONFIG_MAPS"" into main 2023-11-06 22:02:50 +00:00
Colin Cross
6a4fe68cc5 Revert "Add support for PRODUCT_RELEASE_CONFIG_MAPS"
Revert submission 2815850-product-flags

Reason for revert: b/309477343

Reverted changes: /q/submissionid:2815850-product-flags

Bug: 302593603
Bug: 309477343
Change-Id: I5b5a5ade1d85195c43377ed7639deba6eef86e23
2023-11-06 21:32:03 +00:00
LaMont Jones
4c01bb49da Merge "Add support for PRODUCT_RELEASE_CONFIG_MAPS" into main 2023-11-06 14:08:22 +00:00
LaMont Jones
127127b608 Add support for PRODUCT_RELEASE_CONFIG_MAPS
This allows product config to determine what build flags are present in
a product's build, as well as their value.

As product config moves to a declarative specification, this can be
handled with less unnecessary work.

Bug: b/302593603
Bug: b/308849337
Test: manual
Change-Id: If57622059bb7d1c33df7498321db621c0c2d30c0
2023-11-03 19:19:59 +00:00
Colin Cross
f2c51919ad Merge "Remove obsolete java version environment variables" into main 2023-11-02 17:05:06 +00:00
Treehugger Robot
87706098b0 Merge "Revert "Add support for PRODUCT_RELEASE_CONFIG_MAPS"" into main 2023-11-02 08:06:39 +00:00
Colin Cross
f77b2cb534 Revert "Add support for PRODUCT_RELEASE_CONFIG_MAPS"
Revert submission 2787001-product-build-flags

Reason for revert: Possible cause of b/308849337

Reverted changes: /q/submissionid:2787001-product-build-flags

Bug: 302593603
Bug: 308849337
Change-Id: Id3f27eca8d39e854dee232f072a8dc6c561bac63
2023-11-02 00:13:30 +00:00
Colin Cross
f059b2f2fc Remove obsolete java version environment variables
jdk9 and jdk11 are no longer used, remove ANDROID_JAVA9_HOME and
ANDROID_JAVA11_HOME.

Bug: 221270882
Test: treehugger
Change-Id: Iad566bff21055c5e2acd62642ccd579507e21fef
2023-11-01 20:58:30 +00:00
Colin Cross
2650f35faa Merge "Use correct path for soong_build_metrics.pb when dist is enabled" into main 2023-11-01 16:54:14 +00:00
LaMont Jones
c416ddd1cc Merge "Add support for PRODUCT_RELEASE_CONFIG_MAPS" into main 2023-11-01 16:01:08 +00:00
Colin Cross
b67b061baa Use correct path for soong_build_metrics.pb when dist is enabled
When dist is enabled soong_build_metrics.pb is written to $DIST_DIR/logs,
not $OUT_DIR.

Test: m dist nothing, examine out/dist/logs/bulid.trace.gz
Change-Id: Ia41de6c838ecbcf0acca1e7ec7e68ba24dce2b32
2023-10-31 23:52:38 +00:00
LaMont Jones
772e21ba27 Add support for PRODUCT_RELEASE_CONFIG_MAPS
This allows product config to determine what build flags are present in
a product's build, as well as their value.

As product config moves to a declarative specification, this can be
handled with less unnecessary work.

Bug: b/302593603
Test: manual
Change-Id: If6ae19e3da8471eaba8d6e048769f8ac6b244155
2023-10-31 18:47:18 +00:00
Colin Cross
ab766b40e8 Merge "Copy soong_build metrics into build.trace.gz" into main 2023-10-30 19:40:31 +00:00
Colin Cross
aa9a273b40 Copy soong_build metrics into build.trace.gz
Make soong_ui read soong_build_metrics.bp to extract the event timings
and propagate them to Tracer, which will put them in build.trace.gz.
This provides much better visibility into what parts of the build are
contributing to the overly large analysis time.

Test: m nothing and examine build.trace.gz
Change-Id: I473727f1ec044b0d973f2cb4e3eaca96bfca94f6
2023-10-27 19:36:47 -07:00
MarkDacek
39825ead17 Remove Bazel metrics/finalize-bazel-metrics.
Test: m nothing
Change-Id: Idd4610aca77f0a5deeadc3b895869879ded6c16e
2023-10-26 20:14:36 +00:00
Chris Parsons
bf8ab858da disable bazel by default
Test: m nothing, verified bp2build was not run
Test: presubmits
Change-Id: I7476194f8424fa4f199fb94e1cbd0b3c84339fd9
2023-10-26 17:22:27 +00:00
Colin Cross
106d6ef459 Disable Soong tests by default on Linux
Disable running the Soong tests when building on Linux.  The tests
are still enabled when running on Darwin to ensure we have test on
Darwin in CI.  Checkbuilds still depend on the tests, which will
provide test coverage in CI on Linux.

Bug: 269296618
Test: aninja -t path checkbuild out/host/linux-x86/bin/go/soong-java/test/test.passed
Test: m nothing
Change-Id: Ib8359ff6d2a92eeb2ac9a9e95ecd65abc4e40061
2023-10-24 11:00:59 -07:00
Sebastian Pickl
1c4188c9da Revert "Default from-text stub generation in build"
This reverts commit 182b56b870.

Reason for revert: prime suspect for breaking boot tests b/307495247, b/307411752

Bug:307495247
Change-Id: Iea05703b767d2699ca3cf69377eb44b1d21697ad
2023-10-24 11:20:06 +00:00
Jihoon Kang
182b56b870 Default from-text stub generation in build
This change defaults Java stubs to be generated from API text files
during build. Using the `--build-from-source-stubs` flag, users can
toggle between the feature.

Test: m nothing && verify ninja path exists between android_stubs_current and android_stubs_current.from-text, and does not exist between android_stubs_current.from-source, m nothing --build-from-source-stub && verify the opposite
Bug: 274805756
Change-Id: I28834f92c1b1311e3fe0a71a6ea9e8ec2e278d7e
2023-10-20 23:11:19 +00:00
Cole Faust
06ea531ab3 gofmt soong
Test: Presubmits
Change-Id: Ia76c35ba51685eca29df51738000eacd3f43ce20
2023-10-18 17:42:59 -07:00
Jihoon Kang
e100a884e3 Merge changes from topic "enable-hidden-api" into main
* changes:
  Enable hiddenapi check for from-text stub build
  Utilize module lib and test api superset module in hiddenapi
2023-10-12 17:33:35 +00:00
Jihoon Kang
dff2aea212 Enable hiddenapi check for from-text stub build
This change removes the UNSAFE_DISABLE_HIDDENAPI_FLAGS env var setting
during from-text stub build, and enables hiddenapi list during from-text
stub build.

Test: ENABLE_HIDDENAPI_FLAGS=true m --build-from-text-stub
Bug: 275570206
Change-Id: Ic8cd60e376b978ccc658ff43a44d082eb2759fa5
2023-10-11 17:46:29 +00:00
Ramy Medhat
bc06176350 Set download_tmp_dir explicitly to be a directory under the output directory.
This directory is where outputs are download first before being moved
to their final destination to ensure atomicity.

Change-Id: Ic224bf21c07566de00d292d02f1c0f7f727dcb08
2023-10-10 22:33:46 +00:00
Cole Faust
583dfb426f Generate build_number.txt only once
build_number.txt was generated in runKati, and runKati is run multiple
times throughout the build. If a build number is not set, then one will
be generated using the current timestamp, meaning that multiple
different build numbers will be used in different phases of the build.

Instead, generate build_number.txt during SetupOutDir(), so that we
only have 1 build number.

This is a resubmission with a change to ensure out/soong is created
before writing the build number file.

Bug: 297269187
Test: edit a bzl file to force bazel to rerun, change the code so that use_fixed_timestamp=true during partition builds, m installclean, m bazel_sandwich, and observe that the bazel/make partitions are byte-for-byte identical
Change-Id: I47abcca166c701bb66a6a7731aecad5818279244
2023-09-28 17:56:43 -07:00
Roy Luo
f852b72a22 Revert "Generate build_number.txt only once"
This reverts commit ab6e524709.

Reason for revert: DroidMonitor-triggered revert due to breakage https://android-build.corp.google.com/builds/quarterdeck?branch=aosp-main&target=ndk&lkgb=10879153&lkbb=10879224&fkbb=10879224, bug b/302608661.

Change-Id: I07d47a92ea9a9f5d329e813859316decfd75cde1
BUG: 302608661
2023-09-29 00:28:32 +00:00
Cole Faust
ab6e524709 Generate build_number.txt only once
build_number.txt was generated in runKati, and runKati is run multiple
times throughout the build. If a build number is not set, then one will
be generated using the current timestamp, meaning that multiple
different build numbers will be used in different phases of the build.

Instead, generate build_number.txt during SetupOutDir(), so that we
only have 1 build number.

Bug: 297269187
Test: edit a bzl file to force bazel to rerun, change the code so that use_fixed_timestamp=true during partition builds, m installclean, m bazel_sandwich, and observe that the bazel/make partitions are byte-for-byte identical
Change-Id: I5bf3bb6c78e7529749ca9275f67db3f2f9e66af2
2023-09-28 14:11:43 -07:00
Chris Parsons
73f411b72c Delete ApiBp2build soong_build mode
This feature is obsolete.

This makes a large number of codepaths "dead code" (such as
module-specific implementations of ApiBp2build functionality). These
will be deleted in a followup CL.

Bug: 284029211
Test: Presubmits
Change-Id: Ib53b99f1fe8c24380d219caf44e9bb3b96724fa0
2023-09-05 17:09:51 -07:00
Treehugger Robot
3537d8028a Merge "Disable zipbomb detection inside the build" into main 2023-08-09 22:04:03 +00:00
Kousik Kumar
97e9b0cb8e Merge "Revert "Revert "Remove env config fetcher code""" into main 2023-08-08 15:59:56 +00:00
Kousik Kumar
8036b163ef Revert "Revert "Remove env config fetcher code""
This reverts commit d93c67f64c.

Reason for revert: USE_RBE has again been made the default now, so we should no longer need this code.

Change-Id: I4163c61eed90163e763c29e07dd6edfc9c41b9b9
2023-08-04 19:46:16 +00:00
Colin Cross
fe5ed4d6e2 Disable zipbomb detection inside the build
Some versions of Fedora include patches to unzip to enable zipbomb
detection that incorrectly handle zip64 and data descriptors and fail
on large zip files produced by soong_zip.  Disable zipbomb detection
inside the build.

Reported upstream in https://bugzilla.redhat.com/show_bug.cgi?id=2227130.

Bug: 286885495
Test: builds
Change-Id: I8e4438720bbb17a073ff3b5535f01c2827485838
2023-07-28 09:31:03 -07:00
Kousik Kumar
6d1e348cc3 Disable RBE env variable when we cannot support RBE
Enabling the RBE variable can cause issues around pool parallelism when
we set the env variable, but don' run the build with RBE due to auth issues.
So, I am explicitly unsetting the variables in those cases.

Bug: b/292224253
Tested: https://b.corp.google.com/issues/292224253#comment26
Change-Id: I19718b4ee6c058ba1b11d3df260421bbf8c9567e
2023-07-24 17:06:42 +00:00
Jingwen Chen
d7ccde11d6 Don't use RBE for bp2build/queryview/api_bp2build/json_module_graph,
even with USE_RBE=true.

These are alternate modes of soong_build that don't use rbe/reclient.

This lets us to remove USE_RBE=false from the bp2build run.

Test: presubmits
Change-Id: I439f9cf7e92ec85ca56baec5f62a83ee49b510d4
2023-06-30 09:32:32 +00:00
Kousik Kumar
67ad434ffc Ignore USE_RBE when on glinux laptops
When there's no stubby and we are asked to use google prod creds, that's invalid configuration, so don't use RBE

Bug: b/283828386
Change-Id: I1564b4f70e46fb90c87a0432c46616caa1614aac
2023-06-23 18:43:54 +00:00
Andus Yu
f3da8ee95b Merge "Do not check ssh credentials when running gcertstatus" 2023-06-22 19:38:33 +00:00
andusyu
0b3dc03f9d Do not check ssh credentials when running gcertstatus
reproxy only requires LOAS credentials for authentication. Append the
-nocheck_ssh option to gcertstatus so we don't check ssh credentials.

Test: RBE successfully started after running gcertdestroy --nodestroy_loas2
Bug: b/287297140
Change-Id: I413931b10d8e8d9ae2f8acd7ebfe37d8b3bba455
2023-06-21 17:30:21 -04:00
MarkDacek
396491eee4 Add bes-id to soong. Also renames the proto field.
This is for integration with BES for bazel test metrics.
Bug: 287102416
Test: b build libcore:all --invocation_id=`uuidgen`
Test: build/bazel/scripts/analyze_build (to verify it's set)

Change-Id: I541b2d65bfa85718fc916582e0540384ae3810d9
2023-06-21 14:48:17 +00:00
Treehugger Robot
d34e4c72f3 Merge "Increase config fetcher timeout to 20s" 2023-06-15 18:37:53 +00:00
Chris Parsons
21f80277c3 Remove bazel dev mode
This mode is no longer in use, and is dead code.

Fixes: 282809863
Test: Presubmits
Change-Id: I45ab7d7c9dee153f1cbc66262c2cf9f1e9487a28
2023-06-15 14:14:19 +00:00
Kiyoung Kim
eaa55a8ce1 Rename soong.environment.used with product name
From the previous work on renaming build.ninja, it was found that there
are extra dependencies from build.ninja which can be varied by
TARGET_PRODUCT : which is soong.environment.used.<tag>. This change
renames soong.environment.used to have target product between 'used' and
'<tag>' if available.

Bug: 277029044
Test: Test confirmed that build.ninja is not being re-generated
Change-Id: I987b6bd1a8b4f06dac52537e4178d8556251d254
2023-06-08 01:07:47 +00:00
Kousik Kumar
a3a2af6b64 Increase config fetcher timeout to 20s
This is to make sure timeout doesn't show up for users from APAC / EMEA regions

Bug: b/283828386
Change-Id: Ie04352553494634070e4c5e83a68b3004fbdf88c
2023-06-06 17:29:11 -04:00
Kousik Kumar
ac985b9c59 Merge "Revert "Remove env config fetcher code"" 2023-06-06 21:26:36 +00:00
Jeongik Cha
a87506f5e8 Use HINT_FROM_SOONG if ninja_log doesn't exist
In non-incremental build, there is no ninja_log. For this case, use
HINT_FROM_SOONG as an alternative solution.

Bug: 273947040
Test: 1.m after removing out/.ninja_log
      2.check if non-incremental CI build uses HINT_FROM_SOOONG
      3.check if incremental CI build uses NINJA_LOG
      4.check if there is no regression in CUJ
Change-Id: I00cd216df096cb2288eeab233729acefb0d1b73c
2023-06-02 15:50:02 +01:00
Treehugger Robot
9cae81483a Merge "implement USE_BAZEL_VERSION for m" 2023-06-01 22:06:17 +00:00
Chris Parsons
c83398ffb4 implement USE_BAZEL_VERSION for m
This allows USE_BAZEL_VERSION to be set for m builds, which will use
Bazelisk for any Bazel invocations during those builds.

This should be used only for manual debugging, typically to either test
new Bazel features, verify Bazel compatibility with Android, or culprit
find new Bazel breakages.

Test: Manually run builds with USE_BAZEL_VERSION, toggled off and on to
ensure the build was rerun. Tested with a broken commit, a working
commit, and 'last_green' special term

Change-Id: I8b475dca5c8d4bd849ee3724a8c3aca9b631bcb8
2023-06-01 17:42:18 +00:00