Commit graph

997 commits

Author SHA1 Message Date
Cole Faust
16771ec9c2 Merge "Add direct deps on glob result files" into main 2024-01-03 06:14:23 +00:00
Cole Faust
8c0b11ef79 Add direct deps on glob result files
n2 (a reimplemntation of ninja) adds a check that depfiles only add
dependencies on source files, not generated files, because generated
files are not guaranteed to exist before an action runs if there isn't
an explicit dependency between the two.

This is the case for our glob result files. However, we can rework the
build so that they're directly depended on instead of using a depfile.

Bug: 318434287
Test: rm -rf out && m nothing && m nothing
Change-Id: I513fa5536136e6bf19c347710f0722d696199612
2024-01-02 17:02:52 -08:00
Colin Cross
a3759f98a6 Merge "Write raw files to disk instead of the ninja file" into main 2024-01-02 19:44:52 +00:00
Colin Cross
31a674571e Write raw files to disk instead of the ninja file
Writing raw files as rules in the ninja file unnecessarily bloats
the ninja file.  Write files immediately to disk instead to files
based on the hash of the contents, and then emit ninja rules to
copy the files into place during the build.  Delete obsolete files
in a singleton at the end of analysis.

Bug: 306029038
Test: Run: m libc_musl_version.h
           touch build/soong/Android.bp
           m libc_musl_version.h
      libc_musl_version.h/genrule.sbox.textproto is not recopied.
Test: Run: lunch aosp_cf_x86_64_phone-userdebug
           m libc_musl_version.h
	   lunch aosp_x86_64-userdebug
	   m libc_musl_version.h
	   lunch aosp_cf_x86_64_phone-userdebug
	   m libc_musl_version.h
      libc_musl_version.h/genrule.sbox.textproto is recopied but restat prevents rerunning the genrule.
Test: Run: touch out/soong/raw-aosp_cf_x86_64_phone/00/foo
           touch build/soong/Android.bp
	   m nothing
      out/soong/raw-aosp_cf_x86_64_phone/00/foo is removed.
Change-Id: I172869c4d49565504794c051e2e8c1f7cf46486e
2023-12-19 16:33:46 -08:00
Treehugger Robot
2fd4783584 Merge "Revert "Show ETA only if it is smart status"" into main 2023-12-19 15:49:57 +00:00
Jeongik Cha
b6d5ff505d Revert "Show ETA only if it is smart status"
Revert submission 2853087

Reason for revert: not necessary

Reverted changes: /q/submissionid:2853087

Change-Id: I747948ce5bec656851bf9a50e136bf7e3aae8df6
2023-12-19 05:44:26 +00:00
Jeongik Cha
d74c914e36 Fix typo in log in ninja.go
Test: n/a
Bug: 292304818

Change-Id: I6dcb6bb3e84e15292cbd3acc613c744e3ec039f8
2023-12-11 11:41:16 +00:00
Colin Cross
8d411ff8f8 Remove bp2build and bazel from soong_ui
Bug: 315353489
Test: m blueprint_tests
Change-Id: I9df31b18caaae24e3cf2994e56bb90b50523f11e
2023-12-07 17:14:27 -08:00
Colin Cross
31783628ff Merge "Move startRBE error handling to the main goroutine" into main 2023-12-05 05:35:38 +00:00
Jeongik Cha
340a2b8d3f Merge changes Ib785bd1c,Icd05ee3a into main
* changes:
  Show ETA only if it is smart status
  Remove not-used field in smart_status
2023-11-30 08:20:31 +00:00
Jeongik Cha
0ba68e40bb Show ETA only if it is smart status
To prevent breakage in parsing logic in build bot

Test: m
Bug: 313981966
Change-Id: Ib785bd1cf0fec92d9a4cf20ab8a33ae9590282a9
2023-11-30 00:06:26 +00:00
Jeongik Cha
adbdbc33f8 Remove not-used field in smart_status
Bug: 292304818
Test: m nothing
Change-Id: Icd05ee3af49063a01447fef9e9c3daf705f0cc4b
2023-11-30 08:58:26 +09:00
Treehugger Robot
8f7ee30d21 Merge "Support experimental building with OpenJDK 21" into main 2023-11-29 20:25:25 +00:00
Sorin Basca
0760c89882 Support experimental building with OpenJDK 21
Use EXPERIMENTAL_USE_OPENJDK21_TOOLCHAIN=true to build with OpenJDK 21
while still targeting java language version 17.

Bug: 313924276
Test: m EXPERIMENTAL_USE_OPENJDK21_TOOLCHAIN=true
Change-Id: Idc892bb7519e597f1e280ca0765c1a281bb29955
2023-11-29 19:13:55 +00:00
Jeongik Cha
3622b3464b Show estimated build end time during build
Ninja delivers estimated total build time and critical path time from
previous ninja log.

Bug: 292304818
Test: check if ETA shows
Change-Id: I014caaa3e8222a53c8822616b2ae31b88a3b0310
2023-11-29 09:57:35 +09:00
Colin Cross
e7151f971e Move startRBE error handling to the main goroutine
When called from the startRBE goroutine, the panic ctx.Fatalf triggered
by is unhandled and causes the process to exit without cleaning up the
terminal.  Move most of the quick checks to a checkRBERequirements
function called on the main goroutine so they can be reported
immediately before running Soong and Kati, and recover any panics
from startRBE and report them after waiting on rbeCh.

Bug: 293501187
Test: gcertdestroy && m nothing
Change-Id: I62c84c93ab0a7f0e4f2ab2cc64b22e2070dd6e4c
2023-11-28 22:35:37 +00:00
Jihoon Kang
2a929ad250 Revert^2 "Default from-text stub generation in build"
This change defaults Java stubs to be generated from API text files
during build. Using the `--build-from-source-stubs` flag, users can
toggle between the feature.

This change relands aosp/2617457.

Test: m nothing && verify ninja path exists between android_stubs_current and android_stubs_current.from-text, and does not exist between android_stubs_current.from-source, m nothing --build-from-source-stub && verify the opposite
Bug: 301522358
Change-Id: Ic129dafc3231e71470af982374739930947b69cb
2023-11-08 04:25:52 +00:00
LaMont Jones
9a9128610b Readd support for PRODUCT_RELEASE_CONFIG_MAPS
This allows product config to determine what build flags are present in
a product's build, as well as their value.

As product config moves to a declarative specification, this can be
handled with less unnecessary work.

Bug: b/302593603, b/308849337, b/309477343
Test: manual
Change-Id: Ic0551990864bba6d03b6bf49fcfd4ae2523d9c6d
2023-11-07 18:39:40 +00:00
Colin Cross
216ed6c918 Merge "Revert "Add support for PRODUCT_RELEASE_CONFIG_MAPS"" into main 2023-11-06 22:02:50 +00:00
Colin Cross
6a4fe68cc5 Revert "Add support for PRODUCT_RELEASE_CONFIG_MAPS"
Revert submission 2815850-product-flags

Reason for revert: b/309477343

Reverted changes: /q/submissionid:2815850-product-flags

Bug: 302593603
Bug: 309477343
Change-Id: I5b5a5ade1d85195c43377ed7639deba6eef86e23
2023-11-06 21:32:03 +00:00
LaMont Jones
4c01bb49da Merge "Add support for PRODUCT_RELEASE_CONFIG_MAPS" into main 2023-11-06 14:08:22 +00:00
LaMont Jones
127127b608 Add support for PRODUCT_RELEASE_CONFIG_MAPS
This allows product config to determine what build flags are present in
a product's build, as well as their value.

As product config moves to a declarative specification, this can be
handled with less unnecessary work.

Bug: b/302593603
Bug: b/308849337
Test: manual
Change-Id: If57622059bb7d1c33df7498321db621c0c2d30c0
2023-11-03 19:19:59 +00:00
Colin Cross
f2c51919ad Merge "Remove obsolete java version environment variables" into main 2023-11-02 17:05:06 +00:00
Treehugger Robot
87706098b0 Merge "Revert "Add support for PRODUCT_RELEASE_CONFIG_MAPS"" into main 2023-11-02 08:06:39 +00:00
Colin Cross
f77b2cb534 Revert "Add support for PRODUCT_RELEASE_CONFIG_MAPS"
Revert submission 2787001-product-build-flags

Reason for revert: Possible cause of b/308849337

Reverted changes: /q/submissionid:2787001-product-build-flags

Bug: 302593603
Bug: 308849337
Change-Id: Id3f27eca8d39e854dee232f072a8dc6c561bac63
2023-11-02 00:13:30 +00:00
Colin Cross
f059b2f2fc Remove obsolete java version environment variables
jdk9 and jdk11 are no longer used, remove ANDROID_JAVA9_HOME and
ANDROID_JAVA11_HOME.

Bug: 221270882
Test: treehugger
Change-Id: Iad566bff21055c5e2acd62642ccd579507e21fef
2023-11-01 20:58:30 +00:00
Colin Cross
2650f35faa Merge "Use correct path for soong_build_metrics.pb when dist is enabled" into main 2023-11-01 16:54:14 +00:00
LaMont Jones
c416ddd1cc Merge "Add support for PRODUCT_RELEASE_CONFIG_MAPS" into main 2023-11-01 16:01:08 +00:00
Colin Cross
b67b061baa Use correct path for soong_build_metrics.pb when dist is enabled
When dist is enabled soong_build_metrics.pb is written to $DIST_DIR/logs,
not $OUT_DIR.

Test: m dist nothing, examine out/dist/logs/bulid.trace.gz
Change-Id: Ia41de6c838ecbcf0acca1e7ec7e68ba24dce2b32
2023-10-31 23:52:38 +00:00
LaMont Jones
772e21ba27 Add support for PRODUCT_RELEASE_CONFIG_MAPS
This allows product config to determine what build flags are present in
a product's build, as well as their value.

As product config moves to a declarative specification, this can be
handled with less unnecessary work.

Bug: b/302593603
Test: manual
Change-Id: If6ae19e3da8471eaba8d6e048769f8ac6b244155
2023-10-31 18:47:18 +00:00
Christopher Parsons
9e5abfecc9 Merge "Delete bazel-related files while bazel is off" into main 2023-10-31 15:00:17 +00:00
Colin Cross
66d6409e06 Merge "Add performance counter metrics to build.trace.gz" into main 2023-10-30 22:21:14 +00:00
Chris Parsons
b6d6fc9c63 Delete bazel-related files while bazel is off
Bug: 189485959
Test: Manually verified that files are removed
Test: Presubmits
Change-Id: Ia279937d6d335174776ff661ab20c7821b979ca7
2023-10-30 21:33:10 +00:00
Colin Cross
ab766b40e8 Merge "Copy soong_build metrics into build.trace.gz" into main 2023-10-30 19:40:31 +00:00
Colin Cross
46b0c75204 Add performance counter metrics to build.trace.gz
Start a background goroutine at the beginning of soong_build that
captures the CPU usage, heap size, and total system memory every
second.  Propagate the values through soong_build_metrics.pb back
to soong_ui, and then into build.trace.gz.

Test: m nothing, examine build.trace.gz
Change-Id: Iad99f8f1f088f4f7f7d5f76566a38c0c4f4d0daa
2023-10-30 12:39:07 -07:00
Colin Cross
aa9a273b40 Copy soong_build metrics into build.trace.gz
Make soong_ui read soong_build_metrics.bp to extract the event timings
and propagate them to Tracer, which will put them in build.trace.gz.
This provides much better visibility into what parts of the build are
contributing to the overly large analysis time.

Test: m nothing and examine build.trace.gz
Change-Id: I473727f1ec044b0d973f2cb4e3eaca96bfca94f6
2023-10-27 19:36:47 -07:00
MarkDacek
39825ead17 Remove Bazel metrics/finalize-bazel-metrics.
Test: m nothing
Change-Id: Idd4610aca77f0a5deeadc3b895869879ded6c16e
2023-10-26 20:14:36 +00:00
Chris Parsons
bf8ab858da disable bazel by default
Test: m nothing, verified bp2build was not run
Test: presubmits
Change-Id: I7476194f8424fa4f199fb94e1cbd0b3c84339fd9
2023-10-26 17:22:27 +00:00
Colin Cross
106d6ef459 Disable Soong tests by default on Linux
Disable running the Soong tests when building on Linux.  The tests
are still enabled when running on Darwin to ensure we have test on
Darwin in CI.  Checkbuilds still depend on the tests, which will
provide test coverage in CI on Linux.

Bug: 269296618
Test: aninja -t path checkbuild out/host/linux-x86/bin/go/soong-java/test/test.passed
Test: m nothing
Change-Id: Ib8359ff6d2a92eeb2ac9a9e95ecd65abc4e40061
2023-10-24 11:00:59 -07:00
Sebastian Pickl
1c4188c9da Revert "Default from-text stub generation in build"
This reverts commit 182b56b870.

Reason for revert: prime suspect for breaking boot tests b/307495247, b/307411752

Bug:307495247
Change-Id: Iea05703b767d2699ca3cf69377eb44b1d21697ad
2023-10-24 11:20:06 +00:00
Jihoon Kang
182b56b870 Default from-text stub generation in build
This change defaults Java stubs to be generated from API text files
during build. Using the `--build-from-source-stubs` flag, users can
toggle between the feature.

Test: m nothing && verify ninja path exists between android_stubs_current and android_stubs_current.from-text, and does not exist between android_stubs_current.from-source, m nothing --build-from-source-stub && verify the opposite
Bug: 274805756
Change-Id: I28834f92c1b1311e3fe0a71a6ea9e8ec2e278d7e
2023-10-20 23:11:19 +00:00
Cole Faust
06ea531ab3 gofmt soong
Test: Presubmits
Change-Id: Ia76c35ba51685eca29df51738000eacd3f43ce20
2023-10-18 17:42:59 -07:00
Kousik Kumar
298c785766 Allowlist gcertstatus command to be run
With https://team-review.git.corp.google.com/c/foundry-x/re-client/+/2045536, reclient is starting to use gcertstatus instead of prodcertstatus to check for validity of google prod creds.

I'll remove the allowlist for prodcertstatus in a subsequent CL after new reclient is released.

Bug: b/169675226
Change-Id: I0e760c863d534b7a5744daf5f89f530be3d296ff
2023-10-16 20:40:18 +00:00
Treehugger Robot
d278e522f9 Merge "Revert "Add out/partitions_were_clean_at_start_of_build.txt"" into main 2023-10-13 05:32:55 +00:00
Cole Faust
e842a6c874 Revert "Add out/partitions_were_clean_at_start_of_build.txt"
Revert submission 2773149-partition_file_list_check

Reason for revert: b/305103467, b/305103783, b/305103468
Reverted changes: /q/submissionid:2773149-partition_file_list_check

Change-Id: I4116fe908cc914f41dc1bf7e950acb32bc546995
2023-10-13 02:51:08 +00:00
Jihoon Kang
e100a884e3 Merge changes from topic "enable-hidden-api" into main
* changes:
  Enable hiddenapi check for from-text stub build
  Utilize module lib and test api superset module in hiddenapi
2023-10-12 17:33:35 +00:00
Cole Faust
14645b284d Merge "Add out/partitions_were_clean_at_start_of_build.txt" into main 2023-10-12 16:33:11 +00:00
Jihoon Kang
dff2aea212 Enable hiddenapi check for from-text stub build
This change removes the UNSAFE_DISABLE_HIDDENAPI_FLAGS env var setting
during from-text stub build, and enables hiddenapi list during from-text
stub build.

Test: ENABLE_HIDDENAPI_FLAGS=true m --build-from-text-stub
Bug: 275570206
Change-Id: Ic8cd60e376b978ccc658ff43a44d082eb2759fa5
2023-10-11 17:46:29 +00:00
Ramy Medhat
bc06176350 Set download_tmp_dir explicitly to be a directory under the output directory.
This directory is where outputs are download first before being moved
to their final destination to ensure atomicity.

Change-Id: Ic224bf21c07566de00d292d02f1c0f7f727dcb08
2023-10-10 22:33:46 +00:00
Cole Faust
39ec9276a7 Add out/partitions_were_clean_at_start_of_build.txt
Test: Presubmits
Change-Id: I1c7025f2c7594ae84b740ccbae4dcbc933b8cdf6
2023-10-05 13:26:16 -07:00