Commit graph

838 commits

Author SHA1 Message Date
Colin Cross
106d6ef459 Disable Soong tests by default on Linux
Disable running the Soong tests when building on Linux.  The tests
are still enabled when running on Darwin to ensure we have test on
Darwin in CI.  Checkbuilds still depend on the tests, which will
provide test coverage in CI on Linux.

Bug: 269296618
Test: aninja -t path checkbuild out/host/linux-x86/bin/go/soong-java/test/test.passed
Test: m nothing
Change-Id: Ib8359ff6d2a92eeb2ac9a9e95ecd65abc4e40061
2023-10-24 11:00:59 -07:00
Sebastian Pickl
1c4188c9da Revert "Default from-text stub generation in build"
This reverts commit 182b56b870.

Reason for revert: prime suspect for breaking boot tests b/307495247, b/307411752

Bug:307495247
Change-Id: Iea05703b767d2699ca3cf69377eb44b1d21697ad
2023-10-24 11:20:06 +00:00
Jihoon Kang
182b56b870 Default from-text stub generation in build
This change defaults Java stubs to be generated from API text files
during build. Using the `--build-from-source-stubs` flag, users can
toggle between the feature.

Test: m nothing && verify ninja path exists between android_stubs_current and android_stubs_current.from-text, and does not exist between android_stubs_current.from-source, m nothing --build-from-source-stub && verify the opposite
Bug: 274805756
Change-Id: I28834f92c1b1311e3fe0a71a6ea9e8ec2e278d7e
2023-10-20 23:11:19 +00:00
Cole Faust
06ea531ab3 gofmt soong
Test: Presubmits
Change-Id: Ia76c35ba51685eca29df51738000eacd3f43ce20
2023-10-18 17:42:59 -07:00
Kousik Kumar
298c785766 Allowlist gcertstatus command to be run
With https://team-review.git.corp.google.com/c/foundry-x/re-client/+/2045536, reclient is starting to use gcertstatus instead of prodcertstatus to check for validity of google prod creds.

I'll remove the allowlist for prodcertstatus in a subsequent CL after new reclient is released.

Bug: b/169675226
Change-Id: I0e760c863d534b7a5744daf5f89f530be3d296ff
2023-10-16 20:40:18 +00:00
Treehugger Robot
d278e522f9 Merge "Revert "Add out/partitions_were_clean_at_start_of_build.txt"" into main 2023-10-13 05:32:55 +00:00
Cole Faust
e842a6c874 Revert "Add out/partitions_were_clean_at_start_of_build.txt"
Revert submission 2773149-partition_file_list_check

Reason for revert: b/305103467, b/305103783, b/305103468
Reverted changes: /q/submissionid:2773149-partition_file_list_check

Change-Id: I4116fe908cc914f41dc1bf7e950acb32bc546995
2023-10-13 02:51:08 +00:00
Jihoon Kang
e100a884e3 Merge changes from topic "enable-hidden-api" into main
* changes:
  Enable hiddenapi check for from-text stub build
  Utilize module lib and test api superset module in hiddenapi
2023-10-12 17:33:35 +00:00
Cole Faust
14645b284d Merge "Add out/partitions_were_clean_at_start_of_build.txt" into main 2023-10-12 16:33:11 +00:00
Jihoon Kang
dff2aea212 Enable hiddenapi check for from-text stub build
This change removes the UNSAFE_DISABLE_HIDDENAPI_FLAGS env var setting
during from-text stub build, and enables hiddenapi list during from-text
stub build.

Test: ENABLE_HIDDENAPI_FLAGS=true m --build-from-text-stub
Bug: 275570206
Change-Id: Ic8cd60e376b978ccc658ff43a44d082eb2759fa5
2023-10-11 17:46:29 +00:00
Ramy Medhat
bc06176350 Set download_tmp_dir explicitly to be a directory under the output directory.
This directory is where outputs are download first before being moved
to their final destination to ensure atomicity.

Change-Id: Ic224bf21c07566de00d292d02f1c0f7f727dcb08
2023-10-10 22:33:46 +00:00
Cole Faust
39ec9276a7 Add out/partitions_were_clean_at_start_of_build.txt
Test: Presubmits
Change-Id: I1c7025f2c7594ae84b740ccbae4dcbc933b8cdf6
2023-10-05 13:26:16 -07:00
Kousik Kumar
ca390b2f00 Update symlinks in output directory when TOP dir changes
When TOP dir changes and we reuse the same output between the old and new checkouts, we need to rewrite all the symlinks used by Bazel and other tools in the out-directory to point to the new top directory. Otherwise, if the old source dir is deleted, the build will fail.

I used the OUT_DIR/soong/soong.environment.available file to find out the previous PWD.

Tested:
1. Create source dir 1, run build, create source dir 2, remote source dir 1, reuse out dir and rerun build => build succeeded with this change.
2. m libc after moving build TOP. Only the analysis phase was rerun, the actual build was not rerun.

Bug: b/300498226

Change-Id: I196625baa1f4efe7a4734accfa1f0be7c98a7920
2023-10-04 21:07:36 +00:00
Cole Faust
583dfb426f Generate build_number.txt only once
build_number.txt was generated in runKati, and runKati is run multiple
times throughout the build. If a build number is not set, then one will
be generated using the current timestamp, meaning that multiple
different build numbers will be used in different phases of the build.

Instead, generate build_number.txt during SetupOutDir(), so that we
only have 1 build number.

This is a resubmission with a change to ensure out/soong is created
before writing the build number file.

Bug: 297269187
Test: edit a bzl file to force bazel to rerun, change the code so that use_fixed_timestamp=true during partition builds, m installclean, m bazel_sandwich, and observe that the bazel/make partitions are byte-for-byte identical
Change-Id: I47abcca166c701bb66a6a7731aecad5818279244
2023-09-28 17:56:43 -07:00
Roy Luo
f852b72a22 Revert "Generate build_number.txt only once"
This reverts commit ab6e524709.

Reason for revert: DroidMonitor-triggered revert due to breakage https://android-build.corp.google.com/builds/quarterdeck?branch=aosp-main&target=ndk&lkgb=10879153&lkbb=10879224&fkbb=10879224, bug b/302608661.

Change-Id: I07d47a92ea9a9f5d329e813859316decfd75cde1
BUG: 302608661
2023-09-29 00:28:32 +00:00
Cole Faust
ab6e524709 Generate build_number.txt only once
build_number.txt was generated in runKati, and runKati is run multiple
times throughout the build. If a build number is not set, then one will
be generated using the current timestamp, meaning that multiple
different build numbers will be used in different phases of the build.

Instead, generate build_number.txt during SetupOutDir(), so that we
only have 1 build number.

Bug: 297269187
Test: edit a bzl file to force bazel to rerun, change the code so that use_fixed_timestamp=true during partition builds, m installclean, m bazel_sandwich, and observe that the bazel/make partitions are byte-for-byte identical
Change-Id: I5bf3bb6c78e7529749ca9275f67db3f2f9e66af2
2023-09-28 14:11:43 -07:00
Cole Faust
ded796030b Allow actions to read BUILD_BROKEN_INCORRECT_PARTITION_IMAGES
To maintain good backwards compatibility with the legacy partition
building behavior, allow actions to read
BUILD_BROKEN_INCORRECT_PARTITION_IMAGES so that we don't have to rerun
analysis.

Bug: 205632228
Test: Presubmits
Change-Id: I2b55c0143cbdaf010e6b5fd0c3d51d6930a94eff
2023-09-08 15:58:13 -07:00
Firman Prayoga
e2e3172dba Merge "Revert "Allow actions to read BUILD_BROKEN_INCORRECT_PARTITION_I..."" into main 2023-09-08 04:06:57 +00:00
Firman Prayoga
098b237f5e Revert "Allow actions to read BUILD_BROKEN_INCORRECT_PARTITION_I..."
Revert submission 2736954-proper_installclean

Reason for revert: Potential culprit for build breakage b/299545346

Reverted changes: /q/submissionid:2736954-proper_installclean

Change-Id: I96ef912c23f99b7335eeb27c2c09949d655e8fa7
2023-09-08 01:24:39 +00:00
Cole Faust
72d5c104b8 Merge "Allow actions to read BUILD_BROKEN_INCORRECT_PARTITION_IMAGES" into main 2023-09-07 16:41:45 +00:00
Cole Faust
b95e5e873c Allow actions to read BUILD_BROKEN_INCORRECT_PARTITION_IMAGES
To maintain good backwards compatibility with the legacy partition
building behavior, allow actions to read
BUILD_BROKEN_INCORRECT_PARTITION_IMAGES so that we don't have to rerun
analysis.

Bug: 205632228
Test: Presubmits
Change-Id: I8b25a62f24bc6d628fb055239b084f6ea535e38b
2023-09-05 17:48:11 -07:00
Chris Parsons
73f411b72c Delete ApiBp2build soong_build mode
This feature is obsolete.

This makes a large number of codepaths "dead code" (such as
module-specific implementations of ApiBp2build functionality). These
will be deleted in a followup CL.

Bug: 284029211
Test: Presubmits
Change-Id: Ib53b99f1fe8c24380d219caf44e9bb3b96724fa0
2023-09-05 17:09:51 -07:00
Treehugger Robot
b84435c0ab Merge "Revert "Add BUILD_BROKEN_VENDOR_SEAPP_USES_COREDOMAIN"" into main 2023-08-23 06:34:30 +00:00
Inseob Kim
3a61b6edae Revert "Add BUILD_BROKEN_VENDOR_SEAPP_USES_COREDOMAIN"
Revert submission 2715179-BUILD_BROKEN_VENDOR_SEAPP_USES_COREDOMAIN

Reason for revert: not needed

Reverted changes: /q/submissionid:2715179-BUILD_BROKEN_VENDOR_SEAPP_USES_COREDOMAIN

Change-Id: I3bc8f2006b2f82159ae308467ea74ad62a60ec6f
2023-08-23 02:07:40 +00:00
Jeongik Cha
962bcdb18e Merge "Reland "Make glob output include product name"" into main 2023-08-22 04:41:43 +00:00
Treehugger Robot
4f53e94198 Merge "Add BUILD_BROKEN_VENDOR_SEAPP_USES_COREDOMAIN" into main 2023-08-22 02:09:45 +00:00
Inseob Kim
0a7c819498 Add BUILD_BROKEN_VENDOR_SEAPP_USES_COREDOMAIN
Starting from Android V, vendor seapp_contexts files can't assign
coredomain to vendor apps, as it's Treble violation. This build broken
variable is to suppress the enforcement for devices launching with U or
prior.

Bug: 280547417
Test: set BUILD_BROKEN_VENDOR_SEAPP_USES_COREDOMAIN := true and build
Change-Id: Ic4b5309f0d9bab9b93e88988d1a5a942b2de220a
2023-08-18 18:16:26 +09:00
Elliott Hughes
5a92870856 Switch to toybox dd.
Change-Id: I8a0256e8804d0118ae39c63b15a85d9ad5a6ab84
2023-08-17 16:25:30 +00:00
Jeongik Cha
ccf37005ec Reland "Make glob output include product name"
glob pattern could vary, depending on device configuration, but it uses
the same glob output path for now. It makes unnecessary soong
reevaluation due to the previous result is always overwritten whenever
the target is changed.

So make it product specific to avoid this.

And also let bootstrap.ninja use 'new' ninja file for globbing

Bug: 294378814
Bug: 294058160
Test: lunch a && m nothing && lunch b && m nothing && lunch a && m
nothing and then check if there is no glob work. (a and b has different
glob pattern due to RRO configuration or something)
Test: check if bootstreap.ninja include 'new' ninja file for globbing
Test: declare `cc_library { ... srcs: ["*.c"] }` && \
  touch a.c && m nothing && touch b.c && m nothing && \
  rm b.c && m nothing && mv a.c c.c && m nothing
  and then check if m includes 'regenerate glob..' log.

Change-Id: I9337ae3b33cc2c0689f9731bbae5caae67612e3e
2023-08-17 08:04:58 +00:00
Treehugger Robot
3537d8028a Merge "Disable zipbomb detection inside the build" into main 2023-08-09 22:04:03 +00:00
Kousik Kumar
97e9b0cb8e Merge "Revert "Revert "Remove env config fetcher code""" into main 2023-08-08 15:59:56 +00:00
Kousik Kumar
8036b163ef Revert "Revert "Remove env config fetcher code""
This reverts commit d93c67f64c.

Reason for revert: USE_RBE has again been made the default now, so we should no longer need this code.

Change-Id: I4163c61eed90163e763c29e07dd6edfc9c41b9b9
2023-08-04 19:46:16 +00:00
Paul Duffin
ff3fcd1f64 Merge "Revert "Make glob output include product name"" into main 2023-08-03 16:40:47 +00:00
Paul Duffin
d0e8180ea8 Revert "Make glob output include product name"
This reverts commit 03e50cb4d7.

Reason for revert: I think it broke globbing, see b/294378814

Change-Id: I7e81e8e591278bc2d6b6203b93f128351ca6a241
2023-08-03 15:50:16 +00:00
Treehugger Robot
4d730a8f14 Merge "Make glob output include product name" into main 2023-08-03 00:47:32 +00:00
Jeongik Cha
03e50cb4d7 Make glob output include product name
glob pattern could vary, depending on device configuration, but it uses
the same glob output path for now. It makes unnecessary soong
reevaluation due to the previous result is always overwritten whenever
the target is changed.

So make it product specific to avoid this.

Bug: 294058160
Test: lunch a && m nothing && lunch b && m nothing && lunch a && m
nothing and then check if there is no glob work. (a and b has different
glob pattern due to RRO configuration or something)

Change-Id: I7b6b7a326de681b046a55dbdb34d812cace510e6
2023-08-02 11:00:01 +09:00
Colin Cross
fe5ed4d6e2 Disable zipbomb detection inside the build
Some versions of Fedora include patches to unzip to enable zipbomb
detection that incorrectly handle zip64 and data descriptors and fail
on large zip files produced by soong_zip.  Disable zipbomb detection
inside the build.

Reported upstream in https://bugzilla.redhat.com/show_bug.cgi?id=2227130.

Bug: 286885495
Test: builds
Change-Id: I8e4438720bbb17a073ff3b5535f01c2827485838
2023-07-28 09:31:03 -07:00
Kousik Kumar
6d1e348cc3 Disable RBE env variable when we cannot support RBE
Enabling the RBE variable can cause issues around pool parallelism when
we set the env variable, but don' run the build with RBE due to auth issues.
So, I am explicitly unsetting the variables in those cases.

Bug: b/292224253
Tested: https://b.corp.google.com/issues/292224253#comment26
Change-Id: I19718b4ee6c058ba1b11d3df260421bbf8c9567e
2023-07-24 17:06:42 +00:00
Jingwen Chen
d7ccde11d6 Don't use RBE for bp2build/queryview/api_bp2build/json_module_graph,
even with USE_RBE=true.

These are alternate modes of soong_build that don't use rbe/reclient.

This lets us to remove USE_RBE=false from the bp2build run.

Test: presubmits
Change-Id: I439f9cf7e92ec85ca56baec5f62a83ee49b510d4
2023-06-30 09:32:32 +00:00
Kousik Kumar
67ad434ffc Ignore USE_RBE when on glinux laptops
When there's no stubby and we are asked to use google prod creds, that's invalid configuration, so don't use RBE

Bug: b/283828386
Change-Id: I1564b4f70e46fb90c87a0432c46616caa1614aac
2023-06-23 18:43:54 +00:00
Usta (Tsering) Shrestha
a78884edf7 Merge "cosmetic" 2023-06-23 17:39:17 +00:00
Andus Yu
f3da8ee95b Merge "Do not check ssh credentials when running gcertstatus" 2023-06-22 19:38:33 +00:00
andusyu
0b3dc03f9d Do not check ssh credentials when running gcertstatus
reproxy only requires LOAS credentials for authentication. Append the
-nocheck_ssh option to gcertstatus so we don't check ssh credentials.

Test: RBE successfully started after running gcertdestroy --nodestroy_loas2
Bug: b/287297140
Change-Id: I413931b10d8e8d9ae2f8acd7ebfe37d8b3bba455
2023-06-21 17:30:21 -04:00
MarkDacek
396491eee4 Add bes-id to soong. Also renames the proto field.
This is for integration with BES for bazel test metrics.
Bug: 287102416
Test: b build libcore:all --invocation_id=`uuidgen`
Test: build/bazel/scripts/analyze_build (to verify it's set)

Change-Id: I541b2d65bfa85718fc916582e0540384ae3810d9
2023-06-21 14:48:17 +00:00
Lukacs T. Berki
c357c81f48 Handle errors from RunBlueprint().
Bug: 244730498
Test: Presubmits.
Change-Id: I264f1e4bd989b1c219800eec3be007c3f114c6d1
2023-06-21 07:59:52 +00:00
usta
49012eeacb cosmetic
ninja() is only called for soong bootstrap now

Test: NA
Bug: NA
Change-Id: Iededf601dc3bb92d8681b1e008bf20b4da406f4c
2023-06-20 14:00:44 -04:00
Treehugger Robot
d34e4c72f3 Merge "Increase config fetcher timeout to 20s" 2023-06-15 18:37:53 +00:00
Chris Parsons
21f80277c3 Remove bazel dev mode
This mode is no longer in use, and is dead code.

Fixes: 282809863
Test: Presubmits
Change-Id: I45ab7d7c9dee153f1cbc66262c2cf9f1e9487a28
2023-06-15 14:14:19 +00:00
Kiyoung Kim
eaa55a8ce1 Rename soong.environment.used with product name
From the previous work on renaming build.ninja, it was found that there
are extra dependencies from build.ninja which can be varied by
TARGET_PRODUCT : which is soong.environment.used.<tag>. This change
renames soong.environment.used to have target product between 'used' and
'<tag>' if available.

Bug: 277029044
Test: Test confirmed that build.ninja is not being re-generated
Change-Id: I987b6bd1a8b4f06dac52537e4178d8556251d254
2023-06-08 01:07:47 +00:00
Jason Wu
e2df3b5f8f Merge "Add re_analysis environment varibles fields to soong_metrics" 2023-06-07 00:31:46 +00:00