Commit graph

35074 commits

Author SHA1 Message Date
Lukacs T. Berki
d518e1a407 Make bp2build be more correct.
It now handles adding .bp files and changing globs.

In order to do this, depfiles are now written separately from RunBlueprint.

This is necessary due to the confluence of a number of seemingly
unrelated factors:

1. The glob filelist dependencies are discovered in globSingleton
2. Singletons need to be registered because otherwise singleton module
   types panic
3. Singletons don't work because they require mutators bp2build does not
   run

Due to (1), we would need to run the glob singleton. However, due to (2)
and (3), we can't run singletons and have to run Blueprint with
StopBeforeGeneratingBuildActions, which is when the build actions
writing glob files would be generated. So what happens is:

1. When bp2build is run, the glob singleton is disabled
2. At the end of bp2build, the list of glob files is artifically added
   to the depfile of the workspace marker file
3. When build.ninja is generated, the Ninja file containing the glob
   list file is written by the now-active glob singleton

Test: Presubmits.
Change-Id: I3c5898d8c57c554a93520276c64a952afc912dbe
2021-04-15 13:06:16 +02:00
Pedro Loureiro
53d69eb028 Merge "Introduce NewApi lint checks" 2021-04-15 11:04:47 +00:00
Lukács T. Berki
3d76d5d6b8 Merge "Allow running bp2build as part of a regular build." 2021-04-15 11:03:51 +00:00
Paul Duffin
fd105d469c Merge "Move hidden API metadata file rule to platform_bootclasspath" 2021-04-15 09:41:44 +00:00
Paul Duffin
e2243eac3a Merge "Move handling of prebuilt hiddenapi-index.csv to hiddenapi singleton" 2021-04-15 09:41:31 +00:00
Paul Duffin
a4b0d08b83 Merge "Move hidden API index file rule to platform_bootclasspath" 2021-04-15 09:40:37 +00:00
Jooyung Han
1f105f1338 Merge changes from topic "linkerconfig"
* changes:
  Add 'merge' command to conv_linker_config
  make linker_config OutputFileProducer
  Allow uninstallable linker_config to be packaged
2021-04-15 08:49:25 +00:00
Lukacs T. Berki
686965baeb Speed up Soong integration tests.
This is done by running a build before the first test case, tarring up
$TOP (including out/) then untarring it for every test case instead of
rebuilding soong_build each time.

The speedup is approximately 2x.

Also add some cute ANSI escape sequences and delineation of individual
test cases.

Test: The aforementioned integration tests.
Change-Id: I6a98660cfe6a40bcaa978e8d7544cdffad86a7fa
2021-04-15 08:47:39 +02:00
Lukacs T. Berki
f8e2428c5d Allow running bp2build as part of a regular build.
This is done by setting the INTEGRATED_BP2BUILD environment variable
when invoking the build.

Even though the name of the marker file insinuates that a Bazel
workspace is already created, this is not the case yet.

An issue that remains is that a .d file is not written for the marker
file so it won't be rebuilt if a .bp file changes. Fixing this requires
delicate surgery because writing the .d file is the result of delicate
interplay between Soong and Blueprint.

There are also a number of semi-related fixes:

- The name of soong.environment.{used,available} is now on the command
  line of soong_build (soong_docs is still special cased because its
  command line in the Ninja file is taken from the os.Args of
  soong_build so it's not trivial to remove the --{available,used}_env
  from it
- bp2build writes a separate soong.environment.used file
- I had to call SetAllowMissingDependencies() separately when creating
  the android.Context for bp2build so that bp2build runs in the
  integration tests (it was not obvious how not to do this)
- Fixed a number of integration tests where a command with an expected
  exit code of 1 was used as the last one in a test case, thereby
  breaking the test suite

Test: Presubmits.
Change-Id: Ibeb61c26022cf801dcb98505b4039151b3409873
2021-04-15 08:46:07 +02:00
Treehugger Robot
017a1bb7c9 Merge "Add prebuilt_root module" 2021-04-15 06:31:53 +00:00
Treehugger Robot
b06a4bd400 Merge "Transform paths to headers in include dirs to take package boundaries into account." 2021-04-15 05:42:40 +00:00
Treehugger Robot
638830a5a0 Merge changes from topic "userdebug_plat_sepolicy.cil_Android.bp"
* changes:
  Add debug ramdisk variant
  Add path tests for ramdisk
2021-04-15 05:22:35 +00:00
Treehugger Robot
d820c37425 Merge "Fix evaluation order of (Cfi|Memtag) exclude paths." 2021-04-15 03:49:38 +00:00
Treehugger Robot
ccaf57c65a Merge "Truncate vbmeta to 64KB" 2021-04-15 02:06:01 +00:00
Jooyung Han
e134d09831 Add 'merge' command to conv_linker_config
'merge' command can merge multiple configurations.
It just delegates protobuf's msg.MergeFrom(other_msg).

  usage: conv_linker_config merge -o out.pb -i a.pb -i b.pb ...

out.pb will have all keys from multiple inputs.

Bug: 181093750
Test: m
Test: MicrodroidTestCase
Change-Id: Ibf715aa0bcc1e5c82c85a9af8fe7dca8d6ab68ad
2021-04-15 10:58:21 +09:00
Inseob Kim
f84e9c05e2 Add debug ramdisk variant
A module will be installed to debug_ramdisk (or
debug_ramdisk/first_stage_ramdisk if recovery as boot is true) if
debug_ramdisk is set to true.

Bug: 184004542
Test: soong test
Change-Id: Ic5a4d27407e506fffa462de2149e0785f11b2ac7
2021-04-15 00:56:30 +00:00
Rupert Shuttleworth
c143cc50dd Transform paths to headers in include dirs to take package boundaries into account.
This allows the following cc_library_static targets to build with bp2build:

- libc_freebsd
- libc_freebsd_large_stack
- libc_openbsd_ndk
- libc_gdtoa  (* but still fails for mixed builds)
- libc_aeabi
- libc_static_dispatch
- libc_dynamic_dispatch

This also allows a number of other cc_library_static targets to progress further in their builds.

Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/...

Change-Id: I71742565c16594448a41a6428a5c993171ec4cb4
2021-04-14 17:05:36 -04:00
LuK1337
fb545bf2f3 Make sure prebuilt modules are exported before attempting to use them
Fixes an issue where prebuilt module under unexported soong
namespace overrides source modules when flag `prefer` was set.

Test: Create two modules under the same name, use cc_library_shared
      for first one and cc_prebuilt_library_shared with `prefer` set
      to true.
      Then put the prebuilt module under `soong_namespace {}` and
      attempt to compile it without appending the namespace path to
      product namespaces.
      Without this change it should fail to resolve the
      cc_library_shared module even if it should be available.

Change-Id: I506641218e6af94a0b5f048e09e06d83602e50f5
2021-04-14 22:55:22 +02:00
Jooyung Han
a0436a3928 make linker_config OutputFileProducer
so that they can be used in `srcs` of genrule.

Bug: 181093750
Test: m
Change-Id: Ia54554f1e81d766a970c9c4e02906ef9c0d85934
2021-04-15 05:27:48 +09:00
Jooyung Han
c6a91ec6d8 Allow uninstallable linker_config to be packaged
Till now, linker_config has to be installable to be packaged. With this
commit, packaging modules like filesystem now can include uninstallable
linker_config. This can help when linker_config can conflict with other
partitions e.g. system.

Bug: 185391776
Test: build filesystem with uninstallable linker_config
Change-Id: Ic2cece08be73749e6bd1f813152c67f8e3904c20
2021-04-15 05:27:48 +09:00
Ivan Lozano
d466b4a1f5 Merge changes from topic "rust_vendor_support"
* changes:
  Add more Rust vendor image support.
  rust: Use new common image mutator interface.
2021-04-14 19:02:49 +00:00
Ivan Lozano
e8dfbdfb59 Merge "cc: Create a common image mutator interface" 2021-04-14 18:53:18 +00:00
Paul Duffin
85dee5d8fb Move hidden API metadata file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPISingleton into the platform_bootclasspath.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ib25be3618e30a83dc2929a18062eb58eefdabefb
2021-04-14 18:54:55 +01:00
Paul Duffin
22c743170d Move handling of prebuilt hiddenapi-index.csv to hiddenapi singleton
This change moves the handling of the prebuilt hiddenapi-index.csv file
from the hiddenAPIIndexSingleton and removes that as it no longer
needed.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Iab1b39a57f28caa855c48538aa2230795f2c3271
2021-04-14 18:54:45 +01:00
Paul Duffin
00b2bfdea5 Move hidden API index file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPIIndexSingleton into the platform_bootclasspath. It
also moves the corresponding test from java/hiddenapi_singleton_test.go
to java/platform_bootclasspath_test.go.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ia295d0f7ae9b51ea816f16921aa42339ed91704e
2021-04-14 18:54:29 +01:00
Tri Vo
9e7bfdae31 Merge "rust: Switch rust_fuzz to HWASan" 2021-04-14 16:49:27 +00:00
Paul Duffin
08abf7274a Merge "Ensure boot jar modules have been initialized properly for hidden API" 2021-04-14 15:53:15 +00:00
Lukacs T. Berki
b448c34bfd Make integration tests clean up after themselves.
Well, this was embarrassing.

Test: These tests.
Change-Id: Id25f9f1f4bd2d2d01d49d11a5af8eb50dd51fe6e
2021-04-14 16:45:22 +02:00
Jiyong Park
f398995a64 Merge "SdkSpec is fully using ApiLevel" 2021-04-14 13:27:05 +00:00
Inseob Kim
d9580b84a2 Add path tests for ramdisk
Ramdisk path tests existed once, but have been removed for whatever
reason. This change revives the tests.

Test: soong test
Change-Id: Ibade91fbe3e044f772a50df15f448b04aa12d807
2021-04-14 12:24:45 +00:00
Anton Hansson
e9d2f9fcda Merge "Remove special case code for obsolete files" 2021-04-14 09:22:51 +00:00
Paul Duffin
1ba246732d Ensure boot jar modules have been initialized properly for hidden API
Checks to make sure that every module that is part of the platform
bootclasspath has been initialized property for hidden API processing.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ic4368963f2011784b537b8aebf5ef97ea22b2db5
2021-04-14 09:08:02 +01:00
Paul Duffin
35584eeb93 Merge "prebuilt_apex created ApexInfo must not include prebuilt_ prefix" 2021-04-14 08:03:44 +00:00
Jeffrey Vander Stoep
d587ec225a Merge changes from topic "rust_benchmark"
* changes:
  Attach rust_benchmark to atest and tradefed.
  Add rust_benchmark module to soong.
2021-04-14 08:02:46 +00:00
Jiyong Park
da2d6ee7a2 Truncate vbmeta to 64KB
libavb expects to be able to read the maximum vbmeta size, so we must
provide a partition which matches this or the read will fail.

Bug: 181923506
Test: m MicrodroidHostTestCases
libavb tries

Change-Id: Icc5b86ccd98e3ed9c4269e584f35dbfbdf1b0730
2021-04-14 16:42:24 +09:00
Lukács T. Berki
c11f269933 Merge "Delete android/env.go ." 2021-04-14 05:47:58 +00:00
Inseob Kim
27408bf32c Add prebuilt_root module
prebuilt_root supports installing files directly under root.

Bug: 181728482
Test: soong test, manual
Change-Id: Ib1f9a4fd4c9a47094d5f41106fc12696741e2ff1
2021-04-14 13:19:29 +09:00
Inseob Kim
ff5cc064c2 Merge changes from topic "reexport_vndk_from_vendor_snapshot"
* changes:
  Support reexporting vndk from vendor snapshot
  Add flag test for cfi snapshot
2021-04-13 23:46:13 +00:00
Tri Vo
505b0e8991 rust: Switch rust_fuzz to HWASan
Bug: 180495975
Test: example_rust_fuzzer
Change-Id: I26e6f15136ee2d5f4ed1167be5c1c6a14b19421a
2021-04-13 15:58:44 -07:00
Treehugger Robot
6bd0a19b9d Merge "Add tests for request_type ParseResult functions." 2021-04-13 19:08:11 +00:00
Paul Duffin
8f146b99e6 prebuilt_apex created ApexInfo must not include prebuilt_ prefix
As part of the work to modularize the hiddenAPI processing the
generation of the monolithic hidden API index file needs to be moved
to the platform_bootclasspath module type. Doing that broke the
TestBootDexJarsFromSourcesAndPrebuilts tests which checks the inputs to
the rule that creates that file. Fixing that required added a
platform_bootclasspath module to the test fixture for those tests which
highlighted an issue with the prebuilt_apex module.

Previously, when the prebuilt_apex created apex variants it would use
its own name as the apex variant name, even when that name included the
prebuilt_ prefix. That broke the platform_bootclasspath logic as it was
looking for apex variants for "myapex" but the only ones available were
"prebuilt_myapex".

This change ensures that it always uses the unprefixed name and fixes
the TestNoUpdatableJarsInBootImage to match. This also adds some
improved error reporting in platform_bootclasspath which helped debug
this problem.

Bug: 177892522
Test: m nothing
Change-Id: I3e88b5cec767f77dcc0e94b3ae38b499d07eadf0
2021-04-13 19:15:27 +01:00
Paul Duffin
21fb92d7f6 Merge "Improve realism of boot jar tests" 2021-04-13 18:13:34 +00:00
Anton Hansson
fa5e6b50aa Remove special case code for obsolete files
incompatibilities.txt doesn't exist anymore, only
"<module>-incompatibilities.txt". Clean up the associated logic that
mapped incompatibilities.txt to android-incompatibilities.txt.

Bug: 178171189
Test: m
Change-Id: I61eeb5dcc82770fa2af356bad11b7205551d6d83
2021-04-13 19:09:48 +01:00
Paul Duffin
6050c3c31e Merge "Support UNSAFE_DISABLE_HIDDENAPI_FLAGS in platform_bootclasspath" 2021-04-13 17:30:27 +00:00
Evgenii Stepanov
779b64ec5d Fix evaluation order of (Cfi|Memtag) exclude paths.
Before this change, exclude paths disabled sanitization of targets that
would otherwise be enabled by SanitizeDevice product variable (aka
SANITIZE_TARGET).

With this change, in addition to the above logic, exclude path disables
sanitization of targets that would otherwise be enabled by the
corresponding include path.

Effectively, this change disables sanitization of targets that are
covered by *both* include and exclude paths.

Test: build/soong/cc/cc_test.go
Bug: b/184976817
Change-Id: I96f1e2a808d88b352b92b7490fa1df5c010314c3
2021-04-13 10:29:15 -07:00
Paul Duffin
149e19ef70 Merge "Remove noop code from generateHiddenAPIBuildActions" 2021-04-13 17:29:03 +00:00
Ivan Lozano
c08897c1e4 Add more Rust vendor image support.
This adds Rust vendor image support for all module types except
Rust prebuilts.

Bug: 184042776
Test: New Soong tests.
Test: Example cc_library vendor module can depend on rust_ffi_shared.
Test: Example rust_library vendor-only module compiles.

Change-Id: Iaa30ad51fdaedcbf14687da5472581f6af62ff59
2021-04-13 13:17:12 -04:00
Ivan Lozano
699e218324 rust: Use new common image mutator interface.
Refactor rust to use and implement the new common image mutator
interface to handle future image mutations.

Bug: 184042776
Test: m nothing
Change-Id: If6a85e2b8c6a1969d62264eaea6c6b53cae9c039
2021-04-13 13:15:21 -04:00
Liz Kammer
41cca851d9 Merge "Consolidate cc-specific cqueries." 2021-04-13 17:11:34 +00:00
Colin Cross
66d154e436 Merge "Follow blueprint change to return GlobResult from Glob" 2021-04-13 17:01:39 +00:00