Commit graph

35093 commits

Author SHA1 Message Date
Thiébaud Weksteen
45c647d4dc Merge changes I59439b77,I7dbaf8be
* changes:
  bloaty: measure stripped Rust binaries
  rust: do not strip static library
2021-04-15 18:46:32 +00:00
Colin Cross
9e7cea2c0b Merge "Add test for globs on noop incremental builds" 2021-04-15 16:42:08 +00:00
Treehugger Robot
107f92efbe Merge "Handle boot jars on /system_ext correctly." 2021-04-15 16:30:03 +00:00
satayev
76f88384c4 Merge "Generate classpaths.proto config for *CLASSPATH variables." 2021-04-15 16:07:13 +00:00
Paul Duffin
d9d1102b35 Merge "Rename hidden API types ..Augmentation.. to ..FlagFile.." 2021-04-15 15:29:18 +00:00
Paul Duffin
bca75b1f77 Merge "Generalize hiddenAPIAugmentationInfo to make it easier to use" 2021-04-15 15:29:07 +00:00
Ulya Trafimovich
e736e13a8d Handle boot jars on /system_ext correctly.
When Soong looks for boot jars among all modules, it applies certain
constraints to each module that looks like a boot jar (e.g. that it
comes from the right apex or platform). Previously these constraints did
not handle boot jars on /system_ext correctly (they were handled like
apex jars, while they should be handled like platform jars).

Bug: 154976937
Test: m nothing (the modified Soong test would fail previously)
Change-Id: I0746a2fd276ab5ef0400340c5b61cf26c2570e5a
2021-04-15 15:07:06 +01:00
Liz Kammer
bc080da38f Merge "Add test case for too many splits" 2021-04-15 13:40:42 +00:00
Lukács T. Berki
a52ae26e91 Merge "Make bp2build be more correct." 2021-04-15 13:32:30 +00:00
Nikita Ioffe
5b94c8c007 Merge changes I83828f58,I1179235e
* changes:
  Generate hashtree for compressed apexes
  Don't compress apexes produced by apex_test module
2021-04-15 11:40:43 +00:00
Lukacs T. Berki
d518e1a407 Make bp2build be more correct.
It now handles adding .bp files and changing globs.

In order to do this, depfiles are now written separately from RunBlueprint.

This is necessary due to the confluence of a number of seemingly
unrelated factors:

1. The glob filelist dependencies are discovered in globSingleton
2. Singletons need to be registered because otherwise singleton module
   types panic
3. Singletons don't work because they require mutators bp2build does not
   run

Due to (1), we would need to run the glob singleton. However, due to (2)
and (3), we can't run singletons and have to run Blueprint with
StopBeforeGeneratingBuildActions, which is when the build actions
writing glob files would be generated. So what happens is:

1. When bp2build is run, the glob singleton is disabled
2. At the end of bp2build, the list of glob files is artifically added
   to the depfile of the workspace marker file
3. When build.ninja is generated, the Ninja file containing the glob
   list file is written by the now-active glob singleton

Test: Presubmits.
Change-Id: I3c5898d8c57c554a93520276c64a952afc912dbe
2021-04-15 13:06:16 +02:00
Pedro Loureiro
53d69eb028 Merge "Introduce NewApi lint checks" 2021-04-15 11:04:47 +00:00
Lukács T. Berki
3d76d5d6b8 Merge "Allow running bp2build as part of a regular build." 2021-04-15 11:03:51 +00:00
Thiébaud Weksteen
e4dd14b25f bloaty: measure stripped Rust binaries
Modify bloaty's MeasureSizeForPath to allow a module to provide multiple
paths. This is used to measure both unstripped and stripped
libraries/binaries. Add unit test to ensure correct measurements are
generated for Rust.

Test: m out/soong/binary_sizes.pb.gz
Change-Id: I59439b77dbf1cf5ad71e1c02996a6a90938536b4
2021-04-15 12:02:51 +02:00
Paul Duffin
4616977948 Rename hidden API types ..Augmentation.. to ..FlagFile..
Augmentation was too abstract this makes it clearer.

Bug: 177892522
Test: m nothinge
Change-Id: I60ad005e68d9e15b01bcb46bc6a9b7f84d8bfd43
2021-04-15 10:45:39 +01:00
Paul Duffin
e3dc6608cb Generalize hiddenAPIAugmentationInfo to make it easier to use
Previously, each category of flag file had its own property in the
hiddenAPIAugmentationInfo struct that required a lot of repetition
to use. This change moves the flag file specific handling into a
new hiddenAPIFlagFileCategory struct which allows use of the
hiddenAPIAugmentationInfo struct to be parameterized.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: I4413134c0c9382139bef3813f847e453f426692c
2021-04-15 10:45:39 +01:00
Paul Duffin
fd105d469c Merge "Move hidden API metadata file rule to platform_bootclasspath" 2021-04-15 09:41:44 +00:00
Paul Duffin
e2243eac3a Merge "Move handling of prebuilt hiddenapi-index.csv to hiddenapi singleton" 2021-04-15 09:41:31 +00:00
Paul Duffin
a4b0d08b83 Merge "Move hidden API index file rule to platform_bootclasspath" 2021-04-15 09:40:37 +00:00
Jooyung Han
1f105f1338 Merge changes from topic "linkerconfig"
* changes:
  Add 'merge' command to conv_linker_config
  make linker_config OutputFileProducer
  Allow uninstallable linker_config to be packaged
2021-04-15 08:49:25 +00:00
Lukacs T. Berki
686965baeb Speed up Soong integration tests.
This is done by running a build before the first test case, tarring up
$TOP (including out/) then untarring it for every test case instead of
rebuilding soong_build each time.

The speedup is approximately 2x.

Also add some cute ANSI escape sequences and delineation of individual
test cases.

Test: The aforementioned integration tests.
Change-Id: I6a98660cfe6a40bcaa978e8d7544cdffad86a7fa
2021-04-15 08:47:39 +02:00
Lukacs T. Berki
f8e2428c5d Allow running bp2build as part of a regular build.
This is done by setting the INTEGRATED_BP2BUILD environment variable
when invoking the build.

Even though the name of the marker file insinuates that a Bazel
workspace is already created, this is not the case yet.

An issue that remains is that a .d file is not written for the marker
file so it won't be rebuilt if a .bp file changes. Fixing this requires
delicate surgery because writing the .d file is the result of delicate
interplay between Soong and Blueprint.

There are also a number of semi-related fixes:

- The name of soong.environment.{used,available} is now on the command
  line of soong_build (soong_docs is still special cased because its
  command line in the Ninja file is taken from the os.Args of
  soong_build so it's not trivial to remove the --{available,used}_env
  from it
- bp2build writes a separate soong.environment.used file
- I had to call SetAllowMissingDependencies() separately when creating
  the android.Context for bp2build so that bp2build runs in the
  integration tests (it was not obvious how not to do this)
- Fixed a number of integration tests where a command with an expected
  exit code of 1 was used as the last one in a test case, thereby
  breaking the test suite

Test: Presubmits.
Change-Id: Ibeb61c26022cf801dcb98505b4039151b3409873
2021-04-15 08:46:07 +02:00
Treehugger Robot
017a1bb7c9 Merge "Add prebuilt_root module" 2021-04-15 06:31:53 +00:00
Treehugger Robot
b06a4bd400 Merge "Transform paths to headers in include dirs to take package boundaries into account." 2021-04-15 05:42:40 +00:00
Treehugger Robot
638830a5a0 Merge changes from topic "userdebug_plat_sepolicy.cil_Android.bp"
* changes:
  Add debug ramdisk variant
  Add path tests for ramdisk
2021-04-15 05:22:35 +00:00
Treehugger Robot
d820c37425 Merge "Fix evaluation order of (Cfi|Memtag) exclude paths." 2021-04-15 03:49:38 +00:00
Treehugger Robot
ccaf57c65a Merge "Truncate vbmeta to 64KB" 2021-04-15 02:06:01 +00:00
Jooyung Han
e134d09831 Add 'merge' command to conv_linker_config
'merge' command can merge multiple configurations.
It just delegates protobuf's msg.MergeFrom(other_msg).

  usage: conv_linker_config merge -o out.pb -i a.pb -i b.pb ...

out.pb will have all keys from multiple inputs.

Bug: 181093750
Test: m
Test: MicrodroidTestCase
Change-Id: Ibf715aa0bcc1e5c82c85a9af8fe7dca8d6ab68ad
2021-04-15 10:58:21 +09:00
Inseob Kim
f84e9c05e2 Add debug ramdisk variant
A module will be installed to debug_ramdisk (or
debug_ramdisk/first_stage_ramdisk if recovery as boot is true) if
debug_ramdisk is set to true.

Bug: 184004542
Test: soong test
Change-Id: Ic5a4d27407e506fffa462de2149e0785f11b2ac7
2021-04-15 00:56:30 +00:00
Rupert Shuttleworth
c143cc50dd Transform paths to headers in include dirs to take package boundaries into account.
This allows the following cc_library_static targets to build with bp2build:

- libc_freebsd
- libc_freebsd_large_stack
- libc_openbsd_ndk
- libc_gdtoa  (* but still fails for mixed builds)
- libc_aeabi
- libc_static_dispatch
- libc_dynamic_dispatch

This also allows a number of other cc_library_static targets to progress further in their builds.

Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/...

Change-Id: I71742565c16594448a41a6428a5c993171ec4cb4
2021-04-14 17:05:36 -04:00
LuK1337
fb545bf2f3 Make sure prebuilt modules are exported before attempting to use them
Fixes an issue where prebuilt module under unexported soong
namespace overrides source modules when flag `prefer` was set.

Test: Create two modules under the same name, use cc_library_shared
      for first one and cc_prebuilt_library_shared with `prefer` set
      to true.
      Then put the prebuilt module under `soong_namespace {}` and
      attempt to compile it without appending the namespace path to
      product namespaces.
      Without this change it should fail to resolve the
      cc_library_shared module even if it should be available.

Change-Id: I506641218e6af94a0b5f048e09e06d83602e50f5
2021-04-14 22:55:22 +02:00
Nikita Ioffe
bc03588bec Generate hashtree for compressed apexes
Bug: 184615428
Test: flashed build on device, checked compressed apex has hashtree
Change-Id: I83828f581e84504c47047e7692817da5c44a6ec1
2021-04-14 21:35:24 +01:00
Jooyung Han
a0436a3928 make linker_config OutputFileProducer
so that they can be used in `srcs` of genrule.

Bug: 181093750
Test: m
Change-Id: Ia54554f1e81d766a970c9c4e02906ef9c0d85934
2021-04-15 05:27:48 +09:00
Jooyung Han
c6a91ec6d8 Allow uninstallable linker_config to be packaged
Till now, linker_config has to be installable to be packaged. With this
commit, packaging modules like filesystem now can include uninstallable
linker_config. This can help when linker_config can conflict with other
partitions e.g. system.

Bug: 185391776
Test: build filesystem with uninstallable linker_config
Change-Id: Ic2cece08be73749e6bd1f813152c67f8e3904c20
2021-04-15 05:27:48 +09:00
Nikita Ioffe
05b999ecb2 Don't compress apexes produced by apex_test module
AFAIU, apex_test is used by modules to create a test version of their
apex for e2e testing of the update flow. Since it's a regular pattern
for test_apex to use the same apex_defaults as the "production" apex,
and due to the fact that we are only going to compress pre-installed
version of the apex, it makes sense to explicitly not compress
test apexes.

Bug: 185082717
Test: build test_com.android.media verified it's not compressed
Test: atest sdkextensions_e2e_tests
Change-Id: I1179235efe068fefc582369a4218ba967e7efbf3
2021-04-14 21:08:13 +01:00
Ivan Lozano
d466b4a1f5 Merge changes from topic "rust_vendor_support"
* changes:
  Add more Rust vendor image support.
  rust: Use new common image mutator interface.
2021-04-14 19:02:49 +00:00
Ivan Lozano
e8dfbdfb59 Merge "cc: Create a common image mutator interface" 2021-04-14 18:53:18 +00:00
Paul Duffin
85dee5d8fb Move hidden API metadata file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPISingleton into the platform_bootclasspath.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ib25be3618e30a83dc2929a18062eb58eefdabefb
2021-04-14 18:54:55 +01:00
Paul Duffin
22c743170d Move handling of prebuilt hiddenapi-index.csv to hiddenapi singleton
This change moves the handling of the prebuilt hiddenapi-index.csv file
from the hiddenAPIIndexSingleton and removes that as it no longer
needed.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Iab1b39a57f28caa855c48538aa2230795f2c3271
2021-04-14 18:54:45 +01:00
Paul Duffin
00b2bfdea5 Move hidden API index file rule to platform_bootclasspath
This change moves the monolithic hidden API index file creation rule
from the hiddenAPIIndexSingleton into the platform_bootclasspath. It
also moves the corresponding test from java/hiddenapi_singleton_test.go
to java/platform_bootclasspath_test.go.

Bug: 179354495
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ia295d0f7ae9b51ea816f16921aa42339ed91704e
2021-04-14 18:54:29 +01:00
Thiébaud Weksteen
4fab05a2ad rust: do not strip static library
Test: m
Change-Id: I7dbaf8be625acae89b0a62ea13d0b250c6542a79
2021-04-14 19:20:07 +02:00
Tri Vo
9e7bfdae31 Merge "rust: Switch rust_fuzz to HWASan" 2021-04-14 16:49:27 +00:00
Paul Duffin
08abf7274a Merge "Ensure boot jar modules have been initialized properly for hidden API" 2021-04-14 15:53:15 +00:00
Lukacs T. Berki
b448c34bfd Make integration tests clean up after themselves.
Well, this was embarrassing.

Test: These tests.
Change-Id: Id25f9f1f4bd2d2d01d49d11a5af8eb50dd51fe6e
2021-04-14 16:45:22 +02:00
Artur Satayev
97259dc625 Generate classpaths.proto config for *CLASSPATH variables.
Instead of embedding "raw" values from PRODUCT_*_CLASSPATH variables
into /etc/classpath, encode them into classpaths.proto binary format.

Existing platform_bootclasspath{} module is used to generate the whole
monolithic config to begin with. Later, the config will be split
among individual bootclasspath_fragment and
systemserverclasspath_fragment modules.

Bug: 180105615
Test: m && launch_cvd
Change-Id: Ia66f521f8976ff78a62ecf91131d26db21064de7
2021-04-14 15:03:23 +01:00
Jiyong Park
f398995a64 Merge "SdkSpec is fully using ApiLevel" 2021-04-14 13:27:05 +00:00
Inseob Kim
d9580b84a2 Add path tests for ramdisk
Ramdisk path tests existed once, but have been removed for whatever
reason. This change revives the tests.

Test: soong test
Change-Id: Ibade91fbe3e044f772a50df15f448b04aa12d807
2021-04-14 12:24:45 +00:00
Anton Hansson
e9d2f9fcda Merge "Remove special case code for obsolete files" 2021-04-14 09:22:51 +00:00
Paul Duffin
1ba246732d Ensure boot jar modules have been initialized properly for hidden API
Checks to make sure that every module that is part of the platform
bootclasspath has been initialized property for hidden API processing.

Bug: 177892522
Test: verified that the out/soong/hiddenapi/... files are unchanged
      by this change
Change-Id: Ic4368963f2011784b537b8aebf5ef97ea22b2db5
2021-04-14 09:08:02 +01:00
Paul Duffin
35584eeb93 Merge "prebuilt_apex created ApexInfo must not include prebuilt_ prefix" 2021-04-14 08:03:44 +00:00