Commit graph

35624 commits

Author SHA1 Message Date
Martin Stjernholm
0780372ae8 Use oatdump rather than oatdumpd for boot jar boot.*.oatdump.txt files.
oatdumpd isn't available as a prebuilt.

Test: m SOONG_CONFIG_art_module_source_build=false droid
Bug: 172480615
Change-Id: I70317eb8f253272d629f23063cbe265d556caad3
2021-05-18 21:57:28 +01:00
satayev
f1d8819d7d Merge changes I8e8e8b01,Ifb1f54d5,I9986e64f
* changes:
  Add systemserverclasspath_fragments property to apex.
  Add "contents" property to systemserverclasspath_fragment.
  Move classpaths.proto related info into a separate provider.
2021-05-17 22:40:08 +00:00
satayev
3d3a2218a6 Merge "Add no-op systemserverclasspath_fragment module." 2021-05-17 22:37:29 +00:00
Christopher Parsons
72a13e8d40 Merge "bp2build: support generated sources and hdrs" 2021-05-17 22:13:09 +00:00
satayev
333a1732b1 Add systemserverclasspath_fragments property to apex.
- All contents of the fragment are added as java_lib dependencies.
- Generated classpaths.proto is added into etc as required.

Bug: 180105615
Test: m nothing
Change-Id: I8e8e8b019c4ca2909182f205a47deffa946de6da
2021-05-17 21:46:08 +01:00
satayev
9366a053da Add "contents" property to systemserverclasspath_fragment.
Similar to bcp_fragment's contents, this property lists all java library
contributions made by this fragment.

Bug: 180105615
Test: m nothing
Change-Id: Ifb1f54d5db290fffaa31933d15207014bb72d2fb
2021-05-17 21:46:08 +01:00
satayev
14e49130bb Move classpaths.proto related info into a separate provider.
The new info struct can be easily shared with systemserverclasspath
fragments.

Bug: 180105615
Test: m nothing
Change-Id: I9986e64fdf19f4168da63c156de3dc9bcafac8d8
2021-05-17 21:46:04 +01:00
satayev
aa86bac2b0 Add no-op systemserverclasspath_fragment module.
This would allow to start introducing these modules for apexes that
contribute to SYSTEMSERVERCLASSPATH.

In follow up, it will be evolved:
- platform_systemserverclasspath would have "fragments" property to
  list all individual systemserverclasspath_fragments;
- systemserverclasspath_fragment would have "contents" property to list
  contibuting java libs;
- systemserverclasspath_fragment would generate non-empty
  classpaths.proto config within individual apexes.

Bug: 180105615
Test: m nothing
Change-Id: Ibaaa3fae5f1eab9a41ceecc1214a53be6bbc8ba6
2021-05-17 20:53:59 +01:00
Chris Parsons
484e50aa7c bp2build: support generated sources and hdrs
There are two pieces to make this work:
1. Local include paths must include $(BINDIR)-relative paths, to support
generated headers in those directories.
2. The srcs that bp2build outputs for BUILD targets must include labels
for targets given in generated_hdrs and generated_srcs.

Support for exported_generated_hdrs intentionally deferred.

This allows us to remove several targets from the bp2build denylist.
Some are moved to the mixed build denylist, because genreated headers are
still unsupported in mixed builds.
Test: bp2build.sh CI script

Change-Id: Ib4f9dac20f6445487b8dad53b91eac01f437a590
2021-05-17 14:30:07 -04:00
Paul Duffin
1bbd0626f2 Make uncompressDex available for use through hiddenAPIModule
In order for the bootclasspath_fragment to perform dex encoding on its
contents it needs to know whether the dex file is uncompressed or not.
This change makes that information available by passing it to
initHiddenAPI, storing it in hiddenAPI struct and providing access
through the hiddenAPIModule.

Bug: 179354495
Test: m droid
Change-Id: I913416b4836766de194203fd8ed5124b61dfa3dd
2021-05-17 18:38:06 +01:00
Paul Duffin
45897dd663 Remove hiddenAPI.primary field
Now that the individual modules no longer participate in the generation
of the monolithic files it is no longer necessary to select a single
primary module to provide the information they need.

Bug: 179354495
Test: m droid
Change-Id: If09796de710927e3e3f2ccecad0b57ca5fce5dc9
2021-05-17 18:38:06 +01:00
Paul Duffin
74d18d1d6f Merge initHiddenAPI and hiddenAPIUpdatePaths
These two methods did very similar jobs and merging them together
simplifies the behavior.

Bug: 179354495
Test: m droid
Change-Id: Ibe1a23d54105e6a0e5693079cd8743679301fc85
2021-05-17 18:38:06 +01:00
Paul Duffin
66cdbf07ef Remove configurationName from java library and hidden API
The configurationName was intended to separate the name of the module
from the name used in configuration (such as BootJars) so that the
child implementation library of a java_sdk_library on the bootclasspath
would have hidden API encoding performed on it just as for the main
java_library embedded within the java_sdk_library.

While that did use to work it no longer does as the test added in the
preceding change proves. It is not surprising that this regression does
not appear to have caused any issues as the the child implementation
library is only a build time artifact and not used at runtime.

In future the only modules that will require hidden API encoding are
those that are part of a bootclasspath module so there is no point in
maintaining this capability.

Bug: 179354495
Test: m droid
Change-Id: Ief8136fa9e98600cdd8d36108ec22edc2ebd7c69
2021-05-17 18:38:06 +01:00
Paul Duffin
0d586581d1 Add a test for hidden API encoding of java_sdk_library
Fill a hole in the testing of hidden API encoding. Some comments in the
code indicate that the child implementation java_library of a
java_sdk_library should have hidden API flags encoded in its dex jar
just like the embedded java_library. However, this test proves that it
is not working. This will be fixed in a follow up change.

Bug: 179354495
Test: m nothing
Change-Id: Ia581a17f1e48dff252d17f16bf76adf039f46b60
2021-05-17 18:38:06 +01:00
Paul Duffin
e8b0169bb4 Merge "Add temporary restriction on hidden API processing" 2021-05-17 17:16:33 +00:00
Lukács T. Berki
b1caeb0768 Merge "Print free disk space when starting build_test." 2021-05-17 14:47:15 +00:00
Lukács T. Berki
33ebde3bce Merge "Refactor GetTargetProperties()." 2021-05-17 14:17:54 +00:00
Treehugger Robot
38db1b0974 Merge "Don't install rust modules that are never installable" 2021-05-17 13:55:49 +00:00
Lukacs T. Berki
4070bcdcde Print free disk space when starting build_test.
This is so that we know whether the attached bug is caused by the disk
being full.

Bug: 187870338
Test: Presubmits.
Change-Id: Ib09f51d971de556e5737ad2754836f2a12f60281
2021-05-17 14:33:01 +02:00
Lukacs T. Berki
5f51839d0d Refactor GetTargetProperties().
This is so that its implementation is more amenable to implementing
`target: { android_arm: { ... } }` and the like.

In addition, add a Boolean return value to getChildPropertyStruct() and
use it at every call site so that we only merge property structs when it
makes sense.

Bug: 187530594
Test: Presubmits.
Change-Id: I1b441c14b3d90a12b84dc89f82dd053978e89c7e
2021-05-17 14:04:04 +02:00
Rupert Shuttleworth
0096079bbc Make bazel build //external/... work by only allowing "approved" existing BUILD files in the Bazel workspace.
Test: bazel build //bionic/... //external/... //frameworks/... //system/...

Test: ./build/bazel/scripts/run_presubmits.sh

Change-Id: I91865ca87c6535053e7a14d2526ff3ce0991bfea
2021-05-17 06:34:53 -04:00
Paul Duffin
70cfdff3da Add temporary restriction on hidden API processing
Hidden API processing of a bootclasspath_fragment requires the fragment
provides information, such as dependencies on other fragments and flag
files to override the default flags. Failing to do so will cause hidden
API generation to either fail or to generate different flags to that
generated by the hidden API processing done by platform_bootclasspath
which will cause the build to fail.

Previously, this was handled by only performing hidden API processing
for those modules that provide stub libs and relied on there only being
bootclasspath_fragments defined for ART (which already supports hidden
API processing), and Conscrypt and I18n neither of which provide stubs.
Unfortunately, that can no longer be relied upon due to a couple of
recent changes:
1. A java_sdk_library in a bootclasspath_fragment's content property is
   automatically treated a stub library. That avoids duplication for
   most bootclasspath_fragments that provide the implementation and
   stub libraries through the same java_sdk_library. It does not affect
   either ART, conscrypt or i18n as they all define the implementation
   separately to the stubs.

2. bootclasspath_fragment modules have been defined for a number of
   android modules as they are needed for reasons other than hidden API
   processing.

In combination this meant that rules to perform hidden API processing
were being created but they were not currently being used which is
good because they fail.

However, adding the fragment to the platform-bootclasspath will cause
those rules to be used as the platform_bootclasspath module performs a
consistency check on the hidden API flags generated by each of the
fragments to ensure that they are consistent with those it generates
itself.

The dynamic bootclasspath work will need to add fragments to the
platform_bootclasspath and without this change that would be blocked
until all fragments had been switched to generating hidden API flags
properly.

This change adds a new fragments property to the bootclasspath and
disables hidden API processing for everything other than ART and tests
if the fragments property or stub libs is empty.

Bug: 179354495
Test: - Before making this change.
      - Add com.android.os.statds-bootclasspath-fragment to platform-bootclasspath.fragments
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - hidden API processing fails in statsd as it cannot find java.lang.Object.
      - After making this change, run the above command again and it should pass.

Change-Id: Ifbb362f8fcfb2c06595fbd5ae39421b536e329ef
2021-05-17 07:39:10 +01:00
Jiyong Park
bf8147ab66 Don't install rust modules that are never installable
rust modules that return false on EverInstallable don't get installed.
Currently, proc_macro modules are such ones.

Bug: 187810017
Test: m microdroid on aosp_arm64 and inspect that *.dylib.so files
from host x86_64 are not there.

Change-Id: Ia57289a04dbff6e4f2ce2733c60de5338305904c
2021-05-17 13:27:20 +09:00
Treehugger Robot
37ef223972 Merge "Order apex files by destination path not source path" 2021-05-17 04:07:20 +00:00
Paul Duffin
5606029007 Order apex files by destination path not source path
The destination path, i.e. the path the file will occupy in the APEX is
more stable and less likely to change than the source path, i.e. the
path of the file in the out or source directories. This makes it easier
to diff changes in the copy commands.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
      - verify that this does change the order of copy_commands for the above apexes
Change-Id: Ia9ebe8418c59903cd738ee0fb9262b3f32a4ddbf
2021-05-16 22:28:26 +01:00
Elliott Hughes
11bc0eef0c Merge "cc/linker.go: check min_sdk_version." 2021-05-15 00:35:21 +00:00
Treehugger Robot
32ef4e25b8 Merge "Define vsr_min_shipping_api_level to cc_test" 2021-05-15 00:21:11 +00:00
Treehugger Robot
718ac9276b Merge "Add usesTargetFiles option in dexpreopt_gen" 2021-05-14 23:39:49 +00:00
Dan Albert
7f0e9f970c Merge "Add script for building and archiving rustdocs." 2021-05-14 22:52:35 +00:00
Liz Kammer
374623f91f Merge "Define a narrower context for getting arch props" 2021-05-14 21:26:29 +00:00
Elliott Hughes
852edc147a cc/linker.go: check min_sdk_version.
Checking sdk_version broke mainline code that compiles against a current
API level but needs to be able to run on an older API level.

Bug: http://b/187907243
Test: treehugger
Change-Id: If1192ef2809e39b043f0a384775b6c9e3a8bd619
2021-05-14 14:03:53 -07:00
Dan Albert
4d247476ac Add script for building and archiving rustdocs.
Test: OUT_DIR=out DIST_DIR=dist build/soong/scripts/build-rustdocs.sh
Bug: http://b/162741284
Change-Id: Ie94318ed9cc99024a27d414bfecc083b2fa96b57
2021-05-14 13:18:07 -07:00
Liz Kammer
b6dbc87c97 Define a narrower context for getting arch props
go best practices recommend defining an interface where it is used and
limiting the scope

Test: go test soong tests
Change-Id: I96293ba594d37360b1a8910161477133d4aafdd9
2021-05-14 15:14:40 -04:00
Paul Duffin
fdd9743609 Merge "Refactor special handling of hidden API encoding for master-art" 2021-05-14 18:53:43 +00:00
Paul Duffin
b7c78731d4 Merge "Stop generating unnecessary hidden API rules" 2021-05-14 18:53:08 +00:00
Paul Duffin
21f62ef867 Merge "Generate monolithic hidden API files direct from class jars" 2021-05-14 18:52:11 +00:00
Paul Duffin
2bc8b3a646 Merge "Dedup hidden API rule generation" 2021-05-14 18:51:10 +00:00
Paul Duffin
b0f20d134f Merge "Allow shared libraries on bootclasspath" 2021-05-14 17:03:03 +00:00
Bill Peckham
054619d198 Merge "Fix missing headers for vndk snapshotted libc" 2021-05-14 16:52:11 +00:00
Colin Cross
9debc21ca3 Merge "Add order-only dependency on shared libraries" 2021-05-14 16:48:57 +00:00
Jeongik Cha
4dda75e73e Add usesTargetFiles option in dexpreopt_gen
For running dex2oat on the target_files, the paths should be use the
device install path instead of the path starting with $(OUT).
So add usesTargetFiles option and basePath option which indicates
extracted path. With those options, the path is replaced with
$(basePath)/$(device path)

And also, add DexPreoptImageDeviceLocations in the config which refers
to the boot image path(without arch) on the device. Because
DexPreoptImage related device path was missing.

Bug: 158843648
Test: dexpreopt_gen -usesTargetFiles -basePath (extract path) and then
check if paths in the generated shell script are based on on-device
path.

Change-Id: I9667fadbf3b7c6f770e0d1bcbee5d67c1ecd8a3d
2021-05-15 00:29:49 +09:00
Paul Duffin
79fd3d728c Allow shared libraries on bootclasspath
A previous change treated this as an error in order to try and detect
issues with misconfigured java_sdk_library modules and/or
bootclasspath_fragment modules but unfortunately this is not always an
error, e.g. when migrating a library that was a shared library to the
bootclasspath.

This change stops treating that as an error.

Bug: 179354495
Test: m nothing
Change-Id: I4a833ab5f4caf86c6cd340090fc65d2c2f141512
2021-05-14 16:14:17 +01:00
Paul Duffin
b6f53c064e Refactor special handling of hidden API encoding for master-art
Instead of encoding the hidden API with an empty set of flags when the
monolithic flags are not available this simply disables encoding
altogether which should have the same behavior at runtime.

This change also removes the unused flags field in hiddenAPISingleton
which was set but never read.

Bug: 179354495
Test: m nothing
Change-Id: I32d5825e5271829993dd4e5be4d4ee1b22fa7b22
2021-05-14 15:57:10 +01:00
Paul Duffin
afaa47c74a Stop generating unnecessary hidden API rules
The rules to extract hidden API information from an individual module
in order to create module specific files are no longer necessary as
the monolithic files are created directly from the module's class jars
and not the module specific files.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I573ac17f3ea5da5a2a7e4f08718160dacca71c0c
2021-05-14 15:57:10 +01:00
Paul Duffin
537ea3d04c Generate monolithic hidden API files direct from class jars
Previously, the monolithic hidden API files, e.g. hiddenapi-index.csv
file, were generated in two steps. First, each module created its own
files using the information in its class jars. Then, the monolithic
files were created by merging those module specific files into a larger
file.

This change switches to generating the monolithic files directly from
the class jar files and bypassing the intermediate files.

In order to ensure that this change did not change the monolithic files
it is necessary for the hiddenapi-metadata.csv to go through a
reformatting step. Hopefully, this will be able to be removed in a
follow up change.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I5a78e747516014b7c0f402a4b4431b14be6a84b2
2021-05-14 15:57:04 +01:00
Paul Duffin
850e61f234 Dedup hidden API rule generation
Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I54b677c97712a7573ff5eed017c1b2198b80bc51
2021-05-14 14:53:19 +01:00
Inseob Kim
cc12d90753 Fix missing headers for vndk snapshotted libc
VNDK snapshot depends on addExportedGeneratedHeaders to grab generated
headers. This change adds a missing addExportedGeneratedHeaders call to
make libc's headers captured correctly.

Bug: 181326838
Test: VNDK_SNAPSHOT_BUILD_ARTIFACTS=true m dist vndk && unzip -l
out/dist/*.zip

Change-Id: Iaa89b5255c836761a26c77bb27f35f768f49039e
2021-05-14 20:15:55 +09:00
Paul Duffin
098c878838 Make hidden API handle jacoco-stubs consistently
Previously, when code coverage was enabled the monolithic hidden API
processing would add jacoco-stubs to public APIs only as that would
make them accessible for everyone. However, the
art-bootclasspath-fragment added jacoco-stubs to public, system and
test APIs as that was the simplest way of handling it and while the
extract APIs were unnecessary they would not change behavior.

Unfortunately, that lead to a difference in the flags generated which
caused the check that verifies flags are consistent between the
monolithic and modular files to fail.

This change adds jacoco-stubs to system and test APIs for the
monolithic hidden API processing to ensure consistency.

Bug: 179354495
Bug: 188143639
Test: m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true out/soong/hiddenapi/hiddenapi-flags.csv
      - ran before making the change to reproduce the problem and after
        to verify the fix.
Change-Id: Icbd15f0ece871a8c44d9e4a73fd0f7acc3760bba
2021-05-14 10:19:46 +00:00
Justin Yun
a364cfb69d Define vsr_min_shipping_api_level to cc_test
test_options in cc_test can have vsr_min_shipping_api_level to define
the minimum shipping api level that checks vendor api level in
addition to the device shipping api level. It can be used for the
tests that have dependency on the vendor api level.
vsr stands for the vendor software requirements.

Bug: 181284704
Test: manual test
Change-Id: I7f2cecd2818d35d86f7c08cb220e0c9db71fe0a8
2021-05-14 12:47:18 +09:00
Colin Cross
58dd6a92e5 Add order-only dependency on shared libraries
Improve the dependency accuracy by adding an order-only dependency
on the shared library alongside the existing depenency on the
table of contents file.  The dependency won't cause the module
to be rebuilt when the the shared library changes, but will allow
anything using the ninja graph for sandboxing to know the dependency
exists.

Test: manual
Change-Id: I3accbcabee62fa0ad3eb3d1aaedc5a4bffe27308
2021-05-13 18:01:24 -07:00