Commit graph

748 commits

Author SHA1 Message Date
Treehugger Robot
b02128ad0b Merge "Remove libbinder_headers from apex_available allowlist" 2021-02-06 03:20:47 +00:00
Jooyung Han
59d0931de7 Remove libbinder_headers from apex_available allowlist
libbinder_headers now sets apex_available property.

Bug: 150999716
Test: m
Change-Id: If58cfeafcc13fc36be8e44a66f2f472460fd4c8d
2021-02-05 15:32:40 +09:00
Liz Kammer
356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Paul Duffin
a1d6025a49 Add boot_images to apex
Previously, the apex module had to hard code behavior specific to the
art apex module in order to include the art boot image. This change
adds support to the apex module to allow the boot images to be
specified per apex.

In combination with a change to add the "art-boot-image" to the ART
apex this allows the custom code for handling the art boot image in
apex to be removed.

That custom apex code also included the logic to ensure that the
GlobalSoongConfig was initialized for use by the dex_bootjars
singleton. That logic has been moved from the APEX to the boot_image
module. That ensures that it will be run if and only if a boot_image
module is present in the checked out repos. So, limited manifest
checkouts which do not contain the art or frameworks/base repos (which
is where the boot_image modules are defined) will not attempt to run
this logic, which would fail because dex2oat would not be present.

Bug: 177892522
Test: m droid
Change-Id: I02d25fbef6e864e31eb5e0f4eb50358c79486db0
2021-01-30 12:45:07 +00:00
Jooyung Han
ed124c308f apex: checks min_sdk_version for preview/current
If we don't check "current", it won't be checked even in the finalized
branch.

If we don't check "preview", it should be done during the SDK
finalization. It'd be better done before the SDK finalization regarding
that setting min_sdk_version is to get approval from deps library owners.

Bug: 177833148
Test: m (soong tests)
Change-Id: I712b61cfe5a134fbb69c73956d26fb3a1e5c011e
2021-01-26 12:09:07 +09:00
Sophie Zheng
9b6596bb60 Merge "Revert "Revert "Automated NDK API coverage used by Mainline modules build integration.""" 2021-01-26 00:13:46 +00:00
sophiez
6bde0b5f9d Revert "Revert "Automated NDK API coverage used by Mainline modules build integration.""
This reverts commit 556b2ad77e.

Original change-id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15

Reason for revert: Add excutable permission to the gen_ndk_backedby_apex.sh.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Change-Id: Ib587ba200cd7f2f61d478452b43c329d72b2de06
2021-01-25 19:52:10 +00:00
Riddle Hsu
f17b07fc52 Merge "Revert "Automated NDK API coverage used by Mainline modules build integration."" 2021-01-25 06:15:29 +00:00
Riddle Hsu
556b2ad77e Revert "Automated NDK API coverage used by Mainline modules build integration."
This reverts commit 87ff51db84.

Reason for revert: build breakage b/178327796

Change-Id: I51c56938540794097eda579ebe47fa5c0a4d8c12
2021-01-25 01:44:34 +00:00
Sophie Zheng
0bf77198b5 Merge "Automated NDK API coverage used by Mainline modules build integration." 2021-01-22 22:15:21 +00:00
sophiez
87ff51db84 Automated NDK API coverage used by Mainline modules build integration.
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Change-Id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15
2021-01-21 18:08:50 +00:00
Treehugger Robot
d684e04247 Merge "LOCAL_REQUIRED_MODULES from apex has ":32" or ":64" suffix" 2021-01-21 14:34:37 +00:00
Ulyana Trafimovich
0caabb6e27 Merge "Support building boot images with disabled dexpreopt." 2021-01-20 17:46:02 +00:00
Bill Peckham
1c8e5e817e Merge changes from topic "prebuilt-hiddenapi"
* changes:
  Supply LOCAL_SOONG_DEX_JAR for java_import with compile_dex.
  Enable an apex to include a java_import.
  Enable prebuilt hiddenapi CSV files.
2021-01-20 17:43:48 +00:00
Ulya Trafimovich
a4a1c4ee98 Support building boot images with disabled dexpreopt.
Bug: 176171716
Test: build_mainline_modules.sh and ensure that the ART boot image is
    built: `find out -type f -name '*.art'` is nonempty for every arch.
Change-Id: Ibc45581eef2b205c750a30709780cf659ba7cfa1
2021-01-20 14:03:07 +00:00
Jiyong Park
57621b29c4 LOCAL_REQUIRED_MODULES from apex has ":32" or ":64" suffix
Currently, when a module is included in an APEX, the dependencies of the
modules are listed in LOCAL_REQUIRED_MODULES of the APEX. There are two
purposes for this:

1) for native dependencies, they are installed to
$(TARGET_OUT)/apex/<apexname> directories which isn't packaged as an
*.img. However, as a side effect of the installation, their symbol files
are placed under $(TARGET_OUT)/symbols directory to aid debugging.

2) to implement the symlink optimization. When the APEX is not
updatable, the dependencies are not included inside the APEX, but
installed directly to /system partition because the same files might be
used outside of the APEX. The files in the APEX are replaced with
symlinks to the system copy.

So far, the module name like "libfoo" was directly used in
LOCAL_REQUIRED_MODULES. This becomes problematic when only a single arch
variant of the module is used by the APEX. The build system will install
both arch variants to the system partition.

This change fixes the problem by appending ":32" or ":64" suffix
when composing LOCAL_REQUIRED_MODULES.

Bug: N/A
Test: m
Test: Cherry-pick I285c5d1bb9b27265c8589f2588d95eafa324d412 and its
dependencies from internal master. `m nothing` doesn't show the artifact
path requirement error.

Change-Id: I78feae1d5b18f93b0f984d3b1558812fd1689a96
2021-01-20 20:44:35 +09:00
Ryan Prichard
809eac3057 Merge "Switch platform EH unwinder to prebuilt LLVM libunwind" 2021-01-20 00:10:59 +00:00
Bill Peckham
a41a6963b4 Enable an apex to include a java_import.
With the addition of the compile_dex property to the
java_import module it becomes possible to include a
java_import module in an apex. This change allows the
dependency and adds a test.

Bug: 177228901
Test: m nothing
Test: new TestApexWithJavaImport
Change-Id: I9336dade1857109e2fd21f7d57e1dc4abc4a402c
2021-01-19 18:15:03 +00:00
Paul Duffin
064b70c918 Export dex implementation jars from prebuilt_apex
Dexpreopt and boot jars package check all require access to dex
implementation jars created for java_library and java_sdk_library. They
were available when building from source but not when building from
prebuilts, even though they are embedded within the .apex files that
are referenced from prebuilt_apex.

This changes adds support to prebuilt_apex to export the dex
implementation jars and updates java_import to use those exported dex
implementation jars.

In a source build dexpreopt/boot jars package check access the apex (or
platform) specific variant of a java_library, e.g. core-oj, from which
it retrieves the dex implementation jar path.

After this change in a prebuilt build dexpreopt/boot jars package check
behave in the same way except in this case they retrieve the dex
implementation jar path from the apex (or platform) specific variant of
the java_import, e.g. core-oj.

The work to export files from a `.apex` file for use by other modules
is performed by a new `deapexer` module type. It is not used directly
in an `Android.bp` file but instead is created implicitly by
`prebuilt_apex`,

In order to do that this contains the following changes:
* Adds a new `dexapexer` module type to handle the exporting of files
  from the `.apex` file.
* Adds an exported_java_libs property to prebuilt_apex to specify the
  set of libraries whose dex implementation jars need exporting.
* Creates apex specific variants of the libraries listed in the
  exported_java_libs property.
* Adds the set of exported files to the ApexInfo to make them available
  to the apex specific variants.
* Prevents the prebuilt_apex variants from being merged together as
  they will not be compatible.
* Modifies java_import to use the exported file for variants of a
  prebuilt_apex.
* Adds a ninja rule to unpack (using deapexer) the contents of the
  prebuilt_apex's apex file, verify that the required files are present
  and make them available as outputs for other rules to use.
* Some minor refactorings to support these changes.
* Adds tests to cover prebuilt only, prebuilt with source preferred,
  and prebuilt preferred with source.

Test: m nothing
Bug: 171061220
Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
2021-01-15 18:14:10 +00:00
Ryan Prichard
b35a85ee8b Switch platform EH unwinder to prebuilt LLVM libunwind
For the platform libc++ STL, remove the
-Wl,--exclude-libs,libunwind_llvm.a argument, which is redundant with
the same argument in deviceGlobalLdflags.

Bug: http://b/153025717
Test: device boots
Change-Id: Idd7791d52f74aab2d5f59419fb75f841fc29a2eb
2021-01-13 19:51:06 -08:00
Mohammad Islam
fdb9e89642 Merge "Add property to apex soong modules to forcefully compress an APEX" 2021-01-12 13:51:50 +00:00
Jiyong Park
12a719c0fc android_filesystem modules can be included in APEX
android_filesystem modules can be included in APEX via the new
`filesystems` property. The filesystem images are placed at
./etc/fs/<modulename>.img.

Bug: 172413888
Test: m nothing

Change-Id: I215ca7a32ff1988a0de4e1f71397684e189839ea
2021-01-07 15:35:25 +09:00
Jiyong Park
4da079732d Add TARGET_FORCE_APEX_SYMLINK_OPTIMIZATION
The APEX symlink optimization is a build-time trick to save the
storage/RAM usage of APEX by replacing some files in APEX with symlinks
to the files in the system partition. The optimization however is
automatically turned off for 'updatable: true' APEXes because doing the
optimization for them will hide the sys-health implication until when
the APEXes are built unbundled (i.e. prebuilt) and thus the
optimization is impossible.

TARGET_FORCE_APEX_SYMLINK_OPTIMIZATION forcibly disables the safety net.
When it is set to true, the symlink optimization is done regardless of
the 'updatable' property. This is useful for some of the devices like Go
where most APEXes (even the 'updatable: true' ones) should be
effectively non-updatable.

Bug: 175630508
Test: TARGET_FORCE_APEX_SYMLINK_OPTIMIZATION=true m and check that
updatable APEXes have symlinks to system libs

Change-Id: I26f72e5d5ebccc2d1e09c2a2f743db14937eb39a
2021-01-05 21:01:11 +09:00
Jaewoong Jung
18aefc1977 Remove unnecessary snake case variables.
Test: m nothing + TreeHugger
Change-Id: I99f7162944daa6c57c6ae4763261e108bb5cb6b1
2020-12-22 12:38:35 -08:00
Mohammad Samiul Islam
a8008f9595 Add property to apex soong modules to forcefully compress an APEX
This makes creating compressed APEX artifacts for testing simpler.

Bug: 172911820
Test: atest apex_compression_platform_tests
Change-Id: I77b8bea4662c273420e7b37d5fde181b86c9cb78
2020-12-22 10:47:50 +00:00
Colin Cross
e6a83e65d1 Fix NDK build in downstream branches
The NDK build in downstream branches with prebuilt apexes break
with multiple rules to copy the apex after
I8038ed5c6f05c989ac21ec06c4552fb3136b9a7a renamed SkipInstall to
HideFromMake, which is shadowed by an existing HideFromMake method.
Call through from the existing HideFromMake as a quick-fix for the
build.

Fixes: 175911330
Test: NDK build
Change-Id: Id475671d5dad888913bfb1af2e4be1f2454c3211
2020-12-17 18:22:34 -08:00
Treehugger Robot
f1c48afc31 Merge "arch specific dependencies are supported for apex" 2020-12-17 00:15:26 +00:00
Martin Stjernholm
f2635ec434 Always track transitive dependencies for host.
This fixes a regression in https://r.android.com/1515154 when building
the host ART APEX (com.android.art.host).

Test: art/build/apex/runtests.sh
Bug: 166468760
Change-Id: I39a00019b7bfa77a865e7287c66add3566b34d87
2020-12-16 01:34:43 +00:00
Jiyong Park
59140307ec arch specific dependencies are supported for apex
The apex module type now supports arch specific native dependencies.

apex {
    name: "myapex",
    arch: {
        arm64: {
	    native_shared_libraries: ["arm64_only_lib"],
	},
    },
}

Bug: 174639241
Test: m nothing
Change-Id: I3f00aa87c480d4127b27d33e9620c4336824e937
2020-12-14 18:44:04 +09:00
Paul Duffin
d23c726b36 Add RemoveOptionalPrebuiltPrefix() helper function
Test: m nothing
Bug: 171061220
Change-Id: Iaca95efcaf3f02e066751c6e988d609ac40e048a
2020-12-11 18:13:08 +00:00
Paul Duffin
a7d6a89774 Delegate work of apexInfoMutator to ApexInfoMutator interface
Refactor the apexInfoMutator to delegate the work to an implementation
of ApexInfoMutator and then move the existing functionality into the
apexBundle.ApexInfoMutator(). This will allow a follow up change to
customize the behavior of this mutator for prebuilt_apex modules.

Test: m nothing
Bug: 171061220
Change-Id: I7ccd55818e02a606c5494e215f4370b635ddd0a5
2020-12-11 16:07:17 +00:00
Paul Duffin
08703826bd Merge "Rename apexDepsMutator to apexInfoMutator" 2020-12-11 10:06:19 +00:00
Jiyong Park
6f05a73e3e Merge "stub variants also re-exports headers" 2020-12-11 08:50:12 +00:00
Jiyong Park
1ad8e16fbe stub variants also re-exports headers
This change fixes a bug that headers are not re-exported from stub
variants of a library, if the headers are not from header libs, but from
shared or static libs. This is because only header lib dependencies
are respected for stubs variants.

The fix is as follows. 1) dependencies to the shared/static libs are
added even for stubs variants. 2) instead, in depsToPaths, they are
treated like header libs (i.e. don't contribute to linkFile) for the
stubs variants.

Bug: 174558745
Test: m

Change-Id: Iab6c77e7817055d0f2d09cb114186b30164fc231
2020-12-11 13:48:28 +09:00
Jiyong Park
b555609098 Merge "rust ffi libraries can be included in APEX" 2020-12-10 23:54:34 +00:00
Paul Duffin
949abc03eb Rename apexDepsMutator to apexInfoMutator
Generally ...DepsMutator methods add dependencies between modules but
the apexDepsMutator does not which can be confusing. This renames
apexDepsMutator to apexInfoMutator and adds some extra documentation to
clarify its function. It also renames the registered name for the
mutator and its associated providers from apex_deps to apex_info.

Test: m nothing
Bug: 171061220
Change-Id: Ic074a281215b23e982448ccff7ac075236123bee
2020-12-10 20:16:57 +00:00
Steven Moreland
35989ae98b Merge "target.apex.exclude_[shared|static]_libs to cc_* modules" 2020-12-10 19:02:02 +00:00
Jiyong Park
f2cc1b77f3 rust ffi libraries can be included in APEX
This CL fixes a bug that rust_ffi_shared libraries can't be included in
an APEX either directly (via native_shared_libs property) or indirectly.

Bug: 175105284
Test: m
Change-Id: I2e2d3e11b3901f3412b9725e47a8ebf31f93847b
2020-12-09 00:29:54 +09:00
Mohammad Samiul Islam
3cd005d347 Enable soong build tool to handle APEX compression
1. Soong can now detect PRODUCT_COMPRESSED_APEX flag
     We don't want APEX to be compressed on all devices. Only those that
     have explicitely set PRODUCT_COMPRESSED_APEX flag.
2. Handle "compressible" field in soong build rule
     On devices that supports APEX compression, all APEX will be
     compressed by default. If any apex does not want to be compressed,
     they will need to state that by setting "compressible" field to
     false
3. Can use apex_compression_tool to compress APEX
     Note we compress the APEX after it has been signed. That way, when
     we decompress we will get a signed APEX.
4. Place the compressed APEX in system with .capex extension
     This makes it easy to identify. We still preserve the original
     extension so that when we decompress, we can just rename by cuttif
     off the .capex extension.

Note: with this change, we can create a system image with compressed
APEX, but we cannot boot with it since platform doesn't know how to
handle .capex files. Platform support will be added on follow up CLs.

Bug: 172911362
Test: OVERRIDE_PRODUCT_COMPRESSED_APEX=true m (apex_test.go)
Test: observed $OUT/system/apex has .capex files
Change-Id: I20ac4c4ceb521924c751a6017f979b2d808fdded
2020-12-08 13:06:25 +00:00
Jiyong Park
e386754898 target.apex.exclude_[shared|static]_libs to cc_* modules
The property is used to exclude some shared and static libs when the
module is built for an APEX.

Bug: 166468760
Test: m
Change-Id: I0dcaa4ae94c01aa00dc5539c60d3054c57fd8824
2020-12-04 18:09:39 +09:00
Treehugger Robot
d348c41af5 Merge "Don't use module names with "prebuilt_" prefixes in make dependencies." 2020-12-04 06:23:51 +00:00
Martin Stjernholm
2856c66c99 Don't use module names with "prebuilt_" prefixes in make dependencies.
The distinction between foo and prebuilt_foo doesn't exist in make, so
this could create invalid dependencies when both source and prebuilt
modules exist and the prebuilts are preferred.

Test: `m` with ART module prebuilts with prefer:true in the tree
Test: m nothing
Bug: 172480615
Change-Id: I90b76a8f38493882b3330d1b6789159852e59d55
2020-12-04 01:00:23 +00:00
Kiyoung Kim
2ba87f466c Merge "Add linkerconfig to Runtime APEX" 2020-12-04 01:00:22 +00:00
sophiez
c80a2b392e Automated NDK API coverage used by Mainline modules build integration.
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Forrest run: android-build.googleplex.com/builds/forrest/run/L97800000742565481

Change-Id: I39e5aa7c74eff46aba06808642b2bd67ae45ba1e
2020-12-03 05:45:20 +00:00
Kiyoung Kim
4098c7eddf Add linkerconfig to Runtime APEX
Add linkerconfig to Runtime APEX module.

Bug: 165769179
Test: Cuttlefish boot succeeded
Change-Id: I31c4bfa2ce88b64d7100f227ec289416db253b13
2020-12-02 11:33:54 +09:00
Jiyong Park
fdea25781f Merge "rust modules can be included in apex" 2020-11-30 15:41:28 +00:00
Jiyong Park
99644e92c8 rust modules can be included in apex
We will have some APEXes having rust binaries and libraries. So, adding
the support for the types of modules.

rust.Module now inherits from android.ApexModuleBase and implements
the android.ApexModule interface.

Bug: 172414324
Test: m

Exempt-From-Owner-Approval: rebased after +2 from the owner
Change-Id: I356ef4c45f782a6460f001e83af96d1710642d80
2020-11-30 15:40:48 +00:00
Paul Duffin
281bc00bbf Remove redundant setting of DistFiles by apexBundle
A previous change handles dist properties automatically for all module
types and as a result has made the apexBundle setting of DistFiles
redundant so this change removes that.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: If6f741b5b73b3335f4c478a6528f5f311cd633e4
2020-11-27 15:17:44 +00:00
Paul Duffin
74f05598eb Differentiate between no dist tag and an empty dist tag
Change https://r.android.com/1335521 added tag property to the Dist
struct so that it could be used to select one of a number of different
output files to copy to the dist instead of the single file that the
module type made available for dist. The output files were selected
by passing the tag to OutputFiles(tag).

Module types that wanted to support this new approach had to explicitly
set AndroidMkEntries.DistFiles = GenerateTaggedDistFiles(module).
Unfortunately, doing that had a side effect of changing the behavior of
dist entries without a tag.

That was because the change treated a tag that was not specified, as
being the same as "". So, prior to the change no tag meant use the
default dist file but after it meant use the paths returned by
OutputFiles(""). That changed the behavior of the java.Library type
which affected the behavior of the android_app module type.

Prior to the change the java_library would make the
Library.outputFile available for dist when no tag was specified. After
that change it would make Library.outputFile plus
Library.extraOutputFiles. The latter is usually empty except for
android_app which adds some extra files into there which will now be
copied to the dist. That change may have been intentional but there
was no mention of it in the change or the bug. Even if it wasn't
intentional it may still be beneficial.

Any module type that wants to add support for tags in dist runs the
risk of introducing similar changes in behavior. This change
differentiates between the tag not being set and the tag being set to
"" to avoid that possibility and to make the default behavior
explicit for those module types that have switched.

It does so as follows:
* Adds a DefaultDistTag constant that is used when the tag is not set.
  It is a string that is unlikely to be used as an actual tag as it
  does not start with a . and uses some special characters.
* The DefaultDistTag is used in MakeDefaultDistFiles(paths) to indicate
  that the supplied paths are the default ones and and also in
  GenerateTaggedDistFiles() for Dist structures that have no tag
  property set.
* The DefaultDistTag is passed to OutputFiles(tag) just in case the
  module type has explicitly defined the paths to associate with that
  tag in there. If it has then it overrides the legacy behavior. If it
  has not then it is just ignored and falls back to using the previous
  behavior.
* The java.Library.OutputFiles(tag) method explicitly handles the
  DefaultDistTag and returns Library.outputFile for it which restores
  the behavior from before the change that added dist.tag support.
* Similar change was made to apexBundle.OutputFiles(tag) in order to
  preserve its previous behaviour.
* The customModule used by TestGetDistContributions has been modified
  to also preserve its previous behavior after this change.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: Ib8f0d9307751cc2ed34e3d9a5538d3c144666f6d
2020-11-27 15:17:44 +00:00
Jiyong Park
b81b99032d Documenting apex/builer.go
Mostly documentation changes, but includes a few refactorings like
changing the variable names, reordering functions, reordering statements
in logical order, etc.

Bug: 173472337
Test: m
Change-Id: Iefdc8e463bcda60187b98e8c90661e220e8cdf40
2020-11-25 09:53:46 +09:00