Commit graph

2445 commits

Author SHA1 Message Date
Paul Duffin
254aee4843 Merge changes I9e94662d,I4b4e0dc8
* changes:
  Export implementation class jars for java_boot_libs
  Improve module filtering in hiddenapi stubFlagsRule()
2021-02-07 10:52:01 +00:00
Jooyung Han
7701309889 Merge "cc: fix version macro for stubs" 2021-02-06 01:41:31 +00:00
Liz Kammer
b7eab01167 Merge "bp2build: convert paths/module refs to Bazel label" 2021-02-05 13:39:08 +00:00
Paul Duffin
22ff0aaf51 Export implementation class jars for java_boot_libs
Hiddenapi processing currently requires access to the class
implementation jars for libraries on the bootclasspath which means that
they need to be provided as part of the prebuilts. This change modifies
the java_boot_libs property on the sdk to make those files available.

Modularization of the hiddenapi processing will hopefully remove the
need for these to be exported so this should be temporary.

Bug: 178361284
Test: m art-module-sdk
      check generated snapshot zip contains implementation jars
Change-Id: I9e94662dddb0ddb85a477ae6d27e533085147e88
2021-02-05 13:35:25 +00:00
Paul Duffin
b8887127ba Merge "Detect empty apex in ConfiguredJarList" 2021-02-05 09:30:15 +00:00
Treehugger Robot
d0347e4d32 Merge "clean up some hard-coded min_sdk_version" 2021-02-05 02:54:43 +00:00
Jooyung Han
11b0fbdbf6 cc: fix version macro for stubs
When a cc module is built against a stub, compiler passes version macro
of the stub lib. Version macro should be numeric, so codenames or
"current" should be mapped to numbers just like how ndkstubgen maps to.

* "current" -> future (10000)
* codenames -> look up api_level.json
* otherwise -> cast to int

Bug: 179329813
Test: m / soong test / manually check the output build.ninja
Change-Id: Ic0e1dd904984e161694a0b77fad5559c06a4462f
2021-02-05 11:27:57 +09:00
Liz Kammer
356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Paul Duffin
9c3ac96f1f Detect empty apex in ConfiguredJarList
Previously, ConfiguredJarList would accept an empty apex name,
e.g. ":jar" which makes no sense as every apex has to have a non-empty
name. This change makes an empty apex invalid.

In order to improve the test coverage of the TestConfiguredJarList test
this change also changes the implementation of
CreateTestConfiguredJarList([]string) to marshal the supplied strings
into a json list and then unmarshal into a ConfiguredJarList which more
closely matches how it is used at runtime.

Bug: 178361284
Test: m nothing
Change-Id: I7dfec6b4cc1923aa99746e976da0393922ef0791
2021-02-04 10:07:41 +00:00
Jooyung Han
67363e6be7 clean up some hard-coded min_sdk_version
androidx libs don't need hard-coded min_sdk_version any longer.

Bug: 158059172
Test: m
Change-Id: I1ba48b030604562dd74574ae163cb2ec01a47c4a
2021-02-04 15:34:55 +09:00
Inseob Kim
53b2427fc9 Merge "Allow common arch for recovery" 2021-02-04 06:19:28 +00:00
Paul Duffin
7e2fad3f89 Merge changes I0f746a11,I08e61a24
* changes:
  Make OutputPath implement objPathProvider
  Allow CopyDepsToZip() to work on WritablePath
2021-02-03 10:01:39 +00:00
Treehugger Robot
7a1c2b181c Merge changes from topic "mkboot"
* changes:
  Add bootimg module type
  android_filesystem supports compressed cpio format
  add prebuilt_kernel_modules module
  arch.<arch>.deps now works in android_filesystem
2021-02-02 23:26:57 +00:00
Paul Duffin
0267d49255 Make OutputPath implement objPathProvider
Needed to allow OutputPath to replace some usages of ModuleOutPath.

Bug: 179124768
Test: m droid
Change-Id: I0f746a11c53ed8c9c9193b57afc0b06c0249ff75
2021-02-02 13:33:51 +00:00
Paul Duffin
4076a75fd7 Allow CopyDepsToZip() to work on WritablePath
Makes the method more generally usable as it can work with any
WritablePath not just OutputPath. Also, makes it independent of whether
PathForModuleOut() returns ModuleOutPath or OutputPath.

Bug: 179124768
Test: m nothing
Change-Id: I08e61a24722198aa92fa2f8a2d096b1a8bf9c5c2
2021-02-02 13:32:12 +00:00
Inseob Kim
20fb5d4e24 Allow common arch for recovery
Test: build
Change-Id: I9cffd399e0d00b53c344aac6045eadcf5be78cb3
2021-02-02 20:07:58 +09:00
Jiyong Park
6446b62e9e add prebuilt_kernel_modules module
prebuilt_kernel_modules is a build rule for processing and installing
kernel module files. The module files are processed by depmod to build
modules.{load|alias|deps|softdeps} files. Debug symbols are stripped
from the module files and then all the files are installed to
./lib/modules/<kernel_version> directory.

This is the Soong-equivalent of the 'build-image-kernel-modules-dir`
macro and is expected to replace it in the future.

Bug: 178980227
Test: m

Change-Id: I772ac816824921d275869e35b93a083d239a2450
2021-02-02 00:41:57 +09:00
Jiyong Park
2136d15326 arch.<arch>.deps now works in android_filesystem
android_filesystem is a multi targets module type inhering from
PackagingBase. Since it's a multi-targets module, it is mutated only for
the common arch and therefore arch-specific properties are NOT squashed
into the matching top level property (i.e. setArchProperties is no op).

As a result, the values set in arch.<arch>.deps property were ignored.

This fixes the issue by explicitly adding the properties in
arch-specific structs and reading the properties in the matching arch.

Also added a set of tests for non-multi-target modules.

Bug: N/A
Test: m
Change-Id: Ibd4b509e73fa1760cc38b3661a08f83a6f639705
2021-02-02 00:41:28 +09:00
Jingwen Chen
041b184c92 bp2build: flatten *_defaults properties.
This CL brings the defaults mutators into bp2build, reusing the
implementation of flattening defaults from *_defaults module deps into
the modules themselves.

This is an approach to converge defaults semantics with Bazel, which
doesn't have existing equivalent abstractions, other than macros, which
require more infrastructure to be built.

Test: TH
Test: build_conversion_test.go
Change-Id: Ia1443f8e6c3eb277fa861bb96625425a22cfc6d7
2021-02-01 00:33:30 -05:00
Martin Stjernholm
c4e17317d1 Merge "Fix boot jar handling when both source and prebuilt APEXes and modules are present." 2021-01-29 13:15:02 +00:00
Martin Stjernholm
1dc0d6d7f2 Fix boot jar handling when both source and prebuilt APEXes and modules
are present.

1) The boot jar to APEX mapping is maintained by the base names for
both of them. When building with prebuilt modules and APEXes, that
means we need to take care to compare them without regard to any
"prebuilt_" prefixes.

2) VisitAllModules can visit disabled modules and both source and
prebuilt modules, so they need some conditions to skip modules that
aren't applicable for boot jars.

Test: `m droid`
Test: `m droid SOONG_CONFIG_art_module_source_build=false`
  with fresh ART Module prebuilts in place
Bug: 171061220
Change-Id: Iced269d29127bc8b8f9b3171adb60a97d115628b
2021-01-28 20:09:24 +00:00
Treehugger Robot
99afe0d442 Merge "apex: checks min_sdk_version for preview/current" 2021-01-28 13:04:52 +00:00
Jingwen Chen
a42d6417b3 Make bp2buildMutators registration local to TestContext.
The previous implementation relied on the implicit registration of Bp2Build mutators, resulting in test non-hermeticity. Refactor bp2build tests to explicitly specify the bp2build mutators under test.

Test: Soong tests

Test: TH
Change-Id: I9b9674bad1ea533b3bd31b07077a9e02c99b4c1d
2021-01-26 21:57:27 -05:00
Liz Kammer
a830f3abc2 Refactor android/paths.go to allow reuse
Extracts out the relevant pieces of ModuleContext into a minimal
interfaces necessary for path handling for Modules. This will allow
reuse in queryview by allowing it to implement only the parts of the
module interface necessary for path expansion in order to properly
handle path properties.

Test: go test soong tests
Change-Id: Iea39a365e5a2ff55071e435fb64f9e6464bbcafd
2021-01-26 12:51:02 -05:00
Jooyung Han
ed124c308f apex: checks min_sdk_version for preview/current
If we don't check "current", it won't be checked even in the finalized
branch.

If we don't check "preview", it should be done during the SDK
finalization. It'd be better done before the SDK finalization regarding
that setting min_sdk_version is to get approval from deps library owners.

Bug: 177833148
Test: m (soong tests)
Change-Id: I712b61cfe5a134fbb69c73956d26fb3a1e5c011e
2021-01-26 12:09:07 +09:00
Jingwen Chen
ab60f1268a bp2build: comment documentation for BazelTargetModule.
Test: TH
Change-Id: Id193c6305f0264122ffa98f709e3ba4ca95d435c
2021-01-24 21:21:45 -05:00
Jingwen Chen
32b4ece0c3 bp2build: automatically convert all filegroups.
See build_conversion_test.go for expected outputs.

Test: build_conversion_test.go
Test: GENERATE_BAZEL_FILES=true m nothing &&
./build/bazel/scripts/bp2build-sync.sh write && bazel build
//bionic/libc/tools/...
Test: bazel query //... --config=queryview

Change-Id: I3c54b96c0812f1ea4ab2c95da1bff3d7c5cc4006
2021-01-23 02:26:20 -05:00
Jingwen Chen
fc1ad29e0a Merge "bp2build: framework for generating BazelTargetModules." 2021-01-23 06:39:37 +00:00
Colin Cross
9b564c5a22 Merge "Fix mismatch between cc and python for windows prefer32" 2021-01-22 23:34:00 +00:00
Liz Kammer
70b1714ca6 Merge "Add conditions_default for soong config variables." 2021-01-22 13:35:30 +00:00
Jingwen Chen
7385067640 bp2build: framework for generating BazelTargetModules.
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.

See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.

Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.

Change-Id: I33a96365bd439043b13af6db9e439592e9983188
2021-01-21 22:46:11 -05:00
Liz Kammer
432bd598ae Add conditions_default for soong config variables.
Each variable can specify a conditions_default for properties to be used
when the variable is not set, not set to a true value (for bools), or is
set to a value that is not present in the module (for strings).

Test: m nothing
Test: go test soong tests
Change-Id: I76ec026da2369b407f0f530f77760f530e7958fc
2021-01-21 14:25:20 -05:00
Hyundo Moon
2b7d4e1ba4 Merge "Allow service-media-s in apex" 2021-01-21 15:00:13 +00:00
Hyundo Moon
8bee1b00ea Allow service-media-s in apex
Bug: 175511943
Test: Builds successfully
Change-Id: I5d76087d3c537205e8b8ca837c13ca1aeb641b4c
2021-01-21 02:21:23 +00:00
Colin Cross
0cec312660 Move reflect.ValueOf out of product variable loop
Calling reflect.ValueOf for every variable for every module was
using ~3 seconds of CPU time on my AOSP builds.

Test: m checkbuild
Change-Id: Idf459ad8ddf5e07f6c0df0e58e2442aaa6ab3342
2021-01-20 19:32:42 +00:00
Colin Cross
08117deab4 Fix mismatch between cc and python for windows prefer32
Windows prefer32 was implemented in cc, which caused a mismatch
between cc and other packages like python.  This manifested as
missing variant when a python_test_host with compile_multilib: "first"
tried to depend on a cc_binary_host, which implicitly has multilib
"first".

Fixes: 177904375
Test: m checkbuild
Change-Id: Iaaac16ae171c06d90d04d7cac11789d3f39b8d99
2021-01-20 10:26:00 -08:00
Colin Cross
bdeafb93fd Merge "Don't create SDK variants for native bridge modules" 2021-01-20 17:49:36 +00:00
Bill Peckham
1c8e5e817e Merge changes from topic "prebuilt-hiddenapi"
* changes:
  Supply LOCAL_SOONG_DEX_JAR for java_import with compile_dex.
  Enable an apex to include a java_import.
  Enable prebuilt hiddenapi CSV files.
2021-01-20 17:43:48 +00:00
Christopher Parsons
965561ce43 Merge "avoid error if bazel intermediates dir already exists" 2021-01-20 17:41:18 +00:00
Inseob Kim
9bd66d1537 Merge "Implement directed vendor snapshot" 2021-01-20 01:17:25 +00:00
Colin Cross
94e347e6a5 Don't create SDK variants for native bridge modules
Native bridge modules never need to build against NDK stubs, so
don't create SDK variants.  Also clear the sdk_version property
for modules that don't have SDK variants so that later code doesn't
use it to trigger SDK behaviors.

Test: m checkbuild
Change-Id: I1920fa82e9fab06235f01a62624382efa16cc6e3
2021-01-19 15:00:45 -08:00
Chris Parsons
07c1e4aba1 avoid error if bazel intermediates dir already exists
Previously, the build failed if the directory already existed,
so mixed builds had to be null builds.

Test: lunch aosp_flame && USE_BAZEL=1 USE_BAZEL_ANAYSIS=1 m libc &&
  USE_BAZEL=1 USE_ANALYSIS=1 m libc
Change-Id: I167cb82b3e47e9388a1ebca7daffe45f91474125
2021-01-19 17:19:16 -05:00
Bill Peckham
bae4749de2 Enable prebuilt hiddenapi CSV files.
By enabling these hiddenapi CSV files to be prebuilt, it
becomes possible to create a split build that supports
the hiddenapi encode dex step, but doesn't contain all
of the java sources needed to generate the CSV files.

Bug: 175048716
Test: m nothing
Test: new TestHiddenAPISingletonWithPrebuiltCsvFile
Test: local build without prebuilt hiddenapi
Test: local build with prebuilt hiddenapi
Change-Id: I805ea3ec05860d2df211a3985ec025bf36f0d775
2021-01-19 10:07:45 -08:00
Treehugger Robot
de1357378b Merge "Implement code-generation step for bp2build." 2021-01-19 17:41:59 +00:00
Jingwen Chen
daa54bcbba Implement code-generation step for bp2build.
Implement bp2build codegen as a discrete step that runs after an
alternatively registered pipeline of mutators, instead of a
presingleton.

bp2build codegen requires a Context that supports VisitAllModules and
PathContext, so this CL also makes a BpToBuildWrapperContext that
conforms to PathContext by adding two method implementations.

Test: GENERATE_BAZEL_FILES=true m nothing && bazel query //... --config=bp2build | wc -l # 31433
Test: m queryview && bazel query //... --config=queryview # 63638

Change-Id: I0dd359746584b228046d2d0ff00895f28f9bdfc3
2021-01-19 11:03:36 -05:00
Christopher Parsons
ad3421aacf Merge "Improve aquery-related validation and error handling" 2021-01-19 14:59:51 +00:00
Inseob Kim
7cf1465d2e Implement directed vendor snapshot
Vendors can now generate only needed modules by setting the following
Makefile variables:

- DIRECTED_VENDOR_SNAPSHOT: set to true
- VENDOR_SNAPSHOT_MODULES: list of snapshot candidates

e.g.

DIRECTED_VENDOR_SNAPSHOT := true
VENDOR_SNAPSHOT_MODULES := toybox_vendor sh_vendor libbase libcutils ...

Bug: 157967325
Test: m dist vendor-snapshot after setting those in BoardConfig.mk
Change-Id: I6515a43d9358d94483d7c7fa2b066f9dd457f6aa
2021-01-18 20:08:09 +09:00
Chris Parsons
4f069891ae Improve aquery-related validation and error handling
Also introduce test suite for aquery handling

Test: m nothing
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I2493d42782099ea0b575968fca38bce6f0d59015
2021-01-15 13:53:57 -05:00
Paul Duffin
064b70c918 Export dex implementation jars from prebuilt_apex
Dexpreopt and boot jars package check all require access to dex
implementation jars created for java_library and java_sdk_library. They
were available when building from source but not when building from
prebuilts, even though they are embedded within the .apex files that
are referenced from prebuilt_apex.

This changes adds support to prebuilt_apex to export the dex
implementation jars and updates java_import to use those exported dex
implementation jars.

In a source build dexpreopt/boot jars package check access the apex (or
platform) specific variant of a java_library, e.g. core-oj, from which
it retrieves the dex implementation jar path.

After this change in a prebuilt build dexpreopt/boot jars package check
behave in the same way except in this case they retrieve the dex
implementation jar path from the apex (or platform) specific variant of
the java_import, e.g. core-oj.

The work to export files from a `.apex` file for use by other modules
is performed by a new `deapexer` module type. It is not used directly
in an `Android.bp` file but instead is created implicitly by
`prebuilt_apex`,

In order to do that this contains the following changes:
* Adds a new `dexapexer` module type to handle the exporting of files
  from the `.apex` file.
* Adds an exported_java_libs property to prebuilt_apex to specify the
  set of libraries whose dex implementation jars need exporting.
* Creates apex specific variants of the libraries listed in the
  exported_java_libs property.
* Adds the set of exported files to the ApexInfo to make them available
  to the apex specific variants.
* Prevents the prebuilt_apex variants from being merged together as
  they will not be compatible.
* Modifies java_import to use the exported file for variants of a
  prebuilt_apex.
* Adds a ninja rule to unpack (using deapexer) the contents of the
  prebuilt_apex's apex file, verify that the required files are present
  and make them available as outputs for other rules to use.
* Some minor refactorings to support these changes.
* Adds tests to cover prebuilt only, prebuilt with source preferred,
  and prebuilt preferred with source.

Test: m nothing
Bug: 171061220
Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
2021-01-15 18:14:10 +00:00
Paul Duffin
db170e4a92 Add java_boot_libs to sdk
The build has some implicit dependencies (via the boot jars
configuration) on a number of modules, e.g. core-oj, apache-xml, that
are part of the java boot class path and which are provided by mainline
modules (e.g. art, conscrypt, runtime-i18n) but which are not otherwise
used outside those mainline modules.

As they are not needed outside the mainline modules adding them to
the sdk/module-exports as either java_libs, or java_header_libs would
end up exporting more information than was strictly necessary. This
change adds the java_boot_libs property to allow those modules to be
exported as part of the sdk/module_exports without exposing any
unnecessary information.

Some points to note:
* The java_import has to have a valid file for the src property
  otherwise it will be disabled.
* The src property is supposed to reference a jar file but the
  java_boot_libs property will make it reference an empty file (not
  an empty jar) so that any attempt to use that file as a jar, e.g.
  compiling against it, will cause a build failure.
* The name of the file passed to the src property should make it
  clear that the file is not intended to be used.
* The test makes sure that only the jar file is copied to the
  snapshot.

Test: m nothing
Bug: 171061220
Change-Id: I175331e4c8e3874ab70a67cdc2f76ed1576e41eb
2021-01-15 18:14:10 +00:00