Commit graph

32662 commits

Author SHA1 Message Date
Jose Galmes
4c6895e5a3 Implement directed recovery snapshot.
Bug: 177451657
Test: m -j nothing
Change-Id: I0b84c34fb2e9e9340d9f9e7405428cd2cc4d43b5
2021-02-09 08:25:34 -08:00
Paul Duffin
320e170675 Merge "Allow dex jars from prebuilt_apex to be used by hiddenapi" 2021-02-09 13:58:12 +00:00
Paul Duffin
b71a8de0d5 Merge "Refactor the hiddenAPI() method for reusability" 2021-02-09 13:57:52 +00:00
Paul Duffin
abe1abd224 Merge "Extract initHiddenAPI() from hiddenapi()" 2021-02-09 13:57:37 +00:00
Ivan Lozano
4ccfcc2ee4 Merge "rust: Add rust_ffi_static vendor ramdisk Support" 2021-02-09 13:49:28 +00:00
Jingwen Chen
bb483cbcf5 bp2build: rename generated files to BUILD
The checked in files are named BUILD.bazel, while bp2build files are
named BUILD.

Test: GENERATE_BAZEL_FILES=true m nothing
Change-Id: If226f581b5075088f774dc4f5f9ee88259821835
2021-02-09 04:26:18 +00:00
Treehugger Robot
198158b9f7 Merge "bp2build: refactor BazelTargetModule naming boilerplate." 2021-02-09 03:21:44 +00:00
Treehugger Robot
d85d058820 Merge "Pass list of module libraries to gen_ndk_backed_by_apex.sh" 2021-02-09 03:21:36 +00:00
Colin Cross
69f0a24762 Pass list of module libraries to gen_ndk_backed_by_apex.sh
Running find on the entire module directory can lead to flaky failures
when a directory is modified by another rule while find is traversing
it.  Pass a list of libraries in the module into the script instead
of finding them on disk.

Fixes: 178509901
Test: TARGET_BUILD_APPS=com.android.runtime dist
Change-Id: Ie4bf288f0e3d99cbbed5ca8770687a9bbe5377e4
2021-02-08 16:52:07 -08:00
Jingwen Chen
ff8f1ce7d7 Merge "bp2build: make generated BUILD files publicly visible." 2021-02-09 00:38:03 +00:00
Treehugger Robot
ed7bfd9726 Merge "Turn missing jarjar output files into errors" 2021-02-08 23:47:13 +00:00
Treehugger Robot
3b9b0a11e9 Merge "Remove Use_lmkd_stats_log from product variables" 2021-02-08 20:58:19 +00:00
Bill Peckham
5c2406bde9 Merge "Enable android_app_import to supply framework-res." 2021-02-08 19:15:10 +00:00
Paul Duffin
9d67ca6ab7 Allow dex jars from prebuilt_apex to be used by hiddenapi
Invokes hiddenAPIExtractInformation() on the dex jar provided by the
deapexer (on behalf of prebuilt_apex) so that hiddenAPI can extract the
information it needs, if anything, from the dex file (and accompanying
classes implementation file).

The dex file provided by deapexer has already had the hiddenapi
information encoded into it so it does not need to do that again.

This change adds the primary parameter to hiddenAPIExtractInformation()
and checks it (and also the hiddenAPI.active property) before it does
anything. That ensures that it behaves correctly when called directly
as well as when called from hiddenAPIExtractAndEncode().

Bug: 178361284
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
      Also verified that the hiddenapi files created when using the
      prebuilts (using SOONG_CONFIG_art_module_source_build=false) are
      the same as when using the source. There is a slight difference
      in the order but otherwise identical.
Change-Id: I7abb63fd310bb94787ab7f4821e5fd283dc03046
2021-02-08 19:10:50 +00:00
Paul Duffin
4fd997bc13 Refactor the hiddenAPI() method for reusability
A follow up change needs to be able to contribute to the information
the hiddenapi process collates without having a dex file encoded. This
change pushes all the functionality related to information gathering
into the hiddenAPIGenerateCSV() method and then renames it and the
hiddenAPI() method to make it clearer what they do.

Bug: 178361284
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: I04417720216a0fbadcd88e6185e7de6570af6216
2021-02-08 19:10:50 +00:00
Paul Duffin
4103e92c4b Extract initHiddenAPI() from hiddenapi()
Previously, the hiddenapi() method combined both checks to determine
whether a module contributed to/was modified by the hiddenapi process
and logic to create ninja rules to perform those tasks. This change
separates the former out into a new initHiddenAPI() method.

The main purpose of this is to simplify the process of allowing the
CSV generation to be separated from the encoding. That is required
because when a java_import retrieves its dex file from the apex it
has already been encoded.

The initHiddenAPI() method is only called for java.Library (and
indirectly for java.SdkLibrary) and java.Import modules which means
that the hiddenapi() method does nothing for any other module type.
That is consistent with the previous behaviour because while the
hiddenapi() method is called for other module types they fail because
the boot image jars only support java_library, java_sdk_library,
and java_import at the moment. While it will need to support
java_sdk_library_import once any of the libraries that are currently
provided as java_sdk_library modules switches to providing prebuilts
that is outside the scope of this work.

Bug: 178361284
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: Iaa91e0a8e2bffec03296dd894e9662556f4464c0
2021-02-08 19:10:50 +00:00
Paul Duffin
5d0572b7eb Merge "Integrate hiddenapi processing into boot jars test" 2021-02-08 18:43:10 +00:00
Paul Duffin
1a75902832 Merge "Exclude exported_java_libs dependencies from visibility checks" 2021-02-08 18:40:46 +00:00
Treehugger Robot
c6fcf29872 Merge "Remove duplicate entries from hiddenapi files" 2021-02-08 18:16:38 +00:00
Treehugger Robot
41905bf733 Merge "bp2build: add bazel_module: { bp2build_available } prop." 2021-02-08 18:14:20 +00:00
Colin Cross
d97cf63cdf Turn missing jarjar output files into errors
Jarjar doesn't exit with a nonzero return code when there is a syntax
error in a rules file and doesn't write the output file.  Without a
nonzero return code ninja prints a warning but continues, which leads
to stale results on incremental builds where the output file already
exists, or delayed errors on clean builds whre the output file didn't
exist.

Delete the output file before running jarjar, and then check if it
exists after running jarjar.

Fixes: 119516143
Test: m out/target/common/obj/APPS/TeleService_intermediates/classes.jar
Change-Id: I7e9f7ff34de565a986ab0dc6e375f606c91c3846
2021-02-08 10:14:04 -08:00
Jingwen Chen
033bfa6e29 Merge "queryview: blocklist package module type." 2021-02-08 15:18:19 +00:00
Ivan Lozano
e6d3098c1b rust: Add rust_ffi_static vendor ramdisk Support
Similar to our vendor support, this adds support for linking rust static
libraries to vendor ramdisk cc modules.

A bug fix is also included where a restriction against setting rust_ffi
vendor-specific was not being enforced.

Bug: 179397942
Test: Example modules link, Soong tests pass.
Change-Id: I737cdf0c2f49ab349bcea2a0429e6298ebc1313e
2021-02-08 09:36:26 -05:00
Jingwen Chen
1a0485b34e Merge "bp2build: Refactor CreateBazelTargetModule API." 2021-02-08 14:08:05 +00:00
Paul Duffin
a7139425bc Exclude exported_java_libs dependencies from visibility checks
Test: m nothing
Change-Id: Iba45580c0711d48034898e9d42832c7e2b0e6284
2021-02-08 13:13:02 +00:00
Ioannis Ilkos
09ab9b4679 Remove Use_lmkd_stats_log from product variables
Use_lmkd_stats_log used to control LMKD_LOG_STATS when compiling lmkd.

However, for newer Android releases, we mandate that lmkd informs statsd of process kills (change id I8cb123b9488fbc6e88863c2f0e75f1422bcd282e)
After aosp/1555318 we compile lmkd with statsd by default, so this setting is a noop.

Change-Id: Ibcf661e205b3ed2a4a520a42952aaece503e9350
Test: build
Bug: 177985094
2021-02-08 11:18:51 +00:00
Jingwen Chen
fb4692a743 bp2build: refactor BazelTargetModule naming boilerplate.
This CL replaces the "__bp2build__" name prefix boilerplate with a props
creation function, and centralizes the prefixing in there.

Test: TH
Test: soong tests
Change-Id: Ic963199ab60dcce0d3361abff111cfa9acd4c21b
2021-02-08 04:10:18 -05:00
Jingwen Chen
96af35bec6 queryview: blocklist package module type.
The package module type was recently introduced, and its module name is
the full package path. queryview uses the module name as the eventual
target name in the generated BUILD files, so this causes a crash as seen
in b/179605762.

Workaround this by not generating package modules in the queryview
directory.

Test: m queryview && bazel query --config=queryview //...
Fixes: 179605762
Change-Id: Iac2bd79d02bec47d6271583d106c184fb2f16e68
2021-02-08 00:49:32 -05:00
Paul Duffin
f38931c287 Integrate hiddenapi processing into boot jars test
This change registers the hiddenapi singleton so that hiddenapi
processing is performed as part of
TestBootDexJarsFromSourcesAndPrebuilts so that additional hiddenapi
related tests can be added later.

The hiddenapi singleton uses the BootJars/UpdatableBootJars properties
from productVariables not the dexpreopt.GlobalConfig so this change
makes sure that they are consistent.

Performing hiddenapi processing causes the dex file used in "prebuilt
with source apex preferred" test to be encoded with hiddenapi
information which changes the path in the test.

Bug: 178361284
Test: m nothing
Change-Id: I2eb3d8cf11f6fb3bf4a34d6e4ae0c397c890191e
2021-02-07 10:58:24 +00:00
Paul Duffin
219b177279 Remove duplicate entries from hiddenapi files
Prevents the hiddenapi files index and metadata files from including
duplicate entries.

Bug: 178361284
Test: m droid
      Compared hiddenapi files before and after change
      Verified that the only differences were due to the removal of duplicates
      Used sort -u to remove duplicates from before file and compared with after file
Change-Id: I0b9e420aec42cdcf4584099a7ffd2ed9c3715d6d
2021-02-07 10:58:24 +00:00
Paul Duffin
01289a2407 Add test for hiddenAPI index file generation
The index file and the metadata file both currently include duplicate
entries due to them including both the <x> and <x>.impl libraries
created by java_sdk_library in their inputs, plus including both source
and prebuilt versions of the same named module. This change adds a test
to illustrate that behavior. A follow up change will correct the problem
and update the test accordingly.

This change only adds a test for the index file because the metadata
file depends on files from frameworks/base which makes it difficult to
test. Bug 177317659 will fix that and allow the metadata file
generation to be tested too.

Bug: 178361284
Test: m nothing
Change-Id: I33921d7267c9f4bb42726343d73f8a396d536aaa
2021-02-07 10:58:24 +00:00
Paul Duffin
89886cbdb0 Ensure subtest failures are reported on subtest
A subtest (code inside the func passed to t.Run(...)) are passed their
own t *Testing pointer to use to report errors in order to ensure that
they are treated as errors of the subtest and not the containing test.

This change ensures that the subtests in the following tests use the
correct t *Testing.
* TestPrebuiltExportDexImplementationJars
* TestBootDexJarsFromSourcesAndPrebuilts

Bug: 178361284
Test: m nothing
Change-Id: I4e8b166051cb6098c89d8e68a450c81a714f7677
2021-02-07 10:58:24 +00:00
Paul Duffin
254aee4843 Merge changes I9e94662d,I4b4e0dc8
* changes:
  Export implementation class jars for java_boot_libs
  Improve module filtering in hiddenapi stubFlagsRule()
2021-02-07 10:52:01 +00:00
Ivan Lozano
a707092636 Merge "rust: Only pass "-lstatic" for rlibs." 2021-02-06 04:47:03 +00:00
Treehugger Robot
b02128ad0b Merge "Remove libbinder_headers from apex_available allowlist" 2021-02-06 03:20:47 +00:00
Jooyung Han
7701309889 Merge "cc: fix version macro for stubs" 2021-02-06 01:41:31 +00:00
Bill Peckham
a036da9b27 Enable android_app_import to supply framework-res.
By enabling an android_app_import module to supply the
special framework-res module, it becomes possible to
create a split build that excludes the source of over
10,000 files under frameworks/base/core/res, replacing
this source with a prebuilt. However, since other parts
of a monolithic build use the source of framework-res,
this scheme works only if your split build manifest
excludes modules that depend on framework-res source.

Bug: 176851898
Test: lunch aosp_arm64-userdebug && m nothing
Test: new TestAndroidAppImportExportPackageResources
Test: local build without prebuilt framework-res
Test: local build with prebuilt framework-res
Change-Id: If00d9dbc656c8406c038448d09f93f767c1e06d1
2021-02-05 14:30:46 -08:00
Colin Cross
d685adb245 Merge "soong_zip: don't glob RSP file entries" 2021-02-05 21:33:37 +00:00
Jose "Pepe" Galmes
c64b961708 Merge "Generate fake versions of modules excluded in directed vendor snapshot." 2021-02-05 17:54:48 +00:00
Jose Galmes
0a942a0368 Generate fake versions of modules excluded in directed vendor snapshot.
When using the directed vendor snapshot, the build fails because soong
fails to find some variants of the modules that have been excluded
from VENDOR_SNAPSHOT_MODULES, even though those modules are not going
to be used by the build.

The solution implemented here is to generate fake versions of those
modules (empty files) and include them in the generated Android.bp,
so that soong finds the modules, even though trying to use them would
fail.

Bug: 171821997
Bug: 179275601

Test: source build/envsetup.sh
Test: m -j nothing

Change-Id: Ibd3e963ab3e5504c0ac817f7cabbd241bf47a5cb
2021-02-05 17:53:53 +00:00
Ivan Lozano
fb6f36f3e4 rust: Only pass "-lstatic" for rlibs.
Passing '-lstatic' for dylib static dependencies results in some
unexpected bloat as the static libraries are included as
whole-archives.

For now, limit this to rlibs while we investigate.

Bug: 175886967
Bug: 179122837
Test: cd system/bt/; mma
Test: Checked size of dylibs.
Change-Id: Id8328e95771e2ca8503bc8df35248b1c39cabc8e
2021-02-05 12:32:47 -05:00
Treehugger Robot
b3f9025b6e Merge "Support uncompressed cpio" 2021-02-05 15:57:22 +00:00
Ivan Lozano
729d5cf097 Merge "rust: Pass cc static libs to rustc." 2021-02-05 15:53:33 +00:00
Jingwen Chen
1c2317315c bp2build: make generated BUILD files publicly visible.
This is obviously not good build hygiene, but since Soong modules are in a global namespace, making BUILD files fully public will increase our velocity during the initial migration phases.

We'd want to lock visibility down to private by default as soon as possible, but for the purposes of migration, these will be public.

Test: bazel build //bionic/...
Change-Id: I99d62f3f8826712240decd325928435333870352
2021-02-05 09:38:15 -05:00
Jingwen Chen
77e8b7b6d2 bp2build: add bazel_module: { bp2build_available } prop.
This CL adds a per-target allowlist to instruct bp2build on which modules it should generate Bazel targets for.

Test: soong tests
Change-Id: I869e66fce405c2c6689b381569b8cc0118cbcf76
2021-02-05 09:26:22 -05:00
Jingwen Chen
1fd14691dd bp2build: Refactor CreateBazelTargetModule API.
This CL refactors the CreateBazelTargetModule API to minimize boilerplate, and to establish a well defined function signature for the expected metadata about a BazelTargetModule.

Test: soong tests

Test: TH
Change-Id: I474ff5a2b0db8deeed49ba4ca73b416ccb494fdd
2021-02-05 09:24:55 -05:00
Liz Kammer
b7eab01167 Merge "bp2build: convert paths/module refs to Bazel label" 2021-02-05 13:39:08 +00:00
Paul Duffin
22ff0aaf51 Export implementation class jars for java_boot_libs
Hiddenapi processing currently requires access to the class
implementation jars for libraries on the bootclasspath which means that
they need to be provided as part of the prebuilts. This change modifies
the java_boot_libs property on the sdk to make those files available.

Modularization of the hiddenapi processing will hopefully remove the
need for these to be exported so this should be temporary.

Bug: 178361284
Test: m art-module-sdk
      check generated snapshot zip contains implementation jars
Change-Id: I9e94662dddb0ddb85a477ae6d27e533085147e88
2021-02-05 13:35:25 +00:00
Paul Duffin
dd63d6d7bd Improve module filtering in hiddenapi stubFlagsRule()
Previously, it ignored any module that was not a platform apex variant
however that required every module that was referenced from the boot
jars to have a platform variant which is not the case when building
from prebuilts.

This change switches it to explicitly check that the variant is for
either the appropriate apex or the platform depending on what is
configured in the BootJars or UpdatableBootJars.

It partially duplicates some logic from the getBootImageJar() function.
It intentionally does not refactor the getBootImageJar() to allow for
reuse because coupling the hiddenapi and dexpreopt logic would make
refactoring either of them more difficult. Any duplicated code will be
deduped once they have both been refactored.

Bug: 178361284
Test: m droid
Change-Id: I4b4e0dc8ee40c1ba1713d7ef72df13d175e84af6
2021-02-05 13:33:25 +00:00
Treehugger Robot
b89336487e Merge "bp2build: fix BUILD file generation incrementality bug." 2021-02-05 12:05:57 +00:00