Commit graph

71 commits

Author SHA1 Message Date
Justin Yun
a05a4f6f87 Skip snapshot for sanitizer variants with both cfi and hwasan
cfi and hwasan cannot be enabled at the same time. Skip building
vendor snasphots for the snapshot variants with both cfi and hwasan.

Bug: 277174974
Bug: 290779821
Test: HWASAN_INCLUDE_PATHS=frameworks/native/libs/gui m
Change-Id: Iaae913367a70e150d77a614de5f942aea98d1086
2023-09-25 23:07:49 +09:00
Ivan Lozano
22823eed4b Revert "rust: Add SCS sanitizer option for AArch64"
This reverts commit 8827113577.

Reason for revert: hwasan builds breaking

Change-Id: I28094e27d5935c6f371fddddf4b6b0334de9b12c
2023-07-31 18:01:26 +00:00
Ivan Lozano
8827113577 rust: Add SCS sanitizer option for AArch64
SCS has been supported in Rust since 1.64.0.

This adds SCS as a sanitizer option in Rust.

Bug: 168914033
Test: Built module with sanitize: { scs: true }
Change-Id: Ie54ac4693286878b98704cf052649a267777d764
2023-07-28 14:34:28 -04:00
Ivan Lozano
f9780422b3 Revert "rust: Add SCS sanitizer option."
This reverts commit 447f5b1ea9.

Reason for revert: b/293445283

Change-Id: Iadc6c7ed196be2a65329f86423730ba7f851002c
2023-07-27 14:52:20 +00:00
Ivan Lozano
447f5b1ea9 rust: Add SCS sanitizer option.
SCS has been supported in Rust since 1.64.0.

This adds SCS as a sanitizer option in Rust.

Bug: 168914033
Test: Built module with sanitize: { scs: true }
Change-Id: Ibdc116a335339af6d0b0d6dd0bb82c0358fa9087
2023-07-26 13:34:55 -04:00
Ivan Lozano
add122a828 rust: Add vendor and recovery dylib support.
Adds dylib support for vendor and recovery images.

This changes the default linkage for vendor and recovery images to
dylib, which matches the platform default linkage. This also means that
by default, dylib-std variants are used for rlib dependencies.

Bug: 204303985
Test: Soong tests.
Test: m dist vendor-snapshot
Test: RECOVERY_SNAPSHOT_VERSION=current m dist recovery-snapshot
Change-Id: If84074b8615a70c45e7e162abeb853dc8c34d49a
2023-07-14 12:43:09 -04:00
Justin Yun
3cc7846b58 APEX can use any Prebuilt libraries
Vendor snapshot libraries generated from prebuilt libraries set
"apex_inherit" to their min_sdk_version property to allow them to be
used by vendor APEXes.

Bug: 277403349
Test: m vendor-snapshot
Change-Id: Idd9e5f8e94b3fa2adf47a04507bf0c50c59edfb7
2023-05-08 22:33:35 +09:00
Justin Yun
17d0ee2160 VSDK: use relative install path when generating snapshots
Without respecting the relative install path, the snapshot install
path may collide when they use the same stem name.
To avoid this, respect the relative install path when generating the
snapshots.

Bug: 279652606
Test: RECOVERY_SNAPSHOT_VERSION=current m recovery-snapshot
Change-Id: Ic70a5855aeb2a6eda397b4dd06113dddb6ef70f4
2023-05-02 16:29:06 +09:00
Justin Yun
1db9748805 Include license information for vendor and host snapshots
In addition to the license text files, include license kinds
information to json file. Also, use the original paths of license
text files that are copied to NOTICE_FILES directory.
This will be used when generating snapshots from the snapshot build.

Bug: 271539873
Test: m vendor-snapshot
Change-Id: I0c0427bb66f2c1fca322d5554aa66220a3b62fb3
2023-04-25 18:03:32 +09:00
Inseob Kim
5860f8227c Add min_sdk_version prop to snapshots
Bug: 277403349
Test: build vndk/vendor snapshot and build
Change-Id: I226032bb0f9356cdef6747daec6ceb06589d9cbb
2023-04-18 11:30:22 +09:00
Justin Yun
8814fc5c20 VSDK: capture hwasan static libs for vsdk snapshot build
When generating vsdk snapshot with SANITIZE_TARGET=hwaddress option,
include hwasan static libraries to the vendor snapshot.

Bug: 234772527
Test: build against the vsdk with SANITIZE_TARGET=hwaddress
Change-Id: I6fdecefaa8557b5c968745487a3ed7c959e682f9
2022-12-19 17:18:22 +09:00
Inseob Kim
a1888ce78d Add overrides support for snapshots
Overrides properties will now be captured in json flag files, which will
be copied to installed vendor snapshot modules.

Bug: 216567575
Test: soong test && manual install
Change-Id: Ife5e84b126e798fba7802b9cff000c9197756cb9
2022-10-04 14:42:02 +09:00
Colin Cross
d079e0b270 Reformat build/soong for go 1.19
Test: none
Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-08-17 10:43:13 -07:00
Jose Galmes
c1a56bcbe9 Add InstallInRoot property to snapshot binary json file.
Bug: 215295121
Test: Generated snapshot and verified InstallInRoot property is present.
Change-Id: I4f20a454477d1263e9c16e98468d659f89e1e4f1
2022-03-07 14:49:46 -08:00
Rob Seymour
925aa09ea1 Add module based host-tools snapshot
Add new module:
host_snapshot {
    name: "host-snapshot"
    deps: [
      (list of host tools)
    ],
    ...
}

Package host tools using android.PackagingBase to capture host tools and transitive packaging data.
Add JSON meta data to snapshot that allows snapshot to be installed via development/vendor_snapshot/update.py

Add support to generate a fake host snapshot of all host modules that is used to detect required
modules via development/vendor_snapshot/update.py.

Bug: 192556798
Bug: 194799048
Bug: 192896149
Test: m HOST_FAKE_SNAPSHOT_ENABLE=true host-fake-snapshot dist -- check snapshot exists in dist

Change-Id: I849c4db801cd858408f6fe6a3ce69262a23a5be9
2021-09-13 21:29:14 +00:00
Kiyoung Kim
ae11c233b5 Update prebuilt_etc available for snapshot
Make prebuilt_etc module available for the snapshot. This change
includes implementing snapshot interface for the prebuilt_etc module so
it can be added to the snapshot from the fake snapshot, or when the
module is specified in the list.

Bug: 192430376
Test: m nothing passed
Test: Checked if the module is included in the snapshot properly
Change-Id: I3574d2a1b8f8f4e5f083f3913e8768f5088d0c46
2021-07-20 11:24:03 +09:00
Kiyoung Kim
48f3778cb4 Separate snapshot definition
Current snapshot definition is located in the CC module, so it is
difficult to capture non-CC module (such as prebuilt_etc) to the
snapshot. Separate general snapshot definition from cc so other modules
can also define its own snapshot.

Bug: 192430376
Test: m nothing passed
Change-Id: Ifb69fb3d2ec555b629aa31ec03e7ce5831fd3063
2021-07-19 11:42:32 +09:00
Justin Yun
885a7dea68 Share EffectiveLicenseFiles for snapshot modules
License files must be captured with the snapshot modules. NoticeFiles
does not have the information provided by the `license` module.
Reading the `Effective_license_text`, snapshot includes the required
license files.

Bug: 190690041
Test: build
      check the NOTICE files built from snapshots
Change-Id: I96c4579b4bef484cbb39fc06c707742a02a2b90b
2021-07-08 00:02:48 +00:00
Justin Yun
5e03586365 Include static lib information for the snapshot modules
snapshot modules require the list of static libs to find the NOTICE
files for the snapshot modules.
snapshot binaries must include static_executable property to check if
non static binaries have libc as a static lib.

Bug: 190690041
Test: Check NOTICE for libaudiopreprocessing.so built from snapshots.
Change-Id: I53f7549a158ab27ada87833a3f4d58620aa7c3cf
2021-07-01 11:42:28 +00:00
Ivan Lozano
3149e6ed18 Rust rlib vendor snapshot support.
Adds support for snapshotting Rust rlibs. This allows us
vendor-specific code that uses rlib-only linkage until dylib
snapshot support is added.

Bug: 184042776
Test: m nothing # new Soong tests pass
Test: Example test Rust vendor module builds
Test: m dist vendor-snapshot # includes rlibs
Change-Id: I4976d3e1efec0ee778cc97730d45be471dffb678
2021-06-03 08:36:09 -04:00
Ivan Lozano
d1dec54988 Export cc functions for snapshotting Rust rlibs.
Export additional fuctions, structs, and interfaces from cc for use in
the rust package to allow for rlib snapshotting.

Bug: 184042776
Test: m nothing
Change-Id: I4c53b9378d5d5b5973dbd23ab692cdfb2ede60b9
2021-06-03 08:34:47 -04:00
Ivan Lozano
d67a6b0a88 Export cc vendor functions for usage by rust.
This CL exports and refactors some cc vendor-snapshot related functions
so they can be reused by rust modules to support vendor snapshotting.

Bug: 184042776
Test: m nothing
Change-Id: I12706e62ce0ac3b2b4298085fafc1d77b8e0a0c4
2021-06-03 08:31:55 -04:00
Ivan Lozano
d7586b6526 Refactor vendor snapshot to use LinkableInterface.
Refactors the vendor snapshot support to use the LinkableInterface
so that support can be extended to Rust. This CL does not add
vendor snapshot support for Rust; that is left for a follow-on CL.

Bug: 184042776
Test: m nothing
Change-Id: Id0c4970ca00053484a52677d182153cbc454c301
2021-05-12 14:01:10 -04:00
Colin Cross
203b421043 Remove obsolete llndk_library
Remove llndk_library in favor of cc_library with llndk.symbol_file.

Bug: 170784825
Test: m checkbuild
Test: TestLlndkLibrary
Change-Id: I43580976589a7a2a176d7442be53fa043c0c8324
2021-04-26 18:41:00 -07:00
Colin Cross
627280f091 Remove llndk_headers
Replace llndk_headers with cc_library_headers with
llndk.llndk_headers: true.

Bug: 170784825
Test: m checkbuild
Test: compare out/soong/build.ninja
Test: TestLlndkHeaders
Change-Id: I33b411cd4d474318796c2073375176b82dd8c216
2021-04-26 18:40:53 -07:00
Ivan Lozano
3a7d000d12 cc: Create a common image mutator interface
As part of adding Vendor support to Rust, refactor the image mutator in
CC to a new common image mutator interface so this logic can be reused
across both CC and Rust.

Bug: 184042776
Test: m nothing
Change-Id: Ia55d5ad840db7cf1a64d6c65ed86487230cb8742
2021-04-13 11:04:44 -04:00
Tri Vo
6eafc36e60 Export CC HWASan sanitizer type
We need this so that HWASan sanitizer mutator in CC can sanitize Rust.

Bug: 180495975
Test: m nothing
Change-Id: I2c9eb248df4e55a33f5d45083e91588f4c8e3e94
2021-04-08 14:24:06 -07:00
Inseob Kim
1b6fb87654 Automatically set Androidmk suffix of snapshot
Androidmk_suffix has been captured with "m vendor-snapshot". But the
suffix is to avoid collision with core variants, so the suffix should be
based on the existence of core variants. This change sets the suffix by
using OtherModuleDependencyVariantExists. This way we can also avoid
mutators and global states.

Bug: 179665538
Test: m nothing (soong test)
Change-Id: I35343aa2c4c07782057b82e6dd0e705a786e55fc
2021-04-07 17:24:08 +09:00
Colin Cross
70c4741215 Add explicit rspfile argument to RuleBuilderCommand.FlagWithRspFileInputList
Using $out.rsp as the rsp file adds extra complexity around keeping
the $ unescaped.  Make callers to FlagWithRspFileInputList provide
an explicit path for the rsp file instead.

Bug: 182612695
Test: rule_builder_test.go
Change-Id: I3f531d80c1efa8a9d09aac0a63790c5b11a9f0c6
2021-03-16 16:52:56 -07:00
Justin DeMartino
383bfb392f Allow extending of the list of directories to be scanned for VSDK
Bug: 180925851
Bug: 181564692
Test: m nothing, manually
Change-Id: Ifff95db4c9ec362322fecca08f7fd1a7b60755c0
2021-03-15 23:14:47 +00:00
Jose Galmes
0a942a0368 Generate fake versions of modules excluded in directed vendor snapshot.
When using the directed vendor snapshot, the build fails because soong
fails to find some variants of the modules that have been excluded
from VENDOR_SNAPSHOT_MODULES, even though those modules are not going
to be used by the build.

The solution implemented here is to generate fake versions of those
modules (empty files) and include them in the generated Android.bp,
so that soong finds the modules, even though trying to use them would
fail.

Bug: 171821997
Bug: 179275601

Test: source build/envsetup.sh
Test: m -j nothing

Change-Id: Ibd3e963ab3e5504c0ac817f7cabbd241bf47a5cb
2021-02-05 17:53:53 +00:00
Colin Cross
a889080aba Remove global state on module suffixes from vendor and recovery snapshots
Propgate the Android.mk suffix from source modules into the snapshot so
that it can be used for the prebuilt modules.

Bug: 177098205
Test: vendor_snapshot_test.go
Change-Id: Iea151dc91395f714fbcad1df3a6fd0874e5455d9
2021-02-01 20:06:29 +09:00
Justin Yun
e09ac17466 Check vndk-private with the IsVndkPrivate()
*_available will not provide if a module is VNDK-private or not.
Fixed it by using IsVndkPrivate() with private() interface.

By allowing exclude_from_vendor_snapshot for vendor_available
modules, available() interface is no longer required. Tests are
updated for exclude_from_vendor_snapshot on vendor_available modules.

Bug: 175768895
Test: m nothing
Change-Id: I9ff85256170d59b378a4c354d5c007cfbeacabc5
2021-01-23 01:11:58 +00:00
Inseob Kim
7cf1465d2e Implement directed vendor snapshot
Vendors can now generate only needed modules by setting the following
Makefile variables:

- DIRECTED_VENDOR_SNAPSHOT: set to true
- VENDOR_SNAPSHOT_MODULES: list of snapshot candidates

e.g.

DIRECTED_VENDOR_SNAPSHOT := true
VENDOR_SNAPSHOT_MODULES := toybox_vendor sh_vendor libbase libcutils ...

Bug: 157967325
Test: m dist vendor-snapshot after setting those in BoardConfig.mk
Change-Id: I6515a43d9358d94483d7c7fa2b066f9dd457f6aa
2021-01-18 20:08:09 +09:00
Ivan Lozano
3968d8f678 Refactor CC to prep for Rust sanitizable modules.
Adds a PlatformSanitizable interface which both CC and Rust can
implement so that the sanitizer mutators in CC can sanitize Rust
shared/static libraries appropriately.

Bug: 147140513
Test: m nothing
Change-Id: Ib31103b6c4902a4d5df2565c0d7c981298d100a3
2021-01-08 08:53:46 -05:00
Treehugger Robot
56bfef1032 Merge "Implement fake vendor snapshot" 2021-01-08 00:24:51 +00:00
Inseob Kim
0c1ca4c3e4 Remove redundant snapshot functions
Leaving only isSnapshotAware is sufficient and clearer than before.

Test: m nothing
Change-Id: I497ee72ca0cc2386b1b947732473ac63b722621c
2021-01-07 05:15:50 +00:00
Inseob Kim
e9aec6aaca Implement fake vendor snapshot
A fake vendor snapshot is a vendor snapshot whose prebuilt binaries and
captured headers are all empty. It's much faster to be built than the
real vendor snapshot, so users can exploit the fake vendor snapshot to
reduce the size of vendor snapshot they need, by installing the fake
snapshot and then inspecting the ninja dependencies.

Bug: 157967325
Test: m dist vendor-fake-snapshot
Change-Id: I5e16e8dbbf9dd5e753cdd471ca73d06984a6cb2c
2021-01-07 14:09:31 +09:00
Jose Galmes
6f843bc4ba Support for recovery snapshot.
Bug: 171231437

Test: source build/envsetup.sh
Test: ALLOW_MISSING_DEPENDENCIES=true m -j nothing

Change-Id: I74636cf7f97e027a229a5ef7c776f2b7a42ead95
2020-12-23 10:34:44 -08:00
Colin Cross
127bb8b9f6 Don't rewrite LLNDK dependencies with .llndk suffix
Rewriting LLNDK dependencies with .llndk suffix requries referencing
a global data structure to determine if a given library is an LLNDK
library and therefore needs the .llndk suffix.  References to
global data structures from mutators must be removed to support
incremental Soong analysis.  Instead, move the LLNDK stubs rules
into the vendor variant of the implementing cc_library so that
the original name can be used.

As an incremental step, the llndk_library modules are left in
place, and the properties are copied into the cc_library via
the dependency specified by the llndk_stub property.  A followup
will move the LLNDK properties directly into the cc_library and
delete the llndk_library modules.

The global list of LLNDK libraries is kept for now as it is used
to generate the vndk.libraries.txt file.

Bug: 170784825
Test: m checkbuild
Test: compare Soong outputs
Test: all Soong tests
Change-Id: I2a942b21c162541a49e27b2e5833c9aebccff1d0
2020-12-21 17:53:30 -08:00
Colin Cross
a9c8c9f145 Call ctx.InstallFile for uninstallable cc modules
SkipInstall is actually primarily used to prevent making a module
visible to Make, rename it and add new SkipInstall that actually
skips installation without affecting Make.

Call c.SkipInstall() for uninstallable cc modules to allow calling
c.installer.install, which will collect PackagingSpecs for
uninstallable cc modules, allowing them to be used by genrules.

Bug: 124313442
Test: m checkbuild
Change-Id: I8038ed5c6f05c989ac21ec06c4552fb3136b9a7a
2020-12-17 10:02:18 -08:00
Ivan Lozano
f48c89cd39 Merge "Refactor for preliminary Rust vendor image support" 2020-12-11 13:40:20 +00:00
Ivan Lozano
f9e2172aec Refactor for preliminary Rust vendor image support
Refactors parts of CC to prepare for preliminary support for using Rust
static libraries in vendor images. Some previously private functions are
made public, and additional functions are added to LinkableInterface so
GetMakeLinkType can be passed a LinkableInterface.

Bug: 172525289
Test: m
Change-Id: I5fda48e79532fe9ceab255e18d910af58048a123
2020-12-10 10:29:48 -05:00
Inseob Kim
de5744a199 Add some comments for VNDK / vendor snapshots
Also some files are refactored:

- snapshot_prebuilt.go is separated from vendor_snapshot.go. Now
vendor_snapshot.go contains snapshot generation codes, while
snapshot_prebuilt.go contains module definition codes.

- Some helper functions are moved from snapshot_utils.go to util.go.

- Some ambiguous names of types and functions are renamed.

We still can add more detailed comments about the snapshots. They are to
be uploaded in follow-up changes, to avoid making this change too big.

Bug: 173474311
Test: generate vndk and vendor snapshot
Change-Id: I18fa837ccdf44a042b7a78e5c3df25fd2de96d95
2020-12-10 21:12:39 +09:00
Colin Cross
897a5ad507 Merge "Pass pctx and ctx to NewRuleBuilder" 2020-12-03 20:07:30 +00:00
Colin Cross
f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Chris Parsons
bf4f55f180 Improve commenting for cc/builer.go, and kill dead code
Test: m nothing
Change-Id: I836c717d3243e901257120be71246e419de4d28e
2020-12-01 18:28:38 -05:00
Jose Galmes
f729458e69 Support for recovery snapshot.
Bug: 171231437
Test: source build/envsetup.sh
Test: ALLOW_MISSING_DEPENDENCIES=true m -j nothing

Change-Id: I6b35fbb0b90ffffa941ea108cbd31a454b2dd403
2020-11-16 17:09:36 -08:00
Yifan Hong
60e0cfb5cb Add vendor-ramdisk image to Soong.
Add vendor_ramdisk_available and vendor_ramdisk attribute to
various rules. When a vendor_ramdisk variant of a module is
generated, it is installed to $OUT/vendor-ramdisk.

It is similar to a ramdisk image.
Test: m nothing -j

Change-Id: Ib2d16459f3094dbe21c3bdb7c016cb4b2bf62765
2020-10-22 10:26:26 -07:00
Inseob Kim
67be732de1 Set flag exporter provider for vendor snapshots
Also, TestVendorSnapshotUse is amended so it can test whether vndk
snapshot and vendor snapshot modules export their flags correctly.

Bug: N/A
Test: manual test
Change-Id: Ie62ad1206449b1a40db8a2282af00616db693017
2020-10-19 10:15:28 +09:00