Commit graph

2353 commits

Author SHA1 Message Date
Liz Kammer
fe8853d2e6 Add test & documentation for PropertiesToApply
Document PropertiesToApply expectations about props, and suggest it
should generally be generated via CreateProperties.

Test: go soong tests
Test: m nothing
Change-Id: I7cc2590db96865382ad6e0da333d4a4e2c697f45
2020-12-17 18:59:41 +00:00
Chris Parsons
dbcb1ff469 Use aquery to declare bazel actions in the ninja file.
This effectively moves execution of Bazel actions outside of soong_build
and alongside ninja execution of the actual ninja files, whether that be
by ninja or by Bazel itself.

This almost allows for mixed builds and Bazel-as-Ninja-executor to
coexist, but requires hacks explained in b/175307058.

Test: Treehugger
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Test: lunch aosp_flame && USE_BAZEL=1 USE_BAZEL_ANALYSIS=1 m libc,
though this requires a hack of the main BUILD file. See b/175307058

Change-Id: Ia2f6b0f1057e8cea3809de66d8287f13d84b510c
2020-12-15 17:39:23 -05:00
Treehugger Robot
bbbdff3637 Merge "Revert^2 "Always turn on compatible property"" 2020-12-15 12:00:11 +00:00
Treehugger Robot
e223512bb4 Merge "Prebuilt stub not available to platform is handled correctly" 2020-12-15 06:26:33 +00:00
Jiyong Park
f7c3bbe433 Prebuilt stub not available to platform is handled correctly
When a shared library providing stubs is included in an APEX, only the
stub variant (i.e. version:"1", etc.) gets emitted to Android.mk. This
enforces that everything in the Make world to link to the stub providing
only the public APIs of the library. The non-stub variant (i.e.
version:"") isn't exposed, otherwise, others will be able to access
private part of the lib which isn't guaranteed to be stable.

This has been done by unhiding the stub variant when it is known that
the library is actually included in an APEX. Note that stub variants are
by default hidden.

The above mechanism however doesn't work when the shared library is
replaced by a prebuilt and when the APEX is also replaced by a prebuilt.
Then, the fact that the prebuilt library is actually in the APEX gets
lost.  In that case, AnyVariantDirectlyInAnyApex() returns false for the
prebuilt library. As a result, the stub variant remains hidden and not
emitted to Android.mk.

This change fixes the problem by checking if the lib isn't available for
the platform at all. If not available for the platform (e.g.
apex_available doesn't have "//apex_available:platform"), the lib is
assumed to be included in an APEX even when it actually didn't go
through the apex mutator (... because it's a prebuilt).

Bug: 175166063
Test: m nothing

Change-Id: I41fabd5b368baecf4dc3c5a080b466f8bcd79d77
2020-12-15 09:17:36 +09:00
Patrice Arruda
05ab2d0731 Enable bazel profiling in soong_build.
Bazel is executed several times during the execution of soong_build.
For each bazel execution, generate a profile and save under the
BAZEL_METRICS_DIR which is defined in soong_ui.

Bug: b/174479924
Test: * USE_BAZEL_ANALYSIS=1 USE_BAZEL=1 m nothing and checked
        if the cquery and graph build bazel profiles were generated.
      * Verified that the generated bazel profiles were uploaded
        to the local dev metrics pipeline.

Change-Id: I3d20204484dc6c5a1525a5d3eec1d62cfb33535b
2020-12-14 18:48:11 +00:00
Paul Duffin
57fab96e01 Merge changes Iaca95efc,I7ccd5581
* changes:
  Add RemoveOptionalPrebuiltPrefix() helper function
  Delegate work of apexInfoMutator to ApexInfoMutator interface
2020-12-14 10:54:21 +00:00
Treehugger Robot
9ae3c52b20 Merge "Add dependency to list of asset files" 2020-12-12 01:48:52 +00:00
Colin Cross
c20dc8533e Add dependency to list of asset files
We had a dependency on each file in the asset directories, but that
wouldn't cause aapt2 to run if a file was removed.  Add a dependency
on a file that contains the list of files in the asset directories.

Fixes: 172867096
Test: m CarrierConfig && rm packages/apps/CarrierConfig/assets/carrier_config_no_sim.xml && m CarrierConfig
Change-Id: I35f3b85355fa890a3e95eaa6458a21466b6930e4
2020-12-11 19:54:16 +00:00
Paul Duffin
d23c726b36 Add RemoveOptionalPrebuiltPrefix() helper function
Test: m nothing
Bug: 171061220
Change-Id: Iaca95efcaf3f02e066751c6e988d609ac40e048a
2020-12-11 18:13:08 +00:00
Treehugger Robot
f4b00aed6e Merge "Add PackageFile and use it for prebuilt build tools" 2020-12-11 15:42:50 +00:00
Paul Duffin
08703826bd Merge "Rename apexDepsMutator to apexInfoMutator" 2020-12-11 10:06:19 +00:00
Colin Cross
4158950c40 Add PackageFile and use it for prebuilt build tools
Add PackageFile that creates a PackageSpec like InstallFile,
but doesn't create the install rule.  Also improve the documentation
on InstallFile and friends, and on HostToolProvider.

Bug: 124313442
Test: m checkbuild
Change-Id: I30f44a2b3833e48d0bb71bf7c45955f88fb00bc8
2020-12-10 16:39:28 -08:00
Colin Cross
d3b30670f2 Merge changes I1d6750db,Ie2c197bb,I6a51c89a
* changes:
  Add TransitivePackagingSpecs
  Add executable flag to sbox copy requests
  Export files to install as a depset
2020-12-10 21:53:49 +00:00
Paul Duffin
949abc03eb Rename apexDepsMutator to apexInfoMutator
Generally ...DepsMutator methods add dependencies between modules but
the apexDepsMutator does not which can be confusing. This renames
apexDepsMutator to apexInfoMutator and adds some extra documentation to
clarify its function. It also renames the registered name for the
mutator and its associated providers from apex_deps to apex_info.

Test: m nothing
Bug: 171061220
Change-Id: Ic074a281215b23e982448ccff7ac075236123bee
2020-12-10 20:16:57 +00:00
Treehugger Robot
cac133df14 Merge "Remove AndroidMkExtraFootersFunc entries param." 2020-12-10 07:04:02 +00:00
Mohammad Islam
5d2c54f1e2 Merge "Enable soong build tool to handle APEX compression" 2020-12-09 14:34:42 +00:00
Jaewoong Jung
729c0bdb54 Prebuilt is enabled even if the source one is not.
This fix a bug in a recent android_app_import-related change
(I8215aa1a6dad74b503eaa1169a64f8c30dda287c) where it uses an incorrect
module reference, and so disables a prebuilt module if its source-based
counterpart is disabled.

Test: app_test.go
Bug: 152343232
Change-Id: Ic7f5ca40be58f06e44b5ec4945caf209ef2dae96
2020-12-08 19:11:54 -08:00
Colin Cross
69e9230641 Merge "Reimplement DepSet as a wrapper around a generic implementation" 2020-12-08 19:36:22 +00:00
Mohammad Samiul Islam
3cd005d347 Enable soong build tool to handle APEX compression
1. Soong can now detect PRODUCT_COMPRESSED_APEX flag
     We don't want APEX to be compressed on all devices. Only those that
     have explicitely set PRODUCT_COMPRESSED_APEX flag.
2. Handle "compressible" field in soong build rule
     On devices that supports APEX compression, all APEX will be
     compressed by default. If any apex does not want to be compressed,
     they will need to state that by setting "compressible" field to
     false
3. Can use apex_compression_tool to compress APEX
     Note we compress the APEX after it has been signed. That way, when
     we decompress we will get a signed APEX.
4. Place the compressed APEX in system with .capex extension
     This makes it easy to identify. We still preserve the original
     extension so that when we decompress, we can just rename by cuttif
     off the .capex extension.

Note: with this change, we can create a system image with compressed
APEX, but we cannot boot with it since platform doesn't know how to
handle .capex files. Platform support will be added on follow up CLs.

Bug: 172911362
Test: OVERRIDE_PRODUCT_COMPRESSED_APEX=true m (apex_test.go)
Test: observed $OUT/system/apex has .capex files
Change-Id: I20ac4c4ceb521924c751a6017f979b2d808fdded
2020-12-08 13:06:25 +00:00
Colin Cross
ffe6b9d9ba Add TransitivePackagingSpecs
Add TransitivePackagingSpecs to return the PackagingSpecs for a
module and any of its transitive dependencies that have dependency
tags for which IsInstallDepNeeded returns true.

Bug: 124313442
Test: m checkbuild
Change-Id: I1d6750db830d1601d696349674f0b7071372ca11
2020-12-07 12:27:50 -08:00
Colin Cross
96c44127d1 Reimplement DepSet as a wrapper around a generic implementation
Implement depSet as a generic depsets implementation using reflection,
and then make DepSet a type-safe wrapper around it.  This will allow
additional wrappers for depsets that work with other types.  All of
this can be replaced with generics once Go supports them.

Test: depset_test.go
Change-Id: Id9df17bcc76f6c1545e7eb498f298066cf8a7679
2020-12-07 12:27:50 -08:00
Colin Cross
5d5839522c Export files to install as a depset
Export files to install through a depset instead of a list to reduce
the size at each module.

Bug: 124313442
Test: m checkbuild
Change-Id: I6a51c89acc9f0f3a9c3c792d3ef8a7cfd6b8bef2
2020-12-07 12:27:50 -08:00
Jaewoong Jung
02b11a6035 Remove AndroidMkExtraFootersFunc entries param.
I added in case anyone needs to access AndroidMkEntries to generate
footer lines, but nobody uses it, and it only confuses people.

Test: m nothing, TreeHugger
Change-Id: Ic8a450e3c306d9228c1fdec212c7441bd6aaee03
2020-12-07 10:23:54 -08:00
Jaewoong Jung
71debbfcc5 Merge "No matching variant android_app_import fix" 2020-12-07 14:33:17 +00:00
Jingwen Chen
946e0b69d2 Merge "Add bp2build mode to soong_build." 2020-12-07 07:16:15 +00:00
Jaewoong Jung
84f1b80866 No matching variant android_app_import fix
This fixes a bug where android_app_import or android_test_import without
a matching variant or a default apk property value breaks the build.

Test: app_test.go
Fixes: 152343232
Change-Id: I8215aa1a6dad74b503eaa1169a64f8c30dda287c
2020-12-04 11:51:29 -08:00
Jingwen Chen
4133ce6912 Add bp2build mode to soong_build.
This CL adds a new CONVERT_TO_BAZEL env var, and a bp2build goal. It
reuses the queryview architecture, but registers no mutators before
converting to BUILD files. This gives us a blank slate of the module
graph to work with, and create a shadow/alternate pipeline of mutators
to converge the Bazel BUILD graph to be semantically equivalent with the
Android.bp graph (after apply its current set of mutators).

The command to do so is:

    $ CONVERT_TO_BAZEL=true m bp2build

To not clobber with queryview, the generated files are in
out/soong/bp2build.

Test: CONVERT_TO_BAZEL=true m bp2build && bazel query --config=bp2build //...
Test: m queryview && bazel query --config=queryview //..
Test: soong tests
Test: TH presubmit
Fixes: 174465461
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I4dd0ccc73abc345d70a50ca2803d6f400cd8c863
2020-12-03 23:36:57 -05:00
Yifan Hong
4396c01aff Merge "Do not install init.rc for (vendor_)ramdisk variants." 2020-12-03 22:27:53 +00:00
Colin Cross
897a5ad507 Merge "Pass pctx and ctx to NewRuleBuilder" 2020-12-03 20:07:30 +00:00
Treehugger Robot
c0a7b832f3 Merge "WriteFileRule: Chunk long content and merge them to result" 2020-12-03 04:44:02 +00:00
Yifan Hong
919dae1e67 Do not install init.rc for (vendor_)ramdisk variants.
Previously, the vendor_ramdisk variant of a module installs the
init.rc file to /vendor/etc/init.

In first stage init, no init rc files are read, so no init rc files
should be installed to any ramdisks.

This change removes LOCAL_INIT_RC for the (vendor_)ramdisk variant
of a module, so that init rc file is not installed for (vendor_)ramdisk
variant of a module.

Test: m snapuserd snapuserd.vendor_ramdisk
      and manually inspect the out directory
Fixes: 174587489
Change-Id: I71227d807244b6538be0d6acce5e94f129a86065
2020-12-02 18:55:29 -08:00
Jingwen Chen
97d85c8a61 Merge "Remove soong.config, FileConfigurableOptions, MegaDevice config." 2020-12-03 02:07:35 +00:00
Jaewoong Jung
2b2bc8c712 Merge "Bolster comments in android/androidmk.go." 2020-12-02 15:16:27 +00:00
Hans Månsson
d3f2bd79e8 WriteFileRule: Chunk long content and merge them to result
sbox.textproto files are created when handling genrule. The command
for the genrule and output files are written to this file in the
following format:

commands <
  command: "...."
  copy_after: <
    from: "out/asm-generic/auxvec.h"
    to: "out/soong/.intermediates/kernel/msm-4.19/qti_generate_kernel_headers_arm/gen/asm-generic/auxvec.h"
  >
  copy_after: <
    from: "out/asm-generic/bitsperlong.h"
     to:
     "out/soong/.intermediates/kernel/msm-4.19/qti_generate_kernel_headers_arm/gen/asm-generic/bitsperlong.h"
  >
  ....
>

This file grow by one copy_after entry for each output file.

When generating kenrnel headers where the number of output files are
~1000 we run into problems as the contents of sbox.textproto files
are written to disk by generating a shell script using the following
template: /bin/bash -c 'echo -e "$$0" > $out' $content
If $content is very long as in the case of generating kernel headers we
run into the issue where the command line is so long that the shell
script return E2BIG.

Fix this issue by chuking contents into smaller files and then merge
them as a final step.

Test: Build
Issue: 174444967
Change-Id: I1a74023b4222d19672e4df7edb19810a9cf2136f
2020-12-02 15:58:48 +01:00
JaeMan Park
a4d314c91c Merge "Add java sdk library enforcement flag" 2020-12-02 04:39:10 +00:00
Jiyong Park
f10995b7b6 Merge "Remove the arch-based filtering hack in packaging.go" 2020-12-02 01:24:17 +00:00
Kiyoung Kim
e76ec1cbeb Merge "Generate system stub library list at build time" 2020-12-02 00:28:19 +00:00
Colin Cross
f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Treehugger Robot
464e6c71df Merge "Add test suite handling to central androidmk code" 2020-12-01 22:19:26 +00:00
Liz Kammer
57f5b33ad3 Add test suite handling to central androidmk code
MTS is introducing partial MTS test suites that are per-module, with
names of the format: mts-${MODULE}. By centralizing the code for test
suites, we can automatically add "mts" test suite when an
"mts-${MODULE}" test suite is specified, reducing duplication.

Test: m mts
Bug: 170318013
Change-Id: I8ce9d3c252fcc0a937bb5f2826d21cb6c6932d82
2020-12-01 12:23:56 -08:00
Jaewoong Jung
7ef4a900f4 Bolster comments in android/androidmk.go.
Bug: 173398138
Test: N/A
Change-Id: I46048b8398356d658fd5ae0299b59e6e18429198
2020-12-01 06:15:14 -08:00
Treehugger Robot
882c042575 Merge "Document more functions, minor cleanup" 2020-12-01 14:07:35 +00:00
Jiyong Park
d630bdd4dd Remove the arch-based filtering hack in packaging.go
aosp/1501613 has landed. The filtering hack is no longer needed.

Also, fixed a bug that the dependency to the dynamic linker is not
treated as install dep.

Bug: N/A
Test: `m hosttar` and compare the built cvd-host_package.tar.gz with and
without this change. There is no removed file.

Change-Id: I19ef64e10d34bc8be2592c5258505cd632c91af9
2020-12-01 19:25:39 +09:00
Jiyong Park
cc1157cd49 Reformat comments in packaging.go
set textwidth=100

Bug: N/A
Test: N/A

Change-Id: I2fb315c9488010d189b7757a4d93be181672b4a6
2020-12-01 05:27:48 +00:00
Kiyoung Kim
24dfc1fbb2 Generate system stub library list at build time
List of stub libraries from system (and bionic APEX) is currently
managed in static file. This change generates the list at build time and
adds to the system config so linkerconfig can use it.

Bug: 172889962
Test: Build cuttlefish and confirmed list is generated in
/system/etc/linker.config.pb

Change-Id: Ie0400e9d1098b0de19c6a7c25f261384aadde0a3
2020-12-01 14:09:03 +09:00
Jiyong Park
493e7ec3ee Merge "Clients of PackagingBase can customize dependency tag to use" 2020-12-01 03:44:38 +00:00
Jiyong Park
65b6224471 Clients of PackagingBase can customize dependency tag to use
Previously, the dep tag used by PackagingBase was fixed, which prevented
some of its clients (e.g. cvd-host-package) from opting in to
android.InstallAlwaysNeededDependencyTag. Now, PackagingBase.AddDeps
accepts the dependency tag to use.

Also, dependencies toward rust dylib, rlib, and proc_macro are
configured to return true on InstallDepNeeded. This is required to
install shared_lib dependencies of the rust modules when they are
depended on by a rust binary.

Exempt-From-Owner-Approval: a trivial change after +2 from the owner.
This has to land ASAP as many users are affected by acloud being
unavailable.

Bug: N/A
Test: m
Test: acloud create --local-instance --local-image
Change-Id: If22aee7c6f314fcb03b9d4fe6901a2557f1e899c
2020-12-01 03:43:05 +00:00
Alex Humesky
29e3bbe8b8 Update comments in genrule.go.
Test: Noop
Change-Id: Ib341899ea0f04e059d5471ee6e5bff8ebba82583
2020-11-30 12:30:20 -05:00
Sasha Smundak
1e53392425 Document more functions, minor cleanup
Fixes: 173745248
Test: treehugger
Change-Id: I075cc5197fff79179ea976927701290078e4d9ac
2020-11-30 08:53:58 -08:00