Commit graph

204 commits

Author SHA1 Message Date
Dan Willemsen
b13a9487a0 Add apex name to description
Otherwise we see multiple concurrent builds of different actions with
the same description:

    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc

Is now:

    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc [com.android.resolv]
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc [com.android.adbd]
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc [com.android.art.debug]
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc [com.android.media]

Test: m  <watch output>
Change-Id: Ib8fccc144e8fe5fc33cafe8536538b892bd42765
2020-02-14 12:33:52 -08:00
Treehugger Robot
f048446f92 Merge "Add vendor snapshot modules" 2020-02-13 11:01:10 +00:00
Paul Duffin
54d9bb70d0 Improve documentation of the enabled property
Makes it clear that disabling a module is something that should be
avoided if possible.

Test: m soong_docs
Change-Id: Ief11e705f049edfd53b2af66bdf753f24d32aa71
2020-02-12 10:28:40 +00:00
Treehugger Robot
705cd4600d Merge "Implement vendor snapshot" 2020-02-10 23:40:50 +00:00
Colin Cross
eabaedd520 Fix product variables in defaults modules
Product variables structs are generated at runtime to contain only
the properties that apply to the current module.  Defaults modules
always contained all product variable properties. Defaults modules
apply their properties to the target module using
proptools.PrependProperties, which prepends structs that have
matching types.  Filtered property structs had a different type
and were dropped.

Even after adding filtering to the defaults product variable
properties, defaults modules may contain more property structs
than the target module they are applied to, so the product
variables struct for the defaults module could contain more
fields than the product variables struct for the target module.
Use proptools.PrependMatchingProperties when applying defaults
of product variables instead, which will apply matching properties
across types.

Test: defaults_test.go
Test: variable_test.go
Change-Id: I281bdefef92053457a3b7b65383493a4e7d999df
2020-02-06 17:43:29 -08:00
Colin Cross
6961a491a5 Fix product variable zero value check
The zero value check was being done by using reflect.DeepEqual on a
field from the default product variables, but this results in
comparison against a random type when the product variables struct
for the module has been filtered down.  Luckily this will always
fail false, which just removed and optimization but left the
behavior correct.

Use reflect.IsZero instead, which is both faster and correct.

Test: variable_test.go
Change-Id: Ieaaa590c2788ca39230e6695397e8ba8d1c6c103
2020-02-06 17:41:19 -08:00
Inseob Kim
eec88e1de8 Add vendor snapshot modules
This implements four modules (static/shared/header libraries, and
binaries) for vendor snapshot. These modules will override source
modules if BOARD_VNDK_VERSION != current.

Bug: 65377115
Test: 1) VNDK_SNAPSHOT_BUILD_ARTIFACTS=true m dist vndk vendor-snapshot
Test: 2) install snapshot under source tree
Test: 3) set BOARD_VNDK_VERSION and boot cuttlefish
Change-Id: I24ddb4c3aa6abeab60bbfd31bcbd8753e2592dc5
2020-02-07 02:59:24 +09:00
Inseob Kim
8471cdaced Implement vendor snapshot
Vendor snapshot can be captured with "m dist vendor-snapshot". With
vendor snapshot and vndk snapshot, older version of /vendor and newer
version of /system will be able to be built together by setting
BOARD_VNDK_VERSION to past vendor's version.

Only vendor modules under AOSP are to be captured. In detail, modules
under following directories are ignored:
- device/
- vendor/
- hardware/, except for interfaces/, libhardware/, libhardware_legacy/,
  and ril/

Test modules (cc_test, etc.) and sanitized modules are also ignored.

Bug: 65377115
Test: m dist vendor-snapshot
Change-Id: If7a2f6de7f36deee936930c0ccf7c47c4a0cebf6
2020-02-07 02:58:29 +09:00
Colin Cross
b7ec426e01 Merge "Simplify vendor conditionals" 2020-01-27 19:03:23 +00:00
Treehugger Robot
6cc0515e82 Merge "Create a highmem pool and put metalava into it" 2020-01-25 01:09:43 +00:00
Colin Cross
9d34f35815 Simplify vendor conditionals
Support vendor conditionals with no Go code.

Test: TestSoongConfigModule
Change-Id: I42546e7f17324921ada80f4d8e1cd399830f8dfc
2020-01-24 16:43:40 -08:00
Colin Cross
8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Yifan Hong
1b3348d50a Add ramdisk image.
It is similar to recovery image.
Test: m nothing -j

Change-Id: I11389777c6bfb0c0d73bbb4c70091c1e70f44077
2020-01-22 16:54:18 -08:00
Dan Willemsen
40efa1cd31 Fix dependencies through symlinks
Now that ninja uses lstat so that we can get the mtime of the symlink
itself, it's safe to use a real input here rather than an order only
input. This matches how our other install rules work for host files.
Symlinks should be quick to recreate, so I'm not copying the logic to do
order-only dependencies for the device.

Bug: 128577186
Bug: 145278193
Bug: 135515543
Test: m out/target/product/walleye/apex/com.android.art.debug/javalib/arm/boot.art
      rm out/soong/host/linux-x86/bin/dex2oatd64
      m out/target/product/walleye/apex/com.android.art.debug/javalib/arm/boot.art

Change-Id: Ief5ec752ce0d0216aeecd8e904afc59f3f08ed7d
2020-01-16 17:13:35 +00:00
Colin Cross
988414c2cf Sandbox soong_build by changing to root directory
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.

Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
2020-01-11 01:11:46 +00:00
Colin Cross
47e4f9e1e8 Revert "Sandbox soong_build by changing to root directory"
This reverts commit 05c25ccb4a.

Reason for revert: broke absolute OUT_DIR
Bug: 146437378

Change-Id: I523ed79d40e1c1ef040212ba794a7a084abea75d
2020-01-10 18:51:04 +00:00
Colin Cross
05c25ccb4a Sandbox soong_build by changing to root directory
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
2020-01-09 14:19:46 -08:00
Treehugger Robot
64d5ccef7a Merge changes from topic "carry_required"
* changes:
  APEX carries required properties of its dependencies
  *RequiredModuleNames functions are available via Module interface
2020-01-07 03:51:29 +00:00
Colin Cross
c34d2329b4 Use blueprint.EarlyModuleContext
Make android.EarlyModuleContext wrap blueprint.EarlyModuleContext

Test: m checkbuild
Change-Id: I5e05c4f3dfb177811bc244a5cc2d71f42e47613b
2020-01-06 10:57:24 -08:00
Jiyong Park
6a8cf5fab4 *RequiredModuleNames functions are available via Module interface
...so that the required module names can be known to other modules

Bug: N/A
Test: m
Change-Id: Ib032b51c4ea4fd77bdebcf09b2d1c2aec1c8eb27
2020-01-06 10:35:58 +09:00
Colin Cross
1184b647d5 Add EarlyModuleContext for LoadHookContext
Make LoadHookContext embed a new EarlyModuleContext instead of
BaseModuleContext to reduce its API surface in preparation for
moving it to run during parsing instead of mutators.

Test: m checkbuild
Change-Id: I1cd3ff3b636e7e24991a9184d7521903473e505a
2020-01-04 08:40:55 -08:00
Colin Cross
5e70805690 Add helper functions for getting output files from modules
Add OutputFileForModule and OutputFilesForModules to get paths from
OutputFileProducers.

Bug: 122332855
Test: m checkbuild
Change-Id: I2e9aeeb55987742077250390cdb804ba22e79442
2019-12-19 17:52:50 +00:00
Colin Cross
fb0c16e95a Move imageMutator before archMutator
Run the imageMutator between osMutator and archMutator so that
different arch variants can be set for the different partitions.

Bug: 142286466
Test: m checkbuild
Change-Id: I65d05714b75aa462bf9816da60fdc2deda4de593
Merged-In: I65d05714b75aa462bf9816da60fdc2deda4de593
(cherry picked from commit 9c8f687584)
2019-12-17 14:45:11 -08:00
Anton Hansson
f56f74be4a Fix duplicate sentence in visibility documentation
This paragraph was duplicated.

Test: treehugger
Change-Id: I6057b96e310ac0ebf8f6f2a86f93efafd189579e
2019-12-16 11:39:17 +00:00
Paul Duffin
e44358fa8f Add support for host sdk
Adds HostSupported() method to ModuleBase for use by sdk. Adds
host_supported/device_supported to all prebuilt modules and the
sdk snapshot (where necessary).

Adds TestHostSnapshot to verify the behavior is correct.

Bug: 143678475
Test: m nothing
Change-Id: I8b4d097e46d5804f62cb8f651a83069299a3e639
2019-12-02 12:59:38 +00:00
Jooyung Han
6aa363b7ae Merge "Fix: install flattened apex on system_ext" 2019-11-27 07:30:54 +00:00
Jooyung Han
91df2084ad Fix: install flattened apex on system_ext
This build rule is specific to platform APEXes.
For non-platform APEXes, MakeAsSystemExt() is not applied.

This fixes the cases of "soc_specific: true" apexes which fails to
build.

Bug: 139053989
Test: m nothing (soong tests)
Change-Id: I98d0257499647ab41cdaa62a3671d89addbdf833
Exempt-From-Owner-Approval: got +1 before rebasing
2019-11-27 07:30:37 +00:00
Colin Cross
09ef474b6f Merge changes I0dcc9c7b,I9bc40642
* changes:
  Move cc.imageMutator into the android package
  Make CreateVariations return []android.Module
2019-11-25 22:30:17 +00:00
Roland Levillain
97c1f34067 Fix a typo in a comment in android/module.go.
Test: m
Change-Id: I1ea4e3ba70fc5f9e919495442ff94ab1e3b1fce2
2019-11-22 14:20:54 +00:00
Colin Cross
7228ecd5e3 Move cc.imageMutator into the android package
Prepare for making the image mutator available to all modules and
moving it between the os and arch mutators by moving it into the
android package and using an interface implemented by the module
types to control it.

Bug: 142286466
Test: No unexpected changes to out/soong/build.ninja
Change-Id: I0dcc9c7b5ec80edffade340c367f6ae4da34151b
2019-11-20 15:21:32 -08:00
Sundong Ahn
d95aa2d45c Change the partition of outputfile
If some apex module can be installed for both flattened and unflattened,
flattened apex should be installed to system_ext partition.
So add MakeAsSystemExt func to change the partition from system to
system_ext.

Bug: 139053989
Test: m -j
Change-Id: I3e3430413a9045d96130af99e249567b1a26ed7e
2019-11-14 16:04:15 +09:00
Jooyung Han
43240b1191 Remove unused "product_services_specific" prop
product_services_specific is not used any longer.

Bug: 135957588
Test: m
Change-Id: I6c09d3c09773034085a1419cbffe86954066df4e
2019-11-12 17:07:52 +09:00
Ramy Medhat
dd0418a4d4 Run non-RBE supported actions in the local pool when USE_RBE is set.
Bug: 143938974
Test: ran CTS build at -j500 successfully.
Change-Id: I55074bd67308cd716972e24fb56a20bc393d5d9d
2019-11-05 22:57:35 +00:00
Colin Cross
a195f91471 Split out osMutator from archMutator
Split the archMutator into two mutators, osMutator to create
android/linux/darwin variants, and archMutator to then further
split them into arm/arm64, etc.  When combined with alias
variants in Blueprint this will allow us to depend on the
"android" variant without having to know what architectures
were used to build it.

Test: m checkbuild
Change-Id: I0f51d60fc440cf5ae503904ce7849ec797af5fe2
2019-10-16 15:27:23 -07:00
Colin Cross
2334757d24 Merge "Filter product variable property structs" 2019-10-07 20:53:44 +00:00
Treehugger Robot
96ce6ab143 Merge "Use localPool consistently for UseGoma() == true" 2019-10-04 18:36:07 +00:00
Colin Cross
70dda7e3da Separate InstallPath from OutputPath
Create a new type InstallPath that is similar to OutputPath to
differentiate intermediates output paths from installed output
paths.

RelPathString is a poorly defined, undocumented function that is
primarily used to get an install path relative to out/soong to
generate an equivalent install path for Make relative to $(OUT_DIR).
Move it to InstallPath for now, and fix the one remaining user on
OutputPath.

Add a method to create an NDK install path so that ndk_sysroot.go
doesn't have to do it manually with PathForOutput.

Bug: 141877526
Test: m checkbuild
Change-Id: I83c5a0bd1fd6c3dba8d3b6d20d039f64f353ddd5
2019-10-03 10:07:53 -07:00
Colin Cross
90ba5f4e98 Add InstallInRoot to allow modules to install into root partition
If InstallInRoot() returns true the module will be installed to
$OUT/root or $OUT/recovery/root.

Bug: 141877526
Test: m checkbuild
Test: no change to build.ninja or Android-${TARGET_PRODUCT}.mk
Test: TestPathForModuleInstall
Change-Id: Id6e435c6019f11eeb5806528fd464dbf220b88d9
2019-10-03 10:07:46 -07:00
Colin Cross
2e2dbc250a Use localPool consistently for UseGoma() == true
Remove the distinction between pctx.StaticRule and
pctx.AndroidStaticRule so that all of the local rules correctly
get assigned to the localPool.  Also put Module and Singleton
rules into the localPool.

Test: compare out/soong/build.ninja
Change-Id: Id2bb38eff3c7209340fe55bc9006f00bd3661d81
2019-10-01 14:11:20 -07:00
Jaewoong Jung
0949f31657 Fix android_test install path.
Test: m nothing + diff soong mk and ninja files.
Bug: 140795853
Change-Id: I3667eac951dea7e447cf73219ff89199fca9ed63
2019-09-27 16:42:20 -07:00
Colin Cross
18c46807c9 Filter product variable property structs
For each module, take the default product variable properties and
create a new type that contains only the properties that exist in
the current module.  This will provide better errors when attemping
to set product variable properties that will never get used for
a module, and fixes errors when a module attempts to use a product
variable that also contains properties that don't apply to the
current module.

Fixes: 79249983
Test: TestProductVariables
Test: m checkbuild
Change-Id: I4dc24f7781b8ce798651cfc8bf3563005a92c13d
2019-09-26 15:03:07 -07:00
Jooyung Han
344d5439c1 Add "apex_vndk" module type
"apex_vndk" is a variant of "apex" module.

apex_vndk {
  name: "com.android.vndk",
  ..
}

This rule is used to produce a VNDK APEX per vndk version.
It supports automatic inclusion of vndk libs.

If "vndk_version" property is set, the prebuilt vndk libs of
the version will be included in the apex bundle.

apex_vndk {
  name: "com.android.vndk.v29"
  vndk_version: "29",
  ...
}

Otherwise, platform's vndk version is used.

This will replace /system/{lib}/vndk-{ver} and vndk-sp-{ver}.

Bug: 134357236
Bug: 139772411
Test: m com.android.vndk
Change-Id: Ib5c86e625839389670d13c683a7427198ef6852f
2019-09-10 11:22:59 +09:00
Jooyung Han
d48f3c3885 soong: Add tests for depending on disabled module
This will check if direct deps of android.Module type is "Enabled()".
Previously, this is checked only if a module calls VisitDeps*()
functions in GenerateAndroidBuildActions().

Most modules call VisitDeps*() in GenerateAndroidBuildActions(),
but some modules don't. For example, "apex" module calls
WalkDepsBlueprint() or VisitDirectDepsBlueprint() since it
exceptionally depends on non-android.Module modules.

Therefore, when an apex module depends on disabled(enabled:false) module,
build fails with panic, which is fixed by this change.

Test: m # runs soong tests
Change-Id: I81c5c148bbd51a253d2904690eb76ae7b6df1a0f
2019-08-26 13:19:42 +00:00
Treehugger Robot
f02f30952d Merge "java_import can be included in apex" 2019-08-13 04:18:29 +00:00
Jiyong Park
9e6c242856 java_import can be included in apex
java_import can be included in apex via 'java_libs' property.

Bug: 139175488
Test: m (apex_test.go updated)
Change-Id: I3680a47cdac93b0cb2d41da8df3f8defa2bbe670
2019-08-13 09:36:39 +09:00
Colin Cross
607d8587e4 Add InstallBypassMake
Allow modules to mark themselves as InstallBypassMake, which will
cause android.PathForModuleInstall to return a path in $OUT_DIR
instead of $OUT_DIR/soong.  This can be used for modules that
can handle installation to the final location on their own.  The
main blocker for most modules is support for the "required" property,
which requires adding dependencies on the installed location of
other modules.

Bug: 122332855
Test: m checkbuild
Change-Id: I85238d937ff30335167d4b3fec79bbefc734b5e1
2019-08-06 09:45:04 -07:00
Paul Duffin
95d53b584f Add defaults_visibility support
Bug: 130796911
Test: m nothing
Change-Id: I7b9462d3360be2bbeaf6ff38c5328f45ff5b5ebb
2019-07-25 11:48:21 +01:00
Paul Duffin
63c6e183d5 Refactor visibility to support visibility on defaults modules
Existing modules, either general one or package ones have a single
visibility property, called visibility in general, and
default_visibility on package, that controls access to that module, or
in the case of package sets the default visibility of all modules in
that package. The property is checked and gathered during the similarly
named phases of visibility processing.

The defaults module will be different as it will have two properties.
The first, visibility, will not affect the visibility of the module, it
only affects the visibility of modules that 'extend' the defaults. So,
it will need checking but not parsing. The second property,
defaults_visibility, will affect the visibility of the module and so
will need both checking and parsing.

The current implementation does not handle those cases because:
1) It does not differentiate between the property that affects the
   module and those that do not. It checks and gathers all of them with
   the last property gathered overriding the rules for the previous
   properties.

2) It relies on overriding methods in MethodBase in order to change the
   default behavior for the package module. That works because
   packageModule embeds ModuleBase but will not work for
   DefaultsModuleBase as it does not embed ModuleBase and instead is
   embedded alongside it so attempting to override a method in
   MethodBase leads to ambiguity.

This change addresses the issues as follows:
1) It adds a new visibility() []string method to get access to the
   primary visibility rules, i.e. the ones that affect the module.

2) It adds two fields, 'visibilityPropertyInfo []visibilityProperty'
   to provide information about all the properties that need checking,
   and 'primaryVisibilityProperty visibilityProperty' to specify the
   property that affects the module.

The PackageFactory() and InitAndroidModule(Module) functions are
modified to initialize the fields. The override of the
visibilityProperties() method for packageModule is removed and the
default implementations of visibilityProperties() and visibility()
on ModuleBase return information from the two new fields.

The InitDefaultsModule is updated to also initialize the two new
fields. It uses nil for primaryVisibilityProperty for now but that
will be changed to return defaults_visibility. It also uses the
commonProperties structure created for the defaults directly instead
of having to search for it through properties().

Changed the visibilityProperty to take a pointer to the property that
can be used to retrieve the value rather than a lambda function.

Bug: 130796911
Test: m nothing
Change-Id: Icadd470a5f692a48ec61de02bf3dfde3e2eea2ef
2019-07-25 11:31:42 +01:00
Justin Yun
d5f6c8261a Rename product_services to system_ext
Bug: 134359158
Test: build and check if system_ext.img is created
Change-Id: Ice3229baf97a94c24a6eea22e7d4571941d7f843
2019-07-09 08:57:05 +00:00
Colin Cross
9a36223d59 Add a String() method to android.Module
Add a String() method to android.Module for use in debugging.  Store
the name and variations of the module as they are mutated.

Test: TestModuleString
Bug: 136473661
Change-Id: I74e393703dcfc96ed4e21ac4a4419a7858b59216
2019-07-03 15:37:11 -07:00