Currently, installation of a module is defined as an action of copying
the built artifact of the module to an install path like out/soong/host
(for host modules) and out/target/product/<device>/<partition> (for
device modules). After the modules are installed, the installed files
are further processed to create packages like system.img, vendor.img,
cvd-host-package.tar.gz, etc.
This notion of installation seems to have originated from the old time
when system.img is the primary product of the entire build process
(modulo a few more like root.img). Packaging the installed files as the
filesystem image was considered as a post-build step then.
However, this model doesn't seem to fit well to the current and future
environment where we have a lot more filesystem images (system, vendor,
system_ext, product, ...). The filesystem images themselves are even
grouped together to form a higher-level filesystem image like super.img.
Furthermore, things like cvd-host-package.tar.gz requires us to be able
to group some of the host tools in a format that isn't filesystem image.
Lastly, we are expected to have more filesystem images that are subsets
of system.img (and their friends) for the Android-like mini OS that will
be running on on-device virtual machines. These all imply that the
packaging (which we call installation today) is not a global post-build
step, but a part of the build rules for creating the package-like
modules.
A model better fits to the new sitatuation might be this; a module
specifies its built artifact and the path where it should be placed. The
latter path is not rooted at out/. It's a relative path to the root
directory which will be determined by another module that implements the
packaging. For example, cc_library will have ./lib (or ./lib64), not
out/target/product/<device>/<partition>/lib as the path. Then packages
like system.img, cvd-host-package.tar.gz, etc. are explicitly modeled as
modules and they have deps to other modules. Then the modules are placed
at the relative path under the package root, and the entire root
directory finally is packaged as the output file (be it img, tar.gz, or
whatever).
PackagingSpec is the first step to implement the new model. It abstracts
a request to place a built artifact at a certain path in a package. It
has extra information about whether the path should be a symlink or not,
and whether the path is for an executable. It currently is created when
InstallFiles (and its friends) are called, and can be retrieved via
the new method PackagingSpecs().
In this CL, no one is using PackagingSpec. The installation is still
done by the existing rules created in InstallFiles, etc. and the
structs are not used for the filesystem images like system.img.
Bug: 159685774
Bug: 172414391
Test: m
Change-Id: Ie1dec72d1ac14382fc3b74e5c850472e9320d6a3
This removes the need to source bazelenv.sh for USE_BAZEL_ANALYSIS, and
unifies mixed builds to use the checked in tools/bazel and bazelrc.
It also unifies all bazel-related output to be in out/bazel.
Without aosp/1502095, this change still requires toplevel_output_directories to be an empty
list, otherwise there'll be this error:
ERROR: Directories specified with toplevel_output_directories should be
ignored and can not be used as sources.
Test: With aosp/1441774: rm -rf out/ && lunch aosp_cf_x86_auto && USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 results
Change-Id: I69b217ec88da531415792bb6e04b6a194ca4718d
The documentation java_genrule_host states that it creates a single
variant, which would make it work with the single variant fallback
in AddDependency used by the data property, but it actually has a host
and a host-cross windows variant. Modify osMutator to take the
OS-specific enabled properties into account to skip creating variants
that will immediately be disabled so there is a single variant.
Test: m checkbuild
Change-Id: Ic2daab29f4fa3a3797d7a08348fbfcf1036ec5dc
The HostOrDeviceSupported enums have become unwieldy, replace them
with bitfields to simplify checking if a specific trait is set.
Test: Soong arch tests
Test: m checkbuild
Change-Id: I884ddd092d4c6f740def611b6a055007dc24bd6e
This allows mixed builds to continue functioning even given the toplevel
WORKSPACE file containing toplevel_output_directories with out/.
Test: Manually verified on aosp_flame building libc.
Change-Id: I80fc4853421317e2d2c7f03d92d58286df1342ce
In preparation for more complicated sandboxing support in sbox, use
a single implementation of the sbox sandboxing by moving genrule to
use RuleBuilder's sbox support instead of creating an sbox rule
directly.
Also move genrule's input list hash support into RuleBuilder.
Test: genrule_test.go
Test: rule_builder_test.go
Change-Id: I292184d02743c7e6887ebbcd232ba565db2ab0cc
Store ninja file deps from ExistentPathForSource on a
PackageVarContext in the config instead of the PackageContext, as
the PackageContext may be shared between multiple tests running
in parallel.
Test: all soong tests
Change-Id: Ib1809a4dd4a82696e0fe48a87eac21a44684ecb5
Store SingletonMakeVarsProviders in the config instead of a global
variable to avoid races between tests running in parallel.
Test: all soong tests
Change-Id: I2ab64f368b5ac673fd985399d4421ed018abc562
The android.WriteFile rule takes careful escaping to produce the
right contents. Wrap it in an android.WriteFileRule that handles
the escaping.
Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.
Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.
This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.
Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: Ic22603a5c0718b5a21686672a7471f952b4d1017
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.
Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
If SOONG_DELVE_PATH=<path> is set, Soong will use that as the path to
dlv. Otherwise, it will use the one on $PATH as usual.
Test: Manual.
Change-Id: Id4334f05d077a911750f601865bd3096d852d69c
Bazel overlay is an experimental feature. This renames the feature to 'queryview' to better describe its purpose, and also move away from the already overloaded 'overlay' term in Android.
Test: m queryview && bazel query --package_path=out/soong/queryview //...
Change-Id: I8b5068c7db46cb61a03a8e87af9c7c9077ebeff9
This was caught while running the soong_build docs action under Bazel as
ninja executor. Since the config file is not an explicit input to the
action, it's not present in Bazel's execution root during action
execution time, leading to a missing file error. To fix it, add the file
as an input to the action, if it's configured in product variables.
New soong_build statement:
build out/soong/docs/soong_build.html: s.writedocs.soongDocs $
out/.module_paths/Android.bp.list out/soong/soong.variables $
out/soong/dexpreopt.config | out/soong/.bootstrap/bin/soong_build
outDir = out/soong/docs
Test: m
Test: USE_BAZEL=1 m
Bug: 161958189
Fixes: 161958189
Change-Id: I2bf264e848bb29ac91f4196fbe9c4ecf3cea8a2c
Adds a singleton that traverses the module variants finding the ones
that are in the list (updatable and non-updatable) of boot jars and
add a ninja rule to ensure that they only contain packages from an
allowed list.
Replaces a hack that ignored any prebuilt boot jars supplied as dex
file with an equivalent one to ensure that they are still ignored.
A follow up change that switches to checking dex jars will allow the
hack to be removed.
The boot jars check can be strict or lax. If strict then all the boot
jars listed in the configuration must be found, otherwise it will only
check the ones it finds. It is strict by default unless
TARGET_BUILD_UNBUNDLED=true or ALLOW_MISSING_DEPENDENCIES=true.
Moves the script and data file from build/make.
Test: m check-boot-jars - for failing and passing cases
SKIP_BOOT_JARS_CHECK=true - no check-boot-jars target created
ALLOW_MISSING_DEPENDENCIES=true - not strict
TARGET_BUILD_UNBUNDLED=true - not strict
verified manually that apart from path differences the same
files (same check sum) were checked in both old make checks and
the new Soong ones
EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m check-boot-jars
Bug: 171479578
Change-Id: I9d81d6650ba64fc0d48d2dab4ba5a3ba8dd03dec
This adds an extra step to mixed builds: creating a master BUILD/bzl
file pair to facilitate running a single cquery command to analyze
all soong->bazel edges in a single request.
Test: Mixed build tested with aosp/1441774, verified ninja outputs
depend on `bazel-out/` intermediates
Test: Manually verified contents of master BUILD and bzl files
Change-Id: I04803bcc91ac4182578f505b3f42893061ddd167
As a result, one can enable bazel-as-ninja-executor separately from
mixed builds.
Test: Manually verified building image with and without bazelenv.sh.
Change-Id: Ia97806fb41e1de850a80b9483ed8a5ff50d9dab2
By making the Append and RemoveList methods return a new list instead
of modifying the existing list it makes the ConfiguredJarList usages
easier to reason about and safer to use, especially considering that
they are primarily used in global configuration.
Added some tests for Append/RemoveList to ensure that they work and
do not modify the original or result in newly created lists sharing
storage with the original which would lead to corruption.
Bug: 171756871
Bug: 171479578
Test: m nothing
EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m nothing
Change-Id: I541c4686ecdd45c6a0c8b1c93fedf0fcd5952e2b
After previous refactorings the CreateConfiguredJarList function is now
only used in tests and are supplied with a PathContext that will cause
ReportPathErrorf() to panic. So, this change removes the ctx parameter,
calls panic directly on any error and renames the method to make it
clear that it is for testing only.
Bug: 171479578
Test: m nothing
Change-Id: Icfb4bdfe720afa855b64ecf0e74a0b030882d029
This change:
* Switches BootJars/UpdatableBootJars fields of config.productVariables
from []string to ConfiguredJarList.
* Updates BootJars() method to simply concatenate the jars list from
the BootJars/UpdatableBootJars fields.
* Adds an UnmarshalJSON(..) method to ConfiguredJarList to support
unmarshalling from a single string array to avoid having to change the
format of the JSON file from which the configuration is loaded.
* Adds some additional calls to ConfiguredJarList(..) in tests to
convert from []string to ConfiguredJarList. They pass nil as the
ctx argument as there is no suitable PathContext which will cause any
errors to be thrown using panic. That is reasonable for hard coded
values in tests. A follow up change will clean up the calls to
ConfiguredJarList(..).
Bug: 171479578
Test: m nothing
Change-Id: I59b94dafb479ccd8f0471ed802be175af57be271
This change:
* Changes splitConfiguredJarPair to return an error rather than
reporting it through the context.
* Extracts the splitting of a list of pairs into a pair of lists into a
new splitListOfPairsIntoPairOfLists() method for reuse in a follow up
change.
Bug: 171479578
Test: m nothing
Change-Id: I824ed1bc673b47757424563fc37e1190adff2ed1
... and ramdisk_available modules. If a module is both
vendor_ramdisk_available and ramdisk_available, on a device
that mark recovery_as_boot and move_recovery_resources_to_vendor_boot
simultaneously (and incorrectly),
both will be installed to recovery/root/first_stage_ramdisk. Fix the
path conflict of the two variants by moving the vendor_ramdisk variant
to vendor-ramdisk/first_stage_ramdisk instead.
Also update comments for Vendor_ramdisk_available.
Test: m nothing -j
Bug: 156098440
Change-Id: I2b776b6fd8f5a2c361c0f6a89231e3cebc2646f0
By making the Append and RemoveList methods return a new list instead
of modifying the existing list it makes the ConfiguredJarList usages
easier to reason about and safer to use, especially considering that
they are primarily used in global configuration.
Bug: 171479578
Test: m nothing
Change-Id: I102c4fb42f0c54e4ed299d2921fbf5efeb6e99b9
This is only used from within the ConfiguredJarList implementation and
provides no real benefit over directly accessing the apexes slices.
Similarly, uses of Jar(int) from within the implementation are also
replaced with direct slice access.
Bug: 171479578
Test: m nothing
Change-Id: I7e799b1049f4a1da4140e55831c4559751278de6
Install to recovery/root/first_stage_ramdisk if
BOARD_MOVE_RECOVERY_RESOURCES_TO_VENDOR_BOOT,
otherwise vendor-ramdisk. In addition, append /system
if not InstallInRoot().
On devices with dedicated recovery partition,
BOARD_MOVE_RECOVERY_RESOURCES_TO_VENDOR_BOOT is not set,
and this installs to the correct place (under $OUT/vendor-ramdisk).
On devices without a dedicated recovery partition:
- To install a module available before switching root
to /first_stage_ramdisk, e.g. a binary under /system/bin,
use recovery{_available} and install the recovery variant
of the module.
- To install a module available after switching root
to /first_stage_ramdisk, e.g. a binary under
/first_stage_ramdisk/system/bin,
use vendor_ramdisk{_available} and install the vendor_ramdisk
variant of the module.
Test: pass
Bug: 156098440
Change-Id: I1af3f8889891a3e58263cda36f0680ce2b480499
Add vendor_ramdisk_available and vendor_ramdisk attribute to
various rules. When a vendor_ramdisk variant of a module is
generated, it is installed to $OUT/vendor-ramdisk.
It is similar to a ramdisk image.
Test: m nothing -j
Change-Id: Ib2d16459f3094dbe21c3bdb7c016cb4b2bf62765
This change introduces the concept of partition dir for InstallPaths.
It's the path to the partition where the InstallPath is rooted at. For
example, it's out/soong/target/product/<device>/<partitoon> for paths
created for device modules. For host modules, it is defined as
out/soong/host/<host_os>-<host_arch>.
The partition dir is obtained using the new PartitionDir() function.
Another change is that a freshly created InstallPath (usually via
PathForModuleInstall) is the result of joining PartitionDir() and the
remaining path elements. For example, PathForModuleInstall(ctx, "foo",
"bar").Rel() now returns "foo/bar". Previously, that call returned the
relative path from config.buildDir() ("out/soong"). This change is in
line with the behavior of other path-creating functions like
PathForModuleSrc where Rel() returns the path relative to the
contextually determined path like the module source directory.
Notice that the Join() call to InstallPath doesn't change
PartitionDir(), while does change the result of Rel().
p := PathForModuleInstall(ctx, "foo", "bar")
p.PartitionDir() is out/soong/host/linux-x86
p.Rel() is foo/bar
q := p.Join(ctx, "baz")
q.PartitionDir() is still out/soong/host/linux-x86
q.Rel() now returns baz
Bug: N/A
Test: m nothing
Change-Id: I916bb1c782a4bfe0fbd4854e349cd2a2a42f56b6
Prepare for making the relationship between an llndk_library stubs
module and the cc_library implementation module explicit by
adding an llndk_stubs property. Each cc_library will be updated
to point to its llndk_library, and the llndk_library name will
be changed to make the .llndk suffix explicit. Then the implicit
connection and suffix can be removed.
Bug: 170784825
Test: m checkbuild
Change-Id: I6b0482a3f286ec29b2e928551aa4317749f2b499
This CL adds symlink_outputs to various locations in Soong that creates
actions that creates symlink outputs, and explicitly mark them as such.
Test: m
Bug: 160568334
Change-Id: I322751bada52a9f49011c74731d84761586e03e7
Clarify that only __pkg__ and __subpackages__ are valid package
visibility specifications.
Test: go test soong tests
Test: m nothing
Bug: 168926331
Change-Id: I810e9ef92d081d85cd762e7f8c0b0716f0222883
Expands the error to suggest a potential fix, giving the package path
for the module that needs visiblity.
Test: go test soong tests
Bug: 168926331
Change-Id: Ie95dec03f5e2c3190bfaed1f0e5789eeb0ab1878
This retriggers soong_build whenever a new bzl, WORKSPACE, or
BUILD.bazel file is changed or added.
Test: Manually verified on bionic/libc genrules with manual changes to
related BUILD/bzl/WORKSPACE files -- these all retrigger builds.
Test: Updated finder_test.go
Change-Id: I634384f88781a6b6db32f5d6bf9c07e179e14c39
In apex_set and cc_prebuilt_library_*, provide a way to specify an
alternative source to use when build with sanitizers.
Test: prebuilt_test, apex_test
Change-Id: I1ab8091bf696d94da3547cf5248853df489bdee6
Export information about static libraries, shared libraries and
exported flags through Providers instead of accessing the module
directly. Much more is left to be converted, but this significantly
simplifies the dependencies on libraries with stubs by making it easy
for a module to masquerade as another by simply exporting the
providers from the other module. Instead of depending on all the
versions of a library and then picking which one to use later, it
can depend only on the implementation variant and then select the
right SharedLibraryInfo from the variant.
Test: m checkbuild
Test: only expected changes to build.ninja
Change-Id: I1fd9eb4d251cf96ed8398d586efc3e0817663c76
Let the flakes, and the crashes, and the deadlocks, and the leaks, and
the undefined behavior which steals away so many nights find their
numbers decimated in the homogeneous conditions we now rest our data.
To give some intuition why this is not so underperformant, zeroing
memory is one way of priming caches.
This change is actually a no-op, because build/make/core/soong_config.mk
always overrides this setting, but it reflects the spirit and future
direction of this change.
Bug: 131355925
Test: basic simple perf comparisons
Change-Id: I8254c36373de52091839561973c9741e8f85aa09
With this change, users don't need to figure out which libraries
actually hold the resources to be overlaid when targetting apps with a
core lib dependency (e.g. Settings, SystemUI).
Fixes: 169898727
Test: app_test.go
Change-Id: I3c3b9dc0a377b1828db1199858a73d080a173205
Targets[Android][0] is not always the first, preferred target. Add a new
field to config to provide the correct target and begin to use it.
Test: m nothing
Test: build/soong/scripts/build-ndk-prebuilts.sh
Fixes: 156980228
Change-Id: Ib56f52424ebf47f3eee6ac62b9fd9eca2919d63a
stderr is a bytes.Buffer, but the String() method has a pointer
receiver, so stderr does not satisify the Stringer interface
and can't be used for a %s argument. Make stderr a *bytes.Buffer
instead.
Test: go vet ./android
Change-Id: I994402cb954946279375c9d447ad3854380381cc
ctx.PrimaryModule() is wrong in the case of a java_binary that
supports both host and device, use an explicit dependency instead.
Once the dependency exists there is no need to manually request
the jar be installed, it will automatically be installed by the
host installation rules for dependencies.
Test: TestBinary
Change-Id: Iddeea2d08bc574c79d42139020558cd70d718ca1
This reverts commit 323dc60712.
Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests
Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
With this change, bazel_module is a specifiable property on
genrule module definitions. With bazel-enabled mode, soong_build will
defer to Bazel for information on these modules.
source build/soong/bazelenv.sh to enter bazel-enabled mode.
Test: Manually verified on bionic/libc genrules using aosp_cf_x86_phone-userdebug
Change-Id: I3619848186d50be7273a5eba31c79989b981d408
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.
Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
A global variant was used to store the global mapping between
modules and APEXes. Replace it with storing pointers to APEX
contents inside each module so that they can query the contents
of any APEXes they belong to.
Bug: 146393795
Test: all Soong tests
Test: single line change to build.ninja host install dependency ordering
Test: no Android-${TARGET_PRODUCT}.mk, make_vars-${TARGET_PRODUCT}.mk or late-${TARGET_PRODUCT}.mk
Change-Id: Id2d7b73ea27f8c3b41d30820bdd86b65c539bfa4
* changes:
Don't create version variations of sdk modules
Create fewer empty version variants
Don't make SplitPerApiLevel imply UseSdk
Remove vendor crt special case
Fix apex_test.go and add it to Android.bp
apex_test.go wasn't listed in the Android.bp file, which allowed
it to bitrot. Make the API level methods take a PathContext
so that they can be called from a test using configErrorWrapper.
Also fix an int that was converted to a string.
Test: apex_test.go
Change-Id: I1ff87134c837bd5d344d22550baabde10d1b0b2e
Some mainline modules provide an `sdk` and one or more set of
module_exports. When the snapshots are unpacked into separate prebuilts
directories (one per snapshot) then any dependencies from a member of
one snapshot to a member of another may require the latter to have
additional visibility rules.
Previously, these rules had to be added to each source module. This
change allows additional visibility rules to be specified on the
sdk/module_exports that are added to all the prebuilts in its snapshot.
Bug: 155921753
Bug: 168301990
Test: m nothing
Change-Id: Ia3eacb5749981f04770ae9872a8013e43c5c6ef1
This change ensures that each prebuilt in a snapshot explicitly
specifies its visibility even when that visibility is the current
default. This is done for two reasons:
1. It simplifies a follow up change that adds visibility rules to an
existing set of rules.
2. It ensures that the snapshots are independent of the current Soong
default visibility.
The latter is important because we intend to switch from modules being
visible to everyone by default (i.e. //visibility:public) to only being
visible to modules in the same package (i.e. //visibility:private). By
making the snapshots of modules that do not specify any visibility
explicitly specify that they are "//visibility:public" it ensures that
the snapshots will not need to be changed when the default changes.
Bug: 168301990
Test: m nothing
Change-Id: Ia034f4a1e5124c17f46d73b0e9a6c5f2a251038e
Instead of tracking per module and per module variant, track allowed
list of dependecies for all modules combined. This avoids issues with
different products and different downstream branches having different
build graphs.
To compare allowed_deps.txt vs head, run:
:; m -j out/soong/apex/depsinfo/new-allowed-deps.txt.check
To update source allowed_deps.txt, run:
:; build/soong/scripts/update-apex-allowed-deps.sh
Bug: 149622332
Test: m
Change-Id: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
Merged-In: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
(cherry picked from commit 453555083b)
(cherry picked from commit e5207cd9a6)
It will be used by the cvd_host_package module type to gather installed
files and package them in a distributable format.
Bug: 168086242
Test: m
Change-Id: I3af6dcda0bce1f298b1500fbac311c9593ddf375
I2954bb21c1cfdeb305f25cfb6c8711c930f6ed50 switched normalizeVersions
to work on ApiLevels, which inadvertantly caused it to return "current"
instead of "10000" for libraries that specify "current" in their stubs
property. ChooseSdkVersion couldn't handle "current" because it was
manually converting the version to an int. Switch ChooseSdkVersion
to use ApiLevels instead so that it can handle "current".
Test: m checkbuild
Change-Id: Id412359e092483ba419118dd03bc206fae702a96
Mulitple init_rc files may be necessary to support multilib binaries
with different installed names.
Test: m checkbuild
Change-Id: I17d3a3b2346a25b230e5a325deb0d78f0ee045f0
R is out. The platform codename on the main branch is now S and the API
level of the platform is 30, which is R because S hasn't been finalized.
Bug: N/A
Test: rm -rf out; ./build/soong/soong_ui.bash --make-mode --skip-make
Change-Id: I486b9639bccfe17a42681e1f84097373f8f401e7
Both will create a nested property set, that may be merged with an
existing one.
Test: m nothing
Bug: 151303681
Change-Id: I30696ba3eb8960ca6fa54c9ee2cf6229ab9f5da9
linux_bionic_arm64 is the new host target using Bionic as libc. This
change adds a new Soong config Host_bionic_arm64 that enables the
target.
This change also amends firstTarget() function to handle the case when
the available targets for an OS have different arch families. In that
case, first target is selected for each of the arch family. This is
needed in the (rare) care when both linux_bionic_x86_64 and
linux_bionic_arm64 targets are enabled.
Bug: 159685774
Test: add "Host_bionic_arm64": true to out/soong/soong.config and run
./build/soong/soong_ui.bash --make-mode --skip-make
out/soong/host/linux_bionic-arm64/bin/aarch64-linux-bionic/crosvm
Change-Id: I0885e49379f6ee216c6e82adbb1230655171a7a5
A host target is considered as being cross-compiled when the target
can't run natively on the build machine. For example, linux_glibc/x86_64
is a non-cross target on a standard x86/Linux machine, but is a cross
host on Mac. Previously, whether cross or not was a static attribute of
an OsType. For example, Windows was always considered as cross host,
while linux_bionic was not. This becomes a problem when we support more
host targets like linux_bionic/arm64 which should be cross-host on
standard x86/Linux machines.
This change removes HostCross from the OsClass type and instead adds a
property HostCross to the Target type. When a target is being added, it
is initialized to true when the target can't run natively on the current
build machine.
Bug: 168086242
Test: m
Change-Id: Ic37c8db918873ddf324c86b12b5412952b0f2be2
The test case I removed is invalid. The codename has had its int
assigned, but the config claims it is not final.
If this ever does need to be supported it's just a matter of making
sure the Q -> 29 mapping (or whatever) in the finalized codenames map
in android/api_levels.go.
Test: treehugger
Bug: http://b/154667674
Change-Id: I4f42ec2fd4a37750519ee3937938a1c65b6bb1e8
Keeping the int constant around for now as FutureApiLevelInt because
it's still useful in places that haven't adopted ApiLevel yet for
testing if their non-ApiLevel API level is current or not.
Test: treehugger
Bug: http://b/154667674
Change-Id: I47a7012703f41fdeb56f91edf9c83afa93042deb
Can happen if OUT_DIR is set on an absolute path.
Test: env RBE_remote_disabled=true OUT_DIR=/home_2/tmp/out m
Bug: 169048540
Fixes: 169048540
Change-Id: Ia6b91b13142d84e54d0dee8b4e04a91ad2cecc36
Handling of API levels within Soong is currently fairly difficult
since it isn't always clear based on context what kind of API level a
given string represents, how much canonicalizing and error checking
the code receiving the string are expected to do, or how those errors
should be treated.
The API level struct does not export its raw data, so as to keep its
"constructor" private to the android package, and to prevent misuse of
the `number` field, which is only an implementation detail for preview
API levels. API levels can be parsed with either
`android.ApiLevelFromUser`, which returns any errors to the caller, or
`android.ApiLevelOrPanic`, which is used in the case where the input
is trusted and any errors in parsing should panic. Even within the
`android` package, these APIs should be preferred over direct
construction.
For cases where there are context specific parsing requirements, such
as handling the "minimum" alias in the cc module,
`nativeApiLevelFromUser` and `nativeApiLevelOrPanic` should be used
instead.
Test: treehugger
Bug: http://b/154667674
Change-Id: Id52921fda32cb437fb1775ac2183299dedc0cf20
Move PrimaryModule, FinalModule, and VisitAllModuleVariants to
BaseModuleContext so they can be used by mutators.
Test: m checkbuild
Change-Id: I4280333f19283edc8f65fc4af6f49b654af31782
Copy documentation from Blueprint for interface methods that are
wrapped into Soong interfaces.
Test: m checkbuild
Change-Id: Ib58c9201b8e7dfbf900a252e709f0a3cfed73286
Propagate new return values of the BottomUpMutatorContext.Add*
methods from https://github.com/google/blueprint/pull/316 to
Soong.
Test: m checkbuild
Change-Id: I5378db46adf2f31cc4222b4619cc39db53791610
And remove dual meaning from arm_on_x86 target. It was set for x86
targets with arm support and for arm targets with x86 support - and that
2nd one was abused for native_bridge version of RenderScript.
It's much safer and cleaner to rely on __ANDROID_NATIVE_BRIDGE__ there
while leaving arm_on_x86 to mean "x86 binary built in the image with arm
support".
It's only used by bcc compiler which is in canadian-cross situation here:
it's pure x86-64 binary, but it needs to know about ALL supported
architectures on device because it needs to include appropriate codegen
module.
Bug: http://b/153609531
Test: cts-tradefed run cts --abi armeabi-v7a -m CtsRsCppTestCases
Test: cts-tradefed run cts --abi arm64-v8a -m CtsRsCppTestCases
Change-Id: I869212e0b82eeaa30361a4e183d5c862ab40ef12
Previously, disabled modules were able to emit make vars and this could
cause duplicated make var definitions when a prebuilt_build_tools is
enabled for both linux_glibc and linux_bionic.
Bug: 159685774
Test: m
Change-Id: I8570ae5aabb8199e7c3901921886bcfa849037fe
AddFarVariationDependencies was broken, which allowed modules to add
dependencies on bootstrap Go modules using os and arch variations
even though boostrap Go modules did not have any variations. Mutate
bootstrap Go modules in the os and arch mutators so that they have the
expected os and arch variations.
This relands Ida7bc75a51ab1d2d38a6be11f574399097380cc9 with fixes
to also mutate *bootstrap.goPackage modules so that they can find a
matching *bootstrap.goBinary module when creating their pluginFor
dependency, and to create a variation for linux_bionic when it is
enabled so that linux_bionic builds can package Go binaries.
Also requries https://github.com/google/blueprint/pull/320 to
support multiple variants of bootstrap Go modules.
Test: all soong tests
Change-Id: Ibc627aa262c298a076d171a4c21b0b9c2a7b0ada
LinuxBionic now supports arm64 architecture in addition to the existing
x86_64 arch. This is to make it possible to build host tools like adb,
fastboot, crosvm, etc. for Linux/ARM on regular Linux/x86 machines.
The arm64 target can be selected in various ways in Android.bp files:
- target.host (because this is still considered as a host target)
- target.linux (provided that the module is also enabled for host)
- target.linux_bionic (use the OS name directly)
- target.linux_bionic_arm64 (OS name + arch combo)
- target.linux_arm64 (provided that the module is also for host)
- target.not_windows
- arch.arm64
The toolchain for the new target is almost the same as the toolchain
config for Android/ARM64. One notable difference is that the clang
triple is aarch64-linux instead of aarch64-linux-android, so that
__ANDROID__ is not defined for the new OS type.
Bug: 134795810
Test: HOST_CROSS_OS=linux_bionic HOST_CROSS_ARCH=arm64 m nothing
Change-Id: If4300583edfb6d75bd4d984b38f73b6a406b4447
This change fixes some places where host arch is assumed to be x86 to
form a path element like linux-x86. In preparation for non-x86 host
targets, the host arch part is derived from the context.
In addition, InstallForceOS() is refactored so that it can override not
only OsType, but also ArchType. Without this, the paths for the
robolectic modules will be changed from linux-x86 to linux-common, which
breaks several other places where the old paths are expected.
Bug: 134795810
Test: m
Change-Id: Ib38c715948ae546e55021ece82bac1d82e9e5da0
The commonProperties and defaultsVisibilityProperties fields are only
used as containers for default values for properties and only within
the InitDefaultsModule() and so can be removed and replaced with an
empty structure instead.
Test: m checkbuild and TH
Change-Id: Ic5038d6b84b33afdf4859ced08e59871b9cde4a2
Common properties are automatically inherited from a parent module
(i.e. one that calls CreateModule()) to the child module that it
creates. This makes no sense for dist/dists properties so this change
extracts them into their own structure separate to the
commonProperties.
Test: m checkbuild and TreeHugger
Bug: 160448975
Change-Id: Icceb20455e13394dd3b3bce464fb9bb34882d6c3
Similarly to Output, we return the list of Rules that have been
generated for TestingModule. This helps debugging failing tests.
Test: m nothing
Change-Id: I3542f4e4632f94fb84208c2e48e629271a373fd4
If BOARD_CURRENT_API_LEVEL_FOR_VENDOR_MODULES has a numeric value,
it replaces "current" or "system_current" with the version which
the flag indicates.
Bug: 163009188
Test: BOARD_CURRENT_API_LEVEL_FOR_VENDOR_MODULES=29 m, and then check if every vendor
java module's sdk_version is 29 if its sdk_version was current.
Change-Id: I17b49b8e02caf2d1bc57b91648d4420f3ad9fcb9
In order to rely on `-march`/`-mcpu` compiler flags for feature
detection, we introduce a new arch variant based on armv8.2 with the
addition of dot product features.
Test: test-art-target on Pixel 4.
Change-Id: I4d97db6129e2cd718a2b21008d36ec767739f925
AddFarVariationDependencies was broken, which allowed modules to add
dependencies on GoBinaryTool modules using os and arch variations
even though GoBinaryTool modules did not have any variations. Mutate
GoBinaryTool modules in the os and arch mutators so that they have the
expected os and arch variations.
Test: all soong tests
Change-Id: Ida7bc75a51ab1d2d38a6be11f574399097380cc9
Add a new helper SortedIntKeys similar to SortedStringKeys.
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I08a43ec2cae7d1a82531295aca1a0658e3a0dd6f
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.
Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
Use the -printusage flag in R8 to output a list of the unused
methods. Some of the files can be large (2MB for DocumentsUI,
87MB for all of AOSP), so immediately zip them and remove the
originals. The zipped files will be merged and disted.
Bug: 151857441
Test: m TARGET_BUILD_APPS=DocumentsUI dist
Change-Id: I780e84e80eba7fe4d4fa15fec0f461890afd900b
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories. This causes unnecessary
build time and disk space for duplicated work.
Deduplicate APEX variants that would build identically. Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.
This has one significant change in behavior. Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs. After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.
In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs. However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.
This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.
This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.
Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories. This causes unnecessary
build time and disk space for duplicated work.
Deduplicate APEX variants that would build identically. Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.
This has one significant change in behavior. Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs. After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.
In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs. However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.
This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.
Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I0529837476a253c32b3dfb98dcccf107427c742c
In preparation for reusing the same variation for multiple apexes,
rename ApexName to ApexVariationName.
Bug: 164216768
Test: all soong tests
Change-Id: I88f2c5b192ffa27acd38e01952d0cefd413222a0
Only print the list of variants of the matching module when
ctx.ModuleForTesting finds the module but not the variant.
Test: all soong tests
Change-Id: I51ffdde4645db39ec1d37ec018e0dea11d74280e
The modules sources have been moved, update the sdk variant whitelist.
Bug: 156286599
Test: m
Exempt-From-Owner-Approval: cherry-pick from internal
Change-Id: Idbd02b17b1c741f1dc20a3742ff57e8469a846c0
Merged-In: Idbd02b17b1c741f1dc20a3742ff57e8469a846c0
This hasn't worked for a couple years, and continues to bitrot. Just
remove it.
Adds a bpfix rule so that we can eventually remove the
product_variables.pdk definition, which is now always a no-op.
Test: treehugger
Change-Id: I830b54d419b59f6db1d4617b45e61a78234f57a7
Merged-In: I830b54d419b59f6db1d4617b45e61a78234f57a7
Instead of UnbundledBuild, use AlwaysUsePrebuiltSdks
to determine if java modules needs to be built against prebuilt sdks.
And rename UnbundledBuildUsePrebuiltSdks to AlwaysUsePrebuiltSdks to
express its behavior more correctly.(It can be orthgonal to "Unbundled")
Bug: 160390776
Test: TARGET_BUILD_UNBUNDLED_IMAGE=true m vendorimage
Change-Id: I0be7265c1959d8774c295372cd7a9250169f6df9
Includes fixes for out dirs at absolute paths, and symlinks to absolute
paths. Also squashes support for exporting make vars, which previously
happened in a later patch (which got moved first, since it's mostly
infrastructure changes).
Original message:
cc_prebuilt_binary doesn't work well for host tools because they'll
often be using their own versions of shared libraries that may not be
compatible with what we build.
So add a module type that allows genrules to use one of these prebuilts
as a tool. Like other prebuilts, we'll use the source module if we have
it, or the prebuilt otherwise.
It supports adding extra dependencies for shared libraries or other data
files that are necessary to run the tool. Any genrules using the tool
will be rerun if any of the dependencies change.
Bug: 128690776
Test: treehugger
Test: build one-true-awk using genrule with bison
Test: with OUT_DIR=$PWD/out
Test: with OUT_DIR pointing to an absolute symlink
Change-Id: I662296cb55503b6a10ec5e5465b138e9edeb71f3
This allows setting per-module make variables earlier in the build,
particularly for prebuilt_build_tool users like LEX/M4/BISON (which will
come in a later CL). I moved filegroup over because it's a simpler
common interface, but it doesn't strictly need it.
Test: Inspect out/soong/make_vars-*.mk, out/soong/late-*.mk
Test: treehugger
Change-Id: I38631129d82b0aad2438e83610bb6a97568bf932
This is a list of KMI versions for this board. Only gki_apex modules
with matching kmi_version field is enabled.
Test: build GKI apexes
Bug: 162888350
Change-Id: Ieb2116b79e3985ba15155a3cb8a2c340b46473c7
unavailable-to-platform case.
This removes the special case added in https://r.android.com/1274763
from SkipInstall(), so that it doesn't cause conflicting AndroidMk
entries when a cc_prebuilt_library_static module has prefer:true and
the corresponding source module exists.
Test: `m` in a tree with a snapshot created from art-module-sdk where
the libartimagevalues module has prefer:true
Bug: 151303681
Change-Id: I651ae325753b707296892adb4cae80daaddb6af2
The Bazel overlay is a directory at out/soong/bazel_overlay that
replicates the layout of the AOSP Soong module tree, but as a Bazel
workspace. Each Soong module variant is represented as a BUILD target
created with the `soong_module` rule.
To create this overlay, run `m bazel_overlay`.
A `soong_module` target can depend on other `soong_module` targets.
These dependencies replicate each module's `directDeps` in the Blueprint
graph, just before `PrepareBuildActions`.
This enables users to use bazel query as a way to introspect the Soong
module graph. For example,
- Direct reverse dependencies of //bionic/libc:generated_android_ids in
//bionic/libc/...:
$ bazel query 'rdeps(//bionic/libc/...,
//bionic/libc:generated_android_ids, 1)'
//bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static
//bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static
//bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime
//bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static
//bionic/libc:generated_android_ids
- Why does com.android.runtime depend on lzma?
$ bazel query
'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image,
//external/lzma/...)'
//bionic/apex:com.android.runtime--android_common_com.android.runtime_image
//bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime
//system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime
//external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime
- What does the dep graph of //bionic/libc:crtbegin_so look like?
$ bazel query
'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)'
--output=graph > graph.in && dot -Tpng < graph.in > graph.png
https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8
Test: croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel
query //... && bazel query 'rdeps(//bionic/libc/...,
//bionic/libc:generated_android_ids, 1)'
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
With the addition of apexes and /system_ext some of the bootclasspath
and system server jars have moved from /system to the new locations.
This has been implemented by using lists of colon-separated strings
called "apex-jar pairs" (although "apex" was misleading as it could
refer to "platform" or "system_ext", not necessarily a real apex).
Using the colon-separated string representation is inconvenient, as it
requires splitting and reassembling the list components many times,
which harms performance and makes error handling difficult. Therefore
this patch refactors the colon-separated lists into a struct that
hides the implementation details.
Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: Id248ce639a267076294f4d4d73971da2f2f77208
Necessary to avoid problems with implicitly enabled host OS'es, e.g.
linux_glibc getting enabled when we only supply a linux_bionic linker
in runtime-module-host-exports. That will then cause a non-functional
prebuilt to take precedence over source if the prebuilt is preferred.
We don't do this for device since we so far only support a single
device OS (android).
This introduces the notion that SDK member types can be host OS
dependent or not. That way java members with host prebuilts don't get
restricted to a specific host OS.
Test: m nothing
Test: build/soong/scripts/build-aml-prebuilts.sh runtime-module-host-exports
Check that the generated Android.bp correctly disables the bionic
linker prebuilt for linux_glibc.
Test: art/build/apex/runtests.sh
on master-art with an updated runtime SDK snapshot
Test: art/tools/buildbot-build.sh {--host,--target}
on master-art with an updated runtime SDK snapshot
Bug: 160349757
Change-Id: Idad7ef138cdbcbd209d390bf6c10ca8365d4619f
When a module is disabled by calling Disable(), it could be
unexpectedly enabled by the archMutator. For example, a module has
```
enabled : false,
arch : {
arm64 : {
enabled : true,
},
}
```
When this modules is disabled by calling Disable() before the
archMutator, it became enabled again ignoring Disable() command.
By defining the `ForcedDisabled` property, we can disable a module
regardless of `Enabled` property.
Bug: 161565086
Test: m
Change-Id: I443720715edbac25cb6d7c3d9eb1bc9806719161
com.android.gki.* matches any APEX with the prefix com.android.gki.,
like com.android.gki.bar.
Test: use it
Bug: 162267963
Change-Id: Ie46fcb08b031611d26b2b6cde157253f51ba9bfc
dependencyTag uses a set of predefined tags to identify different types
of dependencies. There are already multiple bits of metadata stored
in the dependency tag (Library, Shared, ReexportFlags), and supporting
them all requires a combinatorial explosion of predefined tags and
causes issues when using equality comparisons if a new bit of metadata
is added.
Add a new libraryDependencyTag type that will contain the metadata
bits, and replace the quality comparisons with checks on the metadata
bits.
There are 5 TODOs where modifying the checks identified problems with
the existing checks. These were left in place to produce identical
build output and will be fixed separately.
Bug: 162437057
Test: no change to build.ninja or {Android,make_vars,late}-${TARGET_PRODUCT}.mk
Change-Id: I72d4207dcf381c07c92e00e5a03968ebb5ed8d30
When WITHOUT_CHECK_API=true, the droidstubs apiFile can be nil. Fix this
by defensively checking for nil paths when making the TaggedDistFiles,
and also in android.MakeDefaultDistFiles.
The error message reported in r.android.com/1335521 was:
"internal error: panic in GenerateBuildActions for singleton androidmk
Dist file should not be nil for the default tag in
android.net.ipsec.ike.stubs.source in translateAndroidMkModule for
module android.net.ipsec.ike.stubs.source variant android_common"
Test: WITHOUT_CHECK_API=true m droid dist
Bug: 152834186
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I1b1f7c0b7a0e1c0ed5e15957d0162c47fd3ec197
