Commit graph

27779 commits

Author SHA1 Message Date
Jiyong Park
dc48afd3c5 zipsync handles symlink
This change fixes a bug that zipsync didn't handle symlink correctly;
symlink was extracted as a regular file whose content is the target
path. Fixing the problem by correctly creating the symlink using
os.Symlink.

Bug: N/A
Test: manual

Change-Id: Ib6685c14e1950d1057d89672883cdd9e4879069a
2020-11-19 08:58:02 +09:00
Colin Cross
1357c11948 Merge changes Ic2daab29,I884ddd09
* changes:
  Skip creating variants for disabled OSes
  Use bitfield for HostOrDeviceSupported
2020-11-18 23:36:24 +00:00
Janis Danisevskis
6439a8dae2 Fix Broken build 6978250 on aosp-master
Bug: 173570265
Change-Id: I3425dedf4ed8435420cca9b5917033262ba8b3cc
2020-11-18 14:26:41 -08:00
Colin Cross
151b9ff0cf Rewrite sbox to use a textproto manifest
In preparation for more complicated sandboxing that copies tools
and/or inputs into the sandbox directory, make sbox use a textproto
input that describes the commands to be run and the files to copy
in or out of the sandbox.

Bug: 124313442
Test: m checkbuild
Test: rule_builder_test.go
Test: genrule_test.go
Change-Id: I3b918a6643cea77199fd39577ef71e34cdeacdb1
2020-11-18 13:02:22 -08:00
Treehugger Robot
b08a091502 Merge changes Iba57c949,Ief43ff51,Ib1809a4d,I2ab64f36
* changes:
  Store ndkKnownLibs in the config
  Register the kythe singleton on the Context instead of globally
  Store ninja file deps from PackageVarContext in the config
  Store SingletonMakeVarsProviders in the config
2020-11-18 20:00:31 +00:00
Treehugger Robot
725af86fe7 Merge "Parse else ifxxx statement" 2020-11-18 19:09:09 +00:00
Colin Cross
d4a63eaeb7 Merge "Move genrule on top of RuleBuilder" 2020-11-18 18:44:19 +00:00
Christopher Parsons
2bb2017a30 Merge "Use packagepath and local_repository for mixed builds" 2020-11-18 16:06:35 +00:00
Ivan Lozano
1010f9bd56 Merge "rust: Add header library support to rust_bindgen." 2020-11-18 14:06:09 +00:00
Treehugger Robot
ec40eb0abb Merge "add PackagingSpec" 2020-11-18 12:13:22 +00:00
Will Osborn
f0aa3b9527 Merge "Revert "Always turn on compatible property"" 2020-11-18 11:13:59 +00:00
Inseob Kim
7d13e5b964 Revert "Always turn on compatible property"
This reverts commit f6d606ebea.

Reason for revert: broken targets on the internal branch

Change-Id: I948280e2072789a2ee20e7fe4844c667f58b5182
2020-11-18 11:04:27 +00:00
Jingwen Chen
db892410f7 Merge "soong_ui finder.go: improve comments/documentation." 2020-11-18 07:53:56 +00:00
Treehugger Robot
60447f1ce5 Merge "Always turn on compatible property" 2020-11-18 07:23:39 +00:00
Sasha Smundak
9c35d8bfde Parse else ifxxx statement
More recent versions of make provide "else ifdef/ifndef/ifeq/ifneq" directive:
ifdef FOO
...
else ifdef BAR
...
endif
Fix the parser to handle it. It returns the same Directive as the respective ifxxx
counterpart, only that the Name is set to "elifdef/elifndef/elifeq/elifneq".

Test: treehugger
Change-Id: I74c6a2c7224bce1dd3465012fa84880fae21349b
2020-11-17 23:18:14 -08:00
Alexander Smundak
9240f955cb Merge "Document usage, perform minor cleanups." 2020-11-18 07:08:51 +00:00
Jingwen Chen
27a8719b0f Merge "Integrate bazelenv.sh environment variables into soong_ui environment." 2020-11-18 04:53:22 +00:00
Dan Shi
ebb51503a4 Merge "Add unit_test test option in test configs" 2020-11-18 04:40:37 +00:00
Jiyong Park
073ea55fad add PackagingSpec
Currently, installation of a module is defined as an action of copying
the built artifact of the module to an install path like out/soong/host
(for host modules) and out/target/product/<device>/<partition> (for
device modules). After the modules are installed, the installed files
are further processed to create packages like system.img, vendor.img,
cvd-host-package.tar.gz, etc.

This notion of installation seems to have originated from the old time
when system.img is the primary product of the entire build process
(modulo a few more like root.img). Packaging the installed files as the
filesystem image was considered as a post-build step then.

However, this model doesn't seem to fit well to the current and future
environment where we have a lot more filesystem images (system, vendor,
system_ext, product, ...). The filesystem images themselves are even
grouped together to form a higher-level filesystem image like super.img.
Furthermore, things like cvd-host-package.tar.gz requires us to be able
to group some of the host tools in a format that isn't filesystem image.
Lastly, we are expected to have more filesystem images that are subsets
of system.img (and their friends) for the Android-like mini OS that will
be running on on-device virtual machines. These all imply that the
packaging (which we call installation today) is not a global post-build
step, but a part of the build rules for creating the package-like
modules.

A model better fits to the new sitatuation might be this; a module
specifies its built artifact and the path where it should be placed. The
latter path is not rooted at out/. It's a relative path to the root
directory which will be determined by another module that implements the
packaging. For example, cc_library will have ./lib (or ./lib64), not
out/target/product/<device>/<partition>/lib as the path. Then packages
like system.img, cvd-host-package.tar.gz, etc. are explicitly modeled as
modules and they have deps to other modules. Then the modules are placed
at the relative path under the package root, and the entire root
directory finally is packaged as the output file (be it img, tar.gz, or
whatever).

PackagingSpec is the first step to implement the new model. It abstracts
a request to place a built artifact at a certain path in a package. It
has extra information about whether the path should be a symlink or not,
and whether the path is for an executable. It currently is created when
InstallFiles (and its friends) are called, and can be retrieved via
the new method PackagingSpecs().

In this CL, no one is using PackagingSpec. The installation is still
done by the existing rules created in InstallFiles, etc. and the
structs are not used for the filesystem images like system.img.

Bug: 159685774
Bug: 172414391
Test: m

Change-Id: Ie1dec72d1ac14382fc3b74e5c850472e9320d6a3
2020-11-18 11:30:19 +09:00
Jaewoong Jung
9f98d3f423 Improve soong_ui main.go comments.
And do a minor cleanup while we're there.

Test: TreeHugger
Bug: 173557448
Change-Id: Id1736f82f64c1a0a4b29749f73d3a3724fc501a7
2020-11-17 18:29:58 -08:00
Jingwen Chen
096a3bf08f soong_ui finder.go: improve comments/documentation.
Test: m
Bug: 173474588
Change-Id: I947a34b3d9a5b2cd36fba293b55860a20a11d40b
2020-11-17 19:48:31 -05:00
Jingwen Chen
6d41e995f3 Merge "soong_ui's bazel.go: document/comment with tiny refactoring." 2020-11-18 00:32:18 +00:00
Jingwen Chen
7c6089ad95 Integrate bazelenv.sh environment variables into soong_ui environment.
This removes the need to source bazelenv.sh for USE_BAZEL_ANALYSIS, and
unifies mixed builds to use the checked in tools/bazel and bazelrc.

It also unifies all bazel-related output to be in out/bazel.

Without aosp/1502095, this change still requires toplevel_output_directories to be an empty
list, otherwise there'll be this error:

ERROR: Directories specified with toplevel_output_directories should be
ignored and can not be used as sources.

Test: With aosp/1441774: rm -rf out/ && lunch aosp_cf_x86_auto && USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 results
Change-Id: I69b217ec88da531415792bb6e04b6a194ca4718d
2020-11-17 18:58:03 -05:00
Colin Cross
d976af0cb4 Skip creating variants for disabled OSes
The documentation java_genrule_host states that it creates a single
variant, which would make it work with the single variant fallback
in AddDependency used by the data property, but it actually has a host
and a host-cross windows variant.  Modify osMutator to take the
OS-specific enabled properties into account to skip creating variants
that will immediately be disabled so there is a single variant.

Test: m checkbuild
Change-Id: Ic2daab29f4fa3a3797d7a08348fbfcf1036ec5dc
2020-11-17 14:24:24 -08:00
Colin Cross
34037c66ae Use bitfield for HostOrDeviceSupported
The HostOrDeviceSupported enums have become unwieldy, replace them
with bitfields to simplify checking if a specific trait is set.

Test: Soong arch tests
Test: m checkbuild
Change-Id: I884ddd092d4c6f740def611b6a055007dc24bd6e
2020-11-17 14:24:11 -08:00
Steven Moreland
0029898a84 java link time error improve
Asked about error message including "against private API.Adjust
sdk_version", which is a bit hard to parse due to missing space. Also
tried to make error message less verbose, so that it is more clear what
to do, and fixed a grammar mistake.

Bug: N/A
Test: N/A
Change-Id: Ib9a30d86b5cb0e9b3b7d5576ecb9498a9b316042
2020-11-17 21:57:23 +00:00
Jose "Pepe" Galmes
f77bd8d60d Merge "Support for recovery snapshot." 2020-11-17 20:51:54 +00:00
Chris Parsons
8ccdb6358e Use packagepath and local_repository for mixed builds
This allows mixed builds to continue functioning even given the toplevel
WORKSPACE file containing toplevel_output_directories with out/.

Test: Manually verified on aosp_flame building libc.
Change-Id: I80fc4853421317e2d2c7f03d92d58286df1342ce
2020-11-17 15:42:36 -05:00
Colin Cross
3d68051218 Move genrule on top of RuleBuilder
In preparation for more complicated sandboxing support in sbox, use
a single implementation of the sbox sandboxing by moving genrule to
use RuleBuilder's sbox support instead of creating an sbox rule
directly.

Also move genrule's input list hash support into RuleBuilder.

Test: genrule_test.go
Test: rule_builder_test.go
Change-Id: I292184d02743c7e6887ebbcd232ba565db2ab0cc
2020-11-17 11:23:45 -08:00
Colin Cross
95f1ca07ce Store ndkKnownLibs in the config
Storing ndkKnownLibs prevents multiple tests from running in parallel
as one may be writing to the list while another is reading from it.
Store it in the config so each test has its own copy.

Test: go test -race ./apex
Change-Id: Iba57c9494012c9e0ae9e5ffaa63b9b2bd2c77492
2020-11-17 10:50:19 -08:00
Colin Cross
5789858aa8 Register the kythe singleton on the Context instead of globally
Test: all soong tests
Change-Id: Ief43ff51e66db2d3c895b2a6952385e31b9d669b
2020-11-17 10:50:19 -08:00
Colin Cross
121292911a Store ninja file deps from PackageVarContext in the config
Store ninja file deps from ExistentPathForSource on a
PackageVarContext in the config instead of the PackageContext, as
the PackageContext may be shared between multiple tests running
in parallel.

Test: all soong tests
Change-Id: Ib1809a4dd4a82696e0fe48a87eac21a44684ecb5
2020-11-17 10:50:19 -08:00
Colin Cross
06fa588d1a Store SingletonMakeVarsProviders in the config
Store SingletonMakeVarsProviders in the config instead of a global
variable to avoid races between tests running in parallel.

Test: all soong tests
Change-Id: I2ab64f368b5ac673fd985399d4421ed018abc562
2020-11-17 10:47:24 -08:00
Ivan Lozano
9b44383788 rust: Add header library support to rust_bindgen.
Allow rust_bindgen modules to define dependencies that only provide
headers and may not necessarily need to be linked in.

Bug: 161141999
Test: Soong tests pass.
Test: Example module has appropriate include flags when compiling.
Change-Id: Ic9ce8b1204008ad8dcb18766c914e48bb292d485
2020-11-17 13:40:50 -05:00
Treehugger Robot
beab64ea81 Merge "Add keymint to vndk." 2020-11-17 16:13:50 +00:00
Ivan Lozano
3d0f191c35 Merge "rust: Fix incorrect expected gcno file for dylibs" 2020-11-17 15:18:03 +00:00
Ivan Lozano
d7ddf051d3 rust: Fix incorrect expected gcno file for dylibs
For Rust dylibs we generate a file suffix of .dylib.so. rustc seems to
split the output filename for the gcno file at the first '.', so instead
of libfoo.dylib.gcno (which our build system expects), it produces
libfoo.gcno.

This fixes the mismatch so the build system can find the gcno file that
gets produced for Rust dylibs.

Bug: 170983066
Test: CLANG_COVERAGE=true NATIVE_COVERAGE_PATHS=external/rust/crates \
      m libandroid_log_sys
Change-Id: Icba48bf10ab06c255bd17053d57feb889b1f9ee5
2020-11-17 08:08:21 -05:00
Paul Duffin
969ca6701c Merge "Switch boot jars package check to use dexdump xml output" 2020-11-17 12:16:32 +00:00
Ulyana Trafimovich
d8d8e726a5 Merge changes Iebfbf2ff,Ibd974268
* changes:
  Rename fields and methods to reflect class loader context changes.
  Do not add dependencies of shared SDK libraries to manifest_fixer.
2020-11-17 10:22:00 +00:00
Jingwen Chen
7a5391a979 soong_ui's bazel.go: document/comment with tiny refactoring.
This patch adds information behind the Bazel USE_BAZEL=1 execution
pipeline.

Test: m
Bug: 173474588
Change-Id: I54bdd26fb14c8a371de0922d193419112e8c3ece
2020-11-17 03:46:23 -05:00
Yo Chiang
803c40daa2 prebuilt_etc: Improve comments and slight refactoring
Comment changes:
* Add top level package description.
* Clarify `OutputFileProducer` usage for `prebuilt_etc.src` field.
* Clarify output file basename selection rule.

Refactor changes:
* Don't emit superfluous `LOCAL_UNINSTALLABLE_MODULE := false` lines.
* `for _, i := range list { entry.AddStrings(prop, i) }` =>
  `entry.AddStrings(prop, list...)`

Bug: 173379413
Test: TH presubmit
Change-Id: Iab90aea5ff2497525ebe1600d3ce5a480923760a
2020-11-17 15:46:07 +08:00
Colin Cross
e3d308b5a5 Merge "Revert "Annotate dependency tags for dependencies of installed files"" 2020-11-17 06:32:57 +00:00
Colin Cross
b5ae193b8f Revert "Annotate dependency tags for dependencies of installed files"
This reverts commit 62a0cfd054.

Reason for revert: b/173475545

Change-Id: I4e834200c8e68dfa1b8144dfd1fa95ca68554980
2020-11-17 06:32:06 +00:00
Inseob Kim
f6d606ebea Always turn on compatible property
It's mandatory for devices launching with Android P or later.

Bug: 170082975
Test: m
Change-Id: I22578c3b77d28abc8a09b88324ef3702d0ccafde
2020-11-17 06:29:23 +00:00
Sasha Smundak
8eedba6184 Document usage, perform minor cleanups.
Test: treehugger
Bug: 173466220
Change-Id: I5aa09c2941c0eceb8ae635b9e613ad932d38aebb
2020-11-16 19:00:27 -08:00
Treehugger Robot
0a829f2a0a Merge "Always set apex_available in SDK snapshots." 2020-11-17 01:30:46 +00:00
Joseph Murphy
bac4ee008b Merge "Adds a workaround to fix version inconsistency" 2020-11-17 01:23:39 +00:00
Jose Galmes
f729458e69 Support for recovery snapshot.
Bug: 171231437
Test: source build/envsetup.sh
Test: ALLOW_MISSING_DEPENDENCIES=true m -j nothing

Change-Id: I6b35fbb0b90ffffa941ea108cbd31a454b2dd403
2020-11-16 17:09:36 -08:00
Jaewoong Jung
f2200adccb Add and update comments in env and android_env.go
Bug: 173450116
Test: m nothing
Change-Id: I582a483d11c2ca23fd6e9597e49cdfb7e69152de
2020-11-16 16:01:27 -08:00
Jingwen Chen
556dafcdbf Merge "Set PATH and --action_env=PATH for the Bazel build in soong_ui." 2020-11-16 23:51:10 +00:00