A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3
We avoid both of these by storing the information directly in the
underlying module.
Also as a fringe benefit, removes some necessary boilerplate for
conversion.
For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714
Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
Because it's commonly useful and not cc/ specific
Also export GenerateBazelBuildActions and update uses
Test: Existing tests pass
Change-Id: Ibc6858bb1129afba181a7686dda432defe33b00d
Rtti was already handled by bazel macros, so this change simply
propagates the rtti bit to these macros.
Test: Run bp2build, build //external/libcxxabi:all
Change-Id: I63296db2db868202874c8bd0b1de6310f7bb85c7
- If no system_shared_libs is specified, bp2build writes no attribute
value. In this case, the bazel library macros determine the correct
default behavior.
- If any system_shared_libs is specified for any variant, then bp2build
writes the value verbatim. This includes if an empty list is specified,
as this should override defaulting behavior.
Note this defaulting behavior is incomplete and will be incorrect in
corner cases. For example, if, in an Android.bp, system_shared_libs is
specified for os.linux_bionic but not for os.android, then the bazel
default for os.android will be incorrect. However, there are no current
modules in AOSP which fit this case.
As a related fix, supports static struct for cc_library_static.
Also, removes some elements from the bp2build denylist.
Test: mixed_droid CI
Change-Id: Iee5feeaaf05e8e7209c7a90c913173832ad7bf91
system_shared_libs has been modified to have the same behavior as
the newly added default_shared_libs, remove default_shared_libs in
favor of system_shared_libs.
This reverts Ia2349d84c70e503916f90a5d2702e135248f73df and renames
the default_shared_libs property in cc_object (which never had
system_shared_libs) to system_shared_libs.
Bug: 193559105
Test: m checkbuild
Change-Id: I46672e3a096b6ea94ff4c10e1c31e8fd010a163c
This fixes a bug which was a misunderstanding of soong properties:
Soong's cflags pertain only to C and C++ language, whereas bazel's copts
pertain to all three languages. This change ensures that asflags are
added as specifically asflags, and the 'copts' for the static library
macro pertains only to C and C++ languages.
This requires a somewhat hacky workaround for asflags, however: Since
assembly sources also need includepath-related flags, this duplicates
these flags between copts and asflags. To reduce verbosity of
bp2build-generated targets, this also ensures that asflags are omitted
in cases where there are no assembly sources.
Test: Mixed build droid CI
Change-Id: Ic0babed1f90d6dc82e5788638681ce5b995043f8
This reverts commit cacb972dcd.
Relanding along with one more fix in prebuilts/ndk
Change-Id: I65cbc3c5d72bb47bd00e123c63911c9baa1e5f03
Test: m checkbuild
Building against a locally-built libc besides bionic requires setting
system_shared_libs to avoid circular dependencies, but modules all
over the tree assume that system_shared_libs only affects modules
building against bionic. Add a new default_shared_libs property
that applies to all modules, which will generally be set in
arch-specific clauses to only affect the desired set of modules.
Bug: 190084016
Test: m checkbuild
Change-Id: Ia2349d84c70e503916f90a5d2702e135248f73df
Move the crt objects spread around cc/library.go and cc/binary.go
into the Toolchain. This will simplify adding new toolchains that
have custom crt objects.
Test: m checkbuild
Test: go test ./cc/...
Change-Id: I7fdc1f53769799cb9c10e3e5816dabee0f918768
This makes bp2build generate these attrs into a Starlark dictionary,
passed into the cc_library macro directly. This makes the BUILD target
representation more similar to the Android.bp one, and also makes
it more legible.
Test: TH
Change-Id: I42b427cc4b22c6376d3d24e40b9af1692bb0c692
Add variant_prepend to Export_include_dirs, Export_system_include_dirs
and Generated_headers to match Local_include_dirs so that
architecture-specific headers are found before common headers.
Test: m checkbuild
Test: TestIncludeDirectoryOrdering
Change-Id: I2bf9d4373fd256e500c1f8d6c84cf593e8fdd170
Host bionic will use multiple files for CrtBegin, make CrtBegin and
CrtEnd Paths instead of OptionalPath.
Test: go test ./build/soong/cc/...
Change-Id: Ie2e954cd87808a903617696da443009f6173e312
The host bionic bootstrapping no longer needs an injected symbol.
Replace host_bionic_inject with host_bionic_verify that validates
the resulting binary, and add it as a validation dependency of the
binary.
Test: build and run host bionic binary
Change-Id: I3e303d2a164b6eef851bdc8075e6ee456c05b0a8
This CL adds support to bp2build for generating all strip-related
properties into a strip dictionary as a parameter in the cc_library
macro.
With the dictionary, it's easy to organize related attributes and
directly expand into a stripped_shared_library's attributes.
Test: //build/bazel/tests/bionic:compare_libc_stripping
Test: TH
Fixes: b/187928597
Change-Id: Ifea68d48fe295e71a43b12876cb168c475a62187
There was a reference to the static library provider, but only the
transitive static deps were used, so only store that part. Other members
were stored but unused anywhere.
Test: go test soong tests
Test: m nothing
Change-Id: I12a6b94806c052c3f0df3cab0a10f17042af1c38
This is to simplify the process of resolving label + exclude labels
across the various configuration axes we have and across the various
properties/modules that use this behavior.
Test: ci/bp2build.sh && ci/mixed_droid.sh
Change-Id: I8efae3e75ddb365384f5caaf5bb504a5206618d3
The local diffing behavior is currently flagged off so we can land
this in stages.
Test: pytest cc
Test: treehugger
Test: development/tools/update_ndk_abi.sh
Test: m ndk
Bug: http://b/156513478
Change-Id: Iccb314411bc74ea3ddfea8b85b0539709295f65a
This CL exports and refactors some cc vendor-snapshot related functions
so they can be reused by rust modules to support vendor snapshotting.
Bug: 184042776
Test: m nothing
Change-Id: I12706e62ce0ac3b2b4298085fafc1d77b8e0a0c4
This CL adds support for cc_library to correctly split c, as and cpp
srcs in shared/static nested props, as well as splitting
the *filegroup* deps in those props, where each filegroup is expanded
into its own c, cpp and as srcs filegroups. This ensures that the
correct sources go into cc_library_static's underlying cc_libraries for
c, cpp and as sources respectively.
See the bp2build conversion test for a better visualization.
Bug: 183064430
Test: TH
Change-Id: I29add5140672d042adff65527d8b65f4a5f0a05b
This fixes a test and rolls forward I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
Test: m nothing
Test: See I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
Change-Id: I0e450c28e70087e406e7b562d7e772785f177379
Revert submission 1714835-roboleaf-asm-c
Reason for revert: TestCcLibraryStaticProductVariableSelects fails everywhere
Reverted Changes:
I28cf7437e:Split asm and c flags and srcs in bp2build output
I2b47e6b55:Split libraries by language in cc_library_static
Change-Id: I85d39a462f0a5b3f5ff3d685906813fab9f01358
This allows removal of almost all current items from the mixed build
denylist, which were previously broken due to being unable to separately
control flags for compilations of different languages within the same
target.
Note that this does not appropriately implement asm/c srcs and flags for
either the shared variant or the static variant. This will require a
followup.
Test: bp2build.sh and mixed_libc.sh CI scripts
Test: Updated b2build tests
Change-Id: I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
Test: bp2build and mixed_libc CI
Test: Manually verified that libc_bionic_ndk compilation gets the
appropriate headers (and no extra headers) from downstream
Change-Id: I79eb6e8ec1d415bd50d12105da4cf97101f95474
VNDK snapshot depends on addExportedGeneratedHeaders to grab generated
headers. This change adds a missing addExportedGeneratedHeaders call to
make libc's headers captured correctly.
Bug: 181326838
Test: VNDK_SNAPSHOT_BUILD_ARTIFACTS=true m dist vndk && unzip -l
out/dist/*.zip
Change-Id: Iaa89b5255c836761a26c77bb27f35f768f49039e
Also handle whole_static_libs via a different attribute
(whole_archive_deps), separating these dependencies from regular static
deps.
Test: Build //bionic/libdl with bazel in conjunction with bzl
changes
Change-Id: I45019b6127a0d2731592ec35537ca15e77001cc2
Test: cd bp2build; go test
Test: bazel build //bionic/...
Test: ./build/bazel/scripts/run_presubmits.sh
Change-Id: I71e279470a0d69b243dd0a2b53ce31842fd36ee4
Nothing links against the versioned LLNDK stubs, only current and the
implementation are used. Remove the numbered LLNDK variants.
Also remove llndkStubDepTag, it is never used to add a dependency.
Test: TestLlndkLibrary
Change-Id: Idde62007d77b8e6ceee31144c05756faf9b41f23
This CL adds the ability to filter cc_library modules to only generate
their static variant of their shared variant isn't ready.
For example, the libstdc++ library is buildable as a static library,
which libc depends on. However, the shared variant of libstdc++ depends
on libc, which has to come later.
This CL introduces that abstraction to break up bp2build conversion into
more atomic steps to help with conversion.
Test: TH (bazel build //bionic/... incl. libstdc++'s static variant)
Bug: 186489250
Bug: 186822597
Change-Id: I3e2fe748e4e3d3b656760da4807f342d67c8f45f
Doesn't work when depends on arch/target/etc., but good enough for
libdl_android.
Bug: 186650430
Test: Presubmits.
Change-Id: Ib0facb41a89454717c74663e5e078aedd33d1b9c
This CL adds support for the static and shared props, forwarding
whole_static_libs and srcs to the underlying shared library's
static_deps and static library's srcs respectively.
Test: TH
Fixes: 186609667
Change-Id: I84f838361c7758b256b2a163af894c502de03109
Remove the global list of vendor public library modules used to rewrite
dependencies from the vendor module to the stubs for system modules,
and replace it with building the stubs directly in the system variant
of the vendor module.
Bug: 178231622
Test: vendor_public_library_test.go
Change-Id: I826e69ffd507d7e85fa3d4d85b5157428c642143
This allows us to remove libasync_safe from the mixed builds denylist.
Test: mixed_libc.sh CI script
Change-Id: Ibafd231284864078bf30340f919d39e5098843ce
Not needed anymore for bp2build-incremental since https://android-review.googlesource.com/q/topic:no-include-check.
Not needed for mixed builds either, since cc compile actions aren't sandboxed.
Fixes: 186488830
Test: treehugger and go tests
Change-Id: Ib5d4908dcce6bf910a653c457bb251d726e717d4
Remove llndk_library in favor of cc_library with llndk.symbol_file.
Bug: 170784825
Test: m checkbuild
Test: TestLlndkLibrary
Change-Id: I43580976589a7a2a176d7442be53fa043c0c8324
Set llndk.llndk_headers: true to allow a cc_library_headers
module to be used by a cc_library module with llndk.symbol_file set.
Bug: 170784825
Test: TestLlndkHeaders
Change-Id: Ib65a4b70717dc9a54ae30f2991485bb1bb9b8409
Test: go soong tests
Test: bp2build generate & sync; mixed build libc; mixed build su (su is
an Android.mk target that relies on converted a cc_library_headers)
Bug: 181552740
Change-Id: I9efd587970551fd41f642a208f0aa0a80e8694e0
This CL refactors the cc* bp2build converters to use the common
attribute extractors in cc/bp2build.go.
This also adds include_build_directory to be handled by the compiler
attr extractor to generate recursive headers as inputs.
This also turns include_dirs and local_include_dirs into the
execroot-relative -I flags.
e.g. if a module in bionic/libc has "private" in local_include_dirs,
the "-Ibionic/libc/private" copt is generated for it.
Fixes: 185139955
Test: TH
Test: Forrest for mixed_clean-droid
Change-Id: Ib67056482227e62068fbbea0455035bdf5d56319
Allow cc_library modules to configure themselves as an LLNDK library
using llndk.symbol_file instead of llndk_stubs pointing to an
llndk_library module.
Bug: 170784825
Test: TestEmbeddedLlndkLibrary
Change-Id: Id884cdada7094bbea6809a98e687e039778c5dc6
Remove duplicates from include dir paths before setting
FlagExporterInfoProvider. Duplicates often occur, for example due to
inheriting export_include_dirs from the static variant of a library to
the shared variant of the library that also gets the same
export_include_dirs.
Test: m checkbuild
Change-Id: I71f9e841547db64cbabcd95288ee93ef21fab4fb
There is little overlap at this point, but we expect these to converge
more over time, to handle exported includes, libs, etc., this will allow
those changes to be handled in one place and allow more consolidation of
code.
Test: bp2build generate & sync; mixed build libc
Change-Id: I51685dad9f4fc11a634965a3c9e84f4a0e279ecb
This changes the return value into a compiler/linker attr struct to
standardize callsites and make it easier to retrieve the parsed attrs.
Test: TH
Change-Id: I1e3956e7cb5d924ce8472ece940faea459beef37
This CL contains the converter for libdl_android, a cc_library that
expands into a cc_shared_library and a regular cc_library.
Test: TH
Test: bp2build; bazel test //build/bazel/tests/...
Change-Id: If70641a538211b0d6b2aac0e4d0d06912318304d
Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/... works for more cc_library_static targets (in a parent CL)
Change-Id: Ib487216a4bcbc52958ff948722dae347b0d8b606
Required fixes:
- Set MakeLinkType for libraries, even in mixed build mode.
- Set snapshot header information to empty list, which passes
validation logic for depending modules
Fixing these libraries also requires a Starlark change to
cc_library_static.bzl, which will be submitted separately.
Additionally, this adds better error messaging in the event that
output files are missing from a mixed-build library.
Test: USE_BAZEL_ANALYSIS=1 m libc
Test: USE_BAZEL_ANALYSIS=1 m runtime-module-sdk
Change-Id: Iad2c4d46359986fb0a43263292a15ed45fabbac7
This CL is pretty large, so I recommend starting with reading the newly
added tests for the expected behavior.
This change works in conjunction with the linked CLs in the Gerrit topic.
Those CLs add support for new platform() definitions for OS targets
specified in Soong's arch.go, which are configurable through
Android.bp's `target {}` property. It works similary to previous CLs
adding support for the `arch {}` property.
These configurable props are keyed by the OS: android, linux_bionic,
windows, and so on. They map to `select` statements in label list
attributes, which this CL enables for cc_library_headers' header_libs
and export_header_lib_headers props.
This enables //bionic/libc:libc_headers to be generated correctly, from:
cc_library_headers {
name: "libc_headers",
target: {
android: {
header_libs: ["libc_headers_arch"],
export_header_lib_headers: ["libc_headers_arch"],
},
linux_bionic: {
header_libs: ["libc_headers_arch"],
export_header_lib_headers: ["libc_headers_arch"],
},
},
// omitted props
}
to:
cc_library_headers(
name = "libc_headers",
deps = [] + select({
"//build/bazel/platforms/os:android": [
":libc_headers_arch",
],
"//build/bazel/platforms/os:linux_bionic": [
":libc_headers_arch",
],
"//conditions:default": [],
}),
)
Test: TH
Test: Verify generated //bionic/libc:libc_headers
Fixes: 183597786
Change-Id: I01016cc2cc9a71449f02300d747f01decebf3f6e
So far, when a library `libfoo` has `stubs.versions: ["10", "11"]`, then
`shared_libs: ["libfoo"]` is linked to the version 11 of the stub.
This requires the author of `libfoo` to manually update the property
whenever a new version is introduced. Otherwise, clients are not able
to use the newly added APIs because the latest stub is for an old
version.
This change eliminates the need for manual updating. "current" version
is always implicitly added to `stubs.versions`. It is added even when
nothing is set on the property, if `stubs.symbol_file` is set. i.e.
```
cc_library {
name: "libfoo",
stubs: {
symbol_file: "libfoo.map.txt",
// no versions: [...] needed
},
}
cc_library {
name: "a_client",
shared_libs: ["libfoo"],
apex_available: ["myapex"],
min_sdk_version: "29",
}
apex {
name: "myapex",
native_shared_libraries: ["a_client"],
min_sdk_version: "29",
}
```
`a_client` links to the "current" stub of `libfoo` that has all symbols
shown in the map file.
Note that, above doesn't mean that the client has unlimited access to
APIs that are introduced even after the min_sdk_version of the client
(29 in this example). The use of such APIs still has to be guarded with
`__builtin_available` check.
Bug: N/A
Test: m
Change-Id: I70bb1600c18e74d36c6b24c3569d2149f02aaf96
Also refactor bp2build-related code for cc_library_headers.
(Retry of previous CL after the presubmit failed to detect a merge conflict)
Test: Added unit test.
Test: bp2build-sync.py write; bazel build //bionic/... still works (but bp2build is disabled for most cc_library_static targets for other reasons)
Change-Id: I2f4405c2fea305623bbc6daaaf62808b0c074216
Revert submission 1594391-bp2build-cc_library_static
Reason for revert: Broke the build on aosp-master
Reverted Changes:
Ib16ccf31a:Add cc_library_static macro to help with bp2build ...
I37c856be2:Add bp2build support for cc_library_static.
Change-Id: Ie94d5bc6da81758cd4e0461c08a810a29643c971
Also refactor bp2build-related code for cc_library_headers.
Test: Added unit test.
Test: bp2build-sync.py write; bazel build //bionic/... still works (but bp2build is disabled for most cc_library_static targets for other reasons)
Change-Id: I37c856be20a47b154909338a22a7dba1ab55693f
Background: `min_sdk_version` of a crt object is the oldest SDK version
that the crt object supports. When it's set to for example 16, Soong
creates a number of versioned variants of the crt object starting from
16 to the latest version. The variant for version X is provided to NDK
clients having `min_sdk_version` set to X.
Problem: all versioned variants of a crt object were built with `-target
<arch>-linux-android16`. Therefore they all have been with `#define __ANDROID_API__
16`. This is because the mutated variants still have the same
min_sdk_version property and the clang triple follows min_sdk_version,
not sdk_version. This is too conservative and against our intention to
provide the latest crt object that matches with the min_sdk_version of
the client.
In the other hand, the platform(non-sdk) variant of the crt object
doesn't have such a problem. min_sdk_version is completely ignored.
However, this is a bug and will be fixed by aosp/1640364. As a side
effect of the fixing, the platform variant will begin to built with a
very old __ANDROID_API__ which unnecessarily turns off the new platform
features like the TLS segment over-alignment.
This change fixes the problems:
* For the versioned variants of crt objects, we set both
`min_sdk_version` and `sdk_versio` to the version that the variant is
created for.
* For the platform variant of crt objects, `min_sdk_version` is force
reset to "current".
Bug: 183191008
Test: m
Change-Id: I8c9d0fcea816de8cd1532dac4a47eee4f726c037
Previously, a cc library that included .aidl, .proto and/or .sysprop
files and exported headers generated from at least one of those types
would actually export generated headers from all of them.
While headers generated from .sysprop files are always exported those
generated from .aidl or .proto should only be exported when explicitly
requested.
This change treats them separately as expected. It has the potential
to break the build as it could reduce the set of headers exported and
so a dependent module that needed those would break. The fix in that
case is to simply add one (or both) of the following to the module
that previously exported those headers:
aidl: {
export_aidl_headers: true,
}
proto: {
export_proto_headers: true,
}
Bug: 180712399
Test: m droid
Change-Id: I488182e27dd423d261443612f98d5c112dd3ef8f
This CL introduces a basic bp2build converter for cc_object modules.
cc_objects maps cleanly to cc_library targets, but with -fnoaddrsig.
This CL also demonstrates generating include deps within a macro to
allow the cc_object compilation to depend on a relative-include header
within an include dir.
e.g. if "foo.cc" includes "android/log.h" and the latter is located at
"include/android/log.h", the autogenerated header deps would export
"android/log.h" correctly to the foo.cc upstream target.
Test: GENERATE_BAZEL_FILES=true m nothing && bp2build-sync write && bazel build //bionic/libc:crtbegin_so1
Change-Id: Ifd9e097051ec184ab0a1929d07918f0ff4f24d98
When a cc module is built against a stub, compiler passes version macro
of the stub lib. Version macro should be numeric, so codenames or
"current" should be mapped to numbers just like how ndkstubgen maps to.
* "current" -> future (10000)
* codenames -> look up api_level.json
* otherwise -> cast to int
Bug: 179329813
Test: m / soong test / manually check the output build.ninja
Change-Id: Ic0e1dd904984e161694a0b77fad5559c06a4462f
The property refers to the symbol file that describes APIs across
updatable components like APEXes and system/vendor. Since it's important
to keep the APIs backwards incompatible, any change to the APIs should
be reviewed/gated by the NDK API council. The map.txt suffix is the key
for trigging the NDK API review by gerrit.
Bug: N/A
Test: m nothing
Change-Id: I01eae9c6a106d301863ec86e169111712af74d3f
Dependencies from common arch modules like java libraires to cc modules
may need to retrieve the Target of the library, add it to
SharedLibraryInfo.
Bug: 176593487
Test: cc_test.go
Change-Id: I93ab9da5fab2ccc42c6b483a6d83c071b541d1e6
This reverts commit 758140ceae.
Reason for revert: modifying glob results no longer corrupts cache after aosp/1557261
Change-Id: Id94f1f5835a93e83dc9f465627913b91908d3560
This reverts commit c2aff9e9fd.
Reason for revert: modifying glob results no longer corrupts cache after aosp/1557261
Change-Id: I682b92ad08c82ac4fa5482497cb587bd0223d21b
The vendor snapshot header globs were repeated 9 times, once for each
possible header suffix. Globbing is not a fast operation, so this
took 9.68s of CPU time in my AOSP builds. Replace it instead with a
single glob, followed by filtering down to the desired header
extensions.
Test: m checkbuild
Change-Id: I9c8f8cdad8cbffc0a42217659a48dd9d676e4e4c
