Added directories with deps of adbd that were not converted and added
them to be converted via bp2build.
Test: bp2build.sh
Test: mixed_droid.sh
Change-Id: Iacb634b23e1fdc2767a0923882b71f482ebca53d
Also refactor target.bionic to be handled not as its own configuration
axis, but instead to be grouped into os selects handling.
This allows us to remove libbase and its dependencies from the bp2build
denylist.
Test: mixed_droid.sh CI
Change-Id: I92f30074d286306207653fe37589835ae3db16c4
This should unbreak the build and give me some time to investigate the
root cause.
Test: USE_BAZEL_ANALYSIS=1 m libandroidfw
Change-Id: Ia843ae5b92a70131b4989b4bdebc063866cb1075
This required the following fixes to bp2build:
- Correctly handle Bionic_* and Linux_* targets
- Correctly handle cc_object's selects
- Generate linker_script, stl, and system_dynamic_deps for cc_object in bp2build
Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: I753fd18df8ae551fb69df07e4174527c5388f289
Update BazelOutPath to implement genPathProvider. This allows Bazel
outputs (incl filegroup) to be used as an input to generating a file
(e.g. proto).
Test: bp2build.sh
Test: mixed_build.sh
Bug: 200601772
Change-Id: I5ad67ade193025652100b214996b26ce9ca9bf83
Soong supports export_.*_headers properties, the libraries contained in
this list must also be within a shared/static/whole_static/header libs
property. For bp2build, we eliminate this duplication. The libraries
not listed in an export_.*_headers property will migrate to an attribute
prepended with implementation_, those in export_.*_headers will not have
a prefix.
Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Bug: 198241472
Change-Id: I3eb84c983ec5d241c8a568e411dfd5619d3184a7
We don't currently support propagation of OS in mixed builds, and our
host toolchains are not properly configured.
This allows us to continue to make progress on device OS until host is
supported.
Test: mixed_libc CI
Test: USE_BAZEL_ANALYSI=1 m adbd
Change-Id: I2bf8b3f9ca7928dd33bf2fb3d4fa067da0f593f3
Previously, when referencing a cc_library as a static_lib from a
converted module, the corresponding target would depend on the "root
target" of that cc_library. This resulted in a superfluous dependency on
the shared target of the cc_library dependency. This superfluous
dependency could sometimes result in cycles.
This change ensures the static target name is directly specified in such
cases.
Test: mixed_libc.sh CI
Test: USE_BAZEL_ANALYSIS=1 m fmtlib
Change-Id: I36ee9f0a9017b1e9d73df9e3174669107f0afd4f
Factor out shareable code with cc_static_library
Remove `Linkstatic` field for shared cases,
because it's redundant with the rule
Bug: 198241907
Test: cc_library_shared_conversion_test.go
Test: mixed_{libc,droid}.sh
Change-Id: I5be3b66f812893bce41edb425fbd143ab22d9051
Technically, tzdata apex depends on
//system/sepolicy/apex:com.android.tzdata-file_contexts. Builds suceeded
regardless due to bp2build generating a filegroup for all files in a
directory.
This also requires an update to the check in apex/builder.go to address
the fact that outputs (including filegroups) from Bazel in mixed builds
will be in a Bazel out subdirectory. This change also corrects a
potential bug that would not support all subdirectories under
system/sepolicy as golang's path.Match does not treat ** as recursive.
Test: go test apex tests
Test: build/bazel/ci/mixed_droid.sh
Change-Id: I1076d60cf271e4cdd1ea990156ab0a39fcfa6535
Support three options for converting modules with unconverted
dependencies
1. (default) Warn when converting a module if it has unconverted deps.
2. Error when encountering a module with unconverted deps. (not hooked
up yet)
Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Test: BP2BUILD_ERROR_UNCONVERTED=1 build/bazel/ci/bp2build.sh with
unconverted deps -- get appropriate error
Bug: 181155349
Change-Id: Ifaabf0cd2e43e963366dc137159c705294165c3d
This builds cap_names.list.h, which uses an eponymous filegroup
"generate_cap_names_list.awk" in Soong, but uses the file target
directly in Bazel.
This also improve filegroup support for mixed builds, by issuing a
cquery call _without_ arch. Filegroups in Soong don't have configurable
properties, so don't generate Bazel filegroups into buildroot's
config_nodes (which was x86_64 by default).
The mixed_build_root now looks like this:
```
config_node(...)
config_node(...)
config_node(...)
config_node(...)
...
filegroup(name = "common",
srcs = ["@//bionic/linker:linker_sources_x86",
"@//bionic/libc:kernel_input_headers",
"@//system/timezone/apex:com.android.tzdata-androidManifest",
"@//external/libcap:generate_cap_names_list.awk",
"@//bionic/linker:linker_sources_arm64",
"@//bionic/linker:linker_sources",
"@//bionic/libc:libc_sources_shared_arm",
"@//bionic/linker:linker_sources_x86_64",
"@//bionic/libc:all_kernel_uapi_headers",
"@//build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal-file_contexts",
"@//system/core/libcutils:android_filesystem_config_header",
"@//bionic/libc:libc_sources_static",
"@//bionic/linker:linker_sources_arm",
"@//bionic/libc/tools:bionic-gensyscalls",
"@//bionic/tools:bionic-generate-version-script",
"@//bionic/libc:libc_sources_shared"],
)
mixed_build_root(name = "buildroot",
deps = [":x86",
":arm64",
":arm",
":common",
":x86_64"],
)
```
Test: CI
Fixes: 198595323
Fixes: 198235838
Change-Id: I6df9a14da556cf358d96e6a99b514f66a2638295
* //system/core/libprocessgroup:libprocessgroup_headers
* //external/brotli/... and build libbrotli.
* //external/libcxx/...
This also adds //system/core/libprocessgroup to the denylist, but
non-recursively, because there are other modules in its subpackages that
don't build yet, so CI's bp2build-incremental's `//system/...` build can keep
working.
Fixes: 186826841
Fixes: 198148733
Fixes: 197954422
Test: ran mixed_droid-clean locally
Test: presubmits
Change-Id: I5f21287f5374811c4bfcf7b7d64430fe4f9b2900
This adds //system/core/libprocessgroup to the denylist, but
non-recursively, because there are other modules in its subpackages that
don't build yet, so CI's bp2build-incremental's `//system/...` build can keep
working.
Fixes: 186826841
Test: bp2build-incremental //system/...
Change-Id: I4e778737414364964de6328d7420512da224809c
Revert submission 1789607-java_android_ci
Reason for revert: doesn't work with lunch
Reverted Changes:
I6f82e4984:Exempt prebuilts/jdk and build/bazel/examples/java...
I75a61c24c:Define a java_runtime and toolchain bazel targets ...
I0f107d334:Add a java binary, a java library, and an android ...
Change-Id: I9869424dd44c03b33ac02ab38e4a3e0deee9eb07
Rtti was already handled by bazel macros, so this change simply
propagates the rtti bit to these macros.
Test: Run bp2build, build //external/libcxxabi:all
Change-Id: I63296db2db868202874c8bd0b1de6310f7bb85c7
- If no system_shared_libs is specified, bp2build writes no attribute
value. In this case, the bazel library macros determine the correct
default behavior.
- If any system_shared_libs is specified for any variant, then bp2build
writes the value verbatim. This includes if an empty list is specified,
as this should override defaulting behavior.
Note this defaulting behavior is incomplete and will be incorrect in
corner cases. For example, if, in an Android.bp, system_shared_libs is
specified for os.linux_bionic but not for os.android, then the bazel
default for os.android will be incorrect. However, there are no current
modules in AOSP which fit this case.
As a related fix, supports static struct for cc_library_static.
Also, removes some elements from the bp2build denylist.
Test: mixed_droid CI
Change-Id: Iee5feeaaf05e8e7209c7a90c913173832ad7bf91
The previous build/bazel recursive glob is too aggressive. Not all
directories should be ignored by bp2build.
Test: bp2build-incremental builds //bionic/... and everything it depends on in //build/bazel/...
Bug: 193658718
Change-Id: Icb48fd687f0fe6be144caa5b8c9a4f01d15b6c20
In order to reconcile the mixed builds between
Soong and Bazel, the path for the AOSP Music
BUILD file must be added to the bp2buildKeepExistingBuildFile
map.
Change-Id: Ib895d0f4a09502da2d33fc673026d91269b4d086
apexBundle was missing InitBazelModule for it to have bazel_module properties.
Test: soong unit test
Fixes: 193866723
Change-Id: I5fb38c8b2e8edaecef44162491d399c0223d3fb1
This fixes a bug which was a misunderstanding of soong properties:
Soong's cflags pertain only to C and C++ language, whereas bazel's copts
pertain to all three languages. This change ensures that asflags are
added as specifically asflags, and the 'copts' for the static library
macro pertains only to C and C++ languages.
This requires a somewhat hacky workaround for asflags, however: Since
assembly sources also need includepath-related flags, this duplicates
these flags between copts and asflags. To reduce verbosity of
bp2build-generated targets, this also ensures that asflags are omitted
in cases where there are no assembly sources.
Test: Mixed build droid CI
Change-Id: Ic0babed1f90d6dc82e5788638681ce5b995043f8
Previous changes broke mixed_droid CI, and reverting the denylist
changes should get the build green.
More rigorous testing and fixes to follow.
Test: `m linker` which was previously broken.
Change-Id: I9c009d3456ea0c553148316665c97bfb372606d7
This fixes a test and rolls forward I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
Test: m nothing
Test: See I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
Change-Id: I0e450c28e70087e406e7b562d7e772785f177379
Revert submission 1714835-roboleaf-asm-c
Reason for revert: TestCcLibraryStaticProductVariableSelects fails everywhere
Reverted Changes:
I28cf7437e:Split asm and c flags and srcs in bp2build output
I2b47e6b55:Split libraries by language in cc_library_static
Change-Id: I85d39a462f0a5b3f5ff3d685906813fab9f01358
This change enables checked-in BUILD files like
prebuilts/clang/host/linux-x86/BUILD.bazel to be merged cleanly with the
bp2build generated one into the synthetic workspace.
The checked-in BUILD file contains a package() declaration that bp2build
also generates. To avoid double declaration, the BUILD file writer now
checks if the BazelTargets contain handcrafted targets. If so, it
delegates the package declaration to the handcrafted BUILD file instead.
This change also sorts the bp2build targets before the handcrafted ones,
and adds a section header to demarcate the two sets of targets.
Test: TH
Change-Id: I3ecdeaab3226b895b623daf0791d24a657f7a7c6
This allows removal of almost all current items from the mixed build
denylist, which were previously broken due to being unable to separately
control flags for compilations of different languages within the same
target.
Note that this does not appropriately implement asm/c srcs and flags for
either the shared variant or the static variant. This will require a
followup.
Test: bp2build.sh and mixed_libc.sh CI scripts
Test: Updated b2build tests
Change-Id: I28cf7437ee96cdf2fdbcb1eda2303691cff08ba4
Test: bp2build and mixed_libc CI
Test: Manually verified that libc_bionic_ndk compilation gets the
appropriate headers (and no extra headers) from downstream
Change-Id: I79eb6e8ec1d415bd50d12105da4cf97101f95474
This allows support of generated hdrs / srcs in mixed builds.
Test: Manually verified that libc_bionic_ndk passes compilation (failed
previously due to missing generated heaer)
Test: bp2build and mixed_libc CI scripts
Test: New aquery test
Change-Id: I88e359a4bd9eba383c207d5cf812272725ff0a3d
See b/188503688 for more info on why libjemalloc5 is in the
bp2buildCcLibraryStaticOnlyList.
Bug: 188503688
Fixes: 187012490
Fixes: 186828626
Test: TH
Change-Id: I2b70cf99b2efa81f3f35ab0b61716eb0f6f9ebd6
There are two pieces to make this work:
1. Local include paths must include $(BINDIR)-relative paths, to support
generated headers in those directories.
2. The srcs that bp2build outputs for BUILD targets must include labels
for targets given in generated_hdrs and generated_srcs.
Support for exported_generated_hdrs intentionally deferred.
This allows us to remove several targets from the bp2build denylist.
Some are moved to the mixed build denylist, because genreated headers are
still unsupported in mixed builds.
Test: bp2build.sh CI script
Change-Id: Ib4f9dac20f6445487b8dad53b91eac01f437a590
This CL exports common/global/device/host clang/ld/ldd flags
from their Ninja variable initialization locations in
cc/config/global.go and cc/config/clang.go to make Bazel's cc_toolchain
and Soong's cc actions more consistent with each other.
This does not handle env-dependent or arch-specific toolchain flags
yet (logic in compiler.go and linker.go).
Test: TH
Bug: 187086342
Bug: 187084737
Bug: 186628704
Bug: 187857770
Change-Id: Ie403d7cd23f35160897b9dd902c799cbf1bd7f0c
This now works with --features no_copts_tokenization. With tokenization,
the escaped empty string '\"\"' became an actual empty string '' on the
command line, setting -DWILDABBR to the wrong value.
However, no_copts_tokenization unveiled other problems with Android.bp
flags, such as spaces in existing cflags like
https://cs.android.com/android/platform/superproject/+/master:bionic/libc/Android.bp;l=288;drc=a0a4a6c2967b5b3c02c951ea1145f32ed5564ab9
- this trips up Bazel's copts when generated literally, so the
fix (other than splitting on space and making Soong accept strings that
don't start with dashes as cflags) is to make bp2build split cflags on
spaces before generating them as copts.
Test: bp2build, build bionic/...
Fixes: 186822591
Change-Id: Icf10bd20f6fb81db0b719ca0555fc70c75b91a79
This CL adds the ability to filter cc_library modules to only generate
their static variant of their shared variant isn't ready.
For example, the libstdc++ library is buildable as a static library,
which libc depends on. However, the shared variant of libstdc++ depends
on libc, which has to come later.
This CL introduces that abstraction to break up bp2build conversion into
more atomic steps to help with conversion.
Test: TH (bazel build //bionic/... incl. libstdc++'s static variant)
Bug: 186489250
Bug: 186822597
Change-Id: I3e2fe748e4e3d3b656760da4807f342d67c8f45f
But add it to the mixed builds deny list instead of the bp2build deny list.
Test: Waiting for another postsubmit to run.
Change-Id: Ief51356758304b3f13c95df401e310d4f5e236f7
(Some info has been relocated to blockers in Buganizer)
Also remove libjemalloc5 and fmtlib_ndk from the deny list as they don't seem to need to be there at the moment.
Test: bazel build //bionic/...
Test: build/bazel/scripts/run_presubmits.sh
Change-Id: I87e9dd3d4a83291efbcc15d6b99187b57d45ad18
This combines properties among "multilib" and "arch" stanzas in selects
generated by bp2build.
With this fix, libc_gdtoa may be removed from the denylist.
This change also refactors a portion of arch.go, specifically bp2build's
arch mutator, adding a number of comments along the way, to hopefully
make this code clearer for future readers.
Test: mixed_libc.sh
Change-Id: If2beea672957cfb1af6760406ba507181ec38f77
This CL updates the error messages and reorganizes the bottom half of
the bp2build denylist.
Test: TH
Change-Id: Ib15d019ba9cd006a78474477af34c66b5e418348
This CL adds support for the static and shared props, forwarding
whole_static_libs and srcs to the underlying shared library's
static_deps and static library's srcs respectively.
Test: TH
Fixes: 186609667
Change-Id: I84f838361c7758b256b2a163af894c502de03109
Also update modules in denylist that depend on arm-optimized-routines libraries.
Bug: 186388919
Test: bazel build //external/arm-optimized-routines/...
Test: TH
Change-Id: If449b30288a4f852e5fcf31d923eda6cec1eb043
Unfortunately libdl has to stay on the deny list due to an unrelated clang error in CI.
Test: bazel build //bionic/... now works with libdl and libdl_static both removed from the deny list.
Test: ./build/bazel/ci/bp2build.sh passes locally
Change-Id: I1b3c3dd01bc535c73c2e88c81b7998fa7906b4c5
The comment on the interface of GetDirectDep states if there are
multiple deps with different tags, the first will be returned; however,
the current behavior is to panic if there are multiple deps. The
behavior now:
* a single dep, return the module and tag
* a single module with different tags: return module with first tag
* multiple modules: panic
* no module: return nil, nil
Bug: 186488405
Test: ~/aosp/build/bazel/ci/bp2build.sh
Test: m nothing
Change-Id: Id1e7315e7874b4a683ad7357ed2793822315821f
This allows us to remove libasync_safe from the mixed builds denylist.
Test: mixed_libc.sh CI script
Change-Id: Ibafd231284864078bf30340f919d39e5098843ce
Take two: this time, put libasync_safe on the mixed build denylist
because it seems to break things.
Test: Presubmits + testing the offending action using a Ninja
invocation. "m droid dist" also running.
Change-Id: I7912b6670980776d981d874dab157b0268693ab2
OS-specific exclude_srcs and srcs will be done in a follow-up, due to
complexities from merging multiple select statements together.
Test: TH
Bug: 186153868
Change-Id: I01b881e9a5a7cd41b4a507f8be7e9e65eab37bdc
This reverts commit cba7b0c43c.
Reason for revert: mixed_droid failing: libbacktrace fails due to libasync_safe, linkerconfig (and others) due to duplicate symbols
Change-Id: Ib6dd6bbb6f76048a7abd0970d457cf75921beae6
- libc_openbsd_large_stack
- libc_openbsd
Also update the lib_dns comment as it makes more progress now (still fails).
Test: bp2build; bazel build //bionic/...
Test: USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I54f104d359779420d181f195ecbfc9e1269e30be
This allows us to, for certain modules, enable bp2build, but disable
mixed builds.
This facilitates easier iteration on these two features, as modules can
be incrementally "fixed" for bp2build, even if they are not ready for
mixed builds integration.
Test: USE_BAZEL_ANALYSIS=1 m libc
Test: Manually verified that note_* modules exist in bionic/libc/BUILD
Test: bazel build //bionic/....
Change-Id: I3462c8398680140cfc953443d96196143b8ad0a3
This allows the following cc_library_static targets to build with bp2build:
- libc_freebsd
- libc_freebsd_large_stack
- libc_openbsd_ndk
- libc_gdtoa (* but still fails for mixed builds)
- libc_aeabi
- libc_static_dispatch
- libc_dynamic_dispatch
This also allows a number of other cc_library_static targets to progress further in their builds.
Test: Added unit test
Test: bp2build-sync.py write; bazel build //bionic/...
Change-Id: I71742565c16594448a41a6428a5c993171ec4cb4
This adds the two modules to the bp2build/mixed builds denylist while we
investigate b/185079815.
Test: TH
Change-Id: I6d67c5c24502ac48bdc90a005ddef86c46eab0b7
This CL contains the converter for libdl_android, a cc_library that
expands into a cc_shared_library and a regular cc_library.
Test: TH
Test: bp2build; bazel test //build/bazel/tests/...
Change-Id: If70641a538211b0d6b2aac0e4d0d06912318304d
Required fixes:
- Set MakeLinkType for libraries, even in mixed build mode.
- Set snapshot header information to empty list, which passes
validation logic for depending modules
Fixing these libraries also requires a Starlark change to
cc_library_static.bzl, which will be submitted separately.
Additionally, this adds better error messaging in the event that
output files are missing from a mixed-build library.
Test: USE_BAZEL_ANALYSIS=1 m libc
Test: USE_BAZEL_ANALYSIS=1 m runtime-module-sdk
Change-Id: Iad2c4d46359986fb0a43263292a15ed45fabbac7
Previously, genrules which were used as generated_headers only
propagated the old Soong gen directory as an include path, even if the
genrule was managed by Bazel. With this chance, the gen directory of the
Baze output files are used in mixed-build mode.
Test: USE_BAZEL_ANALYSIS=1 m libc
Test: m libc
Change-Id: I3913c4e5b0354421ba1fe43d1f6ef4d1e729a058
This CL is pretty large, so I recommend starting with reading the newly
added tests for the expected behavior.
This change works in conjunction with the linked CLs in the Gerrit topic.
Those CLs add support for new platform() definitions for OS targets
specified in Soong's arch.go, which are configurable through
Android.bp's `target {}` property. It works similary to previous CLs
adding support for the `arch {}` property.
These configurable props are keyed by the OS: android, linux_bionic,
windows, and so on. They map to `select` statements in label list
attributes, which this CL enables for cc_library_headers' header_libs
and export_header_lib_headers props.
This enables //bionic/libc:libc_headers to be generated correctly, from:
cc_library_headers {
name: "libc_headers",
target: {
android: {
header_libs: ["libc_headers_arch"],
export_header_lib_headers: ["libc_headers_arch"],
},
linux_bionic: {
header_libs: ["libc_headers_arch"],
export_header_lib_headers: ["libc_headers_arch"],
},
},
// omitted props
}
to:
cc_library_headers(
name = "libc_headers",
deps = [] + select({
"//build/bazel/platforms/os:android": [
":libc_headers_arch",
],
"//build/bazel/platforms/os:linux_bionic": [
":libc_headers_arch",
],
"//conditions:default": [],
}),
)
Test: TH
Test: Verify generated //bionic/libc:libc_headers
Fixes: 183597786
Change-Id: I01016cc2cc9a71449f02300d747f01decebf3f6e
Without this fix, mixed builds are silently disabled, as a recent change
disabled mixed builds for all module types which are not listed in this
map. This map should be populated for both bp2build generation and for
mixed builds.
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: Ib2a3fb2de2e7a5204ef01355522ae569e0424dc7
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.
It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.
Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.
Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
modules converted with bp2build_available are will also be available to
be used in mixed builds.
Test: build/bazel/scripts/milestone-2/demo.sh full
Test: go tests
Change-Id: I49f16ec3ba5bb11dfed8066af069c27eb04371fb
If both bp2build_available and label are specified, label will be
preferred.
Initially, we copy the entire BUILD.bazel file. Eventually we may move
this to use bazel query for a more accurate result.
Test: go test *
Test: build/bazel/scripts/milestone-2/demo.sh full
Test: GENERATE_BAZEL_FILES=true m nothing
edit bionic/libc/tools/BUILD.bazel
GENERATE_BAZEL_FILES=true m nothing and verify changes picked up
Bug: 180516554
Change-Id: I43025583300e6b10d2c18032cd4a76237b578d59
This allows more direct access to the bazel label and whether the module
is bp2build available.
Test: go test *
Change-Id: I23375835d20fa53d7d94127b2dc2d5bb20487bfb
