Adds the ability to centrally disable HWASan for multiple modules when
building with SANITIZE_TARGET=hwaddress. Soong version of the patchset.
HWASan takes precedence over CFI and several other sanitizers that it's
incompatible with[1], which can be problematic for modules that require
those sanitizers (e.g. those that depend on vendor prebuilts where only
sanitized variants are provided). This patch adds an easy way to disable
HWASan for such modules while still keeping it globally enabled.
Test: build with HWASAN_EXCLUDE_PATHS set and verify with readelf that
relevant modules have no references to __hwasan symbols
[1] bb31ca1168/core/config_sanitizers.mk (236)
Change-Id: I5824f71f2a400c64cde29e2c7afdd167d851d337
For the trunk stable next build, we have duplicated modules for the
vndk libraries txt files: one from the generated list, the other from
the prebuilt vndk snapshot files.
If the current vndk version provided by the vndk snapshots, use the
txt file from the prebuilt vndk snapshot.
Bug: 296777146
Test: lunch cf_x86_64_phone-next-userdebug && m nothing
Change-Id: I18bd7b7c77bd37c26c5e0b15cb020a59d50f4f2f
I want to make this warning an error in my project and the
current setup is preventing me from doing so.
This has to be -Wno-error=reorder-init-list and not
-Wno-reorder-init-list because the later is overridden by some
modules explicitly enabling warning sets. -Wno-* flags are
overridden by -W* flags coming later in the command line,
but -Wno-error=* flags are only overridden by later -Werror=
flags.
Bug: 145210666
Test: presubmit
Change-Id: Ia08f8a20da37cdc57167324b0cd19413d8786990
Source tree may include prebuilt vndk snapshot libs that are newer
than or equal to the PLATFORM_VNDK_VERSION.
Ignore those prebuilt vndk snapshot libs.
Bug: 296488609
Test: lunch cf_x86_64_phone-next-userdebug; m nothing
Change-Id: I3adaf3b7636f53884f08540959d2ec2fddfb6921
There are three types of tests, deviceless tests, device-driven tests
and host-driven device tests. But currently we don't have information
to get the type of a test and can't generate test targets on desired
types.
Test: b test //platform_testing/tests/example/native:hello_world_test
Test: b test //packages/modules/adb:adbd_test
Bug: 296312548
Change-Id: I3f022ef769636d508e055477623a4d1a6a1d9044
Without an explicit deduping in this helper function, a dependency may
appear multiple times in the generated BUILD files. I noticed this while
converting some internal google tests.
This currently does the deduping for the non-configurable root value. I
ran into some issues deduping the configurable values, and I will
address it in a followup CL.
Test: go test ./bp2build
Change-Id: Ie7b588a9c922d72e4fabfb1414a7cc3dc402ff86
Soong's proto.include_dirs becomes the -I path for aprotoc cpp code
generation. This does not translate well to Bazel because it runs this
action in a sandbox. Even if we construct an -I path, the .proto files
will not be there. This CL attempts to handle this kind of dependency
automatically via bp2build.
For this hypothetical example
```
foo.proto # contains `import bar.proto"
Android.bp # cc_library {srcs:"foo.proto", p.include_dirs:["subdir"]},
subdir/bar.proto
```
Implementation details for CcProtoGen of foo
- Glob the labels of .proto files for each includeDir, and create a
proto_library that encapsulates them.
- Since Bazel poses a contraint that proto_library target needs to be
in the same pacakge as the .proto file, the proto_library might be created
in a subdirectory with an import_prefix
- Add bar's proto_library as transitive deps of foo's cc_proto_library.
This will be added to -I during aprotoc. We cannot add them to `deps`,
otherwise bar's symbols will be statically linked into foo's .a
file.
Implementation details for clang compile of foo
At the end of CcProtoGen, we have converted foo.proto
to .cpp/.h files. To compile them to .a files, we need the .h files
generated from bar.proto. Soong specifies this at the
top-level cc_library soong module, so add those deps as the
implementation deps for clang compile.
(Will add support for java in a follow-up CL)
Test: go test ./bp2build
Test: built some internal modules that were previously blocked by this
Bug: 285140726
Change-Id: I7e1f9f0d1b1ba916a7ba8278f6cfb342b381d695
llndk.libraries.txt file is currently located within the VNDK APEX.
However, this file is still required even if VNDK APEX is deprecated.
This change removes llndk.libraries.txt from VNDK APEX, so it can be
installed within the system image.
Bug: 290160925
Test: aosp_cf build succeeded with llndk.libraries.txt in the system
image
Change-Id: I09a0a43babaa58ff16fc04ea71ab41ab68b54b70
Because enabling ThinLTO makes some constructor functions not called
in arm64 executables.
Bug: 295944813
Test: build
Change-Id: Ic0cab5fd80aa86d0d9ca1131564b747a445c6e46
The orderfile use flags are unused as cflags causing warnings which becomes errors in some order file builds
In addition, I fixed the test cases so it only looks at ldflags not cflags for orderfile use flags.
Test: mma build/soong
Output: #### build completed successfully (05:14 (mm:ss)) ####
Change-Id: I2a2d846d6688fd5256cf753267c000ff054d56f1
- ARM 32-bit targets will be built with 4096 ELF alignment.
- ARM 32-bit targets will only support 4K Page sizes.
- ARM 32-bit targets won't be page agnostic.
Test: source build/envsetup.sh
lunch aosp_cf_x86_64_phone
m
Bug: 295550555
Change-Id: I61cc45fa5f2ea18c494dfc9a0ca44cf9b694dfa3
If `srcs` contains a gensrcs/genrule module, the current bp2build module
will put it in the catch-all `srcs` attribute. This is reserved for .cpp
sources, so if the genrule produces a .proto/.aidl/... file, this will
fail.
This handles genrules that produce .proto files. To implement this, this
creates an additional partition that detects if the other module is a
genrule/gensrc that produces .proto files. If true, it will append it to
the proto partition.
This CL does not handle
- genrule that produce .c/.aidl/.yacc/.... files. They will continue to
be partitioned into the catch-all partition
- java modules
Test: unit tests
Test: TH
Bug: 293205700
Change-Id: Ib720fdf3a053ff5dd256e6faa632e3fa7776966d
Revert submission 2681060-dumpstate-isstrictrun
Reason for revert: This change has been successfully cherry-picked to udc-d1-dev, and can now be reverted in AOSP so that libdumpstateutil doesn't need a vendor variant.
Reverted changes: /q/submissionid:2681060-dumpstate-isstrictrun
Change-Id: Ia337b557c71c5a64d350f45199929440474fa278
Reduce the import instr limit from the LLVM default (100) to 40. This
helps reduce the binary size as well as improving performance slightly.
Size:
Default(100) 40 30
libhwui.so 11059040 11058912 11060872
libart.so 10697576 10697160 10696568
Performance:
40 vs. default: 0.37% improvement
http://go/art-benchmark?p=BootImageProfileId:36054
30 vs. default: 0.36% improvement
http://go/art-benchmark?p=BootImageProfileId:36058
Test: presubmit
Change-Id: Id800ff7818cde908daab784bac0a312c6a71272d
Now migration to STG is complete, hence remove the legacy extraction
mechanisms.
Bug: 156513478
Change-Id: I55b7626e2bfa78c9bb5157ade73d71a97597da72
Signed-off-by: Matthias Maennich <maennich@google.com>
Replace the abidw->stg pipeline by just stg for extraction directly from
ELF/DWARF. This is to migrate to STG entirely.
As a migration tooling, keep the existing method active via
`legacy_use_abidw` as a flag on ndk_library.
Bug: 156513478
Change-Id: Ideaa9908b31591f49f9a167cfa3f3d5c95d8b198
Signed-off-by: Matthias Maennich <maennich@google.com>
STG's native .stg format has been designed with version control in mind
as is the best format to represent STG's internal data representation.
Hence, migrate the ABI representation dump to .stg. That is done by
converting `abidw` extracted xml to stg at build time. STG has builtin
`abitidy` functionality, so `stg` can replace the prior `abitidy` step
to generate the final format.
Bug: 156513478
Change-Id: Ifcac8998ab899637a5d8c54f9e72d79d3e28ec52
Signed-off-by: Matthias Maennich <maennich@google.com>
In addition, I added a test file to check if flags are added and propagated correctly.
Test: mma build/soong
Output: #### build completed successfully (07:24 (mm:ss)) ####
For testing with an actual binary or shared library, steps are in this
README:
https://android.googlesource.com/toolchain/pgo-profiles/+/refs/heads/main/orderfiles/README.md
Change-Id: Idcf169156ef691bcacb8adc92828ef09450085f8
This change adds libdumpstateutil to VndkMustUseVendorVariantList. This
is done to avoid a VNDK re-spin on aosp/2679475 and aosp/2681060, which
add the dumpstate util function IsStrictRun and make it framework-only.
Test: Presubmit; verify that builds still work on internal (refer to
aosp/2681060 test).
Bug: 283326935
Change-Id: I3c60dd919a65182937d6e0b233635ff6ca9414d3
This reverts commit 73fe13c9b1.
Reason for revert: b/293884796, crash on boot of riscv64 aosp phone.
Change-Id: Ie90e0e99ddaa7bb5d496e2b1d0ab0302702cb2bc
If the target SDK of the module is a preview, the current logic passes
it through to llvm-rs-cc, which expects an int. Convert codenames to
their API level first, then extract the final or preview int.
Also simplify the logic to look for 'current' more generically via the
common ApiLevel library functions.
Test: Verified local build of riscv64, confirmed RS attempts to build
with API level 9000+ instead of 'VanillaIceCream'.
Change-Id: I1fa7577181bfd16de0ea4e77f7ab8fbd8fdb55e5
This reverts commit e72ee14006.
Bug: 293532332
Reason for revert: Test failure fixed in https://r.android.com/2681595
Change-Id: Id40079133e50492a6e2f3e2794012695562c39f6
SCS has been supported in Rust since 1.64.0.
This adds SCS as a sanitizer option in Rust.
Bug: 168914033
Test: Built module with sanitize: { scs: true }
Change-Id: Ie54ac4693286878b98704cf052649a267777d764
these two upstream commits fixed the issues in global-isel and fast-isel:
* global-isel: https://reviews.llvm.org/D82615
* fast-isel: https://reviews.llvm.org/D138615
Currently used clang r498229 (defined build/soong/cc/config/global.go)
is based on upstream commit 14f0776550b5a49e1c42f49a00213f7f3fa047bf
which includes these two changes.
Bug: 293297826
Test: build oriole_hwasan and boot
Change-Id: I65fabdf0b5edd248b4bf4e138a999f8142370cd5
cc_test bazel macro will default `gtest` to True (similar to Soong). So
we can skip explicitly setting this in the generated BUILD files. The
benefit will be that it will make the generated files less verbose,
without loss of information.
This will be implemented by changing its datatype to *bool from bool.
Test: go test ./bp2build
Test: TH
Change-Id: I284e10f1d58c8e7893b170209827f7d5084ca95e
SCS has been supported in Rust since 1.64.0.
This adds SCS as a sanitizer option in Rust.
Bug: 168914033
Test: Built module with sanitize: { scs: true }
Change-Id: Ibdc116a335339af6d0b0d6dd0bb82c0358fa9087
This property implicitly adds `libgtest_isolated_main` to the static
libs of the test. bp2build will make this and `liblog` explicit in BUILD
files
Since the deps are made explicit in BUILD files, `isolated` becomes a
no-op for cc_test. Remove this property from cc_test.
Test: unit tests
Bug: 244432609
Change-Id: I189a7b6b62d9064f4b2abad49ac4975468046498
Soong defaults to false, while bp2build defaults to true. This CL fixes
bp2build's default to false to match Soong.
To prevent future differences, bp2build now uses the `isolated()`
function used by Soong. The type of the context object has been changed
to EarlyModuleContext so that the ctx object does not depend on
mutations that might not run in bp2build.
Test: TH
Bug: 244432609
Change-Id: I15346107896312961e8d12270cf2f9a2a48827e1
You can c++filt, but you can't c++unfilt.
By having the linker provide the exact name of symbols
that are used, we can much more easily resolve missing
symbols in version script files. For instance, I spent
2 hours today working around not having this... :(
After a while, you get used to it. I don't even see the
code.
Bug: 206145174
Test: hide all symbols from a library and see beautiful
and lossless raw symbols.
Change-Id: Ie0730cd503358bf2d383a56be902f7aeef1eea01
To support `b test //path/to:my_test` command line, the actual
executable in the cc_test macro now has the __tf_internal suffix in its
name, so that the top level test suite target can be named without any suffixes.
See other CLs in this topic for more info.
Test: presubmits
Fixes: 292159775
Change-Id: I2d0bfc4276717ae881f6f3eb6aefa88bb47f25da
Embedding multiple includes from a genrule may be difficult to read,
instead we generate a header library that contains the headers and all
include dirs that Soong generates.
Test: go test bp2build tests
Change-Id: I590c74c133f015f27cccf5a2fd916153ad9c125e
This is prep work to add additional special handling for genrule
generated headers as there will be similar partitioning for those
headers.
Test: go test soong tests
Change-Id: Ib63e7e4f7554b2b7b7bc78b2825b20c05403216a
This is currently implicitly added by the cc_test macro. This can cause
duplicate issues if users write a BUILD file as
```
cc_test (
name = "mytest",
deps = ["libgtest"],
)
```
Making this dep explicit in BUILD files prevents this issue.
The property `gtest` will not be removed as part of this CL. cc_test
macro will use this property to add some gtest specific copts
Test: unit tests
Bug: 249374572
Change-Id: Ife931e26367fd4aab302bedc709cf907e2c01245
The generated module lets us keep the aconfig code in its own pacakge
and not infect all of the cc package with aconfig. It's also closer
to what bazel is going to do
Bug: 283479529
Test: m aconfig_hello_world_cc && adb push $TOP/out/target/product/panther/system/bin/aconfig_hello_world_cc /system/bin && adb shell aconfig_hello_world_cc
Change-Id: I2fb9e419939c7ca77b111da9c376af077e2348a9
The 'sdk_version' flag should be allowed to specify a preview API level
codename. Convert the raw name into an integer using the common library
functions. This also switches out the comparison logic to use the
ApiLevel as defined by the 'android' library instead of doing
string-to-int conversion on the output.
Test: Verified that setting VIC as an sdk_version does not crash.
Change-Id: I6ed5fb7ff0dcfa3598e74faa656cde7fa2085bae
... similar to isFuzzer.
Test: verify libaudioflinger_timing does not have duplicate -flto flags
Bug: 289475610
Change-Id: Id7e5ec90b1b445f09f8b2ea164289450a77dfbb5
Install VNDK libraries into vendor or product lib when flag
'KEEP_VNDK' is set as false from build. This flag will be used to
test while deprecating VNDK step by step.
Bug: 290157355
Test: build succeded with DEPRECATE_VNDK succeded
Test: Cuttlefish boot succeded with sepolicy disabled
Change-Id: I05b166f054113ae2775df4e310916d0ea4af3fb6
Adds dylib support for vendor and recovery images.
This changes the default linkage for vendor and recovery images to
dylib, which matches the platform default linkage. This also means that
by default, dylib-std variants are used for rlib dependencies.
Bug: 204303985
Test: Soong tests.
Test: m dist vendor-snapshot
Test: RECOVERY_SNAPSHOT_VERSION=current m dist recovery-snapshot
Change-Id: If84074b8615a70c45e7e162abeb853dc8c34d49a
This reverts commit bce5507c02.
The update of LLVM to clang-r498229 resolved the error with sext.w as
resolved in D143594.
Bug: 275574215
Test: Verified boot of riscv64 phone target.
Change-Id: Ie31b7db03004c5aafb6067713ef5f819e0706b64
Build C/C++ targets with ThinLTO with "--lto-O0" by default. This takes
advantage of better dead code elimination and CFG simplification due to
ThinLTO's global view of the program, but do not enable the costly LTO
optimizations. This also makes builds faster because we can avoid doing
codegen for the dead code.
Code size for panther-userdebug:
/system/bin /system/lib /system/lib64
Original 57648 86264 181700
Global ThinLTO 55764 84916 175728
-3.27% -1.56% -3.29%
Build time for aosp_arm64-userdebug:
clang time linker time
Original 56993.87s 1712.36s
Global ThinLTO 52839.18s 3114.93s
saving 4.68% build time in total.
Bug: 169004486
Test: presubmit
Change-Id: Idb0f4675ca4750b8b12b24f4679579419d3448c8
The -g flag is controllable with CLANG_DEFAULT_DEBUG_LEVEL from -g0 to
-g3. The default remains -g
Test: Build with CLANG_DEFAULT_DEBUG_LEVEL=debug_level_1
Change-Id: I913d3a0cb028484f9496a7e0a2298852f9b699cd
Based on: https://chromium-review.googlesource.com/c/chromium/src/+/3988987
```
Savings are almost 330M on 11G
du -s out.diff/target/product/vsoc_arm64/symbols
10751692 out.diff/target/product/vsoc_arm64/symbols
du -s out/target/product/vsoc_arm64/symbols
11086508 out/target/product/vsoc_arm64/symbols
```
Change-Id: I49a5c4cadc77cb0d97e680e95c4efa7156a42f13
Our global default is -fno-strict-aliasing (including for 32-bit x86),
so let's avoid a source of arm32-only bugs.
Bug: http://b/31600139
Test: treehugger
Change-Id: Iaf3069200b2969718593387dc5b756c574a5d68a
Normally, the users of opt-in ABI check do not enable it for all
architectures. The error message instructs the users to update the ABI
reference dumps for the product failing on the check.
Test: make
Bug: 254141417
Change-Id: I19d28c83a3485266a190ba8abc9ddfb8deeb5106
This option is unsupported on the MacOS ld used in Android, leading to
compilation errors when LTO is enabled for binaries.
In particular the error includes:
```
ld64.lld: error: unknown argument '-plugin-opt'
ld64.lld: error: unknown argument '-import-instr-limit=5'
```
Test: m libc++ on a Mac OS machine
Bug: 288983988
Change-Id: I1873e0a77777832f833f211beb42633d462f04d5
This is a revert of aosp/2449906
mixed build currently supports platform and apex variants of allowlisted
cc libraries. However when building apex variants, we would pass an
apexKey only for allowlisted _dcla_ libraries. This meant that for libraries in
mixed build allowlist but not in dcla allowlist, Soong's apex variants
were being replaced with Bazel's platform variants.
Bug: 270339610
Bug: 288906815
Test: Added libnativeloader, libartpalette-system to prod mixed build
allowlist and `m`
Change-Id: I922fed898b3c3961466cd1165c371839a4192237
Propagate api_domain from top-level config_node(s) in mixed builds.
This ensures that cc_libraries are built in the correct config setting
in mixed builds
Test: m com.android.media.swcodec
Test: built and booted google variant of com.android.media.swcodec
Test: m ccodec_unit_test
Change-Id: I14e3c8e8358b5e90e71697584f9b0eceb018bfa9
MTE globals needs four pieces:
1. (done) clang/llvm to produce objfiles with MTE globals metadata
2. (in flight, https://reviews.llvm.org/D152921) lld to process the metadata in the objfiles, and produce information in each DSO describing the global variables that need to be tagged (alongside some other stuff).
3. (this patch) android to handle the new sanitizer
4. (not yet sent) bionic's libc/linker to interpret the information in each DSO, and tag the global variables.
Because #2 will take some time to come through the toolchain rolls, and #3 and #4 can be landed asynchronously, we can land this ahead of time. Should make my life easier by preventing constant rebases.
Bug: N/A
Test: Build {libc, libm, libc++, libm, and libnetd} in internal master
with these patches and an experimental compiler with the lld support.
Also, `SANITIZE_TARGET=memtag_globals m` here with in-tree compiler
without lld support.
Change-Id: Ie7882d474d4d776232de3a3d571a82274df14bf0
Use Go's generics for DepSets so they don't require a type-specific
wrapper and reflection.
Test: depsets_test.go
Change-Id: I22ba0b7d680d37d2cd05230b0f560d166c4dd20b
Bazel does not support using .rscrip/.fs files as cc srcs.
For a module foo with foo srcs, Bp2build will create a bazel
target of rule class rscript_to_cpp with name foo_renderscript
which will have only the renderscript files as srcs.
Bp2build will also create a target foo with the expected cc rule class.
The foo target will have all other src files as srcs and will also
have the target foo_renderscript as another src.
Bug: 210509914
Test: bp2build testing & b build target rstest-latency
Change-Id: Ifdc55051a3595f5fcf54eab8b59e11e9351e141c
We pass mcpu=cortex-a55 for our build, which increases BB to the point
where the LLVM default of 32 is not enough. This caused worse coverage
for use-after-scope detection, and increased stack usage.
Test: build oriole_hwasan
Bug: 229654778
Change-Id: I6ddb47d7224a9c6ddbf2641c043e3430d330c5a9
In Soong, CFI being explicitly set to false overrides presence in
CFI_INCLUDE_PATHS, so this change causes Bazel to mimic the same
precedence.
Bug: 278789968
Test: Unit tests
Change-Id: I827b3d4272eac2002031970f1e67c9321979b14b
This also changes the expectation of ConvertWithBp2build. Each
implementation must either create one or more Bazel target modules, or
mark the module as unconvertible (with a specific reason).
Manually verified no runtime hit in AOSP
In AOSP, the metrics file size increases from 252K to 1.6M
This changes some effective module counts in bp2build metrics:
- Removes "package" modules from the module count list in
metrics, as these will not be converted like regular modules.
- Counts Handcrafted modules as being "unconverted", as bp2build is not
responsible for them.
Bug: 285631638
Test: Verified generated BUILD.bazel files are bit-for-bit identical
with this change
Test: Manually verified one case of each implemented reasonType
Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
...as regular whole archive deps because binaries don't have
implementation whole archive deps.
Also allowlist checkpolicy which requires this change.
Test: bp2build presubmits should now build checkpolicy
Change-Id: I1f98312da249ea1f37004ccba8799449bdbc4718
Cuttlefish still boots to homescreen, all instructions in the extension
are actually seen to be in use, and the bionic tests still pass.
Test: see above
Change-Id: I429729a722cdd7b3bdf257cb7329ee151ec998a0
This reverts commit c1fd399ccd.
Reason for revert: build break for prebuilt host binary in read-only source tree
Change-Id: I9f1792f324acbfbaf53254a972cb0894140efabe
Test: m BUILD_BROKEN_SRC_DIR_IS_WRITABLE=false mkimage-host
Bug: 287168966
lto has different behaviour depending on the GLOBAL_THINLTO env var
value.
Tests for GLOBAL_THINLTO=true will be added in a follow up CL.
Test: presubmit
Bug: 169004486
Change-Id: I45f6a35eb26bdb1beeb3ff5f2c0c0cdc8fd8c41f
Cuttlefish still boots to homescreen, all instructions in the extension
are actually seen to be in use, and the bionic tests still pass.
Test: see above
Change-Id: I6373fa6245115b9b6c845db03ee73b321c07e02a
Rust shared library dependencies are not always bundled in cc_fuzz or
rust_fuzz modules, which can lead to difficult to debug runtime errors
when running these fuzzers. It can also be hard to determine which
dependencies need to be explicitly declared.
This CL makes sure that we bundle the appropriate transitive
dependencies for our fuzzers.
Bug: 249551848
Test: Soong tests
Test: m <fuzzer> # check data/fuzz/<arch>/lib dir contents
Change-Id: I957ca8898079b61e2ff20d750f8c92bf61ac394f
We removed Full LTO support, now LTO is a binary choice. This allows us
to greatly simplify the LTO propagation logic.
Test: m
Test: GLBOAL_THINLTO=true m
Bug: 169004486
Change-Id: Ie4e453d7518c7069a7c755734dab1e776c16e656
-fprofile-sample-accurate allows the compiler to de-optimize cold code
paths according to the profile. However we have a number of micro-
benchmarks that tests these paths and we often trigger regression
reports from those benchmarks.
Turning off the flag mitigates the issue at a cost of ~2MB increase in
system image size. But the more deterministic codegen could reduce the
differential OTA size.
Test: presubmit
Bug: 283235002
Change-Id: I1016f573f21a60223a97c64f5818f0e7e66e41e3
By default, the artifacts generated by cc_binary and cc_library in Soong
track the module name. But Soong supports overidding this using the stem
property. e.g.
```
cc_library {
name: "foo.1.2",
stem: "foo",
}
```
will generate foo.so
This CL adds this property to bp2build
Test: bp2build unit tests
Bug: 240563612
Change-Id: I1dfed870d5bad450511b72c397d2355c01fa3b60
These warnings are not critical. We can fix them later.
Bug: 285361108
Bug: 285356805
Bug: 285356799
Test: build with WITH_TIDY=1 using clang-r498229
Change-Id: I594279a10ffb4fda07dfc33350b44d09c42d3f97
