Commit graph

953 commits

Author SHA1 Message Date
Colin Cross
21c71a35bd Merge "Move checking of minApiForArch for apex into cc" 2022-10-13 20:15:30 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
49b59385da Merge "Support cc_prebuilt_binary building with Bazel" 2022-10-10 21:37:35 +00:00
Spandan Das
4238c65a17 Converters for contributions to systemapi and vendorapi
The module types in scope of this conversion are
1. cc_library and cc_library_shared (non-null llndk or stubs prop)
2. cc_library_headers (all)

For (2), we need some postprocessing on the results of the parser
bp2BuildParseBaseProps. This is necessary because arch and os specific
API exports need to be flattened and added to the generateed API headers
target along NoConfigAxis
e.g.
```
The api equivalent of
cc_library_headers (
  name = "lifoo",
  deps = select({
  	"//build/bazel/platforms/arch:arm": ["arm_deps"],
  	"//build/bazel/platforms/arch:arm64": ["arm64_deps"],
	}),
)
should be
cc_api_library_headers (
  name = "lifoo",
  deps = ["arm_deps", "arm64_deps"],
)
```

For (1), we also need to generate 1:many header api targets so that
arch-specific deps can propagate arch metadata to the top-level
api_domain rule

Test: go test ./bp2build
Test: go test ./cc

Change-Id: Ie40cba1ac8e89f290b3d926c190d5e93abd52859
2022-10-07 23:48:00 +00:00
Colin Cross
8ca61c1820 Move checking of minApiForArch for apex into cc
I49220cbec628f1508709741dc56b62aaac7786d9 attempted to allow
apexes to depend on native code whose min_sdk_version had been
increased to meet the minimum supported API level for a new
architecture.  It wasn't quite right, as it assumed that the
primary architecture of the apex would be the newest, and
it applied to all dependencies, not just ones that were
specfiic to the new architecture.  Move the checking into
cc.ShouldSupportSdkVersion, where it can be specific to an
individual architecture variant.

Bug: 250918230
Test: TestApexMinSdkVersion_MinApiForArch
Change-Id: I303cf485ba54b4c6bf63a9f9b49286ff9b2c9c83
2022-10-07 16:04:50 -07:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
b12ff59f0b Support cc_prebuilt_binary building with Bazel
Bp2build-enable cc_prebuilt_binary -> cc_prebuilt_binary

Bug: 241415823
Test: cc_prebuilt_binary_conversion_test.go
Change-Id: I007deef8d44f68993012f2114314d1cb52cfbb0e
2022-10-06 19:39:09 +00:00
Inseob Kim
a12594a4b6 Merge "Add overrides support for snapshots" 2022-10-05 23:43:00 +00:00
Colin Cross
a497606f1d Merge changes I154a6f3a,I79f0c20a,I605ae9af,I647c7305,I95e96e3e, ...
* changes:
  Move fuzzer's CollectAllSharedDependencies into GenerateAndroidBuildActions
  Support AllowMissingDependencies in prebuilt_apex modules
  Support AllowMissingDependencies for apex dependencies
  Add AllowMissingDependencies support for prebuilt_etc module with no src property
  Make OutputFileForModule work for AllowMissingDependencies
  Fix panics when target arch is riscv64
2022-10-05 21:25:17 +00:00
Spandan Das
e191ae76b6 Merge "Move API providing modules from bp2build to api_bp2build workspace" 2022-10-05 19:40:01 +00:00
Jihoon Kang
0b5f7f153f Merge "Specify jnilib partition in Android-<target>.mk" 2022-10-05 16:19:51 +00:00
Colin Cross
31d89b4e0c Move fuzzer's CollectAllSharedDependencies into GenerateAndroidBuildActions
Make rust and cc fuzzers collect their shared libraries once in
GenerateAndroidBuildActions and store it for later use by the
packaging singleton.  Also use android.OutputFileForModule to get
the paths.  Together this will fix fuzzers that depend on architecture
specific prebuilt shared libraries that are missing a prebuilt for an
architecture when building with AllowMissingDependencies.

Bug: 250918230
Test: lunch aosp_riscv64-userdebug && m ALLOW_MISSING_DEPENDENCIES=true nothing
Change-Id: I154a6f3a767c883e9fe7067003615db73ee78e2d
2022-10-04 19:15:45 -07:00
Spandan Das
8b4a5f352e Move API providing modules from bp2build to api_bp2build workspace
The module types in scope are
1. ndk_library
2. ndk_headers, versioned_ndk_headers
3. api_domain

These modules will no longer generate Bazel targets in bp2build
workspace. A new soong mode `api_bp2build` will be used to generate API
specific targets in a separate Bazel workspace at out/soong/api_bp2build

Test: go test ./bp2build

Change-Id: I01d06b6d8b8364c0d56d6d3f07f8f8bf21db31d7
2022-10-04 17:33:32 +00:00
Inseob Kim
a1888ce78d Add overrides support for snapshots
Overrides properties will now be captured in json flag files, which will
be copied to installed vendor snapshot modules.

Bug: 216567575
Test: soong test && manual install
Change-Id: Ife5e84b126e798fba7802b9cff000c9197756cb9
2022-10-04 14:42:02 +09:00
Jihoon Kang
f78a890183 Specify jnilib partition in Android-<target>.mk
Context
- Android-<target>.mk currently does not contain information about partition for its dependent unembedded jni libraries, but only lists the name of the unembedded jni libraries.
- If an android_app module depends on an unembedded jni library that is located in a different partition, make cannot find the library.

Implementation
- Create a string field partition in jniLib struct.
- Add variable "LOCAL_SOONG_JNI_LIBS_PARTITION_<target>", an array of mappings of the name of the jni library to its partition.

Bug: 154162945
Test: m
Change-Id: I6b8e1272ff59dc70e3dd6ce8c6c8e4686dad76df
2022-10-04 03:26:05 +00:00
Spandan Das
1278c2cb24 Multi-tree API bp2build converter for ndk_library
This is a partial bp2build conversion that only emits the
cc_api_contribution target of ndk_library. We also need to convert this
to cc_stub_suite eventually to enable building vendor/product API
domains with Bazel

Also create an enum for the three known Multi-tree API surfaces (will
likely be expanded in the future)

Test: go test ./bp2build
Test: b cquery //bionic/libc:libc.ndk.contribution --output=starlark
--starlark:expr="providers(target)"

Change-Id: Idb24871ba20aae132b61eb31ef35c917cacae9e1
2022-09-21 19:49:46 +00:00
Trevor Radcliffe
d82e8f6858 Delete SyspropMutator (again)
This CL essentially un-reverts aosp/2119952, now that the issue
has been resolved.

Bug: 226199990
Test: m
Change-Id: I3b828eb7f7cf54023c9e1d3229cf1ff7c85ab6dd
2022-09-15 17:41:58 +00:00
Silvia Vinyes
d4b8f7eb0b Merge "Revert "Delete SyspropMutator"" 2022-09-14 14:00:10 +00:00
Chung-Kai (Michael) Mei
6a777b856f Revert "Delete SyspropMutator"
This reverts commit 2db49e3269.

Reason for revert: Break aosp build on git_tm-dev-plus-aosp and downstream branches

Change-Id: I0a86bf5e1e1f2ddf71308250f9110a9608ac2d8d
Bug: 246355583
2022-09-14 13:16:50 +00:00
Trevor Radcliffe
4c56946644 Merge "Delete SyspropMutator" 2022-09-12 18:59:24 +00:00
Yi Kong
56fc1b6902 Do not enable global ThinLTO for tests
Tests are not shipped, no need to spend extra CPU cycles optimising them
during build.

Test: GLOBAL_THINLTO=true m
Bug: 169004486
Change-Id: I66ede9c01d43b574a7fe9f74f0bc6ba97f51be06
2022-09-06 16:24:00 +08:00
Jingwen Chen
537242c6f1 Convert cc_test targets with bp2build.
Starting with libutils_test, adb_test, libcutils_test, (and ~47 other host_supported tests).

Test: $ b build --config=android_x86_64 //system/core/libutils:libutils_test && adb push bazel-bin/system/core/libutils/. /data/tmp/libutils_test_files/ && adb shell /data/tmp/libutils_test_files/libutils_test
Test: presubmits with newly converted cc_test targets
Test: $ b test //system/... --config=linux_x86_64 --verbose_failures --test_output=errors --keep_going --build_tests_only

Fixes: 242038415

Change-Id: I10051aadb880c3aabeba6d7fb079523e6fb4ac43
2022-09-01 14:59:15 +00:00
Kiyoung Kim
51279d3a7b Define cc_api_headers module
Define cc_api_headers module to import from API surfaces and replace
existing header definition if needed.

Tested with :
cc_api_headers {
  name: "libc_headers",
  export_system_include_dirs: [
    "include",
  ],
  min_sdk_version: "1",
  sdk_version: "1",
  vendor_available: true,
  native_bridge_supported: true,
  ...
}

Bug: 236087698
Test: ALLOW_MISSING_DEPENDENCIES=true m -j vendorimage succeeded
Change-Id: I2c3294fe19a272453a168d8c7beeee9859bd4583
2022-09-01 11:24:35 +09:00
Spandan Das
73bcafcbb0 Add a property in ndk_library for header contributions
The new property will be used to determine the header file contributions of
ndk_library(s) to the Public API surface. This should be a no-op for
regular Soong builds.

This will be used by a future bp2build converter to populate
the BUILD files for Multi-tree `cc_api_contribution` targets

(Also noticed that sdk_test.go was never added to testSrcs, which this
CL should fix)

Test: go test ./cc
Test: TH

Change-Id: Ieea093e4aac68e341c6414b6cafe02c441643cdf
2022-08-30 18:24:40 +00:00
Alix
b5f6d9ea51 Deprecating clang Property
Clang has defaulted to true since 2018. Clang property is deprecated.
Set up a BUILD_BROKEN_CLANG_PROPERTY flag so partners can bypass \
errors from using clang.

Bug: 208980553
Test: m nothing & treehugger
Change-Id: I8fd7d3e6bdbb963279248d3482e8a590238727e3
2022-08-26 20:05:07 +00:00
Yi Kong
c702ebd886 Do not enable ThinLTO for CFI enabled projects
CFI projects already use full LTO.

Test: GLOBAL_THINLTO=true m
Change-Id: I8a57efc237aed977f57dd4040500cc5f84843c3d
2022-08-19 16:05:14 +08:00
Trevor Radcliffe
2db49e3269 Delete SyspropMutator
This CL and its associated CLs point cc modules that rely on
sysprop_library modules to the auto-generated cc_library
module, and then remove the code in Soong which maps the
sysprop_library target to the generated target. This is being done
because Bazel will not be supporting the existing behavior in
Soong.

Bug: 226199990
Test: m
Change-Id: I117c5a4a9bf7c38e6fb7b92272a390df87507e30
2022-08-15 20:10:58 +00:00
Treehugger Robot
3a4088dfd1 Merge "Introduce cc_api_library" 2022-08-10 04:12:32 +00:00
Kiyoung Kim
487689eaee Introduce cc_api_library
Introduce cc_api_library, which reflects imported API from other
inner-tree. This cc_api_library module type will later refer from
the other ninja module which generates stub library from the interface
description.

Tested environment :
* original libc definition has been removed temporarily, to ensure that
  imported api stub library is being used from build
* Added new definition of libc as below
 cc_api_library {
  name: "libc",
  arch: {
    x86: {
      src: "libs/x86/libc.so",
    },
    x86_64: {
      src: "libs/x86_64/libc.so",
    },
  },
  header_libs: [
    "libc_headers",
  ],
  export_header_lib_headers: ["libc_headers"],
  min_sdk_version: "9",
  vendor_available: true,
 }

Bug: 236087698
Test: `ALLOW_MISSING_DEPENDENCIES=true m vendorimage` succeeded
Change-Id: I67070b0f3561aa2afd73b6c1c0fdf4255218baac
2022-08-10 10:12:18 +09:00
Cory Barker
9cfcf6d4e9 Revert^2 "Update the way we build AFL++ fuzz binaries"
Test: Built AFL fuzzers individually and built all using haiku command
and built libfuzzers individually and also by using haiku command. Ran
selected fuzzers manually to ensure fuzzing still worked.

Description: Previously we needed to add cc_afl_fuzz to build an afl fuzz binary,
however, to turn current libFuzzers into AFL fuzzers this would required
an update to each Android.bp file which is a lot of work, and would also
require an approval from each Android.bp file owner, which is even more
work.

To get around this (and also to match how AFL fuzzers are built in G3)
we will build AFL++ fuzz binaries by command line option FUZZ_FRAMEWORK.
When FUZZ_FRAMEWORK=AFL is set, all cc_fuzz modules will be built
for AFL rather than libFuzzer. Devs can also specify if a cc_fuzz module
is only for libFuzzer or AFL by using fuzzing_frameworks. If
fuzzing_frameworks is left blank then it will be assumed that the
cc_fuzz module can be built for all available fuzzing frameworks.

Change-Id: If57d3038f05e52775177eaeb26f8ed2bdc73443a
2022-08-08 20:26:09 +00:00
Cory Barker
74aea6c5ee Revert "Updated the way we build AFL++ fuzz binaries"
Revert submission 2164482-Build AFL from Command line option

Reason for revert: Unit test fails on MAC builds
Reverted Changes:
Ia7a822462:Updated the way we build AFL++ fuzz binaries
I994bde63b:Updated the way we build AFL++ fuzz binaries
Ie863853b4:Updated the way we build AFL++ fuzz binaries
I577d6ae7b:Updated the way we build AFL++ fuzz binaries
Ic4fd34f8d:Updated the way we build AFL++ fuzz binaries

Change-Id: Ic2cda7733d206d7ed178f1db5e38268bc915e57b
2022-08-08 15:55:12 +00:00
Cory Barker
cecb7546db Updated the way we build AFL++ fuzz binaries
Test: Built AFL fuzzers individually and built all using haiku command
and built libfuzzers individually and also by using haiku command. Ran
selected fuzzers manually to ensure fuzzing still worked.

Description: Previously we needed to add cc_afl_fuzz to build an afl fuzz binary,
however, to turn current libFuzzers into AFL fuzzers this would required
an update to each Android.bp file which is a lot of work, and would also
require an approval from each Android.bp file owner, which is even more
work.

To get around this (and also to match how AFL fuzzers are built in G3)
we will build AFL++ fuzz binaries by command line option FUZZ_FRAMEWORK.
When FUZZ_FRAMEWORK=AFL is set, all cc_fuzz modules will be built
for AFL rather than libFuzzer. Devs can also specify if a cc_fuzz module
is only for libFuzzer or AFL by using fuzzing_frameworks. If
fuzzing_frameworks is left blank then it will be assumed that the
cc_fuzz module can be built for all available fuzzing frameworks.

Change-Id: Ia7a8224627f2de61606b410d1d1a56d7bdc0955f
2022-08-05 21:46:26 +00:00
Cory Barker
a1da26fa9b Adding support for building AFLpp
Test: Build AFL fuzzers locally and ran them

Change-Id: Ie4fbd258c87663cf81d7d64d575b3da1d5febc17
2022-07-12 21:31:47 +00:00
Lukács T. Berki
5ad0185b63 Merge "Migrate sanitizers to transition mutators." 2022-06-18 19:26:55 +00:00
Lukacs T. Berki
6c71676d6c Migrate sanitizers to transition mutators.
The logic is not 100% provably the same since HEAD was quite
confusing at some points, but I did make an effort to preserve
functional equivalence.

In case that effort was not enough, it should be pretty easy to
tweak the logic at HEAD since it's still quite malleable.

Bug: 231370928
Test: Presubmits.
Change-Id: I17b2efbfb5c4d0aedd922caed54ff8d857e578df
2022-06-18 06:20:28 +02:00
Colin Cross
3e5e778969 Reland: Fix -Wl,--exclude-libs for clang runtime libraries
The sanitize code was assuming that the names of the clang runtime
library modules were the same as their static library output files,
but that's not true after I39e2cf8ae14edf8510276dab38011afaef85822c.
Use the dependency to get the name of the library to pass to
-Wl,--exclude-libs.

This relands If6ca7838800c76f90105fb02d39e8a68cec96314 with a fix
for skipping tests that don't work on mac.
   
Bug: 235624976
Test: TestUbsan
Change-Id: I32894d10d24473ad48b6afc5663f91fa48a6a0ba
2022-06-17 22:48:22 +00:00
Colin Cross
567d98346c Revert "Fix -Wl,--exclude-libs for clang runtime libraries"
This reverts commit 59de280a48.

Reason for revert: Broke mac tests
Bug: 235624976

Change-Id: I358bb14ec565f106483378f4798e3fc88c491f8f
2022-06-17 22:15:00 +00:00
Colin Cross
59de280a48 Fix -Wl,--exclude-libs for clang runtime libraries
THe sanitize code was assuming that the names of the clang runtime
library modules were the same as their static library output files,
but that's not true after I39e2cf8ae14edf8510276dab38011afaef85822c.
Use the dependency to get the name of the library to pass to
-Wl,--exclude-libs.

Bug: 235624976
Test: TestUbsan
Change-Id: If6ca7838800c76f90105fb02d39e8a68cec96314
2022-06-13 15:55:06 -07:00
Trevor Radcliffe
5d6fa4d857 Enable Mixed Builds for Prebuilt Library
This change provides support for mixed builds for
cc_prebuilt_library modules and allowlists some such modules.

Fixes: 225938765
Test: Unit tests, run mixed builds and verify correctness
Change-Id: I04396c79661df6b9a43907859e4f96d0191e8e1b
2022-06-07 20:36:53 +00:00
Treehugger Robot
5bec8d41ea Merge "Clean up some bp2build technical debt" 2022-05-23 18:01:04 +00:00
Chris Parsons
f874e46153 Refactor mixed builds to only take one pass
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:

1. Queueing calls to Bazel is done in a new mutator instead of a full
   soong_build pass. Normal soong_build flow is interrupted (via a
   functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
   as implementing MixedBuildsBuildable. In this interface, define the
   request that must be queued to Bazel, and then subsequently define
   how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
   improves mixed build performance.

Result:
  A 33% runtime improvement on soong analysis phase with mixed builds.

Caveats:
  C++ BazelHandler handling still remains a bit of a mess; I did what
  I could within this CL's scope, but this may require additional cleanup.

Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.

Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
2022-05-20 10:04:13 -04:00
Liz Kammer
3bf97bd1c0 Clean up some bp2build technical debt
Previously we ran mutators in bp2build mode to add dependencies, now we
look up modules by name directly. Remove workarounds to allow bp2build
mode to not fail when adding/handling dependencies.

Test: m bp2build
Change-Id: Ibf6fd905150cac306e5c395902ef28f609f4df2a
2022-05-13 19:47:48 -04:00
Mark Dacek
390df4c181 Merge "Log information for Mixed Builds modules. Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0" 2022-05-03 20:04:10 +00:00
MarkDacek
ff851b83b6 Log information for Mixed Builds modules.
Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw
Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0

Change-Id: I88780c7cc52a189a72216c5e2e499c96574b3731
2022-05-03 18:11:32 +00:00
Treehugger Robot
7f5b4e3229 Merge "Remove version_selector mutator" 2022-05-03 03:06:57 +00:00
Muhammad Haseeb Ahmad
4e70b7586a Merge "Add jni support to java_fuzz_host" 2022-04-27 16:14:29 +00:00
Muhammad Haseeb Ahmad
7e74405b2d Add jni support to java_fuzz_host
Bug: 219782880
Test: m example_java_fuzzer_with_native_lib, go test -run TestJavaFuzz
Change-Id: I1d05fb449e5378a27a0096869d9c12ca0a1245c6
2022-04-26 18:49:02 +00:00
Trevor Radcliffe
58ea451750 bp2build support for cc_prebuilt_library
while cc_prebuilt_library_(shared|static) were already supported,
the raw cc_prebuilt_library module type was not supported by
bp2build. This CL provides that functionality.

Also did a little bit of refactoring and cleaning.

Bug: 225938765
Test: Confirm output BUILD.bazel files look as expected. Confirm
      that build succeeds.
Change-Id: Ic004f88febc0e5b527f2ef420dbd96ecc55d2e09
2022-04-15 16:29:28 +00:00
Ivan Lozano
f1868af5d5 rust: Don't append '.vendor' to vendor modules.
Rust vendor-only modules would have the '.vendor' subname appended to
them, which meant that 'm <vendor_module>' would not work -- instead
you would need to call 'm <vendor_module>.vendor', which leads to some
confusion.

This CL fixes the behavior by using the same SubName logic as the cc
module.

Bug: 205577906
Test: m <vendor_module> # works without .vendor suffix
Change-Id: I6ba18ce1d7281a1f8342ed6014644b48009d78e0
2022-04-12 13:42:04 -04:00
Liz Kammer
2394224da3 Remove version_selector mutator
This mutator simply set a mutated property immediately before a separate
mutator used the value in the property. Instead of a separate mutator,
condense the two into a single mutator pass.

Test: CI
Change-Id: I26c392cdb96f6182a19fbcf0cf3f88a8fc72f81c
2022-04-08 16:17:12 -04:00
Trevor Radcliffe
f389cb4e35 Add support for test_suites to cc_test_library
Some cc_test_library modules will need to be installed for
compatibility test. This CL adds the capacity to use the
test_suites property to enable this behavior.

Fixes: 226402757
Test: Wrote unit tests, confirmed installation to proper dirs,
      and verified conversion that necessitated this change
Change-Id: I7313d4927d28aacad9e444962fd2a7efc6c3bc1f
2022-04-04 19:10:13 +00:00
Colin Cross
4c4c1be915 Use single module for clang runtime libraries
The clang prebuilts now provide a single module with per-architecture
variants instead of a module per architecture.

Bug: 220019988
Test: m checkbuild
Change-Id: I39e2cf8ae14edf8510276dab38011afaef85822c
Merged-In: I39e2cf8ae14edf8510276dab38011afaef85822c
2022-03-07 14:56:32 -08:00