Commit graph

928 commits

Author SHA1 Message Date
Treehugger Robot
3a4088dfd1 Merge "Introduce cc_api_library" 2022-08-10 04:12:32 +00:00
Kiyoung Kim
487689eaee Introduce cc_api_library
Introduce cc_api_library, which reflects imported API from other
inner-tree. This cc_api_library module type will later refer from
the other ninja module which generates stub library from the interface
description.

Tested environment :
* original libc definition has been removed temporarily, to ensure that
  imported api stub library is being used from build
* Added new definition of libc as below
 cc_api_library {
  name: "libc",
  arch: {
    x86: {
      src: "libs/x86/libc.so",
    },
    x86_64: {
      src: "libs/x86_64/libc.so",
    },
  },
  header_libs: [
    "libc_headers",
  ],
  export_header_lib_headers: ["libc_headers"],
  min_sdk_version: "9",
  vendor_available: true,
 }

Bug: 236087698
Test: `ALLOW_MISSING_DEPENDENCIES=true m vendorimage` succeeded
Change-Id: I67070b0f3561aa2afd73b6c1c0fdf4255218baac
2022-08-10 10:12:18 +09:00
Cory Barker
9cfcf6d4e9 Revert^2 "Update the way we build AFL++ fuzz binaries"
Test: Built AFL fuzzers individually and built all using haiku command
and built libfuzzers individually and also by using haiku command. Ran
selected fuzzers manually to ensure fuzzing still worked.

Description: Previously we needed to add cc_afl_fuzz to build an afl fuzz binary,
however, to turn current libFuzzers into AFL fuzzers this would required
an update to each Android.bp file which is a lot of work, and would also
require an approval from each Android.bp file owner, which is even more
work.

To get around this (and also to match how AFL fuzzers are built in G3)
we will build AFL++ fuzz binaries by command line option FUZZ_FRAMEWORK.
When FUZZ_FRAMEWORK=AFL is set, all cc_fuzz modules will be built
for AFL rather than libFuzzer. Devs can also specify if a cc_fuzz module
is only for libFuzzer or AFL by using fuzzing_frameworks. If
fuzzing_frameworks is left blank then it will be assumed that the
cc_fuzz module can be built for all available fuzzing frameworks.

Change-Id: If57d3038f05e52775177eaeb26f8ed2bdc73443a
2022-08-08 20:26:09 +00:00
Cory Barker
74aea6c5ee Revert "Updated the way we build AFL++ fuzz binaries"
Revert submission 2164482-Build AFL from Command line option

Reason for revert: Unit test fails on MAC builds
Reverted Changes:
Ia7a822462:Updated the way we build AFL++ fuzz binaries
I994bde63b:Updated the way we build AFL++ fuzz binaries
Ie863853b4:Updated the way we build AFL++ fuzz binaries
I577d6ae7b:Updated the way we build AFL++ fuzz binaries
Ic4fd34f8d:Updated the way we build AFL++ fuzz binaries

Change-Id: Ic2cda7733d206d7ed178f1db5e38268bc915e57b
2022-08-08 15:55:12 +00:00
Cory Barker
cecb7546db Updated the way we build AFL++ fuzz binaries
Test: Built AFL fuzzers individually and built all using haiku command
and built libfuzzers individually and also by using haiku command. Ran
selected fuzzers manually to ensure fuzzing still worked.

Description: Previously we needed to add cc_afl_fuzz to build an afl fuzz binary,
however, to turn current libFuzzers into AFL fuzzers this would required
an update to each Android.bp file which is a lot of work, and would also
require an approval from each Android.bp file owner, which is even more
work.

To get around this (and also to match how AFL fuzzers are built in G3)
we will build AFL++ fuzz binaries by command line option FUZZ_FRAMEWORK.
When FUZZ_FRAMEWORK=AFL is set, all cc_fuzz modules will be built
for AFL rather than libFuzzer. Devs can also specify if a cc_fuzz module
is only for libFuzzer or AFL by using fuzzing_frameworks. If
fuzzing_frameworks is left blank then it will be assumed that the
cc_fuzz module can be built for all available fuzzing frameworks.

Change-Id: Ia7a8224627f2de61606b410d1d1a56d7bdc0955f
2022-08-05 21:46:26 +00:00
Cory Barker
a1da26fa9b Adding support for building AFLpp
Test: Build AFL fuzzers locally and ran them

Change-Id: Ie4fbd258c87663cf81d7d64d575b3da1d5febc17
2022-07-12 21:31:47 +00:00
Lukács T. Berki
5ad0185b63 Merge "Migrate sanitizers to transition mutators." 2022-06-18 19:26:55 +00:00
Lukacs T. Berki
6c71676d6c Migrate sanitizers to transition mutators.
The logic is not 100% provably the same since HEAD was quite
confusing at some points, but I did make an effort to preserve
functional equivalence.

In case that effort was not enough, it should be pretty easy to
tweak the logic at HEAD since it's still quite malleable.

Bug: 231370928
Test: Presubmits.
Change-Id: I17b2efbfb5c4d0aedd922caed54ff8d857e578df
2022-06-18 06:20:28 +02:00
Colin Cross
3e5e778969 Reland: Fix -Wl,--exclude-libs for clang runtime libraries
The sanitize code was assuming that the names of the clang runtime
library modules were the same as their static library output files,
but that's not true after I39e2cf8ae14edf8510276dab38011afaef85822c.
Use the dependency to get the name of the library to pass to
-Wl,--exclude-libs.

This relands If6ca7838800c76f90105fb02d39e8a68cec96314 with a fix
for skipping tests that don't work on mac.
   
Bug: 235624976
Test: TestUbsan
Change-Id: I32894d10d24473ad48b6afc5663f91fa48a6a0ba
2022-06-17 22:48:22 +00:00
Colin Cross
567d98346c Revert "Fix -Wl,--exclude-libs for clang runtime libraries"
This reverts commit 59de280a48.

Reason for revert: Broke mac tests
Bug: 235624976

Change-Id: I358bb14ec565f106483378f4798e3fc88c491f8f
2022-06-17 22:15:00 +00:00
Colin Cross
59de280a48 Fix -Wl,--exclude-libs for clang runtime libraries
THe sanitize code was assuming that the names of the clang runtime
library modules were the same as their static library output files,
but that's not true after I39e2cf8ae14edf8510276dab38011afaef85822c.
Use the dependency to get the name of the library to pass to
-Wl,--exclude-libs.

Bug: 235624976
Test: TestUbsan
Change-Id: If6ca7838800c76f90105fb02d39e8a68cec96314
2022-06-13 15:55:06 -07:00
Trevor Radcliffe
5d6fa4d857 Enable Mixed Builds for Prebuilt Library
This change provides support for mixed builds for
cc_prebuilt_library modules and allowlists some such modules.

Fixes: 225938765
Test: Unit tests, run mixed builds and verify correctness
Change-Id: I04396c79661df6b9a43907859e4f96d0191e8e1b
2022-06-07 20:36:53 +00:00
Treehugger Robot
5bec8d41ea Merge "Clean up some bp2build technical debt" 2022-05-23 18:01:04 +00:00
Chris Parsons
f874e46153 Refactor mixed builds to only take one pass
This large refactoring has both immense performance implications and
improves mixed builds complexity / usability. Summary:

1. Queueing calls to Bazel is done in a new mutator instead of a full
   soong_build pass. Normal soong_build flow is interrupted (via a
   functional hook in blueprint) to invoke bazel and parse its response.
2. Implementing mixed build support for additional modules is as simple
   as implementing MixedBuildsBuildable. In this interface, define the
   request that must be queued to Bazel, and then subsequently define
   how to handle the returned bazel cquery metadata.
3. Mixed builds consists of only a single pass. This greatly
   improves mixed build performance.

Result:
  A 33% runtime improvement on soong analysis phase with mixed builds.

Caveats:
  C++ BazelHandler handling still remains a bit of a mess; I did what
  I could within this CL's scope, but this may require additional cleanup.

Test: Treehugger
Test: Verified that aosp_arm ninja file is bit-for-bit identical with or
without this change.

Change-Id: I412d9c94d429105f4ebfafc84100d546069e6621
2022-05-20 10:04:13 -04:00
Liz Kammer
3bf97bd1c0 Clean up some bp2build technical debt
Previously we ran mutators in bp2build mode to add dependencies, now we
look up modules by name directly. Remove workarounds to allow bp2build
mode to not fail when adding/handling dependencies.

Test: m bp2build
Change-Id: Ibf6fd905150cac306e5c395902ef28f609f4df2a
2022-05-13 19:47:48 -04:00
Mark Dacek
390df4c181 Merge "Log information for Mixed Builds modules. Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0" 2022-05-03 20:04:10 +00:00
MarkDacek
ff851b83b6 Log information for Mixed Builds modules.
Test: Output matches expected. https://paste.googleplex.com/5913495636803584?raw
Performance evaluated: https://docs.google.com/spreadsheets/d/1X7eOVBKEZUwUWl5i8CDfBo9yUeZrDPXWi2JYO4BEZt4/edit?resourcekey=0-co8crIFW9dpiedhCMkhAgw#gid=0

Change-Id: I88780c7cc52a189a72216c5e2e499c96574b3731
2022-05-03 18:11:32 +00:00
Treehugger Robot
7f5b4e3229 Merge "Remove version_selector mutator" 2022-05-03 03:06:57 +00:00
Muhammad Haseeb Ahmad
4e70b7586a Merge "Add jni support to java_fuzz_host" 2022-04-27 16:14:29 +00:00
Muhammad Haseeb Ahmad
7e74405b2d Add jni support to java_fuzz_host
Bug: 219782880
Test: m example_java_fuzzer_with_native_lib, go test -run TestJavaFuzz
Change-Id: I1d05fb449e5378a27a0096869d9c12ca0a1245c6
2022-04-26 18:49:02 +00:00
Trevor Radcliffe
58ea451750 bp2build support for cc_prebuilt_library
while cc_prebuilt_library_(shared|static) were already supported,
the raw cc_prebuilt_library module type was not supported by
bp2build. This CL provides that functionality.

Also did a little bit of refactoring and cleaning.

Bug: 225938765
Test: Confirm output BUILD.bazel files look as expected. Confirm
      that build succeeds.
Change-Id: Ic004f88febc0e5b527f2ef420dbd96ecc55d2e09
2022-04-15 16:29:28 +00:00
Ivan Lozano
f1868af5d5 rust: Don't append '.vendor' to vendor modules.
Rust vendor-only modules would have the '.vendor' subname appended to
them, which meant that 'm <vendor_module>' would not work -- instead
you would need to call 'm <vendor_module>.vendor', which leads to some
confusion.

This CL fixes the behavior by using the same SubName logic as the cc
module.

Bug: 205577906
Test: m <vendor_module> # works without .vendor suffix
Change-Id: I6ba18ce1d7281a1f8342ed6014644b48009d78e0
2022-04-12 13:42:04 -04:00
Liz Kammer
2394224da3 Remove version_selector mutator
This mutator simply set a mutated property immediately before a separate
mutator used the value in the property. Instead of a separate mutator,
condense the two into a single mutator pass.

Test: CI
Change-Id: I26c392cdb96f6182a19fbcf0cf3f88a8fc72f81c
2022-04-08 16:17:12 -04:00
Trevor Radcliffe
f389cb4e35 Add support for test_suites to cc_test_library
Some cc_test_library modules will need to be installed for
compatibility test. This CL adds the capacity to use the
test_suites property to enable this behavior.

Fixes: 226402757
Test: Wrote unit tests, confirmed installation to proper dirs,
      and verified conversion that necessitated this change
Change-Id: I7313d4927d28aacad9e444962fd2a7efc6c3bc1f
2022-04-04 19:10:13 +00:00
Colin Cross
4c4c1be915 Use single module for clang runtime libraries
The clang prebuilts now provide a single module with per-architecture
variants instead of a module per architecture.

Bug: 220019988
Test: m checkbuild
Change-Id: I39e2cf8ae14edf8510276dab38011afaef85822c
Merged-In: I39e2cf8ae14edf8510276dab38011afaef85822c
2022-03-07 14:56:32 -08:00
Colin Cross
4dac32ad1a Merge "Fix transitive whole_static_libs on prebuilt static libraries" 2022-02-16 23:32:45 +00:00
Yi Kong
4ef5459f7d Don't reduce inline limit for afdo enabled projects
Test: presubmit
Change-Id: Id964165c7af50da9cd5e92c25ad2c4191eb2147a
2022-02-14 20:02:04 +08:00
Colin Cross
a2bcf2c4b8 Fix transitive whole_static_libs on prebuilt static libraries
whole_static_libs on static library usually uses the .o files from
the dependency, but on a prebuilt static library has to use the .a file.
whole_static_libs on a static library that itself has whole_static_libs
on a prebuilt static library also needs to use the .a file transitively.

Test: TestWholeStaticLibPrebuilts
Change-Id: I9d4aabf72be0941b461a72e610f2b28287251cb2
2022-02-11 13:51:41 -08:00
Jooyung Han
472be14f26 Merge changes I11693286,I8d79a434
* changes:
  VNDK libs use "unique" APEX variants
  use_vndk_as_stable APEX shouldn't include VNDK lib
2022-02-08 01:03:41 +00:00
Jooyung Han
91f9203af4 VNDK libs use "unique" APEX variants
In case there are two vendor apexes(one with "use_vndk_as_stable:true",
and the other with "use_vndk_as_stable:false") a VNDK lib used by both
will have "APEX" variant and the former APEX will use "apex" variation.

For example,

 apex1(use_vndk_as_stable) -> foo -> libvndk
 apex2 -> bar -> libvndk

Since foo, bar and libvndk are mutated into two APEX variations("",
"apex10000"), foo will use the apex variation of libvndk.

To fix this, VNDK libs can use "unique" APEX variations. Then, in the
above example, foo will have "myapex1" variation and libvndk will have
two APEX variations("" and "apex2"). So foo will link to ""(non-APEX)
variation as fallback.

Bug: 216847402
Test: m nothing (soong tests)
Change-Id: I116932860ef79e22dc338a58b251e3ca693ab4f3
2022-02-04 13:07:14 +09:00
Liz Kammer
35ca77edb6 Allowlist additional com.android.runtime blockers
To unblock mixed builds, uses a common mechanism to determine cc module
type across bp2build and mixed builds.

Test: mixed_droid.sh
Change-Id: I1f69ad0858cda549b7c95dbadd596e46df0f23b2
2022-02-03 11:17:05 -05:00
Liz Kammer
718eb27f5b Delete toolchain_library
Test: m && ci
Change-Id: Iab5267e02e3559431d980c70a58b49c2b8b4e6de
2022-01-14 12:52:59 -05:00
Chih-Hung Hsieh
7540a78a35 Generate tidy-* rules unless tidy is disabled
* make tidy-soong_subset, or make tidy-<any_directory>,
  should trigger the same clang-tidy compilations
  with or without global WITH_TIDY=1.
* Normal make should not trigger clang-tidy compilations
  unless global WITH_TIDY=1 or a module has set tidy:true.

Bug: 213918926
Test: NINJA_ARGS="-n" make tidy-soong_subset
Test: NINJA_ARGS="-n" make <some-library>
Change-Id: Iafffd3894abe137c9584c2c01830898422f9a677
2022-01-12 16:48:40 -08:00
Liz Kammer
e844132b85 Merge "Revert "Revert "bp2build: convert host & prebuilt header libraries""" 2022-01-12 13:23:16 +00:00
Liz Kammer
3d3b35c94e Revert "Revert "bp2build: convert host & prebuilt header libraries""
This reverts commit 105deedfc7.

Reason for revert: Revert with forward fix

Change-Id: Ib0b6f6b1cbb011b4c5f2133e1cc15ec76ff29253
2022-01-11 16:02:50 +00:00
Jingwen Chen
ed4900f9d9 Merge "Revert "bp2build: convert host & prebuilt header libraries"" 2022-01-11 14:58:56 +00:00
Jingwen Chen
105deedfc7 Revert "bp2build: convert host & prebuilt header libraries"
This reverts commit d8a72d7dc9.

Reason for revert: root cause for b/214013899

Change-Id: If769708408c419c2fd3d9484fbf40202b7276299
2022-01-11 12:51:27 +00:00
Treehugger Robot
77d93ba871 Merge "bp2build: convert host & prebuilt header libraries" 2022-01-08 00:55:32 +00:00
Yi Kong
37d0a5790a Merge "Introduce afdo" 2021-12-23 04:53:29 +00:00
Liz Kammer
d8a72d7dc9 bp2build: convert host & prebuilt header libraries
Test: bp2build.sh
Change-Id: If8165eac7c17c4eef3686067257aa792121db59d
2021-12-22 16:34:27 -05:00
Chris Parsons
58852a05f3 Handle the 'enabled' property in bp2build
Also fix some bugs pertaining to configurable attribute handling of bool
attributes and label sttributes, so that they may support values across
multiple different axes at the same time.

Test: unit tests for bp2build
Test: mixed_droid

Change-Id: I411efcfddf02d55dbc0775962068a11348a8bb2c
2021-12-21 16:37:32 -05:00
Yi Kong
eb8efc902d Introduce afdo
A new configuration rule for sampling PGO. This differs from the
original pgo.go rule in the following ways:
 * Automatic propagation to static dependencies
 * Simpler configuration (just put `afdo: true` to optimsation targets)

http://go/android-afdo

Test: build
Bug: 79161490
Change-Id: Ie194824cd523bca19e10ced41d2078fc598f13b3
2021-12-21 16:09:23 +08:00
Treehugger Robot
f1228f570a Merge "Remove InstallBypassMake and ToMakePath" 2021-12-17 06:15:16 +00:00
Colin Cross
c68db4b305 Remove InstallBypassMake and ToMakePath
InstallBypassMake and ToMakePath are obsolete, remove them.

Bug: 204136549
Test: m checkbuild
Change-Id: Ie5a6f7254b3d317ed6039e114ed6aec35e1ce273
2021-12-15 15:22:53 -08:00
Liz Kammer
416201d081 Make all defaults modules bazelable
Since these are not being converted directly, this has two primary
impacts:
* defaults modules will no longer appear as unconverted deps.
* defaults modules do not have to be marked individually, which should
  allow all defaults-based soong-config-modules to just convert as
  necessary.

Test: bp2build.sh
Change-Id: I44c99d5a51d1887a95c17dfdf956f91af671b2ea
2021-12-15 13:18:42 -05:00
Liz Kammer
b29852a3da Merge "bp2build: do not convert prebuilt cc binaries" 2021-12-15 15:07:58 +00:00
Wei Li
bafb6d6612 Add the initial implementation of bp2build converter for java_library,
java_library_host, java_binary_host and cc_library_host_shared so
signapk tool can be built with Bazel.

Test: b build //build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal
Test: jarsigner -verify -verbose build.bazel.examples.apex.minimal.apex
Bug: 209876137
Bug: 196204358
Bug: 210158864
Bug: 210159074
Bug: 210158872
Change-Id: I855884159d25e69d8f9623792c376da820a1eb4c
2021-12-14 17:23:19 -08:00
Liz Kammer
bdc922f3fe bp2build: do not convert prebuilt cc binaries
These do not have a conversion process yet, do not convert them. Add
more protection against using the wrong conversion for prebuilts in cc.

Test: allowlist project with prebuilt objs/binaries and verify BUILD
      files
Change-Id: Ib26689fa19d6fbb1f5c73ded3b5836165977e987
2021-12-14 14:27:57 -05:00
Liz Kammer
be46fccc40 Use one mutator for all bp2build conversion.
Each conversion required defining a separate mutator, which will each
operate on _all_ modules and requires each to repeat checks whether the
mutator should operator. Instead, we introduce a single mutator and
modules can define a ConvertWithBp2build to implement bp2build
conversion for that module.

Test: bp2build.sh
Bug: 183079158
Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
2021-12-14 09:37:45 -05:00
Treehugger Robot
f4c2bf6559 Merge changes from topic "revert-1918946-revert-1899703-AMWIVCREEV-CCTEXIRRRV"
* changes:
  Revert^2 "Add dependency license annotations"
  Revert^2 "Build license metadata files in Soong"
2021-12-11 03:32:18 +00:00