Commit graph

38404 commits

Author SHA1 Message Date
Junwei Wang
ea28f4cae7 Revert "Apply pylint to cc/symbolfile"
This reverts commit 73de70264a.

Reason for revert: Broken build 7651218 on aosp-master on aosp_x86_64-userdebug -- b/197190129

Change-Id: I18387e5d2765fecb292127d09e3bbe7cf19b5efd
2021-08-19 08:10:05 +00:00
Treehugger Robot
1c700090e9 Merge "Apply pylint to build/soong/bloaty" 2021-08-19 07:16:04 +00:00
Lukács T. Berki
1e255cf16d Merge "Do globbing-related paperwork explicitly." 2021-08-19 06:17:32 +00:00
Treehugger Robot
4132ef45bb Merge "Collect info on non-cc sources in a cc module." 2021-08-19 05:55:43 +00:00
Matthew Maurer
a8b1ab0370 Merge changes I9682b978,I35465715,I9c5aa5f3,I1c76e620
* changes:
  Support Rust in Ramdisk
  Support Rust in native-bridge
  Support Rust in Product
  Support Rust in Recovery
2021-08-19 02:55:46 +00:00
Colin Cross
c74ea4b77b Support musl builds in Make
Rewrite linux_musl to linux when exporting to Make.

Bug: 190084016
Test: m USE_HOST_MUSL=true adb
Change-Id: I48eed5bd7380a07c5106a9b2bd96899fb58f7a7f
2021-08-18 22:02:54 +00:00
Liz Kammer
380dbb9327 Merge "Don't create a new module for bp2build conversion." 2021-08-18 19:28:20 +00:00
Treehugger Robot
99f323425a Merge "Support product-copy-files-by-pattern macro" 2021-08-18 19:09:19 +00:00
Spandan Das
16c2b8c3d3 Apply pylint to build/soong/bloaty
1. Run black --line-length 80 -S build/soong/bloaty
to fix formatting
2. Annotate # pylint: disable=import-error to skip checks for imports.
The imports are provided by Soong during m <target>

Test: m bloaty_merger_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/bloaty/bloaty_merger.py
build/soong/bloaty/bloaty_merger_test.py
Bug: 195738175

Change-Id: I4579a80203de41d48992424f264dd1cdbafc854c
2021-08-18 17:46:46 +00:00
Spandan Das
30eefd8b84 Apply pylint to build/soong/cc/ndk_api_coverage_parser
1. Run black --line-length 80 -S build/soong/cc/ndk_api_coverage_parser to
fix formatting
2. Annotate # pylint: disable=line-too-long for multi line strings in
test_ndk_api_coverage_parser.py

Test: pytest
build/soong/cc/ndk_api_coverage_parser/test_ndk_api_coverage_parser.py
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/cc/ndk_api_coverage_parser
Bug: 195738175

Change-Id: Ifb6d4cd67399eb3fa201185c5d1ebc2544fa316f
2021-08-18 17:17:47 +00:00
Liz Kammer
5641096e7f Merge "Handle arch-specific/not properties the same." 2021-08-18 15:59:49 +00:00
Ulya Trafimovich
e4432874a9 Propagate <uses-library> dependencies from java_library to apps.
Automatically add `uses_libs`/`optional_uses_libs` properties specified
in `java_library` modules to the apps that transitively depend on these
libraries.

Note that a library may choose to specify its <uses-library> deps as
either optional or required. If this is not the right choice for an app,
it should be able to override it by specifying its own `uses_libs`/
`optional_uses_libs` properties.

Add Soong tests to cover the new cases.

Bug: 196377222
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I601f3e2830ed9699b78d911966ee70812d32b4f6
2021-08-18 16:57:11 +01:00
Jay Patel
eaf5e1b3ec Merge "Allow PRODUCT_CFI_INCLUDE_PATHS to work with blueprints also" 2021-08-18 15:18:41 +00:00
Lukacs T. Berki
809d2ed548 Do globbing-related paperwork explicitly.
In particular, the writing of build-globs.ninja and the collection of
glob list files for build.ninja.d is now done outside of the Blueprint
invocation.

This is useful because RunBlueprint() is sometimes called multiple times
in a single soong_build invocation and because now glob processing is
not hidden by the machinery of Blueprint.

Test: Presubmits.
Change-Id: I6eb19b357901e452e6d1e7ce055f8a3bab4a74f4
2021-08-18 17:11:56 +02:00
Martin Stjernholm
3931b8c70d Merge "Remove fallback workaround for APEX variants which is now unnecessary." 2021-08-18 08:37:06 +00:00
Sasha Smundak
0445308bc3 Support product-copy-files-by-pattern macro
Bug: 193566316
Test: internal
Change-Id: I5ffdbaee231669166aaa3f40f84ecac8f571f981
2021-08-17 18:15:03 -07:00
Aurimas Liutikas
91436f469a Merge "Propagate <uses-library> dependencies from android_library to apps." 2021-08-17 22:49:51 +00:00
Treehugger Robot
68a240badb Merge "Apply pylint to cc/symbolfile" 2021-08-17 22:44:37 +00:00
Matthew Maurer
c6868383f4 Support Rust in Ramdisk
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a ramdisk binary
Change-Id: I9682b978936624133e5a62e94caace0e8958fd0f
2021-08-17 15:08:49 -07:00
Matthew Maurer
a61e31f66a Support Rust in native-bridge
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a native-bridge binary
Change-Id: I3546571530529203d9dbfd62777f20de18c6bd51
2021-08-17 15:08:49 -07:00
Matthew Maurer
52af5b052b Support Rust in Product
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a product binary
Change-Id: I9c5aa5f3a1f323af9aa2aee804635045f1b91bd4
2021-08-17 15:08:49 -07:00
Matthew Maurer
460ee9429e Support Rust in Recovery
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a recovery binary
Change-Id: I1c76e6204019443c567082730a4cf680f4a2a74a
2021-08-17 15:08:49 -07:00
Liz Kammer
74ec1686aa Collect info on non-cc sources in a cc module.
Cc modules in Soong can contain non-cc srcs, collect information on
whether they exist in a module. This is not entirely precise as globs
are not evaluated and srcs from other modules (e.g. filegroups,
generated sources) are not handled; however, it should give a decent
lower bound on how often each type occurs.

Bug: 194938896
Test: SOONG_DUMP_JSON_MODULE_GRAPH=/tmp/json_srcs m nothing and verify
      outputs
Change-Id: I7ad52e863727cf5fa4e8d41cf04d27c86e395c6d
2021-08-17 17:29:51 -04:00
Liz Kammer
2ada09a546 Don't create a new module for bp2build conversion.
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3

We avoid both of these by storing the information directly in the
underlying module.

Also as a fringe benefit, removes some necessary boilerplate for
conversion.

For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714

Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
2021-08-17 15:57:09 -04:00
Liz Kammer
135bf55281 Handle arch-specific/not properties the same.
Refactor arch-handling code to return a "no-config" axis which allows
handling non-arch configed properties the same way as arch-configed
properties.

Test: build/bazel/ci/bp2build.sh
Change-Id: I485b35fd91c28501fe2055234dc9b278488bf4b5
2021-08-17 13:06:54 -04:00
Ulya Trafimovich
42c7f0d173 Propagate <uses-library> dependencies from android_library to apps.
Automatically add `uses_libs`/`optional_uses_libs` properties specified
in `android_library` modules to the apps that transitively depend on
these libraries.

Note that a library may choose to specify its <uses-library> deps as
either optional or required. If this is not the right choice for an app,
it should be able to override it by specifying its own `uses_libs`/
`optional_uses_libs` properties.

Bug: 196377222
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I1e36c56719c0023dcd25fa6aeb92016471702b9f
2021-08-17 16:58:20 +01:00
Lukács T. Berki
0dd067d309 Merge changes from topics "remove-blueprint-main", "remove-some-minibp", "remove-srcdir"
* changes:
  Remove Srcdir() from BootstrapConfig.
  Define command line args in soong_build .
  Move bpglob to .bootstrap from .minibootstrap.
2021-08-17 14:55:09 +00:00
Treehugger Robot
860f42c119 Merge "Most of the TARGET_COPY_OUT_xxx variables are not constant" 2021-08-17 13:05:31 +00:00
Colin Cross
7e7b028b88 Set ANDROID_HOST_MUSL for musl builds
Unlike glibc, musl does not #define __MUSL__ on the assumption that
building against musl will be done using a configure script that should
be testing for individual features rather than assuming them based on
using musl.  We don't use configure-based builds, so add a local
ANDROID_HOST_MUSL macro that will be defined for all host musl builds.

Bug: 190084016
Test: m USE_HOST_MUSL=true adb
Change-Id: I44e3ffd1d727fd0ea8b69c3b074fef7e92e2be41
2021-08-16 14:42:53 -07:00
Sasha Smundak
02183cfc53 Most of the TARGET_COPY_OUT_xxx variables are not constant
Bug: 193566316
Test: internal
Change-Id: I484a88de933904a2ac2e36084509838d7a85262c
2021-08-16 13:39:48 -07:00
Spandan Das
73de70264a Apply pylint to cc/symbolfile
PEP8 recommends using the not operator over comparision to empty string

Test: pytest build/soong/cc/symbolfile/test_symbolfile.py
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/cc/symbolfile
Bug: 195738175

Change-Id: I4ed2bc0680beccc19a63d8b3f383983d4dff1baf
2021-08-16 18:59:57 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
37abc3c343 Merge "Expose Soong's default clang versions to Bazel" 2021-08-16 18:08:03 +00:00
Lukacs T. Berki
f7e36d80ff Remove Srcdir() from BootstrapConfig.
It was always ".".

Test: Presubmits.
Change-Id: I5381002b3f2986122f1b335a88119cead0a86d75
2021-08-16 17:05:09 +02:00
Lukacs T. Berki
f9008075d9 Define command line args in soong_build .
They used to be defined in Blueprint but since it doesn't have a
separate existence anymore, we can consolidate all command line
arguments here.

Note that Blueprint is called directly from soong_ui, but that happens
by a simple Go function call and not by invoking a separate binary, so
command line arguments are not needed there.

Test: Presubmits.
Change-Id: Ifa2c101ddbe424b76fc5a508d3d41c329e7353ca
2021-08-16 15:29:02 +02:00
Lukacs T. Berki
9d7cf6a336 Move bpglob to .bootstrap from .minibootstrap.
Test: Presubmits.
Change-Id: If6315e497cdc0e6c0f348eb9d4b596ebd13d6303
2021-08-16 14:08:57 +02:00
satayev
d34eb0c4a6 Make sure that classpath fragment contents appear in make vars.
The source of truth for what jars are in BOOTCLASSPATH and
SYSTEMSERVERCLASSPATH is make; adding a classpath fragment in soong
with a new jar in the contents without having it declared in make
is wrong (excluding test apexes).

Bug: 191369843
Test: m nothing
Change-Id: Ifd313776ee7ad206031244534ed3870126e4f835
2021-08-16 12:00:25 +01:00
Lukacs T. Berki
56ebaf35b3 Separate the files of bp2build and soong_build.
The following files are forked:

- build-globs.ninja
- The glob list files
- .d files for the glob list files
- The output file (build.ninja or the bp2build marker file)

This makes bp2build and soong_build not overwrite each other's files
they need for proper incrementality.

Test: Presubmits.

Change-Id: I69d192cbd36ecd9677f46f3fa095dfce6f872227
2021-08-16 09:15:28 +02:00
Martin Stjernholm
0e17c1a57e Remove fallback workaround for APEX variants which is now unnecessary.
Test: m nothing
Bug: 191269918
Change-Id: I13091d5ffdd46947899db273b9cb2399dda23907
2021-08-15 22:32:37 +01:00
Aurimas Liutikas
15c82a10de Merge "Fix the way manifest fixer detects optional <uses-library> entries." 2021-08-13 21:00:35 +00:00
Ivan Lozano
0ad64f50fd Merge "rust: Allow modules to use only generated sources" 2021-08-13 19:37:04 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
a38e518eb9 Expose Soong's default clang versions to Bazel
As `CLANG_DEFAULT_*VERSION` static strings

Bug: 192058388
Test: Existing tests pass
Change-Id: I047bc3ad9626bf67c12a0b0ac76943c79fb1b4b0
2021-08-13 17:44:07 +00:00
Ivan Lozano
e4db0036d7 rust: Allow modules to use only generated sources
Removes the need for rust_test modules that test generated source to
define a source file that simply calls include()! to pull in the
generated source.

Bug: 196076408
Test: m libselinux_bindgen_test
Change-Id: I600301ab74f6ece861916fa2545b061c81b0d92d
2021-08-13 13:37:19 -04:00
Ivan Lozano
8711c5cf8c rust: Link libunwind into Rust static executables.
Rust static executables fail to build due to missing unwind symbols.
Make sure we link libunwind when building these.

Bug: 194386505
Test: rust_binary with static_executable: true builds.
Change-Id: I74bab25e186e181bc1ac8fdd608687707c662e4d
2021-08-13 13:15:31 -04:00
Ulya Trafimovich
fc0f6e34ce Fix the way manifest fixer detects optional <uses-library> entries.
Previously manifest_fixer used a naive way to distiniguish optional libs
from required ones: it checked if a library is on the list of optional
compatibility libraries. This works for compatibility libs, but not for
other libs.

Now we properly track optionality through all stages of the build,
starting with the addition of the library as a dependency (here's where
the `uses_libs`/`optional_uses_libs` distinction kicks in), store it in
dependency tag and propagate to class loader context, and from there to
the manifest_fixer.

The tests have been updated accordingly.

Bug: 196377222
Test: lunch bertha_x86_64-userdebug && m droid dist cts mts
Change-Id: I3631ce59ebe47116ce7a9b3d33a86f636846ef0f
2021-08-13 16:10:42 +01:00
Liz Kammer
e21bb67be9 Merge "Add some initial benchmarking for bp2build." 2021-08-13 14:17:11 +00:00
Jingwen Chen
76738b1c45 Merge "Revert "Exempt prebuilts/jdk and build/bazel/examples/java BUILD..."" 2021-08-13 05:18:56 +00:00
Treehugger Robot
6c00d24a1f Merge "Exempt subfolders of packages/apps/Music from bp2build BUILD file overriding." 2021-08-13 03:04:25 +00:00
Alex Humesky
e9b3738bad Revert "Exempt prebuilts/jdk and build/bazel/examples/java BUILD..."
Revert submission 1789607-java_android_ci

Reason for revert: doesn't work with lunch
Reverted Changes:
I6f82e4984:Exempt prebuilts/jdk and build/bazel/examples/java...
I75a61c24c:Define a java_runtime and toolchain bazel targets ...
I0f107d334:Add a java binary, a java library, and an android ...

Change-Id: I9869424dd44c03b33ac02ab38e4a3e0deee9eb07
2021-08-13 02:06:23 +00:00
Treehugger Robot
1e6760fc71 Merge "Update documentation for VisitDirectDeps" 2021-08-12 23:46:36 +00:00
Liz Kammer
32b77cf8a8 Add some initial benchmarking for bp2build.
Test: go test -benchmark=. in bp2build directory
Change-Id: I8275c38461078cc6100fbc89837177de0edc9d68
2021-08-12 15:54:38 -04:00