Commit graph

38520 commits

Author SHA1 Message Date
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
1c92aeff4e Share runBp2BuildTestCase & runPythonTestCase
Pull along bp2buildTestCase as well
Because none are specific to their current hosts

Move `errored` up to the "generic" block vs.
the "custom" block

Test: Existing tests pass
Change-Id: Ie8d5322a5554ebd88f0c142a4c541791687be326
2021-08-23 19:20:29 +00:00
Colin Cross
b3c68004af Merge "Set ANDROID_HOST_MUSL for musl builds" 2021-08-23 18:47:40 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
8f33dcc9e3 Merge "Fix inverted error message" 2021-08-20 22:20:23 +00:00
Treehugger Robot
899b98d508 Merge "rust: doc: Zip up docs when done" 2021-08-20 22:11:17 +00:00
Treehugger Robot
13b3b785ec Merge "rust: Hook up InstallIn functions + Product" 2021-08-20 21:03:18 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
d75507fff7 Fix inverted error message
The code checks to see if the `child` is not a Python library,
but then when reporting the error it uses the module name as
the dependency name and the child (dependency) as the module name

Test: Existing tests pass
Change-Id: Ied606342291312d8485a8fd2ddcc580ad24ae82f
2021-08-20 21:02:43 +00:00
Matthew Maurer
ca68c49621 rust: doc: Zip up docs when done
Fetching artifacts from the build server doesn't go well with large
numbers of files in nested directories.

Produce an additional rustdoc.zip artifact to make this easier.

This change also prevents ${DIST} from receiving the unpacked rustdoc
site. This can be changed if b/188822051 is fixed.

Bug: 162741284
Test: m rustdoc; check resulting zip contents manually
Change-Id: I80b6a8fa6e274d2d8c3419d8734251afd4d7dba7
2021-08-20 13:08:16 -07:00
Matthew Maurer
9f59e8db27 rust: Hook up InstallIn functions + Product
InstallIn functions must be hooked up not only to install to special
images, but also to receive special handling by android/arch.go such
that some images are implicitly multilib: first.

Also hook up more product details.

Bug: 178565008
Bug: 165791368
Test: lunch aosp_arm64; m
Change-Id: I1980d5aa9d55f78c222b98d60a404cd6ea5abbfb
2021-08-20 12:14:54 -07:00
Ulya Trafimovich
0b1c70efbc Don't add uses_libs/optional_uses_libs to the manifest_fixer.
These properties specify libraries that cannot be implicitly inferred by
Soong. If these properties are added to Android.bp, this can only be for
the reason that there is a <uses-library> tag in the manifest which is
unknown to the build system. Adding them to the manifest_fixer doesn't
make sense: if they are not in the manifest, they should be removed from
Android.bp as well.

Bug: 132357300
Test: $ lunch aosp_cf_x86_64_phone-userdebug && m && launch_cvd
      $ adb wait-for-device && adb root && adb logcat \
        | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
        # empty grep output, no errors
Change-Id: Ic6eb5268a954ef3be7f06a181ec72af99000c547
2021-08-20 15:54:42 +01:00
Florian Mayer
7c02c0883a Use new pass manager for HWASan builds.
Now that we have rolled to a sufficiently new clang, HWASan also works
with the new pass manager.

Bug: 135298400
Test: introduce memory safety bug and verify HWASan gets triggered.
Change-Id: I497ca6542971569cbcad949780e4c7df5b0bfb44
2021-08-20 10:07:20 +01:00
Treehugger Robot
99d5a0f5c9 Merge "Revert "Revert "Apply pylint to cc/symbolfile""" 2021-08-19 20:51:40 +00:00
Spandan Das
3f5659f268 Revert "Revert "Apply pylint to cc/symbolfile""
This reverts commit ea28f4cae7.

Reason for revert: Postsubmit failure was unrelated, and was fixed by this revert CL aosp/1802327

Change-Id: I2693b17c2261cfc3dce0ed69baeb2d1f45f784f8
2021-08-19 19:31:54 +00:00
Lukács T. Berki
5470d24ab7 Merge "Use combined.ninja for soong_docs null build test." 2021-08-19 18:04:00 +00:00
Lukacs T. Berki
1a86bd2dc4 Use combined.ninja for soong_docs null build test.
This requires some explanation: turns out, Ninja uses the $builddir
variable declared in the top-level Ninja file to figure out where the
command log should go. This resulted in a loss of incrementality between
the manual Ninja invocation in test_null_build_after_docs because they
used different Ninja files and therefore different values of $builddir.

The reason why this was not a problem before is that the output files of
the main Ninja file and the bootstrap one were separate, but with that
changing, this will start to matter.

Test: Presubmits.
Change-Id: I28959cc7d659e698c7694a58326c8731eac060eb
2021-08-19 16:29:47 +02:00
Treehugger Robot
9ab1e685ee Merge "Revert "Apply pylint to cc/symbolfile"" 2021-08-19 11:43:17 +00:00
Ulyana Trafimovich
abcce6f1c3 Merge "Propagate <uses-library> dependencies from java_library to apps." 2021-08-19 09:14:33 +00:00
Junwei Wang
ea28f4cae7 Revert "Apply pylint to cc/symbolfile"
This reverts commit 73de70264a.

Reason for revert: Broken build 7651218 on aosp-master on aosp_x86_64-userdebug -- b/197190129

Change-Id: I18387e5d2765fecb292127d09e3bbe7cf19b5efd
2021-08-19 08:10:05 +00:00
Treehugger Robot
1c700090e9 Merge "Apply pylint to build/soong/bloaty" 2021-08-19 07:16:04 +00:00
Lukács T. Berki
1e255cf16d Merge "Do globbing-related paperwork explicitly." 2021-08-19 06:17:32 +00:00
Treehugger Robot
4132ef45bb Merge "Collect info on non-cc sources in a cc module." 2021-08-19 05:55:43 +00:00
Matthew Maurer
a8b1ab0370 Merge changes I9682b978,I35465715,I9c5aa5f3,I1c76e620
* changes:
  Support Rust in Ramdisk
  Support Rust in native-bridge
  Support Rust in Product
  Support Rust in Recovery
2021-08-19 02:55:46 +00:00
Colin Cross
c74ea4b77b Support musl builds in Make
Rewrite linux_musl to linux when exporting to Make.

Bug: 190084016
Test: m USE_HOST_MUSL=true adb
Change-Id: I48eed5bd7380a07c5106a9b2bd96899fb58f7a7f
2021-08-18 22:02:54 +00:00
Liz Kammer
380dbb9327 Merge "Don't create a new module for bp2build conversion." 2021-08-18 19:28:20 +00:00
Treehugger Robot
99f323425a Merge "Support product-copy-files-by-pattern macro" 2021-08-18 19:09:19 +00:00
Spandan Das
16c2b8c3d3 Apply pylint to build/soong/bloaty
1. Run black --line-length 80 -S build/soong/bloaty
to fix formatting
2. Annotate # pylint: disable=import-error to skip checks for imports.
The imports are provided by Soong during m <target>

Test: m bloaty_merger_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/bloaty/bloaty_merger.py
build/soong/bloaty/bloaty_merger_test.py
Bug: 195738175

Change-Id: I4579a80203de41d48992424f264dd1cdbafc854c
2021-08-18 17:46:46 +00:00
Spandan Das
30eefd8b84 Apply pylint to build/soong/cc/ndk_api_coverage_parser
1. Run black --line-length 80 -S build/soong/cc/ndk_api_coverage_parser to
fix formatting
2. Annotate # pylint: disable=line-too-long for multi line strings in
test_ndk_api_coverage_parser.py

Test: pytest
build/soong/cc/ndk_api_coverage_parser/test_ndk_api_coverage_parser.py
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/cc/ndk_api_coverage_parser
Bug: 195738175

Change-Id: Ifb6d4cd67399eb3fa201185c5d1ebc2544fa316f
2021-08-18 17:17:47 +00:00
Liz Kammer
5641096e7f Merge "Handle arch-specific/not properties the same." 2021-08-18 15:59:49 +00:00
Ulya Trafimovich
e4432874a9 Propagate <uses-library> dependencies from java_library to apps.
Automatically add `uses_libs`/`optional_uses_libs` properties specified
in `java_library` modules to the apps that transitively depend on these
libraries.

Note that a library may choose to specify its <uses-library> deps as
either optional or required. If this is not the right choice for an app,
it should be able to override it by specifying its own `uses_libs`/
`optional_uses_libs` properties.

Add Soong tests to cover the new cases.

Bug: 196377222
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I601f3e2830ed9699b78d911966ee70812d32b4f6
2021-08-18 16:57:11 +01:00
Jay Patel
eaf5e1b3ec Merge "Allow PRODUCT_CFI_INCLUDE_PATHS to work with blueprints also" 2021-08-18 15:18:41 +00:00
Lukacs T. Berki
809d2ed548 Do globbing-related paperwork explicitly.
In particular, the writing of build-globs.ninja and the collection of
glob list files for build.ninja.d is now done outside of the Blueprint
invocation.

This is useful because RunBlueprint() is sometimes called multiple times
in a single soong_build invocation and because now glob processing is
not hidden by the machinery of Blueprint.

Test: Presubmits.
Change-Id: I6eb19b357901e452e6d1e7ce055f8a3bab4a74f4
2021-08-18 17:11:56 +02:00
Martin Stjernholm
3931b8c70d Merge "Remove fallback workaround for APEX variants which is now unnecessary." 2021-08-18 08:37:06 +00:00
Sasha Smundak
0445308bc3 Support product-copy-files-by-pattern macro
Bug: 193566316
Test: internal
Change-Id: I5ffdbaee231669166aaa3f40f84ecac8f571f981
2021-08-17 18:15:03 -07:00
Aurimas Liutikas
91436f469a Merge "Propagate <uses-library> dependencies from android_library to apps." 2021-08-17 22:49:51 +00:00
Treehugger Robot
68a240badb Merge "Apply pylint to cc/symbolfile" 2021-08-17 22:44:37 +00:00
Matthew Maurer
c6868383f4 Support Rust in Ramdisk
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a ramdisk binary
Change-Id: I9682b978936624133e5a62e94caace0e8958fd0f
2021-08-17 15:08:49 -07:00
Matthew Maurer
a61e31f66a Support Rust in native-bridge
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a native-bridge binary
Change-Id: I3546571530529203d9dbfd62777f20de18c6bd51
2021-08-17 15:08:49 -07:00
Matthew Maurer
52af5b052b Support Rust in Product
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a product binary
Change-Id: I9c5aa5f3a1f323af9aa2aee804635045f1b91bd4
2021-08-17 15:08:49 -07:00
Matthew Maurer
460ee9429e Support Rust in Recovery
Bug: 178565008
Bug: 165791368
Test: Build and link a Rust library into a recovery binary
Change-Id: I1c76e6204019443c567082730a4cf680f4a2a74a
2021-08-17 15:08:49 -07:00
Liz Kammer
74ec1686aa Collect info on non-cc sources in a cc module.
Cc modules in Soong can contain non-cc srcs, collect information on
whether they exist in a module. This is not entirely precise as globs
are not evaluated and srcs from other modules (e.g. filegroups,
generated sources) are not handled; however, it should give a decent
lower bound on how often each type occurs.

Bug: 194938896
Test: SOONG_DUMP_JSON_MODULE_GRAPH=/tmp/json_srcs m nothing and verify
      outputs
Change-Id: I7ad52e863727cf5fa4e8d41cf04d27c86e395c6d
2021-08-17 17:29:51 -04:00
Liz Kammer
2ada09a546 Don't create a new module for bp2build conversion.
A performance improvement for bp2build as Blueprint/Soong no longer have
the overhead of additional modules. The creation of these modules
results in:
* traversal of additional modules for each subsequent mutator
* synchronization over a go channel to collect newly created modules:
https://cs.android.com/android/platform/superproject/+/master:build/blueprint/context.go;l=2594,2600;drc=1602226f23181b8c3fbfcaf3358f0297e839d7d3

We avoid both of these by storing the information directly in the
underlying module.

Also as a fringe benefit, removes some necessary boilerplate for
conversion.

For benchmarks, reduces runtime ~1% for 1% converted, ~24% for 100%
converted. See more: go/benchmarks-for-https:-r.android.com-1792714

Test: ran benchmarks/tests in bp2build
Test: build/bazel/ci/bp2build.sh
Change-Id: Ie9273b8cbab5bc6edac1728067ce184382feb211
2021-08-17 15:57:09 -04:00
Liz Kammer
135bf55281 Handle arch-specific/not properties the same.
Refactor arch-handling code to return a "no-config" axis which allows
handling non-arch configed properties the same way as arch-configed
properties.

Test: build/bazel/ci/bp2build.sh
Change-Id: I485b35fd91c28501fe2055234dc9b278488bf4b5
2021-08-17 13:06:54 -04:00
Ulya Trafimovich
42c7f0d173 Propagate <uses-library> dependencies from android_library to apps.
Automatically add `uses_libs`/`optional_uses_libs` properties specified
in `android_library` modules to the apps that transitively depend on
these libraries.

Note that a library may choose to specify its <uses-library> deps as
either optional or required. If this is not the right choice for an app,
it should be able to override it by specifying its own `uses_libs`/
`optional_uses_libs` properties.

Bug: 196377222
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I1e36c56719c0023dcd25fa6aeb92016471702b9f
2021-08-17 16:58:20 +01:00
Lukács T. Berki
0dd067d309 Merge changes from topics "remove-blueprint-main", "remove-some-minibp", "remove-srcdir"
* changes:
  Remove Srcdir() from BootstrapConfig.
  Define command line args in soong_build .
  Move bpglob to .bootstrap from .minibootstrap.
2021-08-17 14:55:09 +00:00
Treehugger Robot
860f42c119 Merge "Most of the TARGET_COPY_OUT_xxx variables are not constant" 2021-08-17 13:05:31 +00:00
Colin Cross
7e7b028b88 Set ANDROID_HOST_MUSL for musl builds
Unlike glibc, musl does not #define __MUSL__ on the assumption that
building against musl will be done using a configure script that should
be testing for individual features rather than assuming them based on
using musl.  We don't use configure-based builds, so add a local
ANDROID_HOST_MUSL macro that will be defined for all host musl builds.

Bug: 190084016
Test: m USE_HOST_MUSL=true adb
Change-Id: I44e3ffd1d727fd0ea8b69c3b074fef7e92e2be41
2021-08-16 14:42:53 -07:00
Sasha Smundak
02183cfc53 Most of the TARGET_COPY_OUT_xxx variables are not constant
Bug: 193566316
Test: internal
Change-Id: I484a88de933904a2ac2e36084509838d7a85262c
2021-08-16 13:39:48 -07:00
Spandan Das
73de70264a Apply pylint to cc/symbolfile
PEP8 recommends using the not operator over comparision to empty string

Test: pytest build/soong/cc/symbolfile/test_symbolfile.py
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/cc/symbolfile
Bug: 195738175

Change-Id: I4ed2bc0680beccc19a63d8b3f383983d4dff1baf
2021-08-16 18:59:57 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
37abc3c343 Merge "Expose Soong's default clang versions to Bazel" 2021-08-16 18:08:03 +00:00
Lukacs T. Berki
f7e36d80ff Remove Srcdir() from BootstrapConfig.
It was always ".".

Test: Presubmits.
Change-Id: I5381002b3f2986122f1b335a88119cead0a86d75
2021-08-16 17:05:09 +02:00
Lukacs T. Berki
f9008075d9 Define command line args in soong_build .
They used to be defined in Blueprint but since it doesn't have a
separate existence anymore, we can consolidate all command line
arguments here.

Note that Blueprint is called directly from soong_ui, but that happens
by a simple Go function call and not by invoking a separate binary, so
command line arguments are not needed there.

Test: Presubmits.
Change-Id: Ifa2c101ddbe424b76fc5a508d3d41c329e7353ca
2021-08-16 15:29:02 +02:00