Commit graph

38676 commits

Author SHA1 Message Date
Peter Collingbourne
e168f15ab9 Merge "Read LLVM version from $LLVM_PREBUILTS_VERSION if set." 2021-09-09 18:17:11 +00:00
Treehugger Robot
4011ebb12d Merge "Cleanup unused code for bp2build conversion." 2021-09-09 13:12:04 +00:00
Rupert Shuttleworth
c5fa306162 Move genrule tests to genrule_conversion_test.go.
Test: go test

Change-Id: Ice1ed23d008d26cce0a4feca5bfcab51febcea1b
2021-09-09 06:19:02 -04:00
Treehugger Robot
62b843b90f Merge changes I6df9a14d,I830c53ef
* changes:
  bp2build: allowlist //external/libcap/...
  bp2build: don't generate filegroups (or error) if it contains a file with the same name.
2021-09-09 09:51:33 +00:00
Mohammad Islam
385cde82d4 Merge "Relax restriction on suffix of apex_set filename to include .capex" 2021-09-09 08:21:27 +00:00
Jingwen Chen
1e347860d5 bp2build: allowlist //external/libcap/...
This builds cap_names.list.h, which uses an eponymous filegroup
"generate_cap_names_list.awk" in Soong, but uses the file target
directly in Bazel.

This also improve filegroup support for mixed builds, by issuing a
cquery call _without_ arch. Filegroups in Soong don't have configurable
properties, so don't generate Bazel filegroups into buildroot's
config_nodes (which was x86_64 by default).

The mixed_build_root now looks like this:

```
config_node(...)
config_node(...)
config_node(...)
config_node(...)

...

filegroup(name = "common",
    srcs = ["@//bionic/linker:linker_sources_x86",
            "@//bionic/libc:kernel_input_headers",
            "@//system/timezone/apex:com.android.tzdata-androidManifest",
            "@//external/libcap:generate_cap_names_list.awk",
            "@//bionic/linker:linker_sources_arm64",
            "@//bionic/linker:linker_sources",
            "@//bionic/libc:libc_sources_shared_arm",
            "@//bionic/linker:linker_sources_x86_64",
            "@//bionic/libc:all_kernel_uapi_headers",
            "@//build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal-file_contexts",
            "@//system/core/libcutils:android_filesystem_config_header",
            "@//bionic/libc:libc_sources_static",
            "@//bionic/linker:linker_sources_arm",
            "@//bionic/libc/tools:bionic-gensyscalls",
            "@//bionic/tools:bionic-generate-version-script",
            "@//bionic/libc:libc_sources_shared"],
)

mixed_build_root(name = "buildroot",
    deps = [":x86",
            ":arm64",
            ":arm",
            ":common",
            ":x86_64"],
)
```

Test: CI
Fixes: 198595323
Fixes: 198235838
Change-Id: I6df9a14da556cf358d96e6a99b514f66a2638295
2021-09-09 07:54:16 +00:00
Jingwen Chen
5146ac02a1 bp2build: don't generate filegroups (or error) if it contains a file
with the same name.

Also add capability to test for errors raised in bp2build mutators /
contexts.

This CL does two things to filegroups:

1) If the filegroup has only 1 source file with the same name as itself,
don't generate a filegroup target. Instead, dependents will depend
directly on the Bazel file target instead.

2) If the filegroup has more than 1 source file and 1 of them has the
same name as itself, the bp2build mutator will error out. If bp2build
on CI passes, it means that the source tree / product we're testing
against does not have such a case (which seems to be true for most
source trees).

Either way, this will allow us to be unblocked for most of the errant
filegroups (case 1) in the tree.

Test: CI
Test: New test cases in filegroup_conversion_test.go

Fixes: 194762573
Change-Id: I830c53efc8808569afe3c5f9f08436855bcdafed
2021-09-09 07:12:13 +00:00
Lukács T. Berki
4b221b3cf9 Merge "Implement cleanups requested in aosp/1818245:" 2021-09-09 06:09:38 +00:00
Sarah Chin
d2432e668d Merge "Add Radio AIDL to VNDK variant list" 2021-09-09 05:21:43 +00:00
Treehugger Robot
f49df213f8 Merge "Add tidy-soong, tidy-dir-path, module-tidy targets" 2021-09-09 03:22:59 +00:00
Treehugger Robot
7c8174863d Merge "Convert LOCAL_PRODUCT_SERVICES_MODULE to product_specific" 2021-09-09 00:40:58 +00:00
Treehugger Robot
da513b1ec1 Merge "Apply pylint to scripts/manifest_check*.py" 2021-09-08 23:42:25 +00:00
Peter Collingbourne
cbfbd64823 Read LLVM version from $LLVM_PREBUILTS_VERSION if set.
This matches the behavior of the build system.

Change-Id: Idce0a48403acde100e20246896a1ec68df6b73c7
2021-09-08 13:46:01 -07:00
Spandan Das
96d5424440 Merge "Apply pylint to check_boot_jars.py" 2021-09-08 18:32:02 +00:00
Chih-hung Hsieh
095aac0614 Merge "Use tidyExternalVendor for external/googletest" 2021-09-08 18:29:05 +00:00
Romain Jobredeaux
9e09bba237 Exclude directories with manual BUILD files from bp2build.
Test: b build packages/apps/QuickSearchBox
Change-Id: I2534a328e28a6fff1385bfccdc3b461198bc1f88
2021-09-08 18:09:02 +00:00
Sarah Chin
b03e96874e Add Radio AIDL to VNDK variant list
Test: build
Bug: 198331182
Change-Id: I922961b8311d024ebbf3f3339dfdb41c72ee9dd9
Merged-In: I922961b8311d024ebbf3f3339dfdb41c72ee9dd9
2021-09-08 17:31:38 +00:00
Samiul Islam
7c02e26cba Relax restriction on suffix of apex_set filename to include .capex
Compressed APEX files generated by the build system have the .capex
extension. This CL allows us using those .capex files with their actual
suffix.

Bug: 197258691
Test: manual + unit test added
Change-Id: I79173ef942326b48b5e659f1873d80f12b212339
2021-09-08 18:29:47 +01:00
Treehugger Robot
ba839f8070 Merge "Look for the AndroidProduct.mk files in the vendor/ hierarchy, too" 2021-09-08 17:07:08 +00:00
Lukacs T. Berki
e1df43fafe Implement cleanups requested in aosp/1818245:
- Remove useValidations: whenever it was used, it was turned on, so
there was no point in keeping that knob
- Use the new soong_build invocation name constants

Test: Presubmits.
Change-Id: Ia3e8928d30d3de1e0adaa741111771304e56b139
2021-09-08 15:33:37 +02:00
Lukacs T. Berki
89fcdcb788 Refactor the creation of soong_build calls.
They were a bit repetitive.

In addition, make the environment checks more correct; they could
probably use an integration test or two to make sure that when the
environment changes, exactly those outputs are rebuilt that need to be,
but for now, this is an improvement already.

Test: Presubmits.
Change-Id: Idd79b81ca6975d57d00e5bf4699d266152505ff8
2021-09-08 09:36:22 +02:00
Lukacs T. Berki
c6012f36e1 Invoke soong_docs from the bootstrap Ninja file.
This makes soong_ui the only place where soong_build is invoked, thus
greatly simplifying the conceptual model of the build.

It comes with the slight limitation that now soong_docs (and queryview
and the JSON module graph) are not Make targets anymore, but I suppose
that's an acceptable loss.

The only place where someone depended on soong_docs from a Makefile is
removed in a separate change.

Test: Presubmits.
Change-Id: I3f9ac327725c15d84de725d05e3cdde1da3dcbe2
2021-09-08 09:31:52 +02:00
Lukács T. Berki
144bf84677 Merge "Pass StopBefore as an argument to RunBlueprint." 2021-09-08 06:22:29 +00:00
Spandan Das
f880742582 Apply pylint to scripts/manifest_check*.py
1. Run pyformat scripts/<script>.py -s 4 --force_quote_type
none -i to fix formatting
2. Annotate #pylint: disable, where straightforward fix is not available

Test: m manifest_check_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/scripts/manifest_check.py
build/soong/scripts/manifest_check_test.py
Bug: 195738175

Change-Id: I9af498c4abd6ac9f8b9df4f93cbdd4424eacff8e
2021-09-08 03:59:40 +00:00
Justin Yun
0f922ea335 Convert LOCAL_PRODUCT_SERVICES_MODULE to product_specific
LOCAL_PRODUCT_SERVICES_MODULE is interpreted as LOCAL_PRODUCT_MODULE.
When converting Android.mk to Android.bp, it must be converted to
product_specific.

Bug: 135957588
Test: na
Change-Id: I9298991aef10cc7cb82d40cd667c142e64ee50c6
2021-09-08 10:11:57 +09:00
Chih-Hung Hsieh
d0f82fee44 Add tidy-soong, tidy-dir-path, module-tidy targets
* When WITH_TIDY=1, these targets allow quick check of C/C++
  source code with clang-tidy, without building C/C++ binaries.
* For each module with tidy rules, add a module-tidy target, e.g.,
  libart-tidy, libartd-tidy, bionic-benchmarks-tidy, libnativehelper-tidy, etc.
* Add a tidy-soong phony target that depends on all module-tidy targets.
* For each directory X/Y add a tidy-X-Y phony target that depends
  on all *-tidy targets in X/Y and tidy-X-Y-Z for all X/Y/Z directories,
  e.g., tidy-bionic, tidy-bionic-benchmarks, tidy-libnativehelper, etc.
* Only soong modules are collected for now.
  Tidy rules in .mk files will be collected later.
* Some comment lines reformatted by gofmt.

Test: WITH_TIDY=1 make <some_module>-tidy tidy-<some_directory>
Test: WITH_TIDY=1 make tidy-soong
Bug: 199169329
Change-Id: I45aef3875f70288a8e070761e5f083dbbdfa6e94
2021-09-07 14:56:44 -07:00
Chih-hung Hsieh
63d1a0ab94 Merge "Add config/tidy.go to per-file owners." 2021-09-07 20:18:05 +00:00
Chih-Hung Hsieh
f92c7154cd Use tidyExternalVendor for external/googletest
* Do not run expensive clang-analyzer-* checks with googletest/*,
  which should be checked upstream.

Test: make WITH_TIDY=1 CLANG_ANALYZER_CHECKS=1
Bug: 198098397
Change-Id: Ic801ffa802b42d4d55ae89553073c926f9eaa351
2021-09-07 10:22:56 -07:00
Chih-Hung Hsieh
04a5dabf9a Add config/tidy.go to per-file owners.
Test: make
Change-Id: I01b7bcd2cb5a766baf7ea8015996b4a782395875
2021-09-07 10:20:49 -07:00
Liz Kammer
7cbf7738f2 Cleanup unused code for bp2build conversion.
This code to support bp2build-specific modules is no longer necessary as
we do not create bp2build modules after https://r.android.com/1792714

Test: build/bazel/ci/bp2build.sh
Change-Id: I20d3c42f7e3516554cfb537fef05ba40d222dbeb
2021-09-07 13:19:21 -04:00
Linzhao Ye
a0dc31d4a2 Merge "Add android.hardware.wifi.hostapd-V1-ndk_platform to VNDK list." 2021-09-07 17:15:33 +00:00
Lukacs T. Berki
ffc9e8d812 Pass StopBefore as an argument to RunBlueprint.
Its value is a function of the call site, so it doesn't make a lot of
sense to plumb it through the configuration.

Test: Presubmits.
Change-Id: If928b34de075969fd42932212ce9187808cbdf86
2021-09-07 18:14:07 +02:00
Nikita Iashchenko
337b47b81c Merge "Add jdk.internal.math to package_allowed_list.txt (OpenJDK 11)" 2021-09-07 14:22:51 +00:00
Lukacs T. Berki
7edd91c0f4 Make incremental queryview builds work.
The breakage was because:
- aosp/1818236 removed the "rm -fr" invocation from the Ninja file that
removed the old queryview tree
- queryview files were created with permissions 440 to discourage manual
editing

These two together made incremental queryview invocations not be able to
overwrite its output files.

(Of course, it also made them incorrect since old files were lft in place)

Test: Presubmits.
Change-Id: Idaa3e1db82c9eb8325c48e37aca5146e69af7333
2021-09-07 08:47:21 +02:00
Nikita Iashchenko
15c1387710 Add jdk.internal.math to package_allowed_list.txt (OpenJDK 11)
This is part of merging upstream changes from OpenJDK 11.28. This CL
adds jdk.internal.math to package_allowed_list.txt

Bug: 199067710
Test: atest CtsLibcoreOjTestCases
Change-Id: I343bed53ce6b52a6b373a75828b3234073b9cba8
2021-09-06 17:38:34 +01:00
Lukacs T. Berki
3a82169a55 Invoke queryview from the bootstrap Ninja file.
It used to be invoked from out/soong/build.ninja, which required two
soong_build invocations one after the other (ne to generate
out/soong/build.ninja, one to generate the queryview workspace). This
was slower and required some shell-quoted-in-ninja-quoted-in-Go .

Test: Presubmits.
Change-Id: Idda79c067606663b66e9f94626fa24f3b5af4114
2021-09-06 17:10:49 +02:00
Lukacs T. Berki
a1b9372ef7 Make running soong_build in alternate modes nicer.
This includes the JSON graph generator and bp2build.

Before:
GENERATE_BAZEL_FILES=1 m nothing
GENERATE_JSON_MODULE_GRAPH=1 m nothing

Now:
m json-module-graph
m bp2build

They can now also be combined with other targets or each other.

The longer-term goal is to run "m queryview" and "m soong_docs" using
the same infrastructure. There are two alternate approaches:

1. Call soong_build from within the main Ninja invocation. This requires
two sequential soong_build invocations and is thus slower.
2. Do everything requested in the same soong_build invocation. This
would be faster, but one AFAIU can't tell Ninja that multiple possible
actions can build the same output so that doesn't work.

(1) is somewhat more desirable because soong_docs seems to be built
from build/make/core/main.mk ; I assume that that can be worked around
although I haven't checked where the output of "m soong_docs" goes.

Test: Presubmits.
Change-Id: If5ba36490d9f3f60733e6d6be9286eb2b67c3ff5
2021-09-06 16:03:51 +02:00
Treehugger Robot
bd59c1490d Merge "Additional heuristics: variables with names ending with _LIST are lists" 2021-09-06 00:09:39 +00:00
Treehugger Robot
bdd7b3de94 Merge "Fix typo in exempt from licenses property list." 2021-09-03 23:29:13 +00:00
Lukács T. Berki
399dbe0699 Merge "Do not modules in files called "Blueprints"." 2021-09-03 18:20:39 +00:00
Spandan Das
2c0af22ea8 Apply pylint to check_boot_jars.py
1. Run pyformat scripts/check_boot_jars/check_boot_jars.py -s 4
--force_quote_type none -i to fix formatting.
3. use pylint: disable=<X> where fixes are not obvious

Test: m check_boot_jars
Test: pylint --rcfile tools/repohooks/tools/pylintrc
scripts/check_boot_jars/check_boot_jars.py
Bug: 195738175

Change-Id: I0f86d95e0d174c9f87a2872dedaa65cd3e814246
2021-09-03 18:01:49 +00:00
Bob Badour
bae8421fea Fix typo in exempt from licenses property list.
Bug: 151177513

Test: m all
Change-Id: If794ccc814379e383db87f0deb587114b743d952
2021-09-03 17:25:35 +00:00
Treehugger Robot
e267bc8f74 Merge "Allows prebuilts in override_apex." 2021-09-03 16:01:31 +00:00
Matthias Maennich
c2346f1c21 ndk_library: separate abidw and abitidy into separate rules
Both processes might consume a large amount of memory when analyzing
library ABIs. By chaining them via a pipe we keep the same
representation twice in memory. That can introduce a problematic peak
memory consumption. Hence, split them apart into separate rules that
depend on each other.

Bug: 191235788
Test: m out/soong/abi-dumps/ndk/28/x86_64/libc/abi.xml
Signed-off-by: Matthias Maennich <maennich@google.com>
Change-Id: Ia0264a5ede5b2c2a3c2e3fbe968c11d36acf33c2
2021-09-02 21:04:15 +01:00
Sasha Smundak
70f1745f3e Look for the AndroidProduct.mk files in the vendor/ hierarchy, too
Bug: 182418325
Test: N/A
Change-Id: I17048b5fda3ac83ca7c2b359abe94ab5ced91511
2021-09-02 09:34:47 -07:00
Sasha Smundak
468e11fbbe Additional heuristics: variables with names ending with _LIST are lists
Bug: 193540681
Test: internal
Change-Id: Ic23bf0f0eadb159285650f0b7e20307788c12387
2021-09-02 09:34:47 -07:00
Daniel Norman
5a3ce13c6a Allows prebuilts in override_apex.
Bug: 197787336
Test: Create an override_apex that uses a different `prebuilts` than its
      base. Observe built contents.
Test: apex_test.go
Change-Id: I7666ed6cfe3f2fa5dd81e5f8c1961477dabbbd3c
2021-09-02 09:28:52 -07:00
Lukács T. Berki
5eeb6fc3f2 Merge "Make OutDir() and SoongOutDir() be consistent." 2021-09-02 15:46:57 +00:00
Ulyana Trafimovich
2e380e7971 Merge changes I4e7aa1a3,Ic0cd2411
* changes:
  Fix DONT_UNCOMPRESS_PRIV_APPS_DEXS for `android_app_import`.
  Add tests for uncompressed dex for `android_app_import`.
2021-09-02 15:20:58 +00:00
Treehugger Robot
dadcc8af6f Merge "bp2build: allowlist more packages." 2021-09-02 14:48:49 +00:00