Commit graph

38737 commits

Author SHA1 Message Date
Liz Kammer
05d0d24fdc Add TOP to run_cuj so it runs from the correct dir
Test: locally run build/soong/cuj/run_cuj.sh
Change-Id: I89cdc91cc48349fa62840d24baf9b3818284f090
2021-09-10 15:23:14 -04:00
Pirama Arumuga Nainar
1799f9d7b4 [rust/coverage] Pass -z,nostart-stop-gc to the linker
Bug: http://b/199203503

Upstream LLVM change 6d2d3bd0a6 made -z,start-stop-gc the default.  It
drops metadata sections like __llvm_prf_data unless they are marked
SHF_GNU_RETAIN.  https://reviews.llvm.org/D97448 marks generated
sections, including __llvm_prf_data as SHF_GNU_RETAIN.  However this
change is not in the Rust toolchain.  Since we link Rust libs with new
lld, we should use nostart-stop-gc until the Rust toolchain updates past
D97448.

Test: m unicode-xid_device_test_src_lib and verify that they have the
__llvm_prf_data section.

Change-Id: I55eb3622dae9bd789fbacfe708600316102c365f
2021-09-10 10:33:30 -07:00
Colin Cross
748c8c7e09 Make rust builds hermetic
Rust builds were picking up the crt objects and system libraries like
libc.so from /usr/lib/x86_64-linux-gnu/Scrt1.o and
/lib/x86_64-linux-gnu/libc.so.6.  Pass --sysroot to the linker to
point it to the glibc prebuilts.

Test: manual
Change-Id: I6540df8aef0e5c2258de77456d18a6052db627d4
2021-09-10 09:55:54 -07:00
Liz Kammer
8337ea4e7d Cleanup bp2build tests to reduce repetition
Test: go test bp2build tests
Change-Id: Ia96a1a8ed58d5be13fedf31e1667670efdfe3aef
2021-09-10 10:14:09 -04:00
Treehugger Robot
eee562f950 Merge "Avoid duplicate module names in the error from ModuleForTests." 2021-09-10 09:15:06 +00:00
Yi Kong
6afa749d1d Merge "Switch to clang r433403" 2021-09-10 04:21:54 +00:00
Yi Kong
e82732916a Switch to clang r433403
Bug: 197230471
Test: presubmit
Change-Id: I374a66deb785bae99953ea23c67f873f33c2081b
2021-09-10 04:20:49 +00:00
Chih-Hung Hsieh
95dfbce28b Pass TIDY_TIMEOUT to clang-tidy
* Tested with a clang-tidy wrapper that dumps TIDY_TIMEOUT.

Test: make WITH_TIDY=1 TIDY_TIMEOUT=42
Bug: 199451930
Change-Id: I86580225bad7487da786db2ba62c09bd99d70c67
2021-09-09 19:58:41 -07:00
Martin Stjernholm
98e0d88aac Avoid duplicate module names in the error from ModuleForTests.
#codehealth

Test: m nothing
Change-Id: Ia7c3191440452ce04782701bc6ef99251e55af4d
2021-09-09 21:34:02 +01:00
Liz Kammer
5fad501aeb bp2build: Split export_{includes,system_includes}
The specification of exporting includes vs system includes has an impact
on inclusion sort order. Conflating the two caused some symbols to not
be resolved correctly.

Bug: 198403271
Test: build/bazel/ci/bp2build.sh
Test: USE_BAZEL_ANALYSIS=1 m libbacktrace_no_dex succeeds with libc++_*
      modules removed from mixed build denylist (would fail otherwise)
Change-Id: I08aff253d8962dc678ed10214b1c171330e0fe19
2021-09-09 15:59:05 -04:00
Bob Badour
65ee90a362 Change default to require licenses property.
Please do not roll back.

If you have recently created a new Soong module type, please make sure
it supports the `licenses` property.

If your build on an older branch fails due to this change, please
configure your build to set `ANDROID_REQUIRE_LICENSES=false` in the
environment.

If running from the command line, the following will work:

export ANDROID_REQUIRE_LICENSES=false; m -j ...

Previously defaulted to not require property unless overridden in env.

Bug: 151177513

Test: m all

Change-Id: Ib295658f978511d07197c295f04a6f25f7d83686
2021-09-09 18:26:56 +00:00
Peter Collingbourne
e168f15ab9 Merge "Read LLVM version from $LLVM_PREBUILTS_VERSION if set." 2021-09-09 18:17:11 +00:00
Treehugger Robot
4011ebb12d Merge "Cleanup unused code for bp2build conversion." 2021-09-09 13:12:04 +00:00
Rupert Shuttleworth
c5fa306162 Move genrule tests to genrule_conversion_test.go.
Test: go test

Change-Id: Ice1ed23d008d26cce0a4feca5bfcab51febcea1b
2021-09-09 06:19:02 -04:00
Treehugger Robot
62b843b90f Merge changes I6df9a14d,I830c53ef
* changes:
  bp2build: allowlist //external/libcap/...
  bp2build: don't generate filegroups (or error) if it contains a file with the same name.
2021-09-09 09:51:33 +00:00
Mohammad Islam
385cde82d4 Merge "Relax restriction on suffix of apex_set filename to include .capex" 2021-09-09 08:21:27 +00:00
Jingwen Chen
1e347860d5 bp2build: allowlist //external/libcap/...
This builds cap_names.list.h, which uses an eponymous filegroup
"generate_cap_names_list.awk" in Soong, but uses the file target
directly in Bazel.

This also improve filegroup support for mixed builds, by issuing a
cquery call _without_ arch. Filegroups in Soong don't have configurable
properties, so don't generate Bazel filegroups into buildroot's
config_nodes (which was x86_64 by default).

The mixed_build_root now looks like this:

```
config_node(...)
config_node(...)
config_node(...)
config_node(...)

...

filegroup(name = "common",
    srcs = ["@//bionic/linker:linker_sources_x86",
            "@//bionic/libc:kernel_input_headers",
            "@//system/timezone/apex:com.android.tzdata-androidManifest",
            "@//external/libcap:generate_cap_names_list.awk",
            "@//bionic/linker:linker_sources_arm64",
            "@//bionic/linker:linker_sources",
            "@//bionic/libc:libc_sources_shared_arm",
            "@//bionic/linker:linker_sources_x86_64",
            "@//bionic/libc:all_kernel_uapi_headers",
            "@//build/bazel/examples/apex/minimal:build.bazel.examples.apex.minimal-file_contexts",
            "@//system/core/libcutils:android_filesystem_config_header",
            "@//bionic/libc:libc_sources_static",
            "@//bionic/linker:linker_sources_arm",
            "@//bionic/libc/tools:bionic-gensyscalls",
            "@//bionic/tools:bionic-generate-version-script",
            "@//bionic/libc:libc_sources_shared"],
)

mixed_build_root(name = "buildroot",
    deps = [":x86",
            ":arm64",
            ":arm",
            ":common",
            ":x86_64"],
)
```

Test: CI
Fixes: 198595323
Fixes: 198235838
Change-Id: I6df9a14da556cf358d96e6a99b514f66a2638295
2021-09-09 07:54:16 +00:00
Jingwen Chen
5146ac02a1 bp2build: don't generate filegroups (or error) if it contains a file
with the same name.

Also add capability to test for errors raised in bp2build mutators /
contexts.

This CL does two things to filegroups:

1) If the filegroup has only 1 source file with the same name as itself,
don't generate a filegroup target. Instead, dependents will depend
directly on the Bazel file target instead.

2) If the filegroup has more than 1 source file and 1 of them has the
same name as itself, the bp2build mutator will error out. If bp2build
on CI passes, it means that the source tree / product we're testing
against does not have such a case (which seems to be true for most
source trees).

Either way, this will allow us to be unblocked for most of the errant
filegroups (case 1) in the tree.

Test: CI
Test: New test cases in filegroup_conversion_test.go

Fixes: 194762573
Change-Id: I830c53efc8808569afe3c5f9f08436855bcdafed
2021-09-09 07:12:13 +00:00
Lukács T. Berki
4b221b3cf9 Merge "Implement cleanups requested in aosp/1818245:" 2021-09-09 06:09:38 +00:00
Sarah Chin
d2432e668d Merge "Add Radio AIDL to VNDK variant list" 2021-09-09 05:21:43 +00:00
Treehugger Robot
f49df213f8 Merge "Add tidy-soong, tidy-dir-path, module-tidy targets" 2021-09-09 03:22:59 +00:00
Treehugger Robot
7c8174863d Merge "Convert LOCAL_PRODUCT_SERVICES_MODULE to product_specific" 2021-09-09 00:40:58 +00:00
Treehugger Robot
da513b1ec1 Merge "Apply pylint to scripts/manifest_check*.py" 2021-09-08 23:42:25 +00:00
Peter Collingbourne
cbfbd64823 Read LLVM version from $LLVM_PREBUILTS_VERSION if set.
This matches the behavior of the build system.

Change-Id: Idce0a48403acde100e20246896a1ec68df6b73c7
2021-09-08 13:46:01 -07:00
Spandan Das
96d5424440 Merge "Apply pylint to check_boot_jars.py" 2021-09-08 18:32:02 +00:00
Chih-hung Hsieh
095aac0614 Merge "Use tidyExternalVendor for external/googletest" 2021-09-08 18:29:05 +00:00
Romain Jobredeaux
9e09bba237 Exclude directories with manual BUILD files from bp2build.
Test: b build packages/apps/QuickSearchBox
Change-Id: I2534a328e28a6fff1385bfccdc3b461198bc1f88
2021-09-08 18:09:02 +00:00
Sarah Chin
b03e96874e Add Radio AIDL to VNDK variant list
Test: build
Bug: 198331182
Change-Id: I922961b8311d024ebbf3f3339dfdb41c72ee9dd9
Merged-In: I922961b8311d024ebbf3f3339dfdb41c72ee9dd9
2021-09-08 17:31:38 +00:00
Samiul Islam
7c02e26cba Relax restriction on suffix of apex_set filename to include .capex
Compressed APEX files generated by the build system have the .capex
extension. This CL allows us using those .capex files with their actual
suffix.

Bug: 197258691
Test: manual + unit test added
Change-Id: I79173ef942326b48b5e659f1873d80f12b212339
2021-09-08 18:29:47 +01:00
Treehugger Robot
ba839f8070 Merge "Look for the AndroidProduct.mk files in the vendor/ hierarchy, too" 2021-09-08 17:07:08 +00:00
Lukacs T. Berki
e1df43fafe Implement cleanups requested in aosp/1818245:
- Remove useValidations: whenever it was used, it was turned on, so
there was no point in keeping that knob
- Use the new soong_build invocation name constants

Test: Presubmits.
Change-Id: Ia3e8928d30d3de1e0adaa741111771304e56b139
2021-09-08 15:33:37 +02:00
Lukacs T. Berki
89fcdcb788 Refactor the creation of soong_build calls.
They were a bit repetitive.

In addition, make the environment checks more correct; they could
probably use an integration test or two to make sure that when the
environment changes, exactly those outputs are rebuilt that need to be,
but for now, this is an improvement already.

Test: Presubmits.
Change-Id: Idd79b81ca6975d57d00e5bf4699d266152505ff8
2021-09-08 09:36:22 +02:00
Lukacs T. Berki
c6012f36e1 Invoke soong_docs from the bootstrap Ninja file.
This makes soong_ui the only place where soong_build is invoked, thus
greatly simplifying the conceptual model of the build.

It comes with the slight limitation that now soong_docs (and queryview
and the JSON module graph) are not Make targets anymore, but I suppose
that's an acceptable loss.

The only place where someone depended on soong_docs from a Makefile is
removed in a separate change.

Test: Presubmits.
Change-Id: I3f9ac327725c15d84de725d05e3cdde1da3dcbe2
2021-09-08 09:31:52 +02:00
Lukács T. Berki
144bf84677 Merge "Pass StopBefore as an argument to RunBlueprint." 2021-09-08 06:22:29 +00:00
Spandan Das
f880742582 Apply pylint to scripts/manifest_check*.py
1. Run pyformat scripts/<script>.py -s 4 --force_quote_type
none -i to fix formatting
2. Annotate #pylint: disable, where straightforward fix is not available

Test: m manifest_check_test
Test: pylint --rcfile tools/repohooks/tools/pylintrc
build/soong/scripts/manifest_check.py
build/soong/scripts/manifest_check_test.py
Bug: 195738175

Change-Id: I9af498c4abd6ac9f8b9df4f93cbdd4424eacff8e
2021-09-08 03:59:40 +00:00
Justin Yun
0f922ea335 Convert LOCAL_PRODUCT_SERVICES_MODULE to product_specific
LOCAL_PRODUCT_SERVICES_MODULE is interpreted as LOCAL_PRODUCT_MODULE.
When converting Android.mk to Android.bp, it must be converted to
product_specific.

Bug: 135957588
Test: na
Change-Id: I9298991aef10cc7cb82d40cd667c142e64ee50c6
2021-09-08 10:11:57 +09:00
Chih-Hung Hsieh
d0f82fee44 Add tidy-soong, tidy-dir-path, module-tidy targets
* When WITH_TIDY=1, these targets allow quick check of C/C++
  source code with clang-tidy, without building C/C++ binaries.
* For each module with tidy rules, add a module-tidy target, e.g.,
  libart-tidy, libartd-tidy, bionic-benchmarks-tidy, libnativehelper-tidy, etc.
* Add a tidy-soong phony target that depends on all module-tidy targets.
* For each directory X/Y add a tidy-X-Y phony target that depends
  on all *-tidy targets in X/Y and tidy-X-Y-Z for all X/Y/Z directories,
  e.g., tidy-bionic, tidy-bionic-benchmarks, tidy-libnativehelper, etc.
* Only soong modules are collected for now.
  Tidy rules in .mk files will be collected later.
* Some comment lines reformatted by gofmt.

Test: WITH_TIDY=1 make <some_module>-tidy tidy-<some_directory>
Test: WITH_TIDY=1 make tidy-soong
Bug: 199169329
Change-Id: I45aef3875f70288a8e070761e5f083dbbdfa6e94
2021-09-07 14:56:44 -07:00
Chih-hung Hsieh
63d1a0ab94 Merge "Add config/tidy.go to per-file owners." 2021-09-07 20:18:05 +00:00
Chih-Hung Hsieh
f92c7154cd Use tidyExternalVendor for external/googletest
* Do not run expensive clang-analyzer-* checks with googletest/*,
  which should be checked upstream.

Test: make WITH_TIDY=1 CLANG_ANALYZER_CHECKS=1
Bug: 198098397
Change-Id: Ic801ffa802b42d4d55ae89553073c926f9eaa351
2021-09-07 10:22:56 -07:00
Chih-Hung Hsieh
04a5dabf9a Add config/tidy.go to per-file owners.
Test: make
Change-Id: I01b7bcd2cb5a766baf7ea8015996b4a782395875
2021-09-07 10:20:49 -07:00
Liz Kammer
7cbf7738f2 Cleanup unused code for bp2build conversion.
This code to support bp2build-specific modules is no longer necessary as
we do not create bp2build modules after https://r.android.com/1792714

Test: build/bazel/ci/bp2build.sh
Change-Id: I20d3c42f7e3516554cfb537fef05ba40d222dbeb
2021-09-07 13:19:21 -04:00
Linzhao Ye
a0dc31d4a2 Merge "Add android.hardware.wifi.hostapd-V1-ndk_platform to VNDK list." 2021-09-07 17:15:33 +00:00
Lukacs T. Berki
ffc9e8d812 Pass StopBefore as an argument to RunBlueprint.
Its value is a function of the call site, so it doesn't make a lot of
sense to plumb it through the configuration.

Test: Presubmits.
Change-Id: If928b34de075969fd42932212ce9187808cbdf86
2021-09-07 18:14:07 +02:00
Nikita Iashchenko
337b47b81c Merge "Add jdk.internal.math to package_allowed_list.txt (OpenJDK 11)" 2021-09-07 14:22:51 +00:00
Lukacs T. Berki
7edd91c0f4 Make incremental queryview builds work.
The breakage was because:
- aosp/1818236 removed the "rm -fr" invocation from the Ninja file that
removed the old queryview tree
- queryview files were created with permissions 440 to discourage manual
editing

These two together made incremental queryview invocations not be able to
overwrite its output files.

(Of course, it also made them incorrect since old files were lft in place)

Test: Presubmits.
Change-Id: Idaa3e1db82c9eb8325c48e37aca5146e69af7333
2021-09-07 08:47:21 +02:00
Nikita Iashchenko
15c1387710 Add jdk.internal.math to package_allowed_list.txt (OpenJDK 11)
This is part of merging upstream changes from OpenJDK 11.28. This CL
adds jdk.internal.math to package_allowed_list.txt

Bug: 199067710
Test: atest CtsLibcoreOjTestCases
Change-Id: I343bed53ce6b52a6b373a75828b3234073b9cba8
2021-09-06 17:38:34 +01:00
Lukacs T. Berki
3a82169a55 Invoke queryview from the bootstrap Ninja file.
It used to be invoked from out/soong/build.ninja, which required two
soong_build invocations one after the other (ne to generate
out/soong/build.ninja, one to generate the queryview workspace). This
was slower and required some shell-quoted-in-ninja-quoted-in-Go .

Test: Presubmits.
Change-Id: Idda79c067606663b66e9f94626fa24f3b5af4114
2021-09-06 17:10:49 +02:00
Lukacs T. Berki
a1b9372ef7 Make running soong_build in alternate modes nicer.
This includes the JSON graph generator and bp2build.

Before:
GENERATE_BAZEL_FILES=1 m nothing
GENERATE_JSON_MODULE_GRAPH=1 m nothing

Now:
m json-module-graph
m bp2build

They can now also be combined with other targets or each other.

The longer-term goal is to run "m queryview" and "m soong_docs" using
the same infrastructure. There are two alternate approaches:

1. Call soong_build from within the main Ninja invocation. This requires
two sequential soong_build invocations and is thus slower.
2. Do everything requested in the same soong_build invocation. This
would be faster, but one AFAIU can't tell Ninja that multiple possible
actions can build the same output so that doesn't work.

(1) is somewhat more desirable because soong_docs seems to be built
from build/make/core/main.mk ; I assume that that can be worked around
although I haven't checked where the output of "m soong_docs" goes.

Test: Presubmits.
Change-Id: If5ba36490d9f3f60733e6d6be9286eb2b67c3ff5
2021-09-06 16:03:51 +02:00
Treehugger Robot
bd59c1490d Merge "Additional heuristics: variables with names ending with _LIST are lists" 2021-09-06 00:09:39 +00:00
Treehugger Robot
bdd7b3de94 Merge "Fix typo in exempt from licenses property list." 2021-09-03 23:29:13 +00:00