Commit graph

47540 commits

Author SHA1 Message Date
Cole Faust
f8a4bb6d7f Merge "Allow variable-prefixed include statements in mk2rbc" 2022-02-01 18:44:28 +00:00
Gurpreet Singh
ac0a9b00a4 Merge "Add testOnly attribute to AndroidManifest file of apex_test" 2022-02-01 14:25:16 +00:00
Gurpreet Singh
75d65f360c Add testOnly attribute to AndroidManifest file of apex_test
If the build file contains the apex_test module, add the
testOnly attribute to the application element of the
corresponding AndroidManifest file and set its value to true.
If the testOnly attribute is already present and has value
false, then do nothing.

Tests added in manifest_fixer_test.py to check if the updated
AndroidManifest file has the testOnly attribute set to true or not.

Bug: 213310150
Test: atest --host manifest_fixer_test
Test: m nothing
Test: manually checked the AndroidManifest file generated
Change-Id: I36247dbe0261c342d451a4422c314fd8fe0c2369
2022-02-01 12:03:48 +00:00
Treehugger Robot
6a21d959e5 Merge "Remove variableDefinedExpr" 2022-02-01 04:16:02 +00:00
Usta Shrestha
aab5e87431 Merge "redundant statement" 2022-02-01 02:17:46 +00:00
Yuntao Xu
844d8f52d6 Merge "Add LOCAL_GENERATED_SOURCES to androidmk" 2022-02-01 00:12:26 +00:00
Cole Faust
069aba64f6 Allow variable-prefixed include statements in mk2rbc
mk2rbc was already searching the whole android tree
for Makefiles, so allowing variable-prefixed include
statements doesn't affect performance on the file searching
front. On the generated code front, there's already a cap
of 150 potentially-included makefiles that prevents the
performance from getting too bad, and it can be lowered
if necessary.

Bug: 213508006
Test: go test
Change-Id: I3a4e81acb3d97bee08ac3dbe63052a274acf5793
2022-01-31 15:35:28 -08:00
Treehugger Robot
97cffec54e Merge changes from topic "sdk_repo_license"
* changes:
  Generate sdk_repo notice files from license metadata
  Annotate more java dependencies for licensing
  Track transitive packaged license deps of containers
2022-01-31 23:34:59 +00:00
Rob Seymour
4012047331 Merge "Fix host snapshot dependency cycle" 2022-01-31 23:26:39 +00:00
Rob Seymour
0800ef7007 Fix host snapshot dependency cycle
Test: m host-snapshot
Bug: 216351382
Change-Id: Iaa8b7e8373fab0db931cbe7a6328c2adca27d95b
2022-01-31 20:38:00 +00:00
Usta Shrestha
36a0a138ab Merge "cosmetic: 1)typos 2)parameters named for clarity" 2022-01-31 20:19:47 +00:00
Yuntao Xu
680478266b Add LOCAL_GENERATED_SOURCES to androidmk
Added the LOCAL_GENERATED_SOURCES support to the androidmk tool.

Bug: 216527541
Test: TreeHugger
Change-Id: I05b7aa4be6b50e405ec8ec15765d375a4626cc48
2022-01-31 11:54:43 -08:00
Kousik Kumar
dedd43c7b9 Merge "Support fetching config files from experiments fetcher binary" 2022-01-31 18:34:33 +00:00
Treehugger Robot
b8df4b890c Merge "Adding metalava team to OWNERS to be able to disable metalava checks that will break platform build." 2022-01-31 17:49:47 +00:00
Paul Duffin
a54016c3ba Only output annotations properties in snapshots for T+
Marks the `scopeProperties.AnnotationsZip` property as only being
supported on T builds and above.

Bug: 204763318
Test: m nothing
Change-Id: Ie59396287c08da77a6a1f15d4be202429e011e17
2022-01-31 12:28:04 +00:00
Paul Duffin
545c59273d Refactor build_release and test code
Minor restructuring of the build_release pruning code to make it easier
to add logic for handling fields of maps of structs. That includes:
1. Moving some code that is specific to clearing a selected field
   inside the associated if block.
2. Replacing an if with a switch.
3. Improving the error handling by separating the reporting of the
   container that broke from information about which field could not be
   set. That allows follow up code to provide information about the map
   key instead.

The tests were restructed by:
1. Switching from using AssertDeepEquals to compare the structs to
   comparing the output of marshalling the structs to JSON. That was
   for a couple of reasons. Firstly, because JSON will marshal (and so
   allow comparison of) the contents of pointers to structs whereas
   AssertDeepEquals will just compare the pointers themselves.
   Secondly, because JSON can pretty print the output and make it
   easier to read.
2. Using a func to create a new instance of the input structure for
   each test. That is to allow the test to modify the input structure,
   e.g. by clearing a field in a struct that is pointed to by a map.
   The test previously relied on the input structure being immutable
   and passed by value but a follow up change will change that by
   adding a map field that contains pointers to structs.

Bug: 204763318
Test: m nothing
Change-Id: I84dc99621497b7263e30466895b823eb02cb2b56
2022-01-31 12:28:04 +00:00
Paul Duffin
106a3a4bec Allow pruning of unsupported fields in structs in maps
Adds support for traversing into a field that is of type:
   map[...]*struct{...}

This is needed to allow java_sdk_library to mark scope specific
properties, e.g. public.annotations as being target build release
specific.

It was necessary to change the Scope field from:
   Scope map[*apiScope]scopeProperties
to:
   Scope map[*apiScope]*scopeProperties

That is because there is no way in go to change the field of a struct
value of a map. i.e. you cannot do the following, not even using
reflection:
   Scope[apiScopePublic].AnnotationsZip = nil

Bug: 204763318
Test: m nothing
Change-Id: Id103f70f55d4202971321ef4925cbec4b55f8136
2022-01-31 12:28:04 +00:00
Kousik Kumar
5ddefa3df4 Merge "Move env var config loading to be within the config package." 2022-01-31 04:40:50 +00:00
Kousik Kumar
84bd5bf85c Support fetching config files from experiments fetcher binary
The expfetcher binary (added in
https://critique.corp.google.com/cl/424076941), can fetch experiments
configuration from CDPush and load them into Soong as part of its
startup. If the binary exists, then we use it to fetch experiments
config and put them in the out directory so that those env variables can
be loaded by Soong during startup and inturn control whether RBE should
be enabled or not.
Design doc: go/rbe-android-dev-gradual-rollout

Test:
1. time lunch aosp_bramble-userdebug (first lunch when "out" dir doesn't
   exist) - https://paste.googleplex.com/5824218777255936 - took ~9s and
   ran with RBE enabled.
2. time lunch aosp_bramble-userdebug (when "out" dir already exists) - https://paste.googleplex.com/6485045129773056 -
   took ~3.5s and ran with RBE enabled.
3. time lunch aosp_bramble-userdebug (when
   "vendor/google/tools/soong/expfetcher" doesn't exist) - https://paste.googleplex.com/6103083353374720 - took 6.5s
   and not RBE enabled.

Bug: b/215181607
Change-Id: Ie3c085498c59929119534aa98863566eecb8e4eb
2022-01-29 16:59:04 -05:00
Romain Jobredeaux
ea64e62dab Merge "Basic bp2build converter for java_import." 2022-01-29 20:05:08 +00:00
Romain Jobredeaux
428a366703 Basic bp2build converter for java_import.
This only supports the `jars` property/attribute.

Test: b build external/error_prone:error_prone_core_jars
Bug: 215229744
Change-Id: If0d9c8c4e9c1c560d35a501c20bef9361ef45e15
2022-01-29 09:44:37 -05:00
Usta Shrestha
e6b43abd62 redundant statement
Test: m nothing
Bug: n/a
Change-Id: Ida33234b5cf6d915a2d136d8bc0aa6684bfd33c0
2022-01-29 06:37:10 +00:00
Usta Shrestha
c725f47f5d cosmetic: 1)typos 2)parameters named for clarity
Test: n/a
Bug: n/a
Change-Id: I49faa1d8dec4b729409a45e87212b271ecf9e9d1
2022-01-29 06:34:53 +00:00
Usta Shrestha
b3af19caa2 Merge "list all valid flags (commands) to soong_ui upon unknown command" 2022-01-29 06:34:19 +00:00
Usta Shrestha
defcf8d227 Merge "name parameters for clarity" 2022-01-29 06:33:55 +00:00
Treehugger Robot
fd8f0ffee0 Merge "Fix the names of the dynamically inherited modules." 2022-01-29 04:35:39 +00:00
Treehugger Robot
a038c2f269 Merge "Make the Apache 2.0 text available to packages." 2022-01-29 01:41:16 +00:00
Sasha Smundak
845cb29c2b Fix the names of the dynamically inherited modules.
The module name passed to rbld.inherit should be its path without the suffix.

Bug: 215182113
Test: internal
Change-Id: Ic65a5b73037be84f31f8db29f71f793b6c6034bb
2022-01-28 17:41:15 -08:00
Colin Cross
aa1cab0a62 Generate sdk_repo notice files from license metadata
Use the new license metadata files to generate the notice files
for sdk_repo modules.

Bug: 207445310
Test: m sdk_repo_build_tools
Change-Id: I8079061fbdf7417c94eebbb1b31486d3f506f931
2022-01-28 15:56:07 -08:00
Colin Cross
ce56425a6b Annotate more java dependencies for licensing
Annotate more dependencies as runtime linked or toolchain.

Bug: 207445310
Test: m checkbuild
Change-Id: Ia5dc3321a1e476b16058eee94d6dc494fe1e933e
2022-01-28 15:42:06 -08:00
Colin Cross
aff21fbf3c Track transitive packaged license deps of containers
Containers generally package the transitive installable
dependencies of their direct dependencies, track them as license
deps.

Bug: 207445310
Test: m checkbuild
Change-Id: Ic8640152cee0e0cfec5e85a1649a8adfd29d517a
2022-01-28 15:42:01 -08:00
Treehugger Robot
7afffe6af0 Merge "Allow UWB to use Rust in reference HAL" 2022-01-28 22:08:36 +00:00
Treehugger Robot
52ef4c3373 Merge "Limit JIT tier for javac" 2022-01-28 21:59:05 +00:00
Bob Badour
45923719be Make the Apache 2.0 text available to packages.
It might be useful to associate other library names than "Android"
with the Apache 2.0 license text.

Test: m nothing
Change-Id: I3ccbd9ca77f330bb81bf2bf4ec53cfb873befe8b
2022-01-28 12:55:32 -08:00
Cole Faust
71514c07d2 Remove variableDefinedExpr
VariableDefinedExpr was under-developed, and would not
take into account if a variable was from the globals
or product config dictionary.

It also always emitted `g.get("VARIABLE") != None`, which
is not correct behavior. In this example makefile:

```
MY_VAR :=

ifdef MY_VAR
$(info MY_VAR is defined)
else
$(info MY_VAR is not defined)
endif

ifdef MY_UNDEFINED_VAR
$(info MY_UNDEFINED_VAR is defined)
else
$(info MY_UNDEFINED_VAR is not defined)
endif

MY_VAR ?= true
MY_UNDEFINED_VAR ?= true


$(info MY_VAR after ?= is $(MY_VAR))
$(info MY_UNDEFINED_VAR after ?= is $(MY_UNDEFINED_VAR))


.PHONY: all
all:
	@:
```

We get the output:

MY_VAR is not defined
MY_UNDEFINED_VAR is not defined
MY_VAR after ?= is
MY_UNDEFINED_VAR after ?= is true

So we can see that even if a variable was set, it's considered
not defined if it was set to an empty value. However, ?= works
differently, and does require the != None, so that was left
as is.

Just use a variableRefExpr and rely on the fact
that variables will be truthy if they're defined.

Fixes: 216700361
Test: go test
Change-Id: If8944da2579e8658e3fc4f666b1f3b2815f8c8b1
2022-01-28 19:47:34 +00:00
Jared Duke
0849c14aea Limit JIT tier for javac
javac is generally a short-lived JVM invocation that doesn't greatly
benefit from deep JIT tiering. As such, favor faster startup time
by limiting to tier 1 JIT for javac execution. This should greatly
reduce overall CPU usage from Java compilation for an arbitrary
Android build (~20%), and deliver a small but modest improvement in
overall Java build times (~5%).

Representative incremental build times with this change:
  * real 18m3.507s
  * user 563m4.838s
and without:
  * real 19m11.909s
  * user 751m27.790s

Follow-up changes will apply similar flags for other short-lived JVM
build commands.

Bug: 205303544
Test: m
Change-Id: Ic6315b976063f4330902930a6cf5e5cb394c554c
2022-01-28 11:39:47 -08:00
Sam Gilbert
e607988f01 Adding metalava team to OWNERS to be able to disable metalava checks that will break platform build.
Change-Id: Ic560361afc8d982422beef33049278bf480e3964
2022-01-28 18:34:08 +00:00
Jiakai Zhang
e6e90db00a Allow installing boot images outside of APEX for prebuilt.
This change is similar to aosp/1947127, but for prebuilts.

After this change, if `bootImageConfig.installDirOnDevice` is set to a
path outside of the APEX, the build system will build a boot image from
the dex files and the profile extracted from the prebuilt APEX.
Otherwise, it keeps the current behavior: extracting the boot image from
the prebuilt APEX.

This is a no-op change. Current behavior is not affected.

Bug: 211973309
Test: m nothing
Test: -
  On internal master:
  1. Patch aosp/1947128.
  2. Patch ag/16743847 and ag/16746804.
  3. m SOONG_CONFIG_art_module_source_build=false com.google.android.art
  4. See the boot image being installed in `/system/framework/<arch>`.
Change-Id: I24ca525309fecaf3ab7a67960fbf118cd00ecd1d
2022-01-28 15:08:06 +00:00
Keyi Gui
0107901dec Merge "Add new file name to output inputs/outputs of actions of modules into a new file when m json-module-graph is executed." 2022-01-28 14:39:31 +00:00
Pedro Loureiro
e4c255cb04 Merge "Add special case for test-only module in configuredJars method" 2022-01-28 09:50:36 +00:00
kgui
67007248a5 Add new file name to output inputs/outputs of actions of modules into a new file when m json-module-graph is executed.
And also delete the logic in droidstubs to output JSONDataActions.

Test: local.

Change-Id: Ib5d6f1f69c16c41f128e481131775c7699f96464
2022-01-28 13:52:50 +08:00
Treehugger Robot
c7afdefa26 Merge "Add option to generate BTF debug info for bpf programs" 2022-01-28 01:51:05 +00:00
Jiakai Zhang
87847baf41 Remove the primary boot image from the ART APEX.
Move to `/system/framework`.

Bug: 211973309
Test: m
Change-Id: I80ffd9160654a2de96038250b9e9a28ecdba0a81
2022-01-27 23:17:24 +00:00
Chih-hung Hsieh
7892c6d76d Merge "Add tidy.md document" 2022-01-27 23:07:25 +00:00
Treehugger Robot
cd22b62add Merge "Revert "Reland "Enable hwasan use after scope detection.""" 2022-01-27 22:54:24 +00:00
Treehugger Robot
428d5ad909 Merge changes I728208ed,Ifdb50809
* changes:
  [cc] Add per-file OWNERS for cc/coverage.go
  [cc/coverage] Override/disable -Wframe-larger-than
2022-01-27 22:17:29 +00:00
Treehugger Robot
6bf833f428 Merge changes from topic "musl_rust"
* changes:
  Support building rust modules against musl libc
  Support genrules as CrtBegin and CrtEnd in rust
  Support multiple crtbegin and crtend dependencies
2022-01-27 21:39:18 +00:00
Connor O'Brien
2573965c5e Add option to generate BTF debug info for bpf programs
Add "btf" option that generates BTF debug info to support easier map
inspection. This is accomplished by passing the "-g" flag to clang
when compiling the BPF program.

The "-g" option also generates a number of DWARF debug sections which
are not necessary for loading BTF information, so strip these to avoid
increasing file size unnecessarily. bpfloader currently only supports
BTF info for maps, not programs, so we also strip the .BTF.ext section
containing program BTF info.

Bug: 203823368
Test: libbpf_load_test
Test: verify time_in_state.o includes .BTF section iff "btf: true" is
set
Test: verify time_in_state.o still loads if BTF is enabled
Change-Id: Ica25b253bace59d04130b0210350188399889bbe
Signed-off-by: Connor O'Brien <connoro@google.com>
2022-01-27 12:00:45 -08:00
Chih-Hung Hsieh
22c27e8458 Add tidy.md document
Test: make
Change-Id: Ic2198532be4135401faffdda34d0e108c57b241c
2022-01-27 11:02:17 -08:00
Pirama Arumuga Nainar
0d7d3e1647 [cc] Add per-file OWNERS for cc/coverage.go
Test: N/A
Change-Id: I728208ed9ef10e032fd12e495ba67121e4c849e5
2022-01-27 10:49:13 -08:00