Commit graph

27916 commits

Author SHA1 Message Date
Jingwen Chen
3a4b58dc5d Set PATH and --action_env=PATH for the Bazel build in soong_ui.
Ensures that Bazel and Ninja are using the same binaries from PATH, such
as `sort` from the linked bug.

Test: USE_BAZEL=1 m
Test: rm out/soong/apex/depsinfo/new-allowed-deps.txt && USE_BAZEL=1 m
Bug: 161074854
Fixes: 161074854
Change-Id: Icc3e717e6d3cfcecfe90f0fb3a94a86485a11918
2020-11-16 04:47:14 -05:00
Jooyung Han
74f576de58 Remove "apex_uses" mutator
which was added to support built-time common apex. But the common apex
was abandoned and it is used nowhere.

This is a manual revert of 5c998b9ff8.

Bug: 173329465
Test: m
Change-Id: I684e6b8bfe2ef5b53943329f4aaa5a5ccba27738
2020-11-16 09:08:30 +00:00
Colin Cross
cf371cc1f7 Replace android.WriteFile rule with android.WriteFileRule
The android.WriteFile rule takes careful escaping to produce the
right contents.  Wrap it in an android.WriteFileRule that handles
the escaping.

Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
2020-11-14 16:26:00 -08:00
Colin Cross
62a0cfd054 Annotate dependency tags for dependencies of installed files
Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.

Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.

This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.

Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: Ic22603a5c0718b5a21686672a7471f952b4d1017
2020-11-14 16:24:10 -08:00
Colin Cross
a615901b53 Use the the preferred architecture symlink as the tool path if it exists
When a genrule uses a binary with symlink_preferred_arch set as a tool,
use the symlink instead of the binary as the tool, as the symlink is
the more canonical name for the binary.

Test: m checkbuild
Change-Id: I5330b57139f03ca07bf70547261f191ed84ec9b8
2020-11-14 14:38:34 -08:00
Jingwen Chen
1d26d3033a Merge "Add 'pre-production' description to queryview action." 2020-11-14 08:05:35 +00:00
Treehugger Robot
ceed6be00d Merge "Pass Config to NewTestContext instead of ctx.Register" 2020-11-14 05:58:58 +00:00
Patrice Arruda
33e7f6a0ca Remove unnecessary break and default statements in metrics.go file.
break and default statements are not needed inside of a switch
statement.

Bug: b/173028918
Test: USE_BAZEL m nothing
Change-Id: I14aa8cbae3a9b3c859f147c5134600f1e75c6a89
2020-11-13 13:15:09 -08:00
Patrice Arruda
b7cf9ba175 Add tracer to runBazel function.
Added a tracer named "bazel" in runBazel function to capture topline
metrics when running bazel.

Bug: b/173028918
Test: * USE_BAZEL=1 m nothing and checked soong_metrics file using
        printproto command to verify that bazel topline metrics
	were captured.
Change-Id: I5cb0728a4da48545dc701cc79627c318d5d9a29d
2020-11-13 13:14:51 -08:00
Patrice Arruda
18cb70d645 Enable bazel metrics from ninja build.
Generate the bazel profile file that contains the metrics of
executing the ninja build phase of Soong.

Bug: b/173028918
Test: * m nothing and check out dir that there is no bazel_metrics dir.
      * USE_BAZEL=1 m nothing and checked
        out/bazel_metrics/build_bazel_profile.gz existed.
      * m nothing and checked out/bazel_metrics does not exist.
Change-Id: I797d1eecf5c2c260ade8b55dc69c237bfd0fcbd4
2020-11-13 11:37:06 -08:00
Patrice Arruda
af880da0c4 Define the bazel_metrics directory in $OUT.
$OUT/bazel_metrics directory is created in order to dump the
set of bazel profile files being created from a single build
execution. The directory is created from soong_ui.

Bug: b/173028918
Test: * m nothing and check out dir that there is no bazel_metrics dir.
      * USE_BAZEL=1 and checked out/bazel_metrics was created.
      * Ran m nothing and verified that out/bazel_metrics directory is
        deleted to ensure no metrics is uploaded or collected from the
	pipeline.

Change-Id: I1d3c2471065fae6931f35cef7f0f35fd84f76bef
2020-11-13 11:16:21 -08:00
Patrice Arruda
24a086b3dc Add bazel profile related functions in paths.go
Bazel executions occurs in soong_ui and soong_cmd. To collection all
bazel metrics, several path functions are defined to return the path
to store a bazel profile gzipped file.

Bug: b/173028918
Test: m nothing
Change-Id: I78926c66f853233b3977aca22e8a3fcf28edda53
2020-11-13 11:16:21 -08:00
Patrice Arruda
0c1c456643 Define UseBazel in Config
The UseBazel() function will be used in several places to perform
different functionality such as setting up the metrics path.

Bug: b/173028918
Test: m nothing
Change-Id: Ie94073b8f0552ec7528e1cc2f3be74b693dc3135
2020-11-13 11:16:21 -08:00
Christopher Parsons
9414a8be12 Merge "Stop first pass of mixed bazel builds before writing ninja" 2020-11-13 18:18:07 +00:00
Martin Stjernholm
c488743e14 Set up BuildNumberFile in soong.variables.
Necessary for some build steps. aapt2 is one example, although it still
has problems because part of its build logic is in make.

This isn't strictly required for any supported build target at the
moment, but the error when it fails can be confusing so let's add it
anyway.

Test: build/soong/scripts/build-aml-prebuilts.sh art-module-sdk \
  art-module-test-exports conscrypt-module-sdk \
  conscrypt-module-test-exports conscrypt-module-host-exports \
  runtime-module-sdk runtime-module-host-exports \
  i18n-module-test-exports i18n-module-sdk platform-mainline-sdk \
  platform-mainline-test-exports
Bug: 172548608
Change-Id: I88526a85f288287c3a138f8cbb6fd04ad28c3795
2020-11-13 15:40:17 +00:00
Jingwen Chen
478d033499 Add 'pre-production' description to queryview action.
Based on feedback from leadership.

Test: N/A
Change-Id: I4c0debf53a018578a95f074270c330803ef9529b
2020-11-13 10:06:40 -05:00
Chris Parsons
3060ec7731 Stop first pass of mixed bazel builds before writing ninja
This shaves off ~20s, which is ~11%, of `m nothing` with mixed builds.

Test: Manual verification with mixed builds of aosp_flame.
Change-Id: Ief1bc56c92329e899c829fc1374eea9ee9b94fb7
2020-11-12 22:30:46 +00:00
Colin Cross
ae8600b507 Pass Config to NewTestContext instead of ctx.Register
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.

Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
2020-11-12 10:07:49 -08:00
Ivan Lozano
45e0c95f85 Merge "rust: Fix absolute path OUT_DIR bug" 2020-11-12 15:23:39 +00:00
Jingwen Chen
44db598b3e Merge "Export Ninja file paths to lunch repo rule from bazel.go" 2020-11-12 08:32:50 +00:00
Anton Hansson
fabdd28fdb Merge "Make apex modules support tagged dists" 2020-11-12 08:30:55 +00:00
Jingwen Chen
a26ac3caee Export Ninja file paths to lunch repo rule from bazel.go
This ensures that the lunch repo rule always reads the right Ninja file.

Test: USE_BAZEL=1 m
Test: m

Change-Id: I7f670dd538f46c4dae5ea537b857ff04520b60e7
2020-11-12 08:01:33 +00:00
Zach Johnson
c0ea1701cb Merge "rust_grpcio well known types support, default deps" 2020-11-11 23:22:27 +00:00
Rupert Shuttleworth
eb165fd950 Merge "Perform Bazel cleanup using "--batch --max_idle_secs=1" instead of a trap." 2020-11-11 20:56:10 +00:00
Zach Johnson
3df4e6364b rust_grpcio well known types support, default deps
The usage of the well known type Empty requires a hack in the module
above the grpc implementation, this is now the generated stem_mod.rs

This also adds additional implicit dependencies that are required by
the grpc protobuf generated code. This includes the addition of a
'header_libs' property for library dependencies which export include
paths required by protos.

We also now include both the protos and the grpcio in the library
variant via the mod_stem.rs.

Bug: 172592789
Bug: 171504899
Test: m nothing
Test: Example rust_grpcio module build command includes dependencies,
      include paths.
Change-Id: I187a13cd5cdea991828a1020314de16727e4f74e
2020-11-11 12:09:26 -08:00
Ivan Lozano
1776a2ac6f rust: Fix absolute path OUT_DIR bug
The OUT_DIR for rustc is incorrectly calculated when Soong's OUT_DIR
is set to an absolute path through OUT_DIR_COMMON_BASE. This breaks
compilation when rustc is unable to find files that have beein included
through the include! macro.

Bug: 172952634
Test: OUT_DIR_COMMON_BASE=/some/abs/path m
Change-Id: Id447630a7774c1dec655f65dd227c144a159e1b1
2020-11-11 11:25:48 -05:00
Paul Duffin
68ecbaf614 Merge "Switch boot jars package check to using dex jars" 2020-11-11 12:53:23 +00:00
Anton Hansson
82d502a7b2 Make apex modules support tagged dists
Bug: 172951145
Test: m dist my_dist_target (with patch in bug)
Change-Id: I0da4568a49ea48efbf864614a7e7c647f251a7ce
2020-11-11 12:34:20 +00:00
Mathew Inwood
1ef4ba906a Explicitly tag max-target-o APIs as "lo-prio".
This allows the presubmit checker to explicitly allow certain changes to
these APIs, rather than special casing tha max-target-o list. This in turn
will allow other APIs to be tagged as lo-prio in future to get the same
treatment.

Test: m out/soong/hiddenapi/hiddenapi-flags.csv
Bug: 172993934
Change-Id: Ic7cc0243b2b018c96d420a3f266c4b3a6958c32f
2020-11-11 11:01:07 +00:00
Mathew Inwood
82d5b6726b Merge "Temporarily move APIs to blocked list." 2020-11-11 09:13:48 +00:00
Paul Duffin
2d8e1a7e59 Switch boot jars package check to using dex jars
The switch to use dex jars instead of class jars means that a boot jar
that is defined by a dex_import module will now be checked against the
package_allowed_list.txt so it is possible that it will detect
previously unreported problems.

Test: m check-boot-jars - for failing and passing cases
Bug: 171479578
Bug: 125517186
Change-Id: Ie614898dade0fb43c9418d7afb9138169db6f097
2020-11-10 17:37:43 +00:00
Paul Duffin
99a7622885 Merge "Remove support for droidstubs in sdk/module_exports" 2020-11-10 17:34:00 +00:00
Anton Hansson
a1b3314286 Merge "Remove dead code" 2020-11-10 14:35:13 +00:00
Paul Duffin
a3cb6cf2e5 Remove support for droidstubs in sdk/module_exports
The droidstubs support in sdk/module_exports was a temporary measure
to work around the fact that some APIs were being defined by direct use
of droidstubs instead of java_sdk_library.

This change removes that support as those APIs have all been switched
from droidstubs to use java_sdk_library so droidstubs support is no
longer needed.

Bug: 168301990
Test: m nothing
Change-Id: I3517bed29b030438a0423a6cb8c248992a988222
2020-11-10 13:55:17 +00:00
Paul Duffin
37fa489a9e Merge "Use glob for java_sdk_library_import stub_srcs" 2020-11-10 13:54:23 +00:00
Treehugger Robot
13c1df6a2f Merge "Rust bindgen should use r399163b." 2020-11-10 13:42:40 +00:00
Mathew Inwood
59093e3769 Temporarily move APIs to blocked list.
Bug: 170729553
Test: m

Change-Id: I62db78941785fd1bd336cfe81c8e9763e08a0eb0
2020-11-10 13:08:58 +00:00
Treehugger Robot
ccd3f1d66f Merge "Allow extra startup and build args to be passed to Bazel." 2020-11-10 07:22:34 +00:00
Treehugger Robot
8b16e407e2 Merge "add -nostdlibinc for arm64_linux_host" 2020-11-10 06:52:07 +00:00
Kiyoung Kim
9ce4eec815 Merge "Add new properties in linker config format - provideLibs and requireLibs" 2020-11-10 05:49:35 +00:00
Jiyong Park
b304e805d9 add -nostdlibinc for arm64_linux_host
The flag is needed to prevent the compiler from looking into the host
include paths like /usr/include.

Bug: 172869346
Test: inspect the include path by temporarily adding `-v`.
Change-Id: I73a16155a784916a1bd0babcb75d5f9079e13423
2020-11-10 11:55:42 +09:00
Rupert Shuttleworth
2e5bf406f1 Perform Bazel cleanup using "--batch --max_idle_secs=1" instead of a trap.
Test: Manually.
Change-Id: Id28f504fec8c6f7222646fe822264f0a4a746825
2020-11-10 02:37:24 +00:00
Rupert Shuttleworth
f8ae317d3e Allow extra startup and build args to be passed to Bazel.
For example: USE_BAZEL=1 BAZEL_STARTUP_ARGS="--batch" m

Test: Manually.
Change-Id: I76fa8e17d4a9ccb8dda9fca049113f434dacabbe
2020-11-10 02:06:30 +00:00
Stephen Hines
a3faafa377 Rust bindgen should use r399163b.
Bug: http://b/155835175
Test: m
Change-Id: Iedccd3c2258fab5f9fb1034aace7d07da833a0ec
2020-11-09 16:28:23 -08:00
Paul Duffin
7f97957ded Use glob for java_sdk_library_import stub_srcs
Some java_sdk_library modules do not have any stubs for some API
scopes. That results in an empty ".srcjar" being created for them which
ends up not creating a directory for the stubs sources when that
snapshot is unzipped. Previously, that would cause a build failure as
the generated java_sdk_library_import module used the directory, which
did not exist, in its stub_srcs property.

This change switches the stubs_srcs property to use a glob pattern of
"**/*.java" relative to the directory instead of using the directory
directly. When the directory does not exist the glob pattern is
resolved to an empty set of paths and does not break the build.

Bug: 172811712
Test: Add sdkextensions-sdk sdk module (local patch)
      m sdk-extensions (local patch)
      unpack generated snapshot
      m nothing - results in build failure due to missing directory
      Make this change and repeat above except this time the build works.
Change-Id: I691ffbfdc01ba89bbcaf647dcbb7dfebc3c8aec2
2020-11-09 17:42:52 +00:00
Jingwen Chen
b05d62f584 Mark queryview as experimental in the Ninja description.
Test: m queryview
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I08cd5175416013c13ad150885bb918342e79fcff
2020-11-09 08:22:51 -05:00
Jingwen Chen
8024c95ecc Add TODO bug number to stop hardcoding 'droid'
Test: N/A
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I0d60cc4254b3a358085d89e443bd3bfc33b68ed3
2020-11-09 07:03:56 +00:00
Treehugger Robot
25b0780f37 Merge "Define product_available property" 2020-11-09 00:40:50 +00:00
Justin Yun
63e9ec70bb Define product_available property
To make a module available to product variants, it must define
`product_available: true`. `vendor_available: true` will not create
product variants any more.
However, in this CL, we don't change the behavior of
`vendor_available` property. It still creates both variants. After we
update all Android.bp files that need to provide product variants
with `product_available: true`, we may upload the remaining patches.

Bug: 150902910
Test: lunch aosp_arm64-userdebug && m
Change-Id: I0fd5be7bbae2c45d5cab3c3c2ca49f53a9b6f975
2020-11-08 23:53:22 +00:00
Matthew Maurer
40c9798e92 Merge "rust: Fix GRPC generator invocation" 2020-11-06 19:04:16 +00:00