Commit graph

1559 commits

Author SHA1 Message Date
Jingwen Chen
7d7f94547a symlink_outputs: Add .KATI_SYMLINK_OUTPUTS to build/make/core
Test: m
Bug: 160568334

Change-Id: Ie6fb4f16bb77845787d277857e17e6aac0a8c11a
2020-10-21 08:55:10 +00:00
Kousik Kumar
92f32615ed Revert "Remove env-var-allowlist"
Revert "Add -fdebug-compilation-dir option"

Revert submission 1461902-debug-compilation-dir

Reason for revert: "-Xclang" isn't being uniformly respected everywhere. For example, in ".S" compilations, when I pass `-Xclang -fdebug-compilation-dir=.", the assembler seems to be ignoring it and then inserting the `pwd` into the command, however when I pass "-fdebug-compilation-dir=.", it strips out the path to the current working directory.
This indicates that we need to update re-client's input processor so that we can pass -fdebug-compilation-dir=. without "-Xclang" and then remove `PWD` setting.
For now, I'll update this patch to pass both "-fdebug-compilation-dir=." and `PWD` and when RBE side fix is done, I'll remove `PWD` in a separate CL.

Reverted Changes:
Ib0f271e55:Add -fdebug-compilation-dir option
Ifa0592af5:Remove env-var-allowlist

Change-Id: I7c690b3e00d37dbcc8fbaa66dda49f39032be3ab
2020-10-20 05:52:49 +00:00
Kousik Kumar
405847ac19 Remove env-var-allowlist
We no longer need to set PWD variable with the introduction of
`-fdebug-compilation-dir`, so removing env var allowlist of PWD variable
to RBE.

Change-Id: Ifa0592af519b6cc4364ee653f1a2174fd680bac1
2020-10-15 15:53:13 -04:00
Xin Li
37a9f38115 Merge "Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)" into stage-aosp-master 2020-09-01 20:03:45 +00:00
Martin Stjernholm
a24c0fe655 Merge "Preserve symlinks in the Soong prebuilt install steps for host binaries and shared libs." 2020-09-01 10:32:49 +00:00
Martin Stjernholm
49a4e9b52e Preserve symlinks in the Soong prebuilt install steps for host binaries
and shared libs.

If a prebuilt module file is a symlink in the .intermediates tree it is
preserved when copied to the final install location. This preserves
symlinks to prebuilt binaries, so that they are run from their prebuilt
source location and can find libraries in relative paths from there.

Symlinks are converted to absolute, to not fail if they are relative in
the source tree.

Test: m clean && m droid dist checkbuild
  with and without the change, then compare the lists of symlinks in
  the out/ trees
Test: build/soong/soong_ui.bash --make-mode \
    TARGET_PRODUCT=crosshatch_hwasan TARGET_BUILD_VARIANT=userdebug \
    droid SANITIZE_TARGET=hwaddress \
    continuous_instrumentation_tests_api_coverage \
    continuous_native_tests device-tests platform_tests
  check that libclang_rt.hwasan_static-aarch64-android.a isn't a
  symlink in out/target/product/crosshatch/obj
Bug: 145934348
Change-Id: I7600e5b0754f8ea6cd0ffc8e1ba6d39153f182aa
2020-09-01 00:24:54 +01:00
Xin Li
c2fd58fe67 Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: Ie00dbea32e074838466eeed00f316250c2b4f3c2
Change-Id: I9ee3441fd4d063b9dd489cb1b01d3dbc11332457
2020-08-29 01:19:44 -07:00
Jaewoong Jung
046702a6ef Deduplicate my_compat_dist entries globally.
Fixes: 164319611
Test: m TARGET_PRODUCT=aosp_x86_64 mts dist
Change-Id: Ie47e355ded0ce0221988a405fc77546a4a752c38
2020-08-27 13:10:40 -07:00
Colin Cross
6de1a4a273 Merge "Move definition of dist-for-goals before call." am: 80ef0a212c am: f2458a5da7
Original change: https://android-review.googlesource.com/c/platform/build/+/1128914

Change-Id: Ia3736984d5679d017b254a09f86130ffe6df7a68
2020-06-11 20:47:00 +00:00
Colin Cross
80ef0a212c Merge "Move definition of dist-for-goals before call." 2020-06-11 18:36:18 +00:00
Ulyana Trafimovich
5ed9d68b9e Merge "Fix on-device paths to used libraries in dexpreopt." am: cd33e76411 am: c048d40cc9
Original change: https://android-review.googlesource.com/c/platform/build/+/1325806

Change-Id: Ic563dbd7ae789d6a7ad83a4c4dce02053aff3c83
2020-06-11 14:23:22 +00:00
Ulyana Trafimovich
cd33e76411 Merge "Fix on-device paths to used libraries in dexpreopt." 2020-06-11 13:47:19 +00:00
Ulya Trafimovich
2311f61b12 Fix on-device paths to used libraries in dexpreopt.
Also, add function `install-path-to-on-device-path` that converts
its one argument from on-host install path to on-device path by
stripping PRODUCT_OUT prefix.

Test: lunch aosp_cf_x86_phone-userdebug && m
Test: Cherry-pick in internal master and check that on-device path to
    com.google.android.dialer.support.jar now is on /product partition
    (as it should be) and not on /system:
    $ oatdump \
        --instruction-set=x86 \
        --oat-file=out/target/product/vsoc_x86/product/priv-app/GoogleDialer/oat/x86/GoogleDialer.odex \
      | grep '^classpath' \
      | grep -o '[^[]*com.google.android.dialer.support.jar'
    /product/framework/com.google.android.dialer.support.jar

Bug: 132357300
Change-Id: I9bbe45ffbbed09186ac46e067da7ada599ef9678
2020-06-11 10:10:00 +01:00
Treehugger Robot
4c7dc6e511 Merge "base_rules.mk optimizations" am: 8add8cb145 am: ffbee4fbf3
Original change: https://android-review.googlesource.com/c/platform/build/+/1326240

Change-Id: Idd897d57865e3834d3fcd8fb7788b7a2a75e3e1e
2020-06-10 02:53:43 +00:00
Dan Willemsen
8447b8116a base_rules.mk optimizations
A few more misc improvements that I found while analyzing the
performance of base_rules.mk.

This brings an aosp-master/aosp_crosshatch-userdebug kati run from 33.3s
to 28.6s

Bug: 158488548
Test: build-aosp_crosshatch.ninja is the same before/after
Change-Id: If99c31cc7b5d7133d70eb644c6095f19060b71e5
2020-06-09 09:59:24 -07:00
Treehugger Robot
02dcbdb3f9 Merge "More performance improvements" am: 1d1bb3b94f am: 27d4463cf5
Original change: https://android-review.googlesource.com/c/platform/build/+/1326018

Change-Id: Ic783c75beffd9eac7abb208d2f2697f8c4775325
2020-06-09 05:00:29 +00:00
Dan Willemsen
fd1e67b8ce More performance improvements
Instead of using `x := $(x) foo`, make sure `x` is initialized first,
then use `x += foo`.

For a aosp-master/aosp_crosshatch-userdebug build, this takes the build
from 49s to 33s (no significant difference in peak memory)

Bug: 158488548
Test: build-aosp_crosshatch.ninja is the same before/after
Change-Id: I41fb7611030dd3ba824f896a9c9eadbdd218f464
2020-06-08 19:10:23 -07:00
Treehugger Robot
c47b36bbb9 Merge "Remove unused check-api rule" am: fa9ea4b8b0 am: 6231dd8584
Original change: https://android-review.googlesource.com/c/platform/build/+/1322795

Change-Id: Ia66d72bf145458e7b570b47b766a4bc6a4aae164
2020-06-02 19:40:09 +00:00
Aurimas Liutikas
9c5b07966e Remove unused check-api rule
Change-Id: I0297b226d066e99ee4fc4c2b22e160eafc7f5a8a
Test: Treehugger?
2020-06-02 17:01:52 +00:00
Dan Willemsen
f08587c925 Remove AUX support am: 58634e1482 am: c4fb07d615
Change-Id: Ic54cef973b4573886a562690fe0d3c8b35816f4a
2020-04-21 23:11:48 +00:00
Dan Willemsen
58634e1482 Remove AUX support
This was deprecated in R, which has now branched, and there aren't any
users on master.

Test: build-aosp_crosshatch.ninja is the same (except for the removal of the empty auxiliary target)
Test: treehugger
Change-Id: I306156ab7f91cd4a2258554b4215766c99cd12d1
2020-04-18 21:00:18 -07:00
Kousik Kumar
0a8b83d0cb Merge "Use soong_zip instead of jar to generate .jar files in javac actions" am: 1d56417af0 am: 2d1430e51c
Change-Id: Iec9836498edc164265b79034da7a75b588d65057
2020-04-17 22:41:01 +00:00
Kousik Kumar
1d56417af0 Merge "Use soong_zip instead of jar to generate .jar files in javac actions" 2020-04-17 21:51:31 +00:00
Yo Chiang
b301fd5434 Merge "Check ELF prebuilts in PRODUCT_COPY_FILES" am: 2cbcf3bda2 am: 85e68e75f1
Change-Id: Ifb30cc6423b03d9468ec949ba862c61d70085f4f
2020-04-17 01:41:49 +00:00
Kousik Kumar
7d61116cc1 Use soong_zip instead of jar to generate .jar files in javac actions
This is to make the jar-output deterministic so that we can get
cache-hits for r8/d8 actions from RBE.

Change-Id: I911ee702fa5dabfe7122e82b21f78c811a0c960d
Test: m droid checkbuild
2020-04-16 18:23:32 -07:00
Yo Chiang
2cbcf3bda2 Merge "Check ELF prebuilts in PRODUCT_COPY_FILES" 2020-04-17 01:06:32 +00:00
Colin Cross
d77b244a63 Merge "Revert "Revert "Use sdk variant of Soong modules when LOCAL_SDK_..."" am: 65ca0eec96 am: fc85e271c3
Change-Id: I25b78883836b0f189a4106698e6bf0772a8ab00c
2020-04-08 18:42:10 +00:00
Colin Cross
65ca0eec96 Merge "Revert "Revert "Use sdk variant of Soong modules when LOCAL_SDK_..."" 2020-04-08 18:29:30 +00:00
Yo Chiang
0b49c03727 Check ELF prebuilts in PRODUCT_COPY_FILES
Add a check to ban ELF prebuilts in PRODUCT_COPY_FILES.
ELF prebuilts should be defined properly by cc_prebuilt_library_shared
and cc_prebuilt_binary so that the build system can keep track of the
file dependencies.

To opt in this check, set this option in BoardConfig.mk:
  BUILD_BROKEN_ELF_PREBUILT_PRODUCT_COPY_FILES := false

After all devices that have ELF prebuilts in PRODUCT_COPY_FILES are
tagged with BUILD_BROKEN_ELF_PREBUILT_PRODUCT_COPY_FILES := true flip
BUILD_BROKEN_ELF_PREBUILT_PRODUCT_COPY_FILES to default false.

After a device is cleaned up, remove its
BUILD_BROKEN_ELF_PREBUILT_PRODUCT_COPY_FILES := true setting.

Bug: 140560012
Test: Set BUILD_BROKEN_ELF_PREBUILT_PRODUCT_COPY_FILES := false
Test: m # Should report any ELF prebuilt error in PRODUCT_COPY_FILES
Change-Id: Id75dac9417aed9baa7c5e9388430ff9369cf9b4f
2020-04-08 18:01:24 +08:00
Colin Cross
e0c5e44360 Revert "Revert "Use sdk variant of Soong modules when LOCAL_SDK_..."
Revert^2 "Add sdk mutator for native modules"

f8e80229fedb47302e9cfd32990859a6308020cf

Change-Id: Ib686b52339ae5031434a2fb6a0e7f5b0c0dc5641
2020-04-07 16:50:32 +00:00
Colin Cross
a7edce5629 Merge "Revert "Use sdk variant of Soong modules when LOCAL_SDK_VERSION ..."" am: 189a925c8b am: a6be60566f
Change-Id: I467b0b21da0b2078aadee6a09ada0a34f5cca1c8
2020-04-07 04:57:37 +00:00
Colin Cross
79e5a55122 Revert "Use sdk variant of Soong modules when LOCAL_SDK_VERSION ..."
Revert "Add sdk mutator for native modules"

Revert submission 1242911-sdk_version_variant

Reason for revert: b/153394225
Reverted Changes:
Ife99745fb:Use libnativewindow for platform variant of libagq...
I1bae84c43:Use libnativewindow for platform variant of androi...
I6e6021ed3:Use stl to depend on libc++
Ife99745fb:Use libnativewindow for platform variant of libRSS...
I2c9f439b9:Fix static dependency on libprotobuf-cpp-lite-ndk
Iff2aff9cf:Set sdk_version for cc_genrules used by modules wi...
I7d72934aa:Add sdk mutator for native modules
Ief378a007:Use sdk variant of Soong modules when LOCAL_SDK_VE...

Bug: 149591340
Change-Id: I6cd4de221ece29e48d58a8b1297dc2512b2dad13
Fixes: 153394225
2020-04-07 04:21:21 +00:00
Colin Cross
2932e686eb Merge "Use sdk variant of Soong modules when LOCAL_SDK_VERSION is set" am: a2af96b642 am: 4e5fddd0b5
Change-Id: I79b5451254c14dcff631e5dcf5c778c0b6738fcd
2020-04-07 02:35:39 +00:00
Jaewoong Jung
95445e6913 Add signing certificate lineage file support.
Also add multi-cert support to prebuilt apps so that they can benefit
from the new lineage feature.

(This is a cherry-pick change.)

Test: m GoogleServicesFramework w/ modified build rules
Test: m PrebuiltGmsCore w/ modified build rules
Test: apksigner lineage -v --print-certs -in <built_module_path>
Fixes: 152897457
Change-Id: If7d5d4bd308629c8340231520214c76c8a568a65
Merged-In: If7d5d4bd308629c8340231520214c76c8a568a65
2020-04-06 13:42:23 -07:00
Jaewoong Jung
d47e378af2 Add signing certificate lineage file support. am: e2348338e4
Change-Id: Id142f70b02ae46e734ff788e0cc5186138103df6
2020-04-06 20:01:02 +00:00
Jaewoong Jung
e2348338e4 Add signing certificate lineage file support.
Also add multi-cert support to prebuilt apps so that they can benefit
from the new lineage feature.

Test: m GoogleServicesFramework w/ modified build rules
Test: m PrebuiltGmsCore w/ modified build rules
Test: apksigner lineage -v --print-certs -in <built_module_path>
Fixes: 152897457
Change-Id: If7d5d4bd308629c8340231520214c76c8a568a65
2020-04-06 10:19:15 -07:00
Colin Cross
b934116994 Use sdk variant of Soong modules when LOCAL_SDK_VERSION is set
Soong now makes a variant of native modules that set sdk_version.
Use the new variant for native modules or apps with JNI that are
defined in Make and set LOCAL_SDK_VERSION.

Test: m checkbuild
Bug: 149591340
Change-Id: Ief378a007e43b0aea31fd5845410bbffec0ffae6
2020-03-24 10:48:24 -07:00
Jaewoong Jung
39f3811084 Merge "Fix test suite data files issue." am: c0d90ac9ac am: b742f058db
Change-Id: I16973462f2bd1b7eb1271289830808da45f36435
2020-03-20 18:34:48 +00:00
Jaewoong Jung
b5aa16d44f Fix test suite data files issue.
The current test suite rule implementation has a bug where test data
files are excluded from the final suite zip files when a suite doesn't
have a designated testcases output directory, which is represented by
COMPATIBILITY_TESTCASES_OUT_$(suite_name).

Specifically, the rule that collects test data files for each suite
filters out suite output paths that conflict with module output paths,
which happens when a suite doesn't have its own testcases output
directory. It seems to have been done this way just to avoid make rule
conflict errors, but it ended up causing missing test data files.

This change fixes it by adding an indexing purpose only test suit file
variable.

Fixes: 140761783
Test: m general-tests
Test: Compared before vs after, no meaningful differences.
Test: Changed bc-tests to a device test and built general-tests.
Test: Before - data files missing, after - data files present
Change-Id: I2008992f4144c6ee115f4b4e4ac9caa4312bbe34
2020-03-19 06:47:35 -07:00
Jiyong Park
79bf9640f4 system_server_current is accepted
Previously, has-system-sdk-version incorrectly returned true for
the sdk version 'system_server_current', just because it started with
system_. Since system_server_current is a totally different API surface
than system_*, make the macro to return false for system_server_*.

Bug: 146757305
Test: m
Test: add 'sdk_version: "system_server_current" to wifi-service-pre-jarjar
Change-Id: I43700545b0f120fa987696b8f03a4765d97e13e2
2020-03-07 18:34:57 +09:00
Ramy Medhat
c3427499ca Modify D8 template to not have inputs under the output directory.
Test: presubmit

Change-Id: I800664acdeffa04ce83a8446bece411f018af2e2
2020-02-13 07:19:53 -05:00
Ramy Medhat
a9cc25d785 Add RBE support for Javac, R8, and D8 if respective env vars are set.
Test: aosp_crosshatch build with and without the new variables.
Change-Id: I252ff5dd8e372edf8c86e25348c8a2bfab41d70d
2020-01-28 10:25:25 -05:00
Peter Collingbourne
b1d7545e1b Merge "Update make build system for dynamic unwinder." 2020-01-21 21:28:44 +00:00
Peter Collingbourne
86cdf9c2f4 Update make build system for dynamic unwinder.
Change the unwinder linking logic to match soong.

Bug: 144430859
Change-Id: I739980e479d14707b7a3afd6e9d2c817c9b43f81
2020-01-10 11:09:52 -08:00
Dan Willemsen
38dc09d39a Use our prebuilt M4 for flex and bison
And ensure we've got all the proper dependencies.

Bug: 117561006
Test: treehugger
Change-Id: Ia9690b2c0d73a48744c8e33fe83196d02b1e904d
2020-01-08 22:31:12 -08:00
Dan Willemsen
b195e7ab04 Use symlinks in the build graph for jni libs
Now that ninja uses lstat and can support installing arbitrary symlinks,
switch jni lib symlinks from LOCAL_POST_INSTALL_CMDS to real rules.

Bug: 128577186
Test: List of files under PRODUCT_OUT is the same before/after this change
Test: out/target/product/generic/.installable_files now includes the symlinks
Test: m installclean; m NfcNci -> symlinks installed with correct dest
Test: m NfcNci; m NfcNci -> ninja: no work to do
Change-Id: I078dca53ab3d93f74c36fa66d5577e6e3e0640d6
2020-01-06 10:25:56 -08:00
Dan Willemsen
dec6e8e056 Export list of "installable" files to soong_ui
These are a (partial) list of files that we'd install with a default
build. The idea is that if something is removed from this list, soong_ui
can remove it from the installed location before running ninja.

It's okay if there are things missing from this list, it's not intended
to be a 100% solution replacing installclean / CleanSpec.mk, just
something that handles 80% of the cases without user involvement.

In particular, if something is removed from PRODUCT_PACKAGES, we'll
remove it from disk, but not necessarily rebuild the image files. That's
the same as most use cases of CleanSpec.mk today, and often some other
change will trigger the necessary images to be rebuilt.

We should be able to fix that by changing all of the image creation
rules to depend on the (partial) list of files they care about, or by
fixing ninja to rebuild things when their list of dependencies change.
(Other tools run into this same problem)

The list of test files is also included so that we can remove obsolete
tests from their "installed" locations within test suites and the
testcases folders.

Test: remove a module from PRODUCT_PACKAGES, see the print and file removed
Test: change the name of a cts test, see the old one removed from cts
Change-Id: I67f270a6713369099ca523aaf991ee3beb815c0a
2020-01-03 19:19:11 -08:00
Colin Cross
9d789145fa Merge "Set pools manually for rules that can run in RBE or goma" 2019-11-15 18:19:33 +00:00
Tom Cherry
3c697d59fb Merge "Check property type in host_init_verifier" 2019-11-13 18:41:17 +00:00
Colin Cross
23dcf76a36 Set pools manually for rules that can run in RBE or goma
Use .KATI_NINJA_POOL := none to mark rules that can run in RBE or
goma when they are enabled, which along with passing
--default_pool=local_pool will allow moving remoteable jobs into
the remote pool without relying on hacks in kati.

Fixes: 143938974
Test: inspect pools in build-${TARGET-PRODUCT}.ninja for m USE_RBE=true
Test: inspect pools in build-${TARGET-PRODUCT}.ninja for m USE_GOMA=true
Change-Id: I08615ae1f2eaef9ff0ba253b8aa3d83ab57ebb27
2019-11-12 13:59:49 -08:00