Commit graph

32 commits

Author SHA1 Message Date
Treehugger Robot
605eb54154 Merge "Revert "Remove env-var-allowlist"" am: fd8dae609f am: fe4c812bad
Original change: https://android-review.googlesource.com/c/platform/build/+/1465913

Change-Id: I8dd41768ef397889d7cdfecf4f65aaec7482c3a4
2020-10-20 21:04:49 +00:00
Kousik Kumar
92f32615ed Revert "Remove env-var-allowlist"
Revert "Add -fdebug-compilation-dir option"

Revert submission 1461902-debug-compilation-dir

Reason for revert: "-Xclang" isn't being uniformly respected everywhere. For example, in ".S" compilations, when I pass `-Xclang -fdebug-compilation-dir=.", the assembler seems to be ignoring it and then inserting the `pwd` into the command, however when I pass "-fdebug-compilation-dir=.", it strips out the path to the current working directory.
This indicates that we need to update re-client's input processor so that we can pass -fdebug-compilation-dir=. without "-Xclang" and then remove `PWD` setting.
For now, I'll update this patch to pass both "-fdebug-compilation-dir=." and `PWD` and when RBE side fix is done, I'll remove `PWD` in a separate CL.

Reverted Changes:
Ib0f271e55:Add -fdebug-compilation-dir option
Ifa0592af5:Remove env-var-allowlist

Change-Id: I7c690b3e00d37dbcc8fbaa66dda49f39032be3ab
2020-10-20 05:52:49 +00:00
Kousik Kumar
afd93ad912 Merge "Remove env-var-allowlist" am: 3a7a960a86 am: c2c28f9941
Original change: https://android-review.googlesource.com/c/platform/build/+/1461902

Change-Id: Id3e7ce2ee695ff9b760f20007c02e3df7779cf94
2020-10-16 06:26:10 +00:00
Kousik Kumar
405847ac19 Remove env-var-allowlist
We no longer need to set PWD variable with the introduction of
`-fdebug-compilation-dir`, so removing env var allowlist of PWD variable
to RBE.

Change-Id: Ifa0592af519b6cc4364ee653f1a2174fd680bac1
2020-10-15 15:53:13 -04:00
Kousik Kumar
0d0d422359 resolve merge conflicts of 602859b186 to rvc-d1-dev am: 3d723c29d1
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/+/12735596

Change-Id: I9a82c7e89561821e912081790557e987757ba5b3
2020-10-15 04:00:29 +00:00
Kousik Kumar
3d723c29d1 resolve merge conflicts of 602859b186 to rvc-d1-dev
Bug: b/169675226
Change-Id: I5e758b38cfdb71474a95cca6236e505116736586
2020-10-01 06:52:46 -04:00
Kousik Kumar
602859b186 resolve merge conflicts of a7beb10933 to rvc-dev
Bug: b/167405184
Change-Id: Ib35af2813e7b60932df33f905d8e0398727ee545
Merged-In: I4d06b4fa190d6185b15aa98cb6f1a33f9702981e
2020-09-23 04:40:24 -04:00
Kousik Kumar
63e1c37e6f resolve merge conflicts of 22ccf36b1b to rvc-dev
Bug: b/167405184
Change-Id: Ide73513d5822670eeb16e6b4a4332d1dc77567de
Merged-In: Ic8a8adb3fffb357754384ddbf13684d944a9767e
2020-09-22 17:58:24 +00:00
Kousik Kumar
73bcc38ac8 Rename wlist to allowlist
The new release of re-client renames env_var_wlist to env_var_allowlist.
We need to make this change as part of the topic so that with the new
release we are able to deprecate this variable.

Bug: b/167677318
Change-Id: I988381f4f72061f7c6eb99d58593f92cfd618b1b
Merged-In: I988381f4f72061f7c6eb99d58593f92cfd618b1b
2020-09-16 18:20:29 +00:00
TreeHugger Robot
461c0d4620 Merge "Set RBE_DIR default value to point to prebuilts in tree." into rvc-dev 2020-09-15 23:30:18 +00:00
Kousik Kumar
f037e1a969 Remove double quotes in rbe.mk file
These result in weird command line invocations when used with RBE.

Test: Ran an RBE build locally and made sure it still worked as expected
after the change.
Bug: b/163618332

Change-Id: Ic8a8adb3fffb357754384ddbf13684d944a9767e
Merged-In: Ic8a8adb3fffb357754384ddbf13684d944a9767e
2020-09-15 18:43:42 +00:00
Kousik Kumar
0118dd2f83 Add RBE_CXX_COMPARE variable to locally disable compare mode for C++
When we do 64-32 migration, we want compare mode to be present only for
non-c++ actions as it would impact performance on 32-core machines
otherwise.

Bug: b/157414300
Test: Ran a build with "RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" RBE_CXX_COMPARE="false" RBE_compare="true" use_rbe mmma bionic"
and observed that local-executions happened only for link actions and
not for C++ actions.

Change-Id: Ia972c74d804085196a7b6ec755af63042c6b3909
Merged-In: Ia972c74d804085196a7b6ec755af63042c6b3909
2020-09-15 18:35:32 +00:00
Ramy Medhat
83c95dea63 [DO NOT MERGE] Set RBE_DIR default value to point to prebuilts in tree.
The new default points to prebuilts/remoteexecution-client/live.

Also sets the default exec strategy of Javac/R8/D8 to remote local
fallback.

Bug: b/166182389
Test: simple one action build
Change-Id: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
Merged-In: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
2020-09-15 18:32:57 +00:00
Kousik Kumar
edb723a3f8 Rename wlist to allowlist
The new release of re-client renames env_var_wlist to env_var_allowlist.
We need to make this change as part of the topic so that with the new
release we are able to deprecate this variable.

Bug: b/167677318
Change-Id: I988381f4f72061f7c6eb99d58593f92cfd618b1b
2020-09-14 11:10:20 +00:00
Kousik Kumar
136274bfaa [DO NOT MERGE] Update configuration for RemoteExecution actions
1. Specify addition inputs needed for R8 / D8 / Javac actions so that
they can be successfully executed remotely.
2. Cleanup platform configuration so that pool-names are turned into a
common variable and re-used between javac / r8 / d8 actions.

Bug: b/152273259
Change-Id: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
Merged-In: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
2020-08-31 12:37:54 +00:00
Kousik Kumar
8b3850d8e5 [DO NOT MERGE] Add exec_strategy as CLI flags to all rewrapper commands
Bug: b/148798195
Test: The same change is live in master-build-rbe-testing branch where
its not causing any behavior change, and also it works for
RemoteExecution where RBE_CXX_EXEC_STRATEGY variable is set.

Bug: b/166182389
Change-Id: I5efb1c2ab74dbe70dd70bf43d6262f2063d4d378
Merged-In: I745ed4a91c2b070048c0b1fd0881e5f4f28d0148
2020-08-31 12:36:29 +00:00
Ramy Medhat
c9a030aa4d [DO NOT MERGE] Add RBE support for Javac, R8, and D8 if respective env vars are set.
Bug: b/166182389
Test: aosp_crosshatch build with and without the new variables.
Change-Id: I252ff5dd8e372edf8c86e25348c8a2bfab41d70d
Merged-In: I252ff5dd8e372edf8c86e25348c8a2bfab41d70d
2020-08-31 12:36:21 +00:00
Ramy Medhat
77a4879e8c [DO NOT MERGE] Add PWD to environment variable whitelist for RBE builds.
Test: Build aosp with USE_RBE=1.
Bug: b/166182389
Change-Id: I4b51fbde330c6129654aa74f8a58dccf62cfa11d
Merged-In: I4b51fbde330c6129654aa74f8a58dccf62cfa11d
2020-08-31 12:35:52 +00:00
Ramy Medhat
f965526697 [DO NOT MERGE] Add labels to rewrapper cc compile action. Prevent ccwrapper from being used in links.
Test: Built aosp-arm64_eng with and without USE_RBE=1

Bug: b/166182389
Change-Id: I8d04069219a9467c3de9a028eb68e3115173f1e2
Merged-In: I8d04069219a9467c3de9a028eb68e3115173f1e2
2020-08-31 12:35:23 +00:00
Ramy Medhat
7298be1b47 [DO NOT MERGE] Add RBE mk file to support bootstrapping RBE and setting relevant
environment variables.

Test: Built aosp_arm-user with and without USE_RBE. With USE_RBE,
used a simple proxy script in place of rewrapper.

Bug: b/166182389
Change-Id: Ifb6bfd9f7292346f4750a6a409aa0def4e3c0363
Merged-In: Ifb6bfd9f7292346f4750a6a409aa0def4e3c0363
2020-08-31 12:34:55 +00:00
Ramy Medhat
8f8d1126aa Set RBE_DIR default value to point to prebuilts in tree.
The new default points to prebuilts/remoteexecution-client/live.

Also sets the default exec strategy of Javac/R8/D8 to remote local
fallback.

Test: simple one action build
Bug: b/166273085
Change-Id: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
Merged-In: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
2020-08-25 16:31:40 +00:00
Ramy Medhat
ba6d777f25 Set RBE_DIR default value to point to prebuilts in tree.
The new default points to prebuilts/remoteexecution-client/live.

Also sets the default exec strategy of Javac/R8/D8 to remote local
fallback.

Test: simple one action build
Change-Id: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
2020-08-13 11:49:30 -04:00
Kousik Kumar
151dd8fadb Remove double quotes in rbe.mk file
These result in weird command line invocations when used with RBE.

Test: Ran an RBE build locally and made sure it still worked as expected
after the change.
Bug: b/163618332

Change-Id: Ic8a8adb3fffb357754384ddbf13684d944a9767e
2020-08-12 13:38:59 +00:00
Kousik Kumar
2dcf402edc Add RBE_CXX_COMPARE variable to locally disable compare mode for C++
When we do 64-32 migration, we want compare mode to be present only for
non-c++ actions as it would impact performance on 32-core machines
otherwise.

Bug: b/157414300
Test: Ran a build with "RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" RBE_CXX_COMPARE="false" RBE_compare="true" use_rbe mmma bionic"
and observed that local-executions happened only for link actions and
not for C++ actions.

Change-Id: Ia972c74d804085196a7b6ec755af63042c6b3909
Merged-In: Ia972c74d804085196a7b6ec755af63042c6b3909
2020-05-29 08:08:58 -07:00
Kousik Kumar
b0f055db13 Add RBE_CXX_COMPARE variable to locally disable compare mode for C++
When we do 64-32 migration, we want compare mode to be present only for
non-c++ actions as it would impact performance on 32-core machines
otherwise.

Bug: b/157414300
Test: Ran a build with "RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" RBE_CXX_COMPARE="false" RBE_compare="true" use_rbe mmma bionic"
and observed that local-executions happened only for link actions and
not for C++ actions.

Change-Id: Ia972c74d804085196a7b6ec755af63042c6b3909
2020-05-26 11:57:02 -07:00
Kousik Kumar
3fd528e215 Update configuration for RemoteExecution actions
1. Specify addition inputs needed for R8 / D8 / Javac actions so that
they can be successfully executed remotely.
2. Cleanup platform configuration so that pool-names are turned into a
common variable and re-used between javac / r8 / d8 actions.

Bug: b/152273259
Change-Id: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
Merged-In: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
2020-04-02 12:07:47 -07:00
Kousik Kumar
28e7f8cfbc Update configuration for RemoteExecution actions
1. Specify addition inputs needed for R8 / D8 / Javac actions so that
they can be successfully executed remotely.
2. Cleanup platform configuration so that pool-names are turned into a
common variable and re-used between javac / r8 / d8 actions.

Bug: b/152273259
Change-Id: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
Merged-In: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
2020-04-02 07:13:39 -07:00
Kousik Kumar
d463007dfc Add exec_strategy as CLI flags to all rewrapper commands
Bug: b/148798195
Test: The same change is live in master-build-rbe-testing branch where
its not causing any behavior change, and also it works for
RemoteExecution where RBE_CXX_EXEC_STRATEGY variable is set.
2020-02-05 16:06:47 +00:00
Ramy Medhat
a9cc25d785 Add RBE support for Javac, R8, and D8 if respective env vars are set.
Test: aosp_crosshatch build with and without the new variables.
Change-Id: I252ff5dd8e372edf8c86e25348c8a2bfab41d70d
2020-01-28 10:25:25 -05:00
Ramy Medhat
e8de0931a1 Add PWD to environment variable whitelist for RBE builds.
Test: Build aosp with USE_RBE=1.

Change-Id: I4b51fbde330c6129654aa74f8a58dccf62cfa11d
2019-08-27 23:50:31 +00:00
Ramy Medhat
e5839f912b Add labels to rewrapper cc compile action. Prevent ccwrapper from being
used in links.

Test: Built aosp-arm64_eng with and without USE_RBE=1

Change-Id: I8d04069219a9467c3de9a028eb68e3115173f1e2
2019-08-12 14:13:31 -04:00
Ramy Medhat
ec8b8dfa3e Add RBE mk file to support bootstrapping RBE and setting relevant
environment variables.

Test: Built aosp_arm-user with and without USE_RBE. With USE_RBE,
used a simple proxy script in place of rewrapper.

Change-Id: Ifb6bfd9f7292346f4750a6a409aa0def4e3c0363
2019-07-17 14:22:00 +00:00