Commit graph

34 commits

Author SHA1 Message Date
Junfeng Wu
b42c464cf6 Remove the duplicate RBE_CXX_COMPARE config.
The duplication comes from a merge conflict resolution: 0d0d422359:core/rbe.mk;dlc=76ffe4f5ba0c711b118b9df010ffde328866367d

Test: Ran an RBE build locally with RBE_CXX_COMPARE enabled.
Bug: b/214886208

Change-Id: I7104fe3ce0c2861594f42633eb19105ff845ae4d
2022-01-20 23:48:50 +08:00
Kousik Kumar
2b2b87fe80 Ensure RBE_CXX_POOL and RBE_JAVA_POOL variables work as expected
These environment variables are not being passed onto compile actions created from makefiles. This change makes sure that these env variables are passed along so that we are able to update the C++ / Javac pool being used

Test: Ran a build with RBE_CXX_POOL=cpp_next to test the next pool
Bug: b/199774309
Change-Id: Icd47d3647b962a6c3ac712369466d5502b817fd7
2021-09-14 21:31:38 -04:00
Treehugger Robot
605eb54154 Merge "Revert "Remove env-var-allowlist"" am: fd8dae609f am: fe4c812bad
Original change: https://android-review.googlesource.com/c/platform/build/+/1465913

Change-Id: I8dd41768ef397889d7cdfecf4f65aaec7482c3a4
2020-10-20 21:04:49 +00:00
Kousik Kumar
92f32615ed Revert "Remove env-var-allowlist"
Revert "Add -fdebug-compilation-dir option"

Revert submission 1461902-debug-compilation-dir

Reason for revert: "-Xclang" isn't being uniformly respected everywhere. For example, in ".S" compilations, when I pass `-Xclang -fdebug-compilation-dir=.", the assembler seems to be ignoring it and then inserting the `pwd` into the command, however when I pass "-fdebug-compilation-dir=.", it strips out the path to the current working directory.
This indicates that we need to update re-client's input processor so that we can pass -fdebug-compilation-dir=. without "-Xclang" and then remove `PWD` setting.
For now, I'll update this patch to pass both "-fdebug-compilation-dir=." and `PWD` and when RBE side fix is done, I'll remove `PWD` in a separate CL.

Reverted Changes:
Ib0f271e55:Add -fdebug-compilation-dir option
Ifa0592af5:Remove env-var-allowlist

Change-Id: I7c690b3e00d37dbcc8fbaa66dda49f39032be3ab
2020-10-20 05:52:49 +00:00
Kousik Kumar
afd93ad912 Merge "Remove env-var-allowlist" am: 3a7a960a86 am: c2c28f9941
Original change: https://android-review.googlesource.com/c/platform/build/+/1461902

Change-Id: Id3e7ce2ee695ff9b760f20007c02e3df7779cf94
2020-10-16 06:26:10 +00:00
Kousik Kumar
405847ac19 Remove env-var-allowlist
We no longer need to set PWD variable with the introduction of
`-fdebug-compilation-dir`, so removing env var allowlist of PWD variable
to RBE.

Change-Id: Ifa0592af519b6cc4364ee653f1a2174fd680bac1
2020-10-15 15:53:13 -04:00
Kousik Kumar
0d0d422359 resolve merge conflicts of 602859b186 to rvc-d1-dev am: 3d723c29d1
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/+/12735596

Change-Id: I9a82c7e89561821e912081790557e987757ba5b3
2020-10-15 04:00:29 +00:00
Kousik Kumar
3d723c29d1 resolve merge conflicts of 602859b186 to rvc-d1-dev
Bug: b/169675226
Change-Id: I5e758b38cfdb71474a95cca6236e505116736586
2020-10-01 06:52:46 -04:00
Kousik Kumar
602859b186 resolve merge conflicts of a7beb10933 to rvc-dev
Bug: b/167405184
Change-Id: Ib35af2813e7b60932df33f905d8e0398727ee545
Merged-In: I4d06b4fa190d6185b15aa98cb6f1a33f9702981e
2020-09-23 04:40:24 -04:00
Kousik Kumar
63e1c37e6f resolve merge conflicts of 22ccf36b1b to rvc-dev
Bug: b/167405184
Change-Id: Ide73513d5822670eeb16e6b4a4332d1dc77567de
Merged-In: Ic8a8adb3fffb357754384ddbf13684d944a9767e
2020-09-22 17:58:24 +00:00
Kousik Kumar
73bcc38ac8 Rename wlist to allowlist
The new release of re-client renames env_var_wlist to env_var_allowlist.
We need to make this change as part of the topic so that with the new
release we are able to deprecate this variable.

Bug: b/167677318
Change-Id: I988381f4f72061f7c6eb99d58593f92cfd618b1b
Merged-In: I988381f4f72061f7c6eb99d58593f92cfd618b1b
2020-09-16 18:20:29 +00:00
TreeHugger Robot
461c0d4620 Merge "Set RBE_DIR default value to point to prebuilts in tree." into rvc-dev 2020-09-15 23:30:18 +00:00
Kousik Kumar
f037e1a969 Remove double quotes in rbe.mk file
These result in weird command line invocations when used with RBE.

Test: Ran an RBE build locally and made sure it still worked as expected
after the change.
Bug: b/163618332

Change-Id: Ic8a8adb3fffb357754384ddbf13684d944a9767e
Merged-In: Ic8a8adb3fffb357754384ddbf13684d944a9767e
2020-09-15 18:43:42 +00:00
Kousik Kumar
0118dd2f83 Add RBE_CXX_COMPARE variable to locally disable compare mode for C++
When we do 64-32 migration, we want compare mode to be present only for
non-c++ actions as it would impact performance on 32-core machines
otherwise.

Bug: b/157414300
Test: Ran a build with "RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" RBE_CXX_COMPARE="false" RBE_compare="true" use_rbe mmma bionic"
and observed that local-executions happened only for link actions and
not for C++ actions.

Change-Id: Ia972c74d804085196a7b6ec755af63042c6b3909
Merged-In: Ia972c74d804085196a7b6ec755af63042c6b3909
2020-09-15 18:35:32 +00:00
Ramy Medhat
83c95dea63 [DO NOT MERGE] Set RBE_DIR default value to point to prebuilts in tree.
The new default points to prebuilts/remoteexecution-client/live.

Also sets the default exec strategy of Javac/R8/D8 to remote local
fallback.

Bug: b/166182389
Test: simple one action build
Change-Id: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
Merged-In: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
2020-09-15 18:32:57 +00:00
Kousik Kumar
edb723a3f8 Rename wlist to allowlist
The new release of re-client renames env_var_wlist to env_var_allowlist.
We need to make this change as part of the topic so that with the new
release we are able to deprecate this variable.

Bug: b/167677318
Change-Id: I988381f4f72061f7c6eb99d58593f92cfd618b1b
2020-09-14 11:10:20 +00:00
Kousik Kumar
136274bfaa [DO NOT MERGE] Update configuration for RemoteExecution actions
1. Specify addition inputs needed for R8 / D8 / Javac actions so that
they can be successfully executed remotely.
2. Cleanup platform configuration so that pool-names are turned into a
common variable and re-used between javac / r8 / d8 actions.

Bug: b/152273259
Change-Id: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
Merged-In: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
2020-08-31 12:37:54 +00:00
Kousik Kumar
8b3850d8e5 [DO NOT MERGE] Add exec_strategy as CLI flags to all rewrapper commands
Bug: b/148798195
Test: The same change is live in master-build-rbe-testing branch where
its not causing any behavior change, and also it works for
RemoteExecution where RBE_CXX_EXEC_STRATEGY variable is set.

Bug: b/166182389
Change-Id: I5efb1c2ab74dbe70dd70bf43d6262f2063d4d378
Merged-In: I745ed4a91c2b070048c0b1fd0881e5f4f28d0148
2020-08-31 12:36:29 +00:00
Ramy Medhat
c9a030aa4d [DO NOT MERGE] Add RBE support for Javac, R8, and D8 if respective env vars are set.
Bug: b/166182389
Test: aosp_crosshatch build with and without the new variables.
Change-Id: I252ff5dd8e372edf8c86e25348c8a2bfab41d70d
Merged-In: I252ff5dd8e372edf8c86e25348c8a2bfab41d70d
2020-08-31 12:36:21 +00:00
Ramy Medhat
77a4879e8c [DO NOT MERGE] Add PWD to environment variable whitelist for RBE builds.
Test: Build aosp with USE_RBE=1.
Bug: b/166182389
Change-Id: I4b51fbde330c6129654aa74f8a58dccf62cfa11d
Merged-In: I4b51fbde330c6129654aa74f8a58dccf62cfa11d
2020-08-31 12:35:52 +00:00
Ramy Medhat
f965526697 [DO NOT MERGE] Add labels to rewrapper cc compile action. Prevent ccwrapper from being used in links.
Test: Built aosp-arm64_eng with and without USE_RBE=1

Bug: b/166182389
Change-Id: I8d04069219a9467c3de9a028eb68e3115173f1e2
Merged-In: I8d04069219a9467c3de9a028eb68e3115173f1e2
2020-08-31 12:35:23 +00:00
Ramy Medhat
7298be1b47 [DO NOT MERGE] Add RBE mk file to support bootstrapping RBE and setting relevant
environment variables.

Test: Built aosp_arm-user with and without USE_RBE. With USE_RBE,
used a simple proxy script in place of rewrapper.

Bug: b/166182389
Change-Id: Ifb6bfd9f7292346f4750a6a409aa0def4e3c0363
Merged-In: Ifb6bfd9f7292346f4750a6a409aa0def4e3c0363
2020-08-31 12:34:55 +00:00
Ramy Medhat
8f8d1126aa Set RBE_DIR default value to point to prebuilts in tree.
The new default points to prebuilts/remoteexecution-client/live.

Also sets the default exec strategy of Javac/R8/D8 to remote local
fallback.

Test: simple one action build
Bug: b/166273085
Change-Id: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
Merged-In: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
2020-08-25 16:31:40 +00:00
Ramy Medhat
ba6d777f25 Set RBE_DIR default value to point to prebuilts in tree.
The new default points to prebuilts/remoteexecution-client/live.

Also sets the default exec strategy of Javac/R8/D8 to remote local
fallback.

Test: simple one action build
Change-Id: I3392a0f7ac6de818d0c5ba2e113fcb035c6fa0ae
2020-08-13 11:49:30 -04:00
Kousik Kumar
151dd8fadb Remove double quotes in rbe.mk file
These result in weird command line invocations when used with RBE.

Test: Ran an RBE build locally and made sure it still worked as expected
after the change.
Bug: b/163618332

Change-Id: Ic8a8adb3fffb357754384ddbf13684d944a9767e
2020-08-12 13:38:59 +00:00
Kousik Kumar
2dcf402edc Add RBE_CXX_COMPARE variable to locally disable compare mode for C++
When we do 64-32 migration, we want compare mode to be present only for
non-c++ actions as it would impact performance on 32-core machines
otherwise.

Bug: b/157414300
Test: Ran a build with "RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" RBE_CXX_COMPARE="false" RBE_compare="true" use_rbe mmma bionic"
and observed that local-executions happened only for link actions and
not for C++ actions.

Change-Id: Ia972c74d804085196a7b6ec755af63042c6b3909
Merged-In: Ia972c74d804085196a7b6ec755af63042c6b3909
2020-05-29 08:08:58 -07:00
Kousik Kumar
b0f055db13 Add RBE_CXX_COMPARE variable to locally disable compare mode for C++
When we do 64-32 migration, we want compare mode to be present only for
non-c++ actions as it would impact performance on 32-core machines
otherwise.

Bug: b/157414300
Test: Ran a build with "RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" RBE_CXX_COMPARE="false" RBE_compare="true" use_rbe mmma bionic"
and observed that local-executions happened only for link actions and
not for C++ actions.

Change-Id: Ia972c74d804085196a7b6ec755af63042c6b3909
2020-05-26 11:57:02 -07:00
Kousik Kumar
3fd528e215 Update configuration for RemoteExecution actions
1. Specify addition inputs needed for R8 / D8 / Javac actions so that
they can be successfully executed remotely.
2. Cleanup platform configuration so that pool-names are turned into a
common variable and re-used between javac / r8 / d8 actions.

Bug: b/152273259
Change-Id: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
Merged-In: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
2020-04-02 12:07:47 -07:00
Kousik Kumar
28e7f8cfbc Update configuration for RemoteExecution actions
1. Specify addition inputs needed for R8 / D8 / Javac actions so that
they can be successfully executed remotely.
2. Cleanup platform configuration so that pool-names are turned into a
common variable and re-used between javac / r8 / d8 actions.

Bug: b/152273259
Change-Id: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
Merged-In: I0231a96f6e4e094e98d0f89728f6ac3e34f78118
2020-04-02 07:13:39 -07:00
Kousik Kumar
d463007dfc Add exec_strategy as CLI flags to all rewrapper commands
Bug: b/148798195
Test: The same change is live in master-build-rbe-testing branch where
its not causing any behavior change, and also it works for
RemoteExecution where RBE_CXX_EXEC_STRATEGY variable is set.
2020-02-05 16:06:47 +00:00
Ramy Medhat
a9cc25d785 Add RBE support for Javac, R8, and D8 if respective env vars are set.
Test: aosp_crosshatch build with and without the new variables.
Change-Id: I252ff5dd8e372edf8c86e25348c8a2bfab41d70d
2020-01-28 10:25:25 -05:00
Ramy Medhat
e8de0931a1 Add PWD to environment variable whitelist for RBE builds.
Test: Build aosp with USE_RBE=1.

Change-Id: I4b51fbde330c6129654aa74f8a58dccf62cfa11d
2019-08-27 23:50:31 +00:00
Ramy Medhat
e5839f912b Add labels to rewrapper cc compile action. Prevent ccwrapper from being
used in links.

Test: Built aosp-arm64_eng with and without USE_RBE=1

Change-Id: I8d04069219a9467c3de9a028eb68e3115173f1e2
2019-08-12 14:13:31 -04:00
Ramy Medhat
ec8b8dfa3e Add RBE mk file to support bootstrapping RBE and setting relevant
environment variables.

Test: Built aosp_arm-user with and without USE_RBE. With USE_RBE,
used a simple proxy script in place of rewrapper.

Change-Id: Ifb6bfd9f7292346f4750a6a409aa0def4e3c0363
2019-07-17 14:22:00 +00:00