Commit graph

34536 commits

Author SHA1 Message Date
Colin Cross
1661aff8be Run lint actions in sbox
Run lint actions in sbox with RuleBuilder.SandboxInputs.  This
copies all input files into the sandbox, which prevents the lint
tool from finding nearby source files that were not presented to it.

Using SandboxInputs requires use of PathForInput or PathForOutput
anywhere a path is used outside of the RuleBuilderCommand methods
that take paths so that they can be translated to the paths that
will be used in the sandbox.

Bug: 181681346
Test: lint_test.go
Test: m lint-check dist
Test: m USE_RBE=true RBE_LINT=true lint-check dist
Test: m USE_RBE=true RBE_LINT=true RBE_LINT_EXEC_STRATEGY=remote lint-check dist
Change-Id: Iab4e09d961891ef182643583d4d456e413bc5e39
2021-03-18 16:17:34 -07:00
Colin Cross
ef972743e8 Support sbox-in-RBE
Allow passing a remoteexec.REParams to RuleBuilder to configure it to
run the rule remotely through RBE.  Requires the rule to use
SandboxInputs, which ensures that RuleBuilder is aware of all of the
inputs and outputs of the rule.

Running sbox in RBE initially seems unnecessary, as RBE is already a
good sandbox, but reproxy can execute RBE actions locally when configured
for local execution, local fallback or racing.  Using sbox in RBE ensures
that these local actions are also sandboxed, giving consistent results
between directly executed actions, local RBE actions, and remote RBE
actions.

Bug: 182612695
Test: manual
Change-Id: Icf2f24dde8dee833eb680ba22566a8e1c0143b15
2021-03-18 16:17:34 -07:00
Colin Cross
77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Colin Cross
ab020a7333 Support sandboxing inputs in RuleBuilder
When RuleBuilder.SandboxInputs() is called configure sbox to copy
all the input files into the sandbox directory and then change the
working directory there when running the command.

Copying input files into the sandbox directory gets tricky when
the input file is the output file from another rule, and could
be at an arbitrary, possibly absolute path based on the value
of OUT_DIR.  They will need to be copied to a directory in the
sandbox using the path relative to OUT_DIR.

RSP files need special handling, they need to both be copied into
the sandbox as an input, rewritten to contain paths as seen in the
sandbox, and references to them on the command line need to use
sandbox paths.

Bug: 182612695
Test: rule_builder_test.go
Change-Id: Ic0db961961b186e4ed9b76246881e3f04971825c
2021-03-18 16:17:32 -07:00
Jingwen Chen
c5a0089ac4 Merge "bp2build: genrule converter only supports "genrule"." 2021-03-18 22:41:52 +00:00
Jingwen Chen
c7846f38fa bp2build: cc_library_headers converter should only include that module type
Test: bp2build; bazel build //bionic/...
Change-Id: I34768b51098c5b801e7c6d9b3a95ec0f37c3bb09
2021-03-18 18:35:39 -04:00
Colin Cross
a5d09762a6 Merge changes Idcbe2046,I09616402,Ide2f1ed9
* changes:
  Fix sbox in chdir mode
  Check RuleBuilder temporaries for path errors
  Fix WriteFileRule escaping
2021-03-18 22:28:11 +00:00
Tri Vo
0e3758caea Merge "Remove "-C link-dead-code=y" flag" 2021-03-18 21:27:54 +00:00
Treehugger Robot
d19b824e7f Merge "Check for frameworks/base/Android.bp instead of frameworks/base" 2021-03-18 20:56:54 +00:00
Kousik Kumar
47665870d7 Merge "Allowlist RBE_num_retries_if_mismatched variable" 2021-03-18 19:13:29 +00:00
Paul Duffin
813bfcb252 Merge "Convert android/packaging_test.go to test fixtures" 2021-03-18 18:38:34 +00:00
Paul Duffin
387f4ce1ee Merge changes Iccba8caf,I1eb49f25,Id77575f8,I5679cb51,I7f9f3eb6, ...
* changes:
  Convert android/ninja_deps_test.go to test fixtures
  Add NinjaDeps to TestResult
  Convert android/singleton_module_test.go to test fixtures
  Convert android/neverallow_test.go to test fixtures
  Convert android/singleton_module_test.go to test fixtures
  Convert android/variable_test.go to test fixtures
  Convert android/soong_config_modules_test.go to test fixtures
  Convert android/mutator_test.go to test fixtures
  Convert android/deptag_test.go to test fixtures
2021-03-18 18:18:38 +00:00
Paul Duffin
000ae662fa Merge changes I2e69e7b5,I3c9c36e7
* changes:
  Convert sysprop package to fixtures
  Add preparers for sysprop
2021-03-18 18:17:20 +00:00
Paul Duffin
ebcb37bab1 Merge "Migrate sdk package to a per test build directory" 2021-03-18 17:05:11 +00:00
Paul Duffin
c3f89bece1 Merge "Clean up java package use of python build components" 2021-03-18 17:02:44 +00:00
Paul Duffin
e410711cd5 Merge "Convert python package to fixtures" 2021-03-18 17:02:19 +00:00
Paul Duffin
b72dd403de Merge "Prevent ApexInfoMutator from creating unnecessary variants" 2021-03-18 15:13:39 +00:00
Paul Duffin
b01125f92b Merge changes I9aad3e4e,I150889a0
* changes:
  Convert xml package to fixtures
  Add preparers for xml
2021-03-18 14:16:31 +00:00
Paul Duffin
d5065cfc66 Merge "Add prebuilt_platform_compat_config" 2021-03-18 12:28:00 +00:00
Paul Duffin
1afa515929 Migrate android package to a per test build directory
Bug: 182885307
Test: m nothing
Change-Id: I6b868bacc36be1f6768fbb2fb1d7ed4ad6e4118c
2021-03-18 12:18:09 +00:00
Paul Duffin
c3bdd310bd Convert android/packaging_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: Ic2384bfe3f1edb8d252053fda7df66d49c46fd0f
2021-03-18 12:18:09 +00:00
Paul Duffin
d9ec7d2694 Convert android/ninja_deps_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: Iccba8caffc7e5a82ff13d93a032edcf3f748fc40
2021-03-18 12:18:09 +00:00
Paul Duffin
d518b7e2f2 Convert android/path_properties_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: If75f471885b5e4022b406e267e04b7a20e0be080
2021-03-18 12:18:09 +00:00
Paul Duffin
78c36216a8 Add NinjaDeps to TestResult
Bug: 182885307
Test: m nothing
Change-Id: I1eb49f2514b13f52f367cd8f57e5cb0ab3336bd7
2021-03-18 12:18:09 +00:00
Paul Duffin
a9237b6eeb Convert android/singleton_module_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: Id77575f8ceda43545e2fad97fa1bd6b612c967b4
2021-03-18 12:18:09 +00:00
Paul Duffin
3c6a4ea24a Convert android/neverallow_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: I5679cb517151ceaf468f2fe2e0ccab718cc7b4a3
2021-03-18 12:18:09 +00:00
Paul Duffin
d65970072d Convert android/singleton_module_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: I7f9f3eb66279fe5a2447aefa9b636144c32ce92a
2021-03-18 12:18:05 +00:00
Paul Duffin
4bb2b219d6 Convert android/variable_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: I51d3651244b110bf41cb0e4c331ac663bf816f84
2021-03-18 12:17:05 +00:00
Paul Duffin
791302b412 Convert android/soong_config_modules_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: I53231b820154447e359e48afa47cdf76f0ec6c47
2021-03-18 12:17:05 +00:00
Paul Duffin
e8a4ac49fe Convert android/mutator_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: Id7eb915af21ccde5acf1d4a0ec339e0672bc3d57
2021-03-18 12:17:05 +00:00
Paul Duffin
485079130d Convert android/deptag_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: I837c7b9944a9adbe9019ae8a4e69ff3b7086ca5c
2021-03-18 12:17:01 +00:00
Paul Duffin
4b9bb5c057 Merge "Convert android/namespace_test.go to test fixtures" 2021-03-18 12:14:01 +00:00
Paul Duffin
cb460c91f1 Merge "Add FixtureIgnoreErrors" 2021-03-18 12:13:36 +00:00
Paul Duffin
50eb3829f9 Merge "Convert android/rule_builder_test.go to test fixtures" 2021-03-18 12:13:12 +00:00
Artur Satayev
b77b0c5e05 Treat allowed_deps.txt source file as optional.
Not all branches have packages/common/module, which breaks the build
for them.

Bug: 179234385
Test: removed allowed_deps.txt && m apex-allowed-deps-check
Change-Id: I38f47c7200e1afbd899e29843d0214bef826fcf9
2021-03-18 11:15:33 +00:00
Paul Duffin
573989d821 Prevent ApexInfoMutator from creating unnecessary variants
Adds the AlwaysRequireApexVariantTag interface to enable
ApexInfoMutator to differentiate between a tag that is excluded from
apex contents but still requires an apex variant and a tag that is
excluded from apex contents and does not require an apex variant.

That is needed to support the sdkMemberVersionedDepTag which excludes
the target from being added to the APEX but requires an APEX variant.
A more detailed explanation is in the comments.

The AlwaysRequireApexVariant() method follows the pattern used in
ReplaceSourceWithPrebuilt of having a method that returns a bool to
trigger the behavior and not say ExcludeFromApexContentsTag that simply
relies on the tag implementing an interface to trigger. That is because
the former is more flexible and allows a tag type to parameterize the
behavior if necessary.

The tags that this will exclude from creating an apex variant are:
* PrebuiltDepTag - by the time the apex variant has been created any
  preferred prebuilts will have replaced the sources so there is no
  need to create an APEX variant if the only dependency path from the
  APEX to the prebuilt is via this tag.
* hiddenApiAnnotationsDependencyTag - the target of which is a purely
  build time artifect and MUST NEVER end up in the APEX.

It will also stop calling DepIsInSameApex for any dependency created
by the sdkMemberVersionedDepTag. Which will fix the issue reported in
the bug.

Bug: 182992071
Test: m nothing
Change-Id: I9569e488d6446ca45d3ea8f32a9b74524eb865df
2021-03-18 09:05:28 +00:00
Paul Duffin
0700f77806 Merge "Convert android/paths_test.go to test fixtures" 2021-03-18 08:58:54 +00:00
Paul Duffin
e9ae3616a2 Merge "Convert android/module_test.go to test fixtures" 2021-03-18 08:58:19 +00:00
Lukács T. Berki
734b215d66 Merge "Compile soong_build for debugging if needed." 2021-03-18 08:58:16 +00:00
Paul Duffin
7f09fd298a Merge "Convert android/defaults_test.go to test fixtures" 2021-03-18 08:57:31 +00:00
Paul Duffin
c14e8aa6a8 Merge "Add AssertSame" 2021-03-18 08:56:21 +00:00
Paul Duffin
28f0e9e2cb Merge "Convert android/csuite_config_test.go to test fixtures" 2021-03-18 08:55:55 +00:00
Paul Duffin
3bf545fa6b Merge "Convert android/arch_test.go to test fixtures" 2021-03-18 08:55:29 +00:00
Paul Duffin
abc6ec9d16 Merge "Add OptionalFixturePreparer" 2021-03-18 08:55:04 +00:00
Paul Duffin
81e7e2b9a3 Merge "Remove android/apex_test.go use of buildDir" 2021-03-18 08:54:35 +00:00
Paul Duffin
aa8270eca7 Merge "Convert android/androidmk_test.go to test fixtures" 2021-03-18 08:52:15 +00:00
Paul Duffin
21d1b9a52c Merge "Migrate bpf package to a per test build directory" 2021-03-18 08:51:45 +00:00
Paul Duffin
603fa13e08 Merge "Correct typo in the name of compatConfigTag" 2021-03-18 08:48:36 +00:00
Paul Duffin
c0a5ccbf3b Merge "Allow apex dependencies to be restricted to source modules only" 2021-03-18 08:48:05 +00:00
Paul Duffin
533cf74318 Merge "Separate metadata provider from apex contents provider" 2021-03-18 08:46:49 +00:00