Commit graph

34655 commits

Author SHA1 Message Date
Colin Cross
5bedfa2d45 Simplify lint rules using improved RuleBuilder rsp support
With the improved RuleBuilder rsp support a manual resources.list
file is not necessary, use FlagWithRspFileInputList instead.

The switch to RBE support in RuleBuilder in
Iab4e09d961891ef182643583d4d456e413bc5e39 obsoleted tracking remoteInputs
and remoteRSPInputs, remove them.

writeLintProjectXML was written to allow it to be applied to a separate
rule than the one that ran lint, but it is not used that way.  Using
the same rule for both means that manual tracking of the input
dependencies described by the project.xml rule but read by the lint
rule is not necessary, just treat them as inputs to the single rule.

Test: m lint-check
Test: m USE_RBE=true RBE_LINT=true RBE_LINT_EXEC_STRATEGY=remote lint-check
Change-Id: If1827b9dede3ebcd0792b6b4b8114d3199f6570b
2021-03-25 11:06:45 -07:00
Colin Cross
ce3a51dc96 Support multiple rsp files in RuleBuilder
The lint rule is manually creating a second rsp file because Ninja
only supports on per rule.  Move the support into RuleBuilder so
that it can apply the same rewrites that it does to the primary
one.

Test: TestRuleBuilder_Build
Change-Id: Iec250a2d60e74ccf1b4ad085a960fec6867ea559
2021-03-25 11:06:45 -07:00
Colin Cross
e55bd423df Pass rsp files into sbox and rewrapper
The current implementation causes inputs listed in an rsp file used with
sbox to be duplicated 3 times in the build.ninja file; once as a
dependency of the rule, once in the rspfile_content field of the rule
with the paths rewritten to be relative to the sandbox, and once in the
rule to write the sbox manifest.  When RBE is enabled it also gets a
fourth copy in the list of files to be treated as inputs by rewrapper.

Reduce this to a single copy by using "$in" for the rspfile_content so
that the files only have to be listed in the input dependencies of the
rule, and then add support to sbox to rewrite the rsp file while copying
it into the sandbox, and pass it to rewrapper as well.

Test: m lint-check
Change-Id: I3f46f61119508d39a8bb231c99fc130153fb6f04
2021-03-25 11:06:45 -07:00
Colin Cross
045bfd9640 Add test for sbox input sandboxing
Add a test that was dropped in Ic0db961961b186e4ed9b76246881e3f04971825c.

Test: rule_builder_test.go
Change-Id: Idf136919939ad28eb5260dd8d686abe6948f47c7
2021-03-25 11:06:45 -07:00
Colin Cross
a4eafddc41 Support multiple rsp files in REParams
rewrapper supports a comma separate list of rsp files, replace
REParams.RSPFile with REParmas.RSPFiles.

Test: remoteexec_test.go
Change-Id: I7850c071c23d368d6fad4480dd527d146c13c6d3
2021-03-25 11:06:45 -07:00
Colin Cross
fd708b5651 Move response file handling to a separate package
sbox is going to need to read and write response files, move ReadRspFile
to its own package.

Test: response_test.go
Change-Id: Iecb5486b4aaeb2531828743ad8ef784df675e18e
2021-03-25 11:06:45 -07:00
Paul Duffin
6c968b1f0f Merge "Fix TestDroiddoc's reliance on absolute paths" 2021-03-25 18:02:26 +00:00
Paul Duffin
301deecd6c Merge "Add Path.RelativeToTop" 2021-03-25 18:01:31 +00:00
Pirama Arumuga Nainar
4c0b11a16e Merge "[pgo] Do not explicitly link profile runtime" 2021-03-25 17:57:49 +00:00
Roland Levillain
737ad608ab Merge "Fix a typo in the definition of varApexNames in java/dexpreopt_bootjars.go." 2021-03-25 17:25:44 +00:00
Jingwen Chen
6400f20430 Merge "bp2build: add a per-module denylist." 2021-03-25 15:02:57 +00:00
Paul Duffin
618eeff010 Merge "Remove config from basePath" 2021-03-25 14:45:43 +00:00
Paul Duffin
85484e344a Merge changes I12e1854c,I08f7dba4
* changes:
  Add buildDir to WritablePath implementations
  Add srcDir to SourcePath
2021-03-25 14:44:51 +00:00
Paul Duffin
4dd6cdb3bc Fix TestDroiddoc's reliance on absolute paths
TestDroiddoc compares the value returned by OutputFiles(""), which will
usually be absolute paths including the temporary buildDir, against
paths returned from TestingBuildParam.RelativeToTop(), which does not
currently change the Path contents and so will include absolute
temporary paths. However, a follow up change to this will make the
TestingBuildParam.RelativeToTop() also change the Path contents at
which point this test would be comparing relative to absolute paths.

So, this change makes sure that they are all converted to relative to
top paths before comparison.

Bug: 183650682
Test: m droid
Change-Id: Ia4478f527af27a920945f5849525e5031cc5b8b6
2021-03-25 13:14:37 +00:00
Paul Duffin
85d8f0dcd9 Add Path.RelativeToTop
Makes Path.RelativeToTop the source of truth as to what `relative to
top` means.

Uses it in PathRelativeToTop so that it will be tested by the existing
tests that use AssertPathRelativeToTop.

Also adds RelativeToTop() to WritablePaths and Paths.

Bug: 183650682
Test: m droid
Change-Id: I88dc56afd1314e51b4e41795a2448cab4ce8a899
2021-03-25 13:14:37 +00:00
Ulyana Trafimovich
2258cbbe42 Merge "Allow using updatable boot jars in dexpreopt (but don't use them yet)." 2021-03-25 10:13:45 +00:00
Jingwen Chen
5d72cba25c bp2build: add a per-module denylist.
This speeds up development of experimental converters.

Test: TH
Change-Id: I1623d3dbf63c8ed777646d64560d77b38edf9213
2021-03-25 09:29:26 +00:00
Paul Duffin
74abc5d4ee Remove config from basePath
Bug: 183650682
Test: m droid
Change-Id: I60eb2b8ed0317e874ecdafd5cb37d7c997d41bb8
2021-03-25 09:18:29 +00:00
Paul Duffin
d65c58b204 Add buildDir to WritablePath implementations
First, the buildDir() method was renamed to getBuildDir() to avoid
clashing with the buildDir field.

Then, a buildDir was added to both `OutputPath` and `InstallPath` but
not to `PhonyPath` as it does not contain any path components. Instead
the `PhonyPath.getBuildDir()` was changed to simply return "".

Bug: 183650682
Test: m droid
Change-Id: I12e1854c829b980c5c01205753c62c00dc0a4774
2021-03-25 09:18:29 +00:00
Paul Duffin
580efc8716 Add srcDir to SourcePath
Bug: 183650682
Test: m droid
Change-Id: I08f7dba449538d289a6c4d6b5325c1a833cff834
2021-03-25 09:18:29 +00:00
Paul Duffin
2b1d2b6e4d Merge "Cleanup some issues left over from previous reviews" 2021-03-25 09:16:40 +00:00
Paul Duffin
e742ee14f8 Merge "Convert test that disallows non existent paths to use fixtures" 2021-03-25 09:16:20 +00:00
Treehugger Robot
f7e79a591c Merge "Enable safe ICF for all devices" 2021-03-25 08:59:00 +00:00
Jingwen Chen
8b086655cd Merge "Revert "Add bp2build support for cc_library_static."" 2021-03-25 06:14:03 +00:00
Jingwen Chen
f9410540a8 Revert "Add bp2build support for cc_library_static."
Revert submission 1594391-bp2build-cc_library_static

Reason for revert: Broke the build on aosp-master
Reverted Changes:
Ib16ccf31a:Add cc_library_static macro to help with bp2build ...
I37c856be2:Add bp2build support for cc_library_static.

Change-Id: Ie94d5bc6da81758cd4e0461c08a810a29643c971
2021-03-25 06:11:59 +00:00
Rupert Shuttleworth
d3e10104b0 Merge "Add bp2build support for cc_library_static." 2021-03-25 05:39:39 +00:00
Paul Duffin
3cb2c06b44 Cleanup some issues left over from previous reviews
Bug: 183235980
Test: m nothing
Change-Id: I187765c56b889445de3a4729405ee745e5f73729
2021-03-24 22:08:05 +00:00
Paul Duffin
76e5c8a37f Convert test that disallows non existent paths to use fixtures
This change needed to add some additional files to the registered
files for PrepareForTestWithJavaDefaultModules because otherwise they
would fail when "TestAllowNonExistentPaths = false". Those files were
being added by the TestJavaLintRequiresCustomLintFileToExist (albeit in
some cases in different locations to that required by the default
modules but as the files are needed by the modules defined in
PrepareForTestWithJavaDefaultModules they should be defined in it.

A couple of other places also provided some files so moving them into
PrepareForTestWithJavaDefaultModules caused some conflicts which needed
to be resolved.

Bug: 183184375
Test: m nothing
Change-Id: I76ce9f1673c1c1c4000635b76b8377d582224bf1
2021-03-24 22:08:05 +00:00
Paul Duffin
958408d921 Merge "Cleanup usages of Dex2oatDepTag" 2021-03-24 22:04:58 +00:00
Jingwen Chen
95c6eb3959 Merge "bp2build: add allowlist for package-level conversions." 2021-03-24 21:38:07 +00:00
Nicolas Geoffray
ed35a90d2c Merge "Fix "filename too long" issue on Luci caused by test name" 2021-03-24 20:45:38 +00:00
Paul Duffin
688ba5bc2b Merge "Remove testSdkWithJava" 2021-03-24 17:59:09 +00:00
Paul Duffin
c8ad86479f Merge "Convert java_sdk_test.go tests to fixtures" 2021-03-24 17:57:47 +00:00
Lukács T. Berki
f2926e751c Merge "Make null builds always be null builds." 2021-03-24 15:48:02 +00:00
Paul Duffin
ac94726ef2 Remove testSdkWithJava
Bug: 181070625
Test: m nothing
Change-Id: I9ef906a386cc87f69b166ec88e0b6c7388c3d06a
2021-03-24 15:47:28 +00:00
Paul Duffin
9ec86b14e7 Convert java_sdk_test.go tests to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I6a1fabdd2c8385e5fbaef6985047f9d0bdceb209
2021-03-24 15:47:28 +00:00
Paul Duffin
a35178b105 Fix "filename too long" issue on Luci caused by test name
Stop including fields in the test name for TestJavaSdkLibraryEnforce to
reduce its length to avoid the filename too long limit.

Test: m nothing
      Check the lengths before (~240) and after (~74) to make sure
      there was a sizeable reduction.
Change-Id: I275a1110e5102b8ea8376759f28c7c6333a5efee
2021-03-24 15:09:37 +00:00
Paul Duffin
b506c9dc11 Cleanup usages of Dex2oatDepTag
Creates a new deptag type for it so that it can implement the marker
interfaces that will exclude it from being added to the APEX and from
visibility enforcement. The latter is probably not an issue ATM because
the dependencies are added after visibility checks are enforced but
this code is undergoing lots of refactoring so that may change.

Bug: 177892522
Test: m nothing
Change-Id: Ibd167d557adec761a2e3eed78f4d334c40a04fb9
2021-03-24 14:34:40 +00:00
Roland Levillain
d603c7dbae Fix a typo in the definition of varApexNames in java/dexpreopt_bootjars.go.
Change-Id: If1267cf18dcd307316cce8d2cac7c6f5c173f3d4
Test: `m nothing` in internal tree
Bug: 180325915
2021-03-24 14:24:04 +00:00
Rupert Shuttleworth
21e743df41 Add bp2build support for cc_library_static.
Also refactor bp2build-related code for cc_library_headers.

Test: Added unit test.

Test: bp2build-sync.py write; bazel build //bionic/... still works (but bp2build is disabled for most cc_library_static targets for other reasons)

Change-Id: I37c856be20a47b154909338a22a7dba1ab55693f
2021-03-24 13:56:44 +00:00
Paul Duffin
a1063c09f6 Merge changes I94f66e3e,I233a4fe1,Idbb37485
* changes:
  Group all the preparations needed for testing dexpreopt
  Separate methods used for fixture based and legacy tests
  Use more inclusive language in dexpreopt/testing.go
2021-03-24 13:41:56 +00:00
Thiébaud Weksteen
70204f9fc9 Merge "rust: Drop libgcc dependency" 2021-03-24 12:44:09 +00:00
Ulya Trafimovich
9023b02c00 Allow using updatable boot jars in dexpreopt (but don't use them yet).
This CL handles updatable boot jars in the same hacky way as we handle
non-updatable boot jars: it creates a set of predefined paths to the dex
jars in a global config, then traverses all modules in a singleton
context, finds updatable boot jars and adds copy rules from these jars
to the predefined paths. A proper way would be to register dependencies
of the dexpreopted modules on the boot jars and extracting paths to dex
files by walking these dependencies.

Bug: 178467404
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Test: added new Soong test
Change-Id: I87f764109315f79315d73bf43799b70eb010fc0b
2021-03-24 11:16:11 +00:00
Ulyana Trafimovich
8ef5b4f95f Merge "Split functions for compiling boot jars to allow future code reuse." 2021-03-24 10:48:51 +00:00
Lukacs T. Berki
c99c947c88 Make null builds always be null builds.
Previously, soong.environment.used was written after build.ninja and if
the amount of time that passed between the two was long enough, Ninja
would decide that build.ninja is older than soong.environment.used and
rebuild it.

Test: test_null_build in bootstrap_test.sh in a loop.
Change-Id: I5467da487e8e8f2646644b8a7fb9549b9ff18276
2021-03-24 10:51:05 +01:00
Paul Duffin
9fc9f53423 Group all the preparations needed for testing dexpreopt
Make it easier to test dexpreopt functionality by grouping all the
fixture preparations together.

Bug: 177892522
Test: m nothing
Change-Id: I94f66e3ec82efc4fd791f4fdab678d298565e452
2021-03-24 08:30:27 +00:00
Paul Duffin
3c84eaaa98 Separate methods used for fixture based and legacy tests
The fixture mechanism makes it easy to refactor by splitting up an
existing preparer into separate ones and then combining them back
together. Unfortunately, that becomes slightly more tricky when
preparers and legacy tests use the same functions to register build
components and define default modules.

This change splits the RegisterRequiredBuildComponentsForTest and
GatherRequiredDepsForTest methods into two methods each, with the
existing method used for legacy tests and calling the new method that
is used for the preparer.

At the moment all the functionality is in the new methods but over
time, as functionality is extracted into separate preparers, the
functionality can also be moved from the method that is common to both
legacy and fixture based tests into the legacy only method.

Bug: 177892522
Test: m nothing
Change-Id: I233a4fe1fb072a00292acc2bb20821ec82a9bd67
2021-03-24 08:27:04 +00:00
Paul Duffin
f8e3d8341a Use more inclusive language in dexpreopt/testing.go
Bug: 177892522
Test: m nothing
Change-Id: Idbb37485a573ddd25c4da48ab88f9c559fca5434
2021-03-24 08:26:48 +00:00
Paul Duffin
b6535d329f Merge "Remove javaFixtureFactory" 2021-03-24 08:25:20 +00:00
Paul Duffin
9d33cf237a Merge "Remove unused test methods and buildDir" 2021-03-24 08:23:31 +00:00