Commit graph

32120 commits

Author SHA1 Message Date
Jiyong Park
99644e92c8 rust modules can be included in apex
We will have some APEXes having rust binaries and libraries. So, adding
the support for the types of modules.

rust.Module now inherits from android.ApexModuleBase and implements
the android.ApexModule interface.

Bug: 172414324
Test: m

Exempt-From-Owner-Approval: rebased after +2 from the owner
Change-Id: I356ef4c45f782a6460f001e83af96d1710642d80
2020-11-30 15:40:48 +00:00
Ivan Lozano
57f434e858 rust: Add support for multiple protos per module.
This further emulates the rust-protobuf gen_mod_rs flag by providing
support for generating a single module containing multiple protobuf
definitions.

Bug: 171361369
Test: New Soong tests.
Test: Example module containing multiple protos works.
Change-Id: I815f9628a8289ae512758073dac49bc4535abf01
2020-11-30 14:55:00 +00:00
Paul Duffin
dd60120cfb Merge "java_sdk_library: Copy removed API specification file to dist" am: 8abcbdc0b3
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511916

Change-Id: I1181a19f5102c2b7c837492332186f12320c831f
2020-11-30 13:41:37 +00:00
Paul Duffin
ba5ddbd78c Merge "Remove redundant setting of DistFiles by apexBundle" am: 11402caa47
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511915

Change-Id: Ib12f4c17bd41dad2a005a28dd060ebbe1d245f06
2020-11-30 13:41:27 +00:00
Paul Duffin
e9b3c66e22 Merge "Remove redundant setting of DistFiles by java.Library" am: a67ff601d9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511914

Change-Id: I2795ea81390fe4532e10c4c208a47ab725b972d4
2020-11-30 13:41:15 +00:00
Paul Duffin
8abcbdc0b3 Merge "java_sdk_library: Copy removed API specification file to dist" 2020-11-30 13:24:19 +00:00
Paul Duffin
11402caa47 Merge "Remove redundant setting of DistFiles by apexBundle" 2020-11-30 13:24:00 +00:00
Paul Duffin
a67ff601d9 Merge "Remove redundant setting of DistFiles by java.Library" 2020-11-30 13:23:50 +00:00
Paul Duffin
fed1c082d0 Merge "Generalize the handling of dist tags" am: 5e95589fe9
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511913

Change-Id: If811b96f5ae891e85f7fbca7a7cf642568e2d808
2020-11-30 13:05:45 +00:00
Paul Duffin
982093faaf Merge "Differentiate between no dist tag and an empty dist tag" am: 4326e0e094
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511911

Change-Id: I9044ee88fbf8bd7b0fc800ea11f4369816bb23b9
2020-11-30 13:05:01 +00:00
Paul Duffin
5e95589fe9 Merge "Generalize the handling of dist tags" 2020-11-30 12:42:55 +00:00
Paul Duffin
4326e0e094 Merge "Differentiate between no dist tag and an empty dist tag" 2020-11-30 12:31:53 +00:00
Mathew Inwood
17a8cd9634 Add 'lo-prio' tag to temp blocklist.
This is a temporary measure to allow these APIs to be identified by
downstream tooling. Before S is finished, these APIs will be moved
to the max-target-R list anyway.

Adding the lo-prio flag should have no semantic impact on these APIs
but does allow them to be distinguished from the regular blocklist
APIs.

Test: m
Bug: 174455522
Change-Id: I031dcecbdef34a4f93a0fec1a92d95f769918486
2020-11-30 11:47:30 +00:00
Thiébaud Weksteen
db1f2483da Merge "Use 1.48.0 prebuilts" am: c9b71a446a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512614

Change-Id: I03d2c3fdc6753a840557c94540bd610928e55bbb
2020-11-30 09:51:20 +00:00
Thiébaud Weksteen
c9b71a446a Merge "Use 1.48.0 prebuilts" 2020-11-30 09:29:23 +00:00
Paul Duffin
61a4f20964 Improve test coverage of dist processing code am: 62269498be
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512612

Change-Id: Id74f0ee3d7b4077ca3ff60292530ec12c0696b3c
2020-11-27 19:13:06 +00:00
Paul Duffin
879488716b Expand dist property checks to cover dists am: 89968e3dec
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511912

Change-Id: Ie88a749d00c712cc92ff9f7d2376ab0cc30d03c4
2020-11-27 19:13:02 +00:00
Paul Duffin
f3f916c2a6 Remove duplicate tests from TestGetDistForGoals am: d83988dbad
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512611

Change-Id: Ic06bb2b734c237bd51aec0ad961358d8a5260880
2020-11-27 19:12:58 +00:00
Paul Duffin
04159e7961 Separate dist processing from make specific output am: 8b0349c652
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512610

Change-Id: Ied0373d2789934815175f7b761e331cc5106553c
2020-11-27 19:12:28 +00:00
Ulya Trafimovich
3bfabf2626 Documenting java/dexpreopt_bootjars.go
Test: m nothing
Bug: 173092919
Change-Id: I20018e1df6ad5e21172bb91435ae2c9cad7024e6
2020-11-27 18:10:22 +00:00
Ulya Trafimovich
480d174a16 Documenting dexpreopt/class_loader_context.go.
Test: m nothing
Bug: 173092919
Change-Id: Iec64d7449a66b45be547e0d42bb3808215750cd1
2020-11-27 18:10:22 +00:00
Paul Duffin
62269498be Improve test coverage of dist processing code
This change adds some additional tests to cover the different
combinations of setting OutputFile and DistFiles properties in the
AndroidMkEntry.

These tests will prevent follow up changes from inadvertently changing
the behavior that some modules rely on.

Test: m nothing
Bug: 174226317
Change-Id: I6ab6f2711a5fa416ea2b84a1e0b8754d525d7a0c
2020-11-27 15:17:44 +00:00
Paul Duffin
040e906fcf java_sdk_library: Copy removed API specification file to dist
Uses the dists property to copy the removed API specification file for
each API scope alongside the API specification file in the dist.

Bug: 158544979
Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences other than
      the after now contains -removed.txt files for each scope of each
      java_sdk_library
Change-Id: If23c594d29a12e49d2c5a8945ad488874bb1590c
2020-11-27 15:17:44 +00:00
Paul Duffin
281bc00bbf Remove redundant setting of DistFiles by apexBundle
A previous change handles dist properties automatically for all module
types and as a result has made the apexBundle setting of DistFiles
redundant so this change removes that.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: If6f741b5b73b3335f4c478a6528f5f311cd633e4
2020-11-27 15:17:44 +00:00
Paul Duffin
620680516d Remove redundant setting of DistFiles by java.Library
A previous change handles dist properties automatically for all module
types and as a result has made the java.Library setting of DistFiles
redundant so this change removes that and the tests that duplicate
tests of the general mechanism.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Bug: 174226317
Change-Id: Ib79a3bdd46897efd84a9c456c37c374bd6036303
2020-11-27 15:17:44 +00:00
Paul Duffin
af970a2e7a Generalize the handling of dist tags
Previously in order to make use of dist entries with tag properties it
was necessary for each module type to be modified to store the results
of GenerateTaggedDistFiles(module) in the AndroidMkEntry.DistFiles
property. This change generalizes that mechanism to work across all
module types.

This change does improve the behavior of a couple of tests where
previously a dist with tag property was ignored because tagged dist
files were not available now it is used correctly.

Some module types do not implement OutputFileProducer interface and so
they cannot handle dist with tag properties. This change makes it an
error to attempt to do that. That necessitated adding OutputFiles(tag)
method to customModule in queryview_test.go as some of the tests in
that file used dist entries with tag properties.

Follow up changes will remove the code that was made redundant by this.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Bug: 174226317
Change-Id: Ifc54d67db10ce0d0fe8179c05b97a2be2113be4e
2020-11-27 15:17:44 +00:00
Paul Duffin
74f05598eb Differentiate between no dist tag and an empty dist tag
Change https://r.android.com/1335521 added tag property to the Dist
struct so that it could be used to select one of a number of different
output files to copy to the dist instead of the single file that the
module type made available for dist. The output files were selected
by passing the tag to OutputFiles(tag).

Module types that wanted to support this new approach had to explicitly
set AndroidMkEntries.DistFiles = GenerateTaggedDistFiles(module).
Unfortunately, doing that had a side effect of changing the behavior of
dist entries without a tag.

That was because the change treated a tag that was not specified, as
being the same as "". So, prior to the change no tag meant use the
default dist file but after it meant use the paths returned by
OutputFiles(""). That changed the behavior of the java.Library type
which affected the behavior of the android_app module type.

Prior to the change the java_library would make the
Library.outputFile available for dist when no tag was specified. After
that change it would make Library.outputFile plus
Library.extraOutputFiles. The latter is usually empty except for
android_app which adds some extra files into there which will now be
copied to the dist. That change may have been intentional but there
was no mention of it in the change or the bug. Even if it wasn't
intentional it may still be beneficial.

Any module type that wants to add support for tags in dist runs the
risk of introducing similar changes in behavior. This change
differentiates between the tag not being set and the tag being set to
"" to avoid that possibility and to make the default behavior
explicit for those module types that have switched.

It does so as follows:
* Adds a DefaultDistTag constant that is used when the tag is not set.
  It is a string that is unlikely to be used as an actual tag as it
  does not start with a . and uses some special characters.
* The DefaultDistTag is used in MakeDefaultDistFiles(paths) to indicate
  that the supplied paths are the default ones and and also in
  GenerateTaggedDistFiles() for Dist structures that have no tag
  property set.
* The DefaultDistTag is passed to OutputFiles(tag) just in case the
  module type has explicitly defined the paths to associate with that
  tag in there. If it has then it overrides the legacy behavior. If it
  has not then it is just ignored and falls back to using the previous
  behavior.
* The java.Library.OutputFiles(tag) method explicitly handles the
  DefaultDistTag and returns Library.outputFile for it which restores
  the behavior from before the change that added dist.tag support.
* Similar change was made to apexBundle.OutputFiles(tag) in order to
  preserve its previous behaviour.
* The customModule used by TestGetDistContributions has been modified
  to also preserve its previous behavior after this change.

Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
      Test the effect on the apex by following instructions in
      http://b/172951145
Bug: 174226317
Change-Id: Ib8f0d9307751cc2ed34e3d9a5538d3c144666f6d
2020-11-27 15:17:44 +00:00
Paul Duffin
89968e3dec Expand dist property checks to cover dists
Previously, only the dist property's nested properties were checked for
correctness. This change also checks the dists property's nested dist
structures and adds some tests to verify that the checks are run and
correctly report the location of the incorrect property even when it is
within a slice of dist structs.

Test: m nothing
Bug: 174226317
Change-Id: If5a19360e1e4c98ee3b5afc813e35349d1fc6f6f
2020-11-27 15:17:44 +00:00
Paul Duffin
d83988dbad Remove duplicate tests from TestGetDistForGoals
A previous change duplicated the test cases from TestGetDistForGoals()
to test the getDistContributions() method. This change removes the
duplicate tests and leaves a single test to verify that
GetDistForGoals() uses getDistContributions() and
generateDistContributionsForMake() correctly.

Test: m nothing
Bug: 174226317
Change-Id: I545501016b932f9af0d5fda2f452a3a09932d3fb
2020-11-27 15:17:44 +00:00
Paul Duffin
8b0349c652 Separate dist processing from make specific output
Previously, the GetDistForGoals(Module) func combined the processing
of the dist properties with generating the make specific rules for
generating that dist. That has a couple of problems:
1. It combines two pieces of functionality into one method which is
   bad practice.
2. It makes it hard to test because the make specific output ends up
   containing absolute paths to temporary directories created by the
   test.
3. It makes switching to a non-make output difficult and fragile as
   changing the output will also require changing the tests.

This change adds an intermediate data structure to contain the result
of the dist processing. That processing is done by the new method
getDistContributions(Module) which returns the new intermediate
structure. It also adds generateDistContributionsForMake(..) to
generate the make output. The GetDistForGoals(Module) func uses them to
implement the previous behavior.

It adds identical tests to those in TestGetDistForGoals() but leaves
those tests alone to show that this refactoring does not change the
behavior. Follow up changes will clean up TestGetDistForGoals(). It
also adds a test for generateDistContributionsForMake(..).

Bug: 174226317
Test: m nothing
      m dist sdk - before and after this change, compare result to
      make sure that there are no significant differences.
Change-Id: I458b7c8e4485bf66d3498f50df85a8d65fc2ee00
2020-11-27 15:17:38 +00:00
Thiébaud Weksteen
c6fe19d637 Use 1.48.0 prebuilts
Bug: 173721343
Test: build external/rust/ and crosvm
Change-Id: Ifde24640fc262c5eebdcd623556e925e7d78ae85
2020-11-27 14:42:57 +01:00
Paul Duffin
bc71c946d8 Merge "Extract method to process bp file for androidmk_test.go" am: 3e9614198a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1512609

Change-Id: I21e3286f6967560d17bb71b489a848bb569a2972
2020-11-27 12:28:55 +00:00
Paul Duffin
3e9614198a Merge "Extract method to process bp file for androidmk_test.go" 2020-11-27 11:53:46 +00:00
Justin Yun
7a44479e0a map list files must have newline characters
module_names.txt and module_paths.txt did not end with the newline
character in each line. Instead it had literal '\n'.
The character must use '\n' without escape character.

Test: m vndk
Change-Id: I1de1d0993994c8a95b02426777f62bfb2bed7b6f
2020-11-27 20:30:44 +09:00
Ulyana Trafimovich
38352b8f69 Merge "Assume any <uses-library> is shared, add only toplevel ones to manifest." am: af07f732e6
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511271

Change-Id: Ia0dd1ab9cd3f89dcaed1b8a54f336103f4732a7a
2020-11-27 11:16:28 +00:00
Ulyana Trafimovich
af07f732e6 Merge "Assume any <uses-library> is shared, add only toplevel ones to manifest." 2020-11-27 10:44:54 +00:00
Paul Duffin
103aaae715 Extract method to process bp file for androidmk_test.go
Test: m nothing
Bug: 174226317
Change-Id: Ifc9ee5088fe766d971718fc112d1e95719a2b825
2020-11-27 01:24:02 +00:00
Treehugger Robot
024e2442bc Merge "Documenting apex/builer.go" am: 9c4c534d25
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1509688

Change-Id: I5c65d5e0bbc591de5591f6d1c9228042f036675f
2020-11-27 01:08:31 +00:00
Treehugger Robot
9c4c534d25 Merge "Documenting apex/builer.go" 2020-11-27 00:55:40 +00:00
Ulf Adams
b73daa588d Soong: use deterministic temp dir names in sbox
The sbox tool is used to wrap genrule actions, and run them in a
temporary directory. Afterwards, the outputs are moved into their
proper location in the output tree. However, some tools embed the file
name of the output file (as passed to the tool) into the output file.

For example, the perfetto code generator script uses the output file
name to generate a C header guard when it generates
perfetto_src_base_version_gen_h/gen/perfetto_version.gen.h.

When using remote execution / remote caching, these genrule actions are
run locally whenever the output file doesn't exist, effectively making
them unique (across time and between users).

They then cause cache misses on all actions depending on these output
files as well as on transitive actions. In the above example, this
causes libperfetto.so to differ, which then causes all actions
depending on libperfetto.so to be rerun unnecessarily.

As a fix, this commit changes the sbox tool to use the sha1 hash of the
manifest path. The tool already seems to assume that there are no
concurrent runs of the same tool with the same manifest (otherwise
it would fail flakily depending on exact timing). It seems therefore
safe to use a temporary path deterministically derived from the
manifest path.

Test: ran multiple builds w/ remote execution; observe proper caching
Change-Id: I5b73ffd3b7f85cbb0336dfa1675de7ac0e2fd1a8
2020-11-26 22:11:05 +01:00
Paul Duffin
80b40d415a Merge "Improve TestGetDistForGoals debuggability" am: d35d92a7b5
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511910

Change-Id: I32162832b2fe085ded36f671cd4abd2982f25105
2020-11-26 17:38:59 +00:00
Paul Duffin
d35d92a7b5 Merge "Improve TestGetDistForGoals debuggability" 2020-11-26 17:16:53 +00:00
Thiébaud Weksteen
e006241e6f Merge "Ignore variant dependencies in rust-project.json" am: 463ab7a27e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511272

Change-Id: Iddf43a1dcd27f625656bfe75ac824f980e034833
2020-11-26 10:12:17 +00:00
Thiébaud Weksteen
463ab7a27e Merge "Ignore variant dependencies in rust-project.json" 2020-11-26 09:15:09 +00:00
A. Cody Schuffelen
4e5fc3b422 Allow rust code in device/google/cuttlefish
Test: TreeHugger
Change-Id: If7bce48e101bd49389d952556bfdf95d7899f01b
2020-11-25 18:29:39 -08:00
Treehugger Robot
c8e3ceb4b5 Merge "Remove hardlink optmization in sbox" am: 5bbec1a371
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1511711

Change-Id: I85d7c64d249c7ec813c2636f9a767ec909968054
2020-11-26 00:20:34 +00:00
Liz Kammer
0a1221c89a Merge "Refactor rust to use AndroidMkEntries" am: 0fd0e95e1b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1510112

Change-Id: I993479a405c6eea4b448ef039911fcb35c39af4e
2020-11-26 00:20:24 +00:00
Liz Kammer
b1084e3f6c Merge "Refactor python to use AndroidMkEntries." am: c55dcfcc2b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1510111

Change-Id: I61e4f606a50161ea82a0d18f3b21ab44118effa7
2020-11-26 00:20:16 +00:00
Liz Kammer
adc711564e Merge "Convert csuite config to use AndroidMkEntries" am: 6953adeb95
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1510569

Change-Id: If091d2cfadc2c41a7b15fe2ef24d89e63c630e85
2020-11-26 00:20:08 +00:00
Treehugger Robot
5bbec1a371 Merge "Remove hardlink optmization in sbox" 2020-11-26 00:20:01 +00:00