Commit graph

1423 commits

Author SHA1 Message Date
Liz Kammer
18f643090f Add blueprint owners: joeo, eakammer, patricearruda
Change-Id: I1ec308a022b6a19cd4bd2fac24646862925c55e9
2020-10-06 08:42:52 -07:00
Colin Cross
f94644fe3d Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Add Providers to Blueprint

Test: m checkbuild
Change-Id: I4d722c70ce347b16e6258ecd4c3df3adc9d17035
2020-09-23 10:40:18 -07:00
colincross
8a747a0864
Merge pull request #309 from colincross/providers
Add Providers to Blueprint
2020-09-23 10:27:40 -07:00
Colin Cross
2da8492b9d Add Providers to Blueprint
Providers are a new concept for Blueprint, based on providers in Bazel:
https://docs.bazel.build/versions/master/skylark/rules.html#providers

Providers aim to simplify the interaction between modules by replacing
type asserting to arbitrary interfaces with requesting optional data
objects from modules.  This will also move Blueprint closer to supporting
incremental analysis by serializing the providers and only rerunning
the analysis phase on modules whose inputs have changed.

Change-Id: I39f5f78b372412a7dbf151ceccb3f917f6c874bf
2020-09-22 18:20:18 -07:00
Colin Cross
f38eda59a8 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Return dependency modules from dependency-adding methods.
  Support pausing parallelVisit

Test: m checkbuild
Change-Id: I5a01454e5d66d8fe00ef2f363d0fcbcfcee9fef9
2020-09-22 11:05:03 -07:00
colincross
66fa73dd6e
Merge pull request #316 from skvadrik/add-dep-ret-mod
Return dependency modules from dependency-adding methods.
2020-09-22 10:59:46 -07:00
colincross
ca601ee4fb
Merge pull request #318 from colincross/pause_parallel_visit
Support pausing parallelVisit
2020-09-22 10:58:03 -07:00
Ulya Trafimovich
9577bbc922 Return dependency modules from dependency-adding methods.
The motivaion for this change is to allow writing code that uses the
newly added dependency module in the same mutator pass, for example to
add more dependencies. Like this:

  for _, m := range ctx.AddVariationDependencies(nil, tag, deps...) {
      if someModuleProperty(m); ok {
          ctx.AddVariationDependencies(nil, tag, otherDep)
      }
  }

Note that there is no guarantee that the returned module has already
been processed by the current mutator.

The patch does not add runtime overhead on findng dependency modules,
as this has already been done previously.

Test: go test
2020-09-17 11:43:15 +01:00
Colin Cross
c4773d90a2 Support pausing parallelVisit
Pass a channel to visitor functions called by parallelVisit that
allows them to pause the current visitor until a given visitor
has finished.  This allows parallelVisit to work on a dependency
graph while it is being mutated.

Test: Test_parallelVisit
Change-Id: Id8b1542c22ac9914439310e31d992ae0d7318d69
2020-09-16 12:58:39 -07:00
Colin Cross
60bb204d8f Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Relax check in moduleMatchingVariant
  Fix AddFarVariationDependencies subset checks
  Maintain ordering between variants and aliases
  Add tests for findVariant

Test: m checkbuild
Change-Id: I4c166ff93a09fe4c6d700c82f79548f0b4afdae0
2020-09-11 20:20:41 -07:00
colincross
4b3efdc5b8
Merge pull request #321 from colincross/far_variations
Relax check in moduleMatchingVariant
2020-09-11 20:15:21 -07:00
Colin Cross
edbdb8c2d3 Relax check in moduleMatchingVariant
Pull request #317 made the equality check in moduleMatchingVariant
more correct by comparing the variant maps instead of the names.
Using only the names effectively ignores any variation with an
empty name.  Unfortuantely this broke a current user of
ReplaceDependenciesIf.  Go back to the relaxed check for now.

Test: build on a branch with sdk modules
Change-Id: I11016c8df7bd91fd022d04fd3033756f54d7fa0b
2020-09-11 19:24:59 -07:00
colincross
277bac9904
Merge pull request #317 from colincross/far_variations
Fix AddFarVariationDependencies
2020-09-11 14:57:09 -07:00
Colin Cross
5dc6759951 Fix AddFarVariationDependencies subset checks
AddFarVariationDependencies claims to take the first variant that
matches all the requested variations, but it did nothing of the sort.
It took the first variant that either matched or did not contain each
of the requested variations.  A module with no variations was always
matched, and requesting variations that didn't apply to a module
still matched (for example, requesting an image variation for a
host module that was ignored by the image mutator).

Fix AddFarVariationDependencies by making subset actually check
for subsets.

Test: Test_findVariant
Change-Id: I10063fec342db2a1c0685a7db08e4a650d14bd4e
2020-09-09 18:29:15 -07:00
Colin Cross
5df74a8e38 Maintain ordering between variants and aliases
AddFarVariationDependencies takes the first matching variant.  To
maintain sensible behavior on a module with aliases, the ordering
of aliases and module variants needs to be maintained so that
AddFarVariationDependencies can find an earlier matching alias
instead of a more specific variant.

Test: go test .
Change-Id: I78f4e96edd98159f3a62d94e240e5d652667bec4
2020-09-09 18:29:15 -07:00
Colin Cross
39644c0903 Add tests for findVariant
Add tests for findVariant behavior that provides the matching
behaviors of AddVariationDependencies, AddFarVariationDependencies,
etc.

Test: Test_findVariant
Change-Id: I3494d57179c8b3d62f7d32e5a1b43c9b9672c2df
2020-09-09 18:27:32 -07:00
Colin Cross
8dfb36edbf Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Enable variants of bootstrap modules
  Make VisitAllModuleVariants, PrimaryModule and FinalModule available to mutators
  bpmodify: handle nested properties

Test: m checkbuild
Change-Id: I9785802ed33fa819b705317cd50c38432972e94c
2020-09-09 15:18:51 -07:00
colincross
ad63e18efe
Merge pull request #320 from colincross/bootstrap_variants
Enable variants of bootstrap modules
2020-09-09 15:01:12 -07:00
Colin Cross
d2458a23bf Enable variants of bootstrap modules
The primary builder may want to create variants of bootstrap
modules if they need to fit in to the primary builder's dependency
graph.  Enable arbitrary variants of bootstrap modules by only
running the module's actions on the primary variant and then
copying the result to any other variants that exist.

Test: m checkbuild
Change-Id: I24b97771bb11faeacab4079ed8cf69aef59da140
2020-09-09 14:32:42 -07:00
Colin Cross
18926521dc Make VisitAllModuleVariants, PrimaryModule and FinalModule available to mutators
Move the VisitAllModuleVariants, PrimaryModule and FinalModule methods to
baseModuleContext so they can be used by mutators.

Test: m checkbuild
Change-Id: I1827ce2fc75f017460a7f6a53b1dab6a81109435
2020-09-09 13:25:54 -07:00
Dan Willemsen
34e8ddd0a5
Merge pull request #294 from silverneko/master
bpmodify: handle nested properties
2020-09-02 17:28:12 -07:00
Xin Li
32feafd96e Mark Android R (rvc-dev-plus-aosp-without-vendor@6692709) as merged
Bug: 166295507
Merged-In: I0b10c9cb0b11dfa1677b16e472a26208932d4c08
Change-Id: I6e1add57781b620195e34a7feae07c1e82996a9f
2020-08-27 09:44:04 -07:00
Colin Cross
ba9a3b9ea8 Merge remote-tracking branch 'aosp/upstream' am: ce522b4826
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1401130

Change-Id: Ib14cd39cef93cefdcbcb78564767b936c77856af
2020-08-14 23:00:02 +00:00
Colin Cross
ce522b4826 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Add CreateAliasVariation
  Combine variant fields into variant struct

Bug: 164216768
Test: m checkbuild
Change-Id: Ie46adc27a7af6a35803f2189ed7dbc154de79541
2020-08-14 14:05:56 -07:00
colincross
bc973ffe1e
Merge pull request #314 from colincross/alias_variations
Add CreateAliasVariation
2020-08-14 13:47:14 -07:00
Colin Cross
279489c017 Add CreateAliasVariation
CreateAliasVariation creates a new variation for the current mutator
that is an alias to a variation that was created with CreateVarations
by the same mutator.  It is similar to AliasVariation, except that
AliasVariation creates an alias from the pre-mutated variant to one
of the new variations, while CreateAliasVariation creates an alias
from a new post-mutated variation to one of the new variations.

Bug: 164216768
Test: TestCreateAliasVariation
Change-Id: I8847b8d6fc1d3248abc910b2fe2399881f9bdaee
2020-08-14 11:15:58 -07:00
Colin Cross
edc41769fe Combine variant fields into variant struct
Combine variant, variations and dependencyVariations into a single
struct.

Test: blueprint tests
Change-Id: I76811bb2bdf76f1b4cffc6a7b9bc7362ecb5f7b9
2020-08-14 11:15:58 -07:00
Colin Cross
ec4ae78c09 Merge remote-tracking branch 'aosp/upstream' into inclusive am: c69182870e
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1380886

Change-Id: I5ca3261464e85a1bcf413f6d9fa5addee84f39e1
2020-08-06 20:06:02 +00:00
Colin Cross
c69182870e Merge remote-tracking branch 'aosp/upstream' into inclusive
* aosp/upstream:
  Android inclusive language fixit.  Make comment more succinct, clear and remove use of term "sanity".
  Update bpdocs to filter nested properties by tags (#312)

Bug: 162536543
Test: m checkbuild
Change-Id: I4b40b1637e142d2f2fd513f62357213a9832bbaa
2020-07-30 17:28:24 -07:00
colincross
576b0fd577
Merge pull request #313 from colincross/inclusive
Android inclusive language fixit.  Make comment more succinct, clear …
2020-07-30 16:51:50 -07:00
Daniel Cardenas
5de3dc04e0 Android inclusive language fixit. Make comment more succinct, clear and remove use of term "sanity".
Bug: 162536543
Change-Id: I4b40b1637e142d2f2fd513f62357213a9832bbaa
2020-07-30 16:48:42 -07:00
Liz Kammer
9810dcd4b9
Update bpdocs to filter nested properties by tags (#312)
* Update bpdocs to filter nested properties by tag

Prior to this change nested properties tagged with `blueprint:"mutated"`
are erroneously included in documentation, this corrects that behavior
and adds testing to verify.

Tested: go test bpdocs tests
Change-Id: I822c9a98276634d2f584d8709e83003824cdffd5
2020-07-28 09:30:46 -07:00
Colin Cross
b06c753d91 Merge remote-tracking branch 'aosp/upstream' am: 9b15d1a888 am: d5cf2cb6be
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1364798

Change-Id: I0b10c9cb0b11dfa1677b16e472a26208932d4c08
2020-07-16 22:25:46 +00:00
Colin Cross
d5cf2cb6be Merge remote-tracking branch 'aosp/upstream' am: 9b15d1a888
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1364798

Change-Id: I9fcc87af3d45dcc134e8b32cacbab1d0157448dc
2020-07-16 22:08:46 +00:00
Colin Cross
9b15d1a888 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Add flag to use validations for tests
  Add Validations support to Blueprint
  Remove unused Context.ModulePath

Test: m checkbuild
Change-Id: Ib63d651e79801121681dff449136067d1ea3b321
2020-07-16 13:00:57 -07:00
colincross
94594808c5
Merge pull request #308 from colincross/modulepath
Remove unused Context.ModulePath
2020-07-16 12:54:17 -07:00
colincross
ab003ffcee
Merge pull request #311 from colincross/validations
Add Validations support to Blueprint
2020-07-09 19:20:01 -07:00
Colin Cross
0cdec99c81 Add flag to use validations for tests
Using validations for tests ensures the tests run without blocking
the critical path.

Change-Id: Icb21a52e96f70d815f7df86882351c13f5575cf5
2020-07-09 14:28:20 -07:00
Colin Cross
9ece72b055 Add Validations support to Blueprint
The Android fork of Ninja supports "validations", a node that is
added to the top level of the build graph whenever another node
is in the build graph.  Add support in Blueprint to specify them
on build statements and write them to the ninja.

Test: ninja_writer_test.go
Change-Id: I87cd1281dbd2ed113cc26a265c50d20c65118c91
2020-07-09 14:28:20 -07:00
Colin Cross
e9e15a5059 Merge remote-tracking branch 'aosp/upstream' am: 726d2e6085 am: 8ea9686962
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1354383

Change-Id: Id1e1f9f30ccf683dd7e8ecf5a0b4f10c64d13628
2020-07-08 18:33:53 +00:00
Colin Cross
8ea9686962 Merge remote-tracking branch 'aosp/upstream' am: 726d2e6085
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1354383

Change-Id: Ic720c2289434125580ff1e832a32ddf1438f1f30
2020-07-08 18:22:28 +00:00
Colin Cross
8e454c5575 Remove unused Context.ModulePath
Context.ModulePath is a duplicate of Context.BlueprintFile, and is
misleading because it returns the path to the Android.bp, and not
the path to the directory that contains the Android.bp file like
ModuleContext.ModuleDir.

Change-Id: I505136fce3c3d37595cd4b4e08de5e2691a2a0f6
2020-07-06 13:34:42 -07:00
Colin Cross
726d2e6085 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Add ReplaceDependeciesIf to allow for conditional replacement

Test: m checkbuild
Change-Id: Iff96881da42c7ea3c3594d2ff2451215c38c4cdc
2020-07-02 09:04:03 -07:00
colincross
e635dc0568
Merge pull request #307 from paulduffin/master
Add ReplaceDependeciesIf to allow for conditional replacement
2020-06-30 09:37:59 -07:00
Paul Duffin
8969cb6170 Add ReplaceDependeciesIf to allow for conditional replacement
Adds support for making the replacement of one dependency with another
conditional on the from and to module as well as the tag used.
2020-06-30 12:15:26 +01:00
Dan Willemsen
ae72f8c943 Merge remote-tracking branch 'aosp/upstream' am: d877dde242 am: bc3c09c88d
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1349285

Change-Id: I279abc2a2c2a088e7fee4e573ff6447b009678b5
2020-06-25 01:09:56 +00:00
Dan Willemsen
bc3c09c88d Merge remote-tracking branch 'aosp/upstream' am: d877dde242
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1349285

Change-Id: I5847b94560a809797bee3b7cbff9de1c87a8b1a8
2020-06-25 00:54:17 +00:00
Dan Willemsen
d877dde242 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Stop writing depfiles from Glob singleton

Bug: 159845846
Test: manually check that changes to directories cause glob re-execution
Test: second build after a clean does not rerun soong_build
Change-Id: I1f257f3f11f70cb407397854fd85292dd0d0b08e
2020-06-24 16:22:27 -07:00
Dan Willemsen
fe13954f3c
Merge pull request #306 from danw/globs
Stop writing depfiles from Glob singleton
2020-06-24 14:35:11 -07:00
Dan Willemsen
55252655d2 Stop writing depfiles from Glob singleton
This makes a minimal difference on my AOSP tree (which is on a fast NVME
drive), but it saves ~10-15% of the time in our primary builder
(soong_build) on larger trees on slower drives.

We're always going to run the glob tool at least once outside of the
primary builder, and ninja only loads depfiles created after running the
tool, so this should be a no-op change.

We still need to write the list file (if it has changed), since that's
the file we're adding as a dependency of the primary builder, so the
effective timestamp must be earlier than the build.ninja timestamp.

Change-Id: I872b7581da50e7f2089daa7e248ca05b6703f019
2020-06-24 14:23:22 -07:00