Commit graph

35 commits

Author SHA1 Message Date
Usta Shrestha
2bae13b095 Use phony ninja outputs to reduce file-size
1. scan if any set of order-only deps are repeated
  2. if so extract them as a phony output to be shared

Test: m libc
Bug: NA
Change-Id: I0689111b97bbbd1f3b26650e8ae2e0a4ffb5085e
2023-03-07 01:39:27 -05:00
Sam Delmerico
26e44b7b78 apply gofmt
Change-Id: I2416e246b3d8485a6b7810b998cff2f45f6a0494
2023-02-21 15:11:20 -05:00
Spandan Das
ed4af01be6 Conditional inclusion of blueprint modules
This introduces a new `blueprint_package_includes` module type. If
present, other blueprint modules in that file will be analyzed
if and only if the requested include tags are met

example syntax:
```
Android.bp
blueprint_packgage_includes {
  match_all: ["tag1", "tag2", ...],
}

other_module1 {...}
other_module2 {...}
```
other_module1 and other_module2 will not be analyzed unless
tag1,tag2, ... are set

This also adds a new object of type `IncludeTags` to the Context object,
which is a container for these string keys.

Test: In build/blueprint, go test ./
Test: TH

Change-Id: I79de0d7da3224a5b2025c27a5137f39d00c7382e
2022-12-02 01:46:28 +00:00
Liz Kammer
9967d62fad Update CreateModule to require a type name
This makes the LoadHook and TopDownMutatorContext interfaces for
CreateModule the same.

Test: CI
Change-Id: I30122099bdf109cda40f9d81a574dd6fa861f459
2022-04-26 09:15:13 -04:00
kgui
a78b020089 Support writing inputs/outputs of actions of modules into a file from the moduleInfo.actionDefs.
An example module variant in the module-actions.json:
{
	"Name": "metalava-gradle-plugin-deps",
	"Variations": null,
	"DependencyVariations": null,
	"Deps": [
		{
			"Name": "prebuilts_gradle-plugin_license",
			"Variations": null,
			"DependencyVariations": null,
			"Tag": ""
		}
	],
	"Type": "",
	"Blueprint": "prebuilts/gradle-plugin/Android.bp",
	"Module": {
		"Actions": [
			{
				"Inputs": [
					"prebuilts/gradle-plugin/com/android/tools/lint/lint-api/30.1.0-alpha13/lint-api-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/lint/lint-checks/30.1.0-alpha13/lint-checks-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/lint/lint/30.1.0-alpha13/lint-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/lint/lint-gradle/30.1.0-alpha13/lint-gradle-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/lint/lint-model/30.1.0-alpha13/lint-model-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/common/30.1.0-alpha13/common-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/sdk-common/30.1.0-alpha13/sdk-common-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/sdklib/30.1.0-alpha13/sdklib-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/external/org-jetbrains/uast/30.1.0-alpha13/uast-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/external/com-intellij/intellij-core/30.1.0-alpha13/intellij-core-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/external/com-intellij/kotlin-compiler/30.1.0-alpha13/kotlin-compiler-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/repository/30.1.0-alpha13/repository-30.1.0-alpha13.jar",
					"prebuilts/gradle-plugin/com/android/tools/build/manifest-merger/30.1.0-alpha13/manifest-merger-30.1.0-alpha13.jar"
				],
				"Outputs": [
					"out/soong/.intermediates/prebuilts/gradle-plugin/metalava-gradle-plugin-deps/linux_glibc_common/combined/metalava-gradle-plugin-deps.jar"
				]
			},
			{
				"Inputs": null,
				"Outputs": [
					"out/soong/.intermediates/prebuilts/gradle-plugin/metalava-gradle-plugin-deps/linux_glibc_common/meta_lic"
				]
			}
		]
	}
}

Test: local
Change-Id: Icbb7236507251e257f6773b110ae8a0788eef41e
2022-01-28 14:50:24 +08:00
kgui
20f19a5d9b Add JSON data related struct and function into context.
Test: local test
Change-Id: I7a2000b458378f240cd6481066c15dec98d110ea
2022-01-10 14:55:06 +08:00
Lukacs T. Berki
eef5685c65 Rename Blueprints to Android.bp .
This was the only one in the source tree.

Side cleanup: remove some dead code that I assume comes from the time
where Blueprint files had to specify what subdirectories other Blueprint
files are in.

Test: Presubmits.
Change-Id: If84c4e85bc5516f30da97c1be29b56e50dddb3c4
2021-09-02 11:48:19 +02:00
Colin Cross
e5ff770c95 Add variant to dependency cycle errors
Use moduleInfo.String() to print dependency cycle errors so they
contain the variant.

Test: Test_parallelVisit
Change-Id: I2bddaa35c8abb57c42b4c424e861361a8813d588
2021-04-28 09:40:47 -07:00
Colin Cross
7d4958d84c Fix detecting cycles in parallelVisit
Fix detecting cycles in parallelVisit when the first alphabetical
module is not part of the cycle, but depends on the cycle.  Instead of
starting from the first alphabetical module, check every module in a
determinsitic order and return the first time a cycle is found.

Test: Test_parallelVisit
Change-Id: I03726f838ec42975251088ba75158103940115c2
2021-02-08 15:56:43 -08:00
Colin Cross
c4773d90a2 Support pausing parallelVisit
Pass a channel to visitor functions called by parallelVisit that
allows them to pause the current visitor until a given visitor
has finished.  This allows parallelVisit to work on a dependency
graph while it is being mutated.

Test: Test_parallelVisit
Change-Id: Id8b1542c22ac9914439310e31d992ae0d7318d69
2020-09-16 12:58:39 -07:00
Colin Cross
5dc6759951 Fix AddFarVariationDependencies subset checks
AddFarVariationDependencies claims to take the first variant that
matches all the requested variations, but it did nothing of the sort.
It took the first variant that either matched or did not contain each
of the requested variations.  A module with no variations was always
matched, and requesting variations that didn't apply to a module
still matched (for example, requesting an image variation for a
host module that was ignored by the image mutator).

Fix AddFarVariationDependencies by making subset actually check
for subsets.

Test: Test_findVariant
Change-Id: I10063fec342db2a1c0685a7db08e4a650d14bd4e
2020-09-09 18:29:15 -07:00
Colin Cross
5df74a8e38 Maintain ordering between variants and aliases
AddFarVariationDependencies takes the first matching variant.  To
maintain sensible behavior on a module with aliases, the ordering
of aliases and module variants needs to be maintained so that
AddFarVariationDependencies can find an earlier matching alias
instead of a more specific variant.

Test: go test .
Change-Id: I78f4e96edd98159f3a62d94e240e5d652667bec4
2020-09-09 18:29:15 -07:00
Colin Cross
39644c0903 Add tests for findVariant
Add tests for findVariant behavior that provides the matching
behaviors of AddVariationDependencies, AddFarVariationDependencies,
etc.

Test: Test_findVariant
Change-Id: I3494d57179c8b3d62f7d32e5a1b43c9b9672c2df
2020-09-09 18:27:32 -07:00
Paul Duffin
72bab1707e WalkDeps - only record module visited when it has been recursed into
Previously, WalkDeps() would record that a module was visited after the
first time it encountered the module irrespective of whether it recursed
into or not. This change moves the recording so it happens only after it
has been recursed into.

Added TestWalkDepsDuplicates_IgnoreFirstPath to test the change. Without
the change the test fails because it does not visit E.

Test refactoring:
* A depsMutator was added instead of relying on blueprintDepsMutator to
  allow different tags to be used for different dependency types.
* Modified barModule and fooModule to support the new depsMutator and
  add support for another type of dependency that is ignored by the
  walking code.
* Extracted walkDependencyGraph() function to reuse common code.
2020-04-02 10:56:13 +01:00
Colin Cross
da70fd0b84 Move LoadHooks from Soong to Blueprint
Move LoadHooks from Blueprint and run them during ParseBlueprintsFiles.
This will allow them to add scoped module types that are visible to the
rest of the Blueprints file.  Requires passing the config object to
ParseBlueprintsFiles.

Test: all blueprint tests
Change-Id: Ia2a2c9a0223d5458bfd48bd22ebed0fdbd0156c6
2020-01-02 20:32:51 -08:00
Colin Cross
d03b59d03e Use module groups more widely
Use module groups instead of passing around the list of modules
extracted from a module group.

Test: blueprint tests
Change-Id: I02a79950c6377441c49129ebeb5f12be257df668
2019-11-14 12:58:52 -08:00
Jaewoong Jung
ccc3494088 Make sure all modules have a name. (#226)
Throw an error if run into a module without a name when generating
contexts.

Test: context_test.go
Change-Id: I3976d86d1f15b8ac10a7a38aa42ae277740a8f3b
2018-10-11 13:01:05 -07:00
Colin Cross
526e02f0c6 Prevent duplicate visit calls in WalkDeps
WalkDeps was following every possible path through the dependency
tree, which can be enormous.  Modify it to only call visit for
any particular (child, parent) pair once for each direct dependency
by not recursing into child if visitDown returns true but child
has already been visited.

Test: TestWalkDeps, TestWalkDepsDuplicates
Change-Id: Ieef28399bd10e744417cdeb661dfa04fbeb4ec60
2018-06-21 13:41:42 -07:00
Colin Cross
9607a9f248 Allow multiple dependencies on the same module
Allow adding multiple dependencies on the same module.  Adds a flag
to Context.walkDeps to determine whether dependencies should be
visited multiple times, and clarifies the behavior of
VisitDepsDepthFirst (continues to visit each module once, even if
there are multiple dependencies on it) and VisitDirectDeps (may
visit a module multiple times).

Test: TestWalkDepsDuplicates, TestVisit
Change-Id: I58afbe90490aca102d242d63e185386e1fe55d73
2018-06-20 14:10:18 -07:00
Jeff Gaston
a7e408af0a prevent file=nil panic if syntax error in Blueprints
Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
      'build/blueprint:work^^^' 'build/blueprint:work'
Test: put a syntax error in a file and see that the
      reported error reports the location of the violation

Change-Id: Iaeedb91ea8e816cb8e9ee954f21cd6c6bc4afa48
2017-12-05 16:09:02 -08:00
Jeff Gaston
656870fbca Have handleOneFile(child) wait for handleOneFile(parent).
Bug: 65683273
Test: m -j nothing # which runs unit tests

Change-Id: I850e78ebcdbee68637f66d81fc127f19dd28508a
2017-12-04 17:16:21 -08:00
Jeff Gaston
d70bf75491 Extract module naming into an interface
in facilitate moving name resolution to Soong

Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
      --products=aosp_arm \
      'build/blueprint:work^' 'build/blueprint:work'
      # and see that the only changes were:
      # 1. adding the name_interface.go file
      # 2. changing some line numbers

Change-Id: Ifa7603cca59b3b3d592f2f146fdafe57012bd4b9
2017-11-29 12:01:09 -08:00
Jeff Gaston
c3e2844dfe Support for a custom list of Blueprints files to parse
Bug: 64363847
Test: BLUEPRINT_LIST_FILE=out/.module_paths/Android.bp.list minibp

Change-Id: Id7f8cb1ab3a6684b3f8265d77bb32413957f1c93
2017-10-30 15:00:19 -07:00
Jeff Gaston
5f763d0511 Clarify context.go somewhat
in preparation to support parsing a predefined list of Blueprints files

Bug: 64363847
Test: m -j
Change-Id: I98cc98dc17ecaf69b95272d32cd83adbdb0e8c5b
2017-10-30 15:00:19 -07:00
Colin Cross
874a346904 Let mutators use ctx.AddNinjaFileDeps
Propagate extra ninja file deps through mutators so that they
can use ctx.AddNinjaFileDeps.

Test: blueprint tests
Change-Id: I299a0665c3f63b020ae345889fd78b91b91b215a
2017-08-01 15:12:12 -07:00
Colin Cross
af4fd215eb Add TopDownMutatorContext.CreateModule
Allow a module to create other modules as if they were specified
in a blueprint file.  CreateModule takes the name of a module type,
calls the factory function the module type, and then inserts
properties into the property structs returned by the factory.

Bug: 64161749
Test: TestCreateModule
Change-Id: Ic1903305d6b08eae1edc7f0fb4137e85544aac0f
2017-08-01 15:12:12 -07:00
Colin Cross
127d2eae8b Import globbing from Soong
Add globbing with dependency checking to blueprint.  Calling
ModuleContext.GlobWithDeps or SingletonContext.GlobWithDeps will return
a list of files that match the globs, while also adding efficient
dependencies to rerun the primary builder if a file that matches the
glob is added or removed.

Also use the globbing support for optional_subdirs=, subdirs= and build=
lines in blueprints files.  The globbing slightly changes the behavior
of subname= lines, it no longer falls back to looking for a file called
"Blueprints".  Blueprint files that need to include a subdirectory with
a different name can use build= instead of subdir= to directly include
them.  The Blueprints file is updated to reset subname="Blueprints" in
case we want to include subdirectories inside blueprint and the primary
builder has changed the subname.

Also adds a new test directory that contains a simple primary builder
tree to test regeneration for globbing, and runs the tests in travis.

Change-Id: I83ce525fd11e11579cc58ba5308d01ca8eea7bc6
2016-11-03 13:54:03 -07:00
Colin Cross
0b7e83e11a Ask primary builder for module names
Remove the assumed Name and Deps properties.  Instead, ask the Module
for the Name, and require them to use the existing replacements for
Deps.

Change-Id: I729045d86277686078b3aa0bba71c67d612ead2c
2016-10-12 13:33:37 -07:00
Colin Cross
bafd5f5a2b Reimplement VisitDirectDeps[If] in on Context.walkdeps
VisitDirectDeps[If] need to know the parent in order to support
OtherModuleDependencyTag(), remove Context.visitDirectDeps[If] and
reimplement [Module]Context.VisitDirectDeps[If] using Context.walkdeps.

Change-Id: I2faac2d100d2a0ad2a4f1d8250997eed1a7ef6f0
Test: TestVisit
2016-08-08 17:30:38 -07:00
Colin Cross
d7b0f60802 Allow tests to mock filesystem
Tests can call Context.MockFileSystem to pass in a map of filenames to
contents.

Change-Id: Idd67c68f7cb43bc2117cc78336347db7e2f21991
2016-08-08 17:26:57 -07:00
Colin Cross
763b6f18fa Deprecate EarlyMutator and DynamicDependencies
DynamicDependencies can be implemented more flexibly by a
BottomUpMutator.  If there are no DynamicDependencies, then
EarlyMutators are identical to BottomUpMutators.  Deperecate both, and
reimplement DynamicDependencies inside a BottomUpMutator that is
guaranteed to be registered first.
2015-10-29 16:03:37 -07:00
Yuchen Wu
f995846a4b Added test for walkDeps. 2015-10-09 17:57:00 -07:00
Colin Cross
1fef536ea6 Add support for build= variable
build is a magic variable that acts similarly to subdirs, but specifies
files to build instead of subdirs.  Using files provides a few advantages:
files can be named something more appropriate for the project than
"Blueprints", whatever you want instead of Blueprints, and multiple
Blueprints files can exist in the same directory.

A new variable is used instead of putting filenames into subdirs to avoid
unexpected behavior when a glob matches both files and directories.

subdirs= and build= entries that don't match any directories or files
are now reported as errors.

Change-Id: Id329504ace251eab4ccea1081a3c8665a4c52f5a
2015-04-20 17:25:17 -07:00
Colin Cross
8e0c51192a Add license headers and LICENSE file
Change-Id: I6f7c7374093c0745ee4aa677480376a06648b358
2015-01-23 14:23:27 -08:00
Colin Cross
3e8e74f276 Move blueprint/* up a directory
Make integrating with go tools easier by putting the blueprint package
files in the top level directory of the git project instead of in a
subdirectory called blueprint.

Change-Id: I35c144c5fe7ddf34e478d0c47c50b2f6c92c2a03
2015-01-23 14:23:27 -08:00
Renamed from blueprint/context_test.go (Browse further)