Adding a dependency on a module with variants can be problematic
if the code adding the dependency is not aware of every mutator
that has created variants. Add an AliasVariations to
BottomUpMutatorContext, which allows a mutator to alias the
original variant of a module to one of the new variants of the
module, which will allow future dependencies to be added using
the original list of variations. The aliases are transient,
and only exist until the next mutator that calls CreateVariations
when visiting the module without also calling AliasVariations.
Test: TestAlises
Change-Id: Ieaa04b5a6bdcb5a1ff5114b1e03460de795d4479
Modules are created with a nil variantionMap, which avoids allocating
an empty map if the module will never be split.
Context.addVariationDependencies constructs a variationMap to
compare against the variationMap of each variant, but constructed
an empty variationMap if no variations were specified. A nil
map is not the same as an empty map, so consistently use a nil
map and create it when populating the first entry.
Change-Id: I48b604659f9cdb23326b504a093cdfe5a3eb4f68
SetDefaultDependencyVariation sets the variation name that will be used
when a dangling dependency is found while a module is being split. A
dangling dependency can occur if a module is split to a variant that one
of its dependencies is not split into. When the default variation is not
set, such dangling dependency is a hard error. But with the new
function, the default variation can be set and subsequent calls to
CreateVariations and its variations on the same context uses the default
variation when necessary.
(If even the default variation does not exist for the dependent module,
it is an hard error)
Note that this is different from calling SetDependencyVariation("foo")
followed by CreateVariations("foo", "bar"). In that case, regardless of
whether a dependency of the current module has the variant 'bar' or not,
only the 'foo' variant is chosen.
With SetDefaultDependencyVariation("foo") followed by
CreateVariations("foo", "bar"), 'foo' variant is used only when the
'bar' variant of the current module depends on a module that does not
have 'bar' variant.
Bug: 138103882
Test: m
Change-Id: I4520ca87487994de024fdbacda3bef6636225f0d
The interface is visible in godoc, not the implementation, so
move the documentation onto the interface methods.
Test: view godoc
Change-Id: I3460003eed6bf57d8ae4dabe70ab306f049a0060
Move most of the mutator context methods into BaseModuleContext
so they are available to both top down and bottom up mutators.
Only CreateModule is unique to TopDownMutatorContext, and the
dependency and variation adding methods are unique to the
BottomUpMutatorContext.
The dependency visiting methods are now available on
BottomUpMutatorContext, which requires delaying making newly
added dependencies visible to maintain the invariant that
the mutator has been called on the dependency before the
dependency can be visited by its parents.
Test: m checkbuild
Change-Id: Ie14afc02ac76d0b5a66b0e52de2aa9e17fd1bec0
Add OtherModuleDir, OtherModuleSubDir, and OtherModuleType to
ModuleContext and TopDownMutatorContext, and add ModuleType to
BaseModuleContext.
Change-Id: If5c873a2620ef10333c0bdba5ab89d4256e5fdf2
GetDirectDeWithTag would panic if the first possible directDep did not
have the correct tag. It should check to see if any version of that
dependency has the right tag.
Test: atest CtsJdwpTunnelHostTestCases
Bug: 124507633
Bug: 125933724
Change-Id: I00d269130e9f136a93fd30c58b8fd929372d5b37
WalkDeps was following every possible path through the dependency
tree, which can be enormous. Modify it to only call visit for
any particular (child, parent) pair once for each direct dependency
by not recursing into child if visitDown returns true but child
has already been visited.
Test: TestWalkDeps, TestWalkDepsDuplicates
Change-Id: Ieef28399bd10e744417cdeb661dfa04fbeb4ec60
Allow adding multiple dependencies on the same module. Adds a flag
to Context.walkDeps to determine whether dependencies should be
visited multiple times, and clarifies the behavior of
VisitDepsDepthFirst (continues to visit each module once, even if
there are multiple dependencies on it) and VisitDirectDeps (may
visit a module multiple times).
Test: TestWalkDepsDuplicates, TestVisit
Change-Id: I58afbe90490aca102d242d63e185386e1fe55d73
This makes it easy for users of Glob to detect whether the match is a
file or directory. Doing the check at this level means that the filelist
file used as a dependency will be updated if a directory is replaced
with a file of the same name, or vice versa.
Change-Id: I79ebba39327218bcdcf50b393498306119de9d6c
To allow it to validate that the filename equals Android.bp
Bug: 65683273
Test: m -j nothing # which runs tests
Change-Id: I171dddd102590df558053b615d39c75c00b6ac6e
in facilitate moving name resolution to Soong
Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
--products=aosp_arm \
'build/blueprint:work^' 'build/blueprint:work'
# and see that the only changes were:
# 1. adding the name_interface.go file
# 2. changing some line numbers
Change-Id: Ifa7603cca59b3b3d592f2f146fdafe57012bd4b9
When a mutator calls CreateModule, set the blueprints file and line
number to the values of the module that the mutator was visiting.
Change-Id: I4526f14a744be944e42582bc46f11a9a7b502f03
Propagate extra ninja file deps through mutators so that they
can use ctx.AddNinjaFileDeps.
Test: blueprint tests
Change-Id: I299a0665c3f63b020ae345889fd78b91b91b215a
Allow a module to create other modules as if they were specified
in a blueprint file. CreateModule takes the name of a module type,
calls the factory function the module type, and then inserts
properties into the property structs returned by the factory.
Bug: 64161749
Test: TestCreateModule
Change-Id: Ic1903305d6b08eae1edc7f0fb4137e85544aac0f
The next patch will introduce creating new modules, so rename the
current variable that contains new variations of existing modules
to avoid confusion.
Test: blueprint tests
Change-Id: Ic1d3824e54e51c282a08f8ecef8fc658cc503a65
It is not allowed to directly use BaseDependencyTag as customized
user dependency tag passed down to BP since it might cause issues
that different type of modules will be mixed when fetched based on
Tag.
Add globbing to filesystem mocking so that more code can be tested
against the mock. Also moves the filesystem mock to pathtools,
and renames pathtools.GlobWithExcludes to pathtools.Glob, replacing
the existing pathtools.Glob.
Test: blueprint tests
Change-Id: I722df8121bc870c4a861d7c249245c57dcb607be
Instead of forcing every mutator or module to use VisitDirectDeps
to iterate through its deps, allow it to request a dependency by
name to return the Module.
Change-Id: I18b23bffa324bb9d93a7743b6e2a19c07058c775
Mutator context goroutines appending directly to the global Context's
replacements list causes a data race. Send them over a channel
instead.
The renames and replacements are local to the mutator, so move them
out of Context and into the runMutator method.
Change-Id: I797edb1e27ee29f8946c58101b40fcfb50a32eb9
Add globbing with dependency checking to blueprint. Calling
ModuleContext.GlobWithDeps or SingletonContext.GlobWithDeps will return
a list of files that match the globs, while also adding efficient
dependencies to rerun the primary builder if a file that matches the
glob is added or removed.
Also use the globbing support for optional_subdirs=, subdirs= and build=
lines in blueprints files. The globbing slightly changes the behavior
of subname= lines, it no longer falls back to looking for a file called
"Blueprints". Blueprint files that need to include a subdirectory with
a different name can use build= instead of subdir= to directly include
them. The Blueprints file is updated to reset subname="Blueprints" in
case we want to include subdirectories inside blueprint and the primary
builder has changed the subname.
Also adds a new test directory that contains a simple primary builder
tree to test regeneration for globbing, and runs the tests in travis.
Change-Id: I83ce525fd11e11579cc58ba5308d01ca8eea7bc6
ReplaceDependencies allows a module to replace dependencies on a
matching variant of a target module with itself.
Change-Id: I22946dec23c38ed5e1ad23b87121d72668268c01
Allow modules to be renamed by mutators. This will allow modules to
dynamically adapt to the presence or absence of modules with the same
name. For example, if a source module does not exist, a prebuilt module
could rename itself to the name of the source module so that
dependenices on the module name are satisified.
Change-Id: I44004604c6d9db041bb7d38fe6c1ca877bc7d6f1
Allow dependency modifying mutators to check if another module exists
based on its name. Allows primary builders to have custom handling for
missing-module errors, for example by falling back to another module.
Change-Id: I04ddd66be43c514f1f8928e15b2a79cadebcfc17
Remove the assumed Name and Deps properties. Instead, ask the Module
for the Name, and require them to use the existing replacements for
Deps.
Change-Id: I729045d86277686078b3aa0bba71c67d612ead2c
Allow BottomUpMutators to run in parallel by calling Parallel() on the
return value of RegisterBottomUpMutator. To avoid locking, moves
updates of global state into a separate goroutine that receives updates
over channels from the mutator goroutines.
Change-Id: Ic59b612da9b406cf59ec44940f0c1dee0c051a51
VisitDirectDeps[If] need to know the parent in order to support
OtherModuleDependencyTag(), remove Context.visitDirectDeps[If] and
reimplement [Module]Context.VisitDirectDeps[If] using Context.walkdeps.
Change-Id: I2faac2d100d2a0ad2a4f1d8250997eed1a7ef6f0
Test: TestVisit
It can be useful for a BottomUpMutator to visit its dependencies. Embed
all of TopDownMutatorContext into BottomUpMutatorContext.
Change-Id: I7c7262fbe3d8fa9cc0d26899eee704cc862835df
Variants of a module may need to depend on other variants, for example
to reuse results between variants. Support AddInterVariantDependency to
add an explicit dependency from one variant to another, along with a
dependency tag. Both modules must have just been returned by a call to
CreateVariants.
Change-Id: I8f4878f94ced74dd00cfac8303b15ef70cdebf36
Primary builder logic is becoming complicated due to the two pass nature
of mutators that add dependencies and GenerateBuildActions that handle
the dependencies. The reason why the dependency was added is lost by
the time GenerateBuildActions is called, resulting in build logic that
has to recreate all the dependencies and try to match them up to the
modules returned by VisitDirectDeps.
Change the API of AddDependency to take a DependencyTag interface, which
is satisifed by anything that embeds BaseDependencyTag. Mutators and
GenerateBuildActions that call VisitDirectDeps can pass each Module to
ctx.OtherModuleDependencyTag to retreive the DependencyTag that was
passed when adding the dependency.
Change-Id: I0814dcd26d1670302d340b77e8dc8704ed7b60bf
Catch panics in the build logic in order to provide extra data on when
the panic occurred, for example which module had GenerateBuildActions
called on it.
In some cases the primary builder may need to handle missing
dependencies. Add Context.SetAllowMissingDependencies to cause
Blueprint to store the list of missing dependencies without immediately
emitting an error, and ModuleContext.GetMissingDependencies to return
the missing dependencies to the primary builder. If the primary builder
does not call ModuleContext.GetMissingDependencies Blueprint will emit
dependency errors.
Turn PackageContext into an interface so that build systems can wrap it
to add more custom helpers.
This does introduce an API change, though it should be fairly simple.
NewPackageContext used to provide an opaque *PackageContext struct, now it
provides a PackageContext interface.
Change-Id: I383c64a303d857ef5e0dec86ad77f791ba4c9639
AddReverseDependencies would add modules the target's dependency list in
a non-deterministic order based on the order the modules were parsed.
Redefine AddReverseDependencies to collect dependencies until the end of
the mutator and then add them sorted by name.
PropertyErrorf gets called from build logic, and shouldn't panic if the
build logic reports an error on an unset property. Fall back to using
the module position if the property wasn't set.
Also put the property name into the error message.
DynamicDependencies can be implemented more flexibly by a
BottomUpMutator. If there are no DynamicDependencies, then
EarlyMutators are identical to BottomUpMutators. Deperecate both, and
reimplement DynamicDependencies inside a BottomUpMutator that is
guaranteed to be registered first.
walkDeps performs a pre-order DFS (unlike visitDepsDepthFirst which is
a post-order DFS). The visit function takes in both a parent and child
node and returns a bool indicating if the child node should be
traversed.
Now that we have multi-stage bootstrapping, we can make the primary
builder build more dynamic. Add the concept of plugins that will be
linked and loaded into bootstrap_go_binary or bootstrap_go_package
modules. It's expected that the plugin's init() functions will do
whatever registration is necessary.
Example Blueprint definition:
bootstrap_go_binary {
name: "builder",
...
}
bootstrap_go_package {
name: "plugin1",
pluginFor: ["builder"],
}
A package may specify more than one plugin if it will be inserted into
more than one go module.
Change-Id: I109835f444196b66fc4018c3fa36ba0875823184
AddVariationDependencies is used to add dependency modules that
have the same variations as the depending module, but with additional
variataions. It cannot be used to add a dependency that is unrelated
to the depending module, for example a dependency on a code generator
that needs to run on the host to generate a target source file.
Add AddFarVariationDependencies, which adds a dependency on a module
that contains all the passed variations, but ignores the variations
of the depending module, as well as any unspecified variations on
the dependency module.
Change-Id: Ief696ec85cf33ad5fb187227d215c1c2e894f962
"Variant" has been used to mean two slightly concepts in Blueprint.
The first and most obvious is a variant of a module, meaning a list
of all the ways that on version of a module differs from other
versions of the same modules, for example "arch:arm, link:static".
The other is for the specific way that a version of a module differs,
"arch:arm".
Rename all of uses of "variant" as the second meaning to "variation",
and update the documentation to clarify "variant" vs. "variation".
This modifies the build logic api to convert CreateVariants to
CreateVariations.
Change-Id: I789ef209ae6ddd39ec12e0c032f9f47f86698fe6
The mutators that run after dependencies are resolved can be too late
to support build logic that needs to vary the dependencies based on
the mutated axis, for example architecture. This patch provides an
EarlyMutator interface that can be used to mutate modules before
any dependencies have been resolved.
In order for dependencies to be satisifed in a later pass, all
dependencies of a module must either have an identical variant,
must have a single variant, or must be inserted using
DynamicDependencyModuleContext.AddVariantDependency.
Change-Id: Ic6ae57e98edfd6c8c09a7788983128d3e4e992f0
