in facilitate moving name resolution to Soong
Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
--products=aosp_arm \
'build/blueprint:work^' 'build/blueprint:work'
# and see that the only changes were:
# 1. adding the name_interface.go file
# 2. changing some line numbers
Change-Id: Ifa7603cca59b3b3d592f2f146fdafe57012bd4b9
PreSingletons use the same interface as Singletons, but run after
parsing all the blueprint files but before running any mutators
or GenerateBuildActions. They can be used to perform global setup,
including generating rules, that may be referenced by later
mutator or generate passes.
Test: m checkbuild
Change-Id: I4b93425a724e739f5b8843262efb8804b3bf3531
because there some symlinks named Android.bp that point to
files named Android.bp that contain <subdir> entries that
are supposed to be interpreted relative to the original
symlink. It's more convenient to just skip following subdirs.
Bug: 64363847
Test: make -j
Change-Id: Ib15f1d1c7d511191fd841b1a29ceed559dd6cdbb
Bug: 64600838
Test: mkdir errtest \
&& ln -s /tmp/dontexist errtest/Android.bp \
# and add errtest to ./Android.bp \
&& m nothing \
# and check that the error message mentions a symlink
Change-Id: I841ec12d613f61ccc3396538062bee48c8c1ca27
Propagate extra ninja file deps through mutators so that they
can use ctx.AddNinjaFileDeps.
Test: blueprint tests
Change-Id: I299a0665c3f63b020ae345889fd78b91b91b215a
Allow a module to create other modules as if they were specified
in a blueprint file. CreateModule takes the name of a module type,
calls the factory function the module type, and then inserts
properties into the property structs returned by the factory.
Bug: 64161749
Test: TestCreateModule
Change-Id: Ic1903305d6b08eae1edc7f0fb4137e85544aac0f
The next patch will introduce creating new modules, so rename the
current variable that contains new variations of existing modules
to avoid confusion.
Test: blueprint tests
Change-Id: Ic1d3824e54e51c282a08f8ecef8fc658cc503a65
Darwin has a default limit of 256 open files per process. Parsing
too many blueprint files in parallel can hit the limit. Cap the
concurrency at 200.
Test: manual testing with limit set to 32
Change-Id: Ic64d21d2c0ffd7c86bf3f02fb51216ee5684a80c
Instead of just saying:
error: .../Android.bp:48:1: dependency "libc++" of "libtest" missing variant "arch:android_arm_armv7-a, link:shared, vndk:"
Include a list of currently existing variants:
error: .../Android.bp:48:1: dependency "libc++" of "libtest" missing variant:
arch:android_arm_armv7-a, link:shared, vndk:
available variants:
arch:android_arm_armv7-a, link:shared
arch:linux_x86, link:shared
arch:linux_x86_64, link:shared
arch:windows_x86, link:shared
arch:windows_x86_64, link:shared
This still isn't the best experience for users, but it at least provides
enough information for someone more familiar with the build to
understand the problem.
It is not allowed to directly use BaseDependencyTag as customized
user dependency tag passed down to BP since it might cause issues
that different type of modules will be mixed when fetched based on
Tag.
Add globbing to filesystem mocking so that more code can be tested
against the mock. Also moves the filesystem mock to pathtools,
and renames pathtools.GlobWithExcludes to pathtools.Glob, replacing
the existing pathtools.Glob.
Test: blueprint tests
Change-Id: I722df8121bc870c4a861d7c249245c57dcb607be
Mutator context goroutines appending directly to the global Context's
replacements list causes a data race. Send them over a channel
instead.
The renames and replacements are local to the mutator, so move them
out of Context and into the runMutator method.
Change-Id: I797edb1e27ee29f8946c58101b40fcfb50a32eb9
Bypassing c.glob() and using filepath.Glob() directly for non-glob
paths does not add dependencies on directories that contain missing
files. For optional_subdirs, this means no dependency is added to
rerun the primary builder when an Android.bp file is added to an
optional_subdirs directory. Always use c.glob(), for the non-optional
case it will not insert any dependencies if the file exists (as tested
by glob_test.go's no-wild tests), and if the file doesn't exist the
len(matches) == 0 will error out.
Change-Id: I370479c6e89f5ff590897702e256256a4dca6952
Add globbing with dependency checking to blueprint. Calling
ModuleContext.GlobWithDeps or SingletonContext.GlobWithDeps will return
a list of files that match the globs, while also adding efficient
dependencies to rerun the primary builder if a file that matches the
glob is added or removed.
Also use the globbing support for optional_subdirs=, subdirs= and build=
lines in blueprints files. The globbing slightly changes the behavior
of subname= lines, it no longer falls back to looking for a file called
"Blueprints". Blueprint files that need to include a subdirectory with
a different name can use build= instead of subdir= to directly include
them. The Blueprints file is updated to reset subname="Blueprints" in
case we want to include subdirectories inside blueprint and the primary
builder has changed the subname.
Also adds a new test directory that contains a simple primary builder
tree to test regeneration for globbing, and runs the tests in travis.
Change-Id: I83ce525fd11e11579cc58ba5308d01ca8eea7bc6
ReplaceDependencies allows a module to replace dependencies on a
matching variant of a target module with itself.
Change-Id: I22946dec23c38ed5e1ad23b87121d72668268c01
Allow modules to be renamed by mutators. This will allow modules to
dynamically adapt to the presence or absence of modules with the same
name. For example, if a source module does not exist, a prebuilt module
could rename itself to the name of the source module so that
dependenices on the module name are satisified.
Change-Id: I44004604c6d9db041bb7d38fe6c1ca877bc7d6f1
Remove the assumed Name and Deps properties. Instead, ask the Module
for the Name, and require them to use the existing replacements for
Deps.
Change-Id: I729045d86277686078b3aa0bba71c67d612ead2c
Blueprint files included with build= statements were not adding a
dependency to cause regenerating when they were changed.
Bug: 32085516
Test: touch external/boringssl/sources.bp && mmma -j external/boringssl
Change-Id: Id4fdf3b6788ae5c1e94547dc63ec6b55424a66a0
Don't add a "_" to the variant name if the variation name is empty. For
example, when splitting a variant with name "foo" into variations "" and
"bar", the new variants would be named "foo" and "foo_bar" instead of
"foo_" and "foo_bar".
Change-Id: I82342d57e2a8e9f2d65a7d8d2872dcb7b3512899
runMutators was sending the wrong value into errsCh, hiding all errors.
parallelVisit had a data race on error, send the cancel signal over a
channel instead of writing to a bool from a goroutine.
Change-Id: I18919d71464c094db5846badbfac80f99366d70c
Convert parallelVisitAllBottomUp into a generic parallelVisit that takes
a visitOrderer to select top down or bottom up. Combine
runTopDownMutator and runBottomUpMutator into runMutators that takes a
mutatorDirection to select the visitOrderer for parallelVisit and which
function pointer in the mutatorInfo to run. Optimize out the
updateDependencies to only run if dependencies have been modified to
avoid running it after TopDownMutators that cannot modify dependencies.
Change-Id: Ib00302db1108ebab2ce8e01b20aa026140d382a4
cloneModules duplicates every module, which can be slow. Spawn a new
goroutine for each module to spread the factory and CopyProperties calls
over multiple cpus.
Change-Id: I2597f921dadfd321ccb705aec9a904c71b75f9ef
Allow BottomUpMutators to run in parallel by calling Parallel() on the
return value of RegisterBottomUpMutator. To avoid locking, moves
updates of global state into a separate goroutine that receives updates
over channels from the mutator goroutines.
Change-Id: Ic59b612da9b406cf59ec44940f0c1dee0c051a51
Get rid of runEarlyMutators, and implement EarlyMutators as a shim
around BottomUpMutators. Removes mostly-duplicate code between
runEarlyMutators and runBottomUpMutator to allow parallelizing
runBottomUpMutator.
Change-Id: I66a051cb8710770b47da21d35e55aa49c84e9be8
VisitDirectDeps[If] need to know the parent in order to support
OtherModuleDependencyTag(), remove Context.visitDirectDeps[If] and
reimplement [Module]Context.VisitDirectDeps[If] using Context.walkdeps.
Change-Id: I2faac2d100d2a0ad2a4f1d8250997eed1a7ef6f0
Test: TestVisit
In order to implement some build statistics in Soong, expose the module
type (the string used to define the module) to singletons.
Change-Id: I441d12c7782bcf338b3654cfe907b8d2a7253594
Pos is going to be part of the Node interface, rename the Pos member
of structs to be more specific.
Change-Id: Ibd31119863b96d38bf8dac216e026200a54bbe18
It wasn't adding anything useful, and it resulted in Name.Name to get to
the identifier. Replace Name Ident with Name string; NamePos
scanner.Position.
Change-Id: Idf9b18b31dd563a18f27c602c2d14298955af371
Refactor the blueprint parser Value object, which contained a Type enum
and members to hold every possible type, into an interface (now called
Expression). Rename the existing Expression object that represented a binary
operator Operator.
Also adds and fixes some new printer test cases with mulitline expressions.
Change-Id: Icf4a20f92c8c2a27f18df8ca515a9d7f282ff133
Variants of a module may need to depend on other variants, for example
to reuse results between variants. Support AddInterVariantDependency to
add an explicit dependency from one variant to another, along with a
dependency tag. Both modules must have just been returned by a call to
CreateVariants.
Change-Id: I8f4878f94ced74dd00cfac8303b15ef70cdebf36
Primary builder logic is becoming complicated due to the two pass nature
of mutators that add dependencies and GenerateBuildActions that handle
the dependencies. The reason why the dependency was added is lost by
the time GenerateBuildActions is called, resulting in build logic that
has to recreate all the dependencies and try to match them up to the
modules returned by VisitDirectDeps.
Change the API of AddDependency to take a DependencyTag interface, which
is satisifed by anything that embeds BaseDependencyTag. Mutators and
GenerateBuildActions that call VisitDirectDeps can pass each Module to
ctx.OtherModuleDependencyTag to retreive the DependencyTag that was
passed when adding the dependency.
Change-Id: I0814dcd26d1670302d340b77e8dc8704ed7b60bf
A common pattern in mutators is to set a property on a module to be used
later in GenerateBuildActions. A common anti-pattern is to set a member
variable instead of a property. Setting member variables will appear to
work until a mutator calls CreateVariations after the member variable is
set. The first variant will have the member variables, but any other
variants will have zero values for all member variables.
To catch this common case early, replace all modules with clones after
running all the mutators but before GenerateBuildActions.
This catches the anti-pattern used in bootstrap, replace the buildStage
member variable with a property.
Change-Id: I6ff37580783a6227ebba2b46d577b443566a79bb
Our use case is to write out some configuration variables to give to our
legacy build system. But in order to do that, we either need to keep a
Go copy of all of the configuration, and still have all the calls to
convert them to ninja variables, or evaluate our ninja variables.
Change-Id: If9dda305ed41bc8aabe57dd750a74d8b9af1d1a4
visited map was not being updated when the dependency was being
visited, only if it was being walked. This led to cases where
duplicate traversals were possible.
Catch panics in the build logic in order to provide extra data on when
the panic occurred, for example which module had GenerateBuildActions
called on it.
