Adds a CheckBlueprintSyntax(...) method to check the syntax of a
Blueprint file.
Changes processModuleDef and newModule from being method on *Context to
being standalone functions. That ensures that CheckBlueprintSyntax(...)
does not need to take a context and so there is no chance that it can
change its state.
Previously a LoadHook could not modify the name of a module because the
module was registered before the LoadHooks were run. That made it very
complicated (requiring mutators and auto generated names) to create a
module type whose name was determined by say the directory in which it
is defined.
This change moves the LoadHook execution slightly earlier so it runs
before registration of the module.
That caused one slight side problem which was that the
moduleInfo.Name() would fail when called in a LoadHook. That was
because that gets the name from group.name but was group was nil
because it is only set when the module is registered.
Modifying the moduleInfo.Name() method to get the name from the module
logicModule.Name() if group is nil fixed that. The reason for getting
the name from the group.name rather than the logicModule.Name() is that
the former tracks renames but the latter does not. However that is not
an issue in this case as there has been no opportunity for the module
to be renamed until after the LoadHook has returned.
Previously, WalkDeps() would record that a module was visited after the
first time it encountered the module irrespective of whether it recursed
into or not. This change moves the recording so it happens only after it
has been recursed into.
Added TestWalkDepsDuplicates_IgnoreFirstPath to test the change. Without
the change the test fails because it does not visit E.
Test refactoring:
* A depsMutator was added instead of relying on blueprintDepsMutator to
allow different tags to be used for different dependency types.
* Modified barModule and fooModule to support the new depsMutator and
add support for another type of dependency that is ignored by the
walking code.
* Extracted walkDependencyGraph() function to reuse common code.
Mutators were not propagating the results of ctx.AddNinjaFileDeps.
Test: examine out/soong/build.ninja.d
Fixes: 150689149
Change-Id: Ia1e69ebc9dfa94a05f4ecd9cc2a8691ee63c9dd5
There are 8935901 *ninjaString objects generated in an AOSP
aosp_blueline-userdebug build, and 7865180 of those are a literal
string with no ninja variables.
Each of those *ninjaString objects takes a minimum of 48 bytes for
2 slices, plus 8 bytes for the pointer to the ninjaString. For
the literal string case, one of those slices has a single element,
(costing another 16 bytes for the backing array), and the other
slice is empty, for a total of 72 bytes.
Replace *ninjaString with a ninjaString interface. This increases
the size of the reference from 8 bytes to 16 bytes, but using
a type alias of a string for the literal string implementation uses
only 16 bytes, saving 40 bytes per literal string or 314 MB.
Test: ninja_strings_test
Change-Id: Ic5fe16ed1f2a244fe6a8ccdf762919634d825cbe
The proptools functions took an inconsistent variety of
struct and *struct types. Some methods even took a struct
but returned a *struct. Make all the exported methods
take a *struct, with internal helpers for the ones that need
to take a struct.
Test: proptools tests
Change-Id: I60ce212606e96adcef66c531d57f69c39e1a1638
Add RegisterScopedModuleType to LoadHookContext that registers
a module type factory that will be visible for the remainder of
the file. Also add ModuleFactories that returns the globally
register module factories.
Test: all blueprint tests
Change-Id: If646a73befe3b8e45c4b0984531c6a39ddc8d066
Move LoadHooks from Blueprint and run them during ParseBlueprintsFiles.
This will allow them to add scoped module types that are visible to the
rest of the Blueprints file. Requires passing the config object to
ParseBlueprintsFiles.
Test: all blueprint tests
Change-Id: Ia2a2c9a0223d5458bfd48bd22ebed0fdbd0156c6
Adding a dependency on a module with variants can be problematic
if the code adding the dependency is not aware of every mutator
that has created variants. Add an AliasVariations to
BottomUpMutatorContext, which allows a mutator to alias the
original variant of a module to one of the new variants of the
module, which will allow future dependencies to be added using
the original list of variations. The aliases are transient,
and only exist until the next mutator that calls CreateVariations
when visiting the module without also calling AliasVariations.
Test: TestAlises
Change-Id: Ieaa04b5a6bdcb5a1ff5114b1e03460de795d4479
Modules are created with a nil variantionMap, which avoids allocating
an empty map if the module will never be split.
Context.addVariationDependencies constructs a variationMap to
compare against the variationMap of each variant, but constructed
an empty variationMap if no variations were specified. A nil
map is not the same as an empty map, so consistently use a nil
map and create it when populating the first entry.
Change-Id: I48b604659f9cdb23326b504a093cdfe5a3eb4f68
Use module groups instead of passing around the list of modules
extracted from a module group.
Test: blueprint tests
Change-Id: I02a79950c6377441c49129ebeb5f12be257df668
This fixes a bug that reverse dependency can't be made for modules
having local variations. Previously, when module A having local variants
calls AddReverseDependency to module B having local variants, the match
is tested between the non-local variants of module A against all
variants of module B, which can never be successful.
This change fixes it by using all variants of module A when
findMatchingVariants is called for AddReverseDependency.
Test: m
Change-Id: Ib289188a5dd58c72bd6ba07e3c0f825f8b1c6b1b
SetDefaultDependencyVariation sets the variation name that will be used
when a dangling dependency is found while a module is being split. A
dangling dependency can occur if a module is split to a variant that one
of its dependencies is not split into. When the default variation is not
set, such dangling dependency is a hard error. But with the new
function, the default variation can be set and subsequent calls to
CreateVariations and its variations on the same context uses the default
variation when necessary.
(If even the default variation does not exist for the dependent module,
it is an hard error)
Note that this is different from calling SetDependencyVariation("foo")
followed by CreateVariations("foo", "bar"). In that case, regardless of
whether a dependency of the current module has the variant 'bar' or not,
only the 'foo' variant is chosen.
With SetDefaultDependencyVariation("foo") followed by
CreateVariations("foo", "bar"), 'foo' variant is used only when the
'bar' variant of the current module depends on a module that does not
have 'bar' variant.
Bug: 138103882
Test: m
Change-Id: I4520ca87487994de024fdbacda3bef6636225f0d
Move most of the mutator context methods into BaseModuleContext
so they are available to both top down and bottom up mutators.
Only CreateModule is unique to TopDownMutatorContext, and the
dependency and variation adding methods are unique to the
BottomUpMutatorContext.
The dependency visiting methods are now available on
BottomUpMutatorContext, which requires delaying making newly
added dependencies visible to maintain the invariant that
the mutator has been called on the dependency before the
dependency can be visited by its parents.
Test: m checkbuild
Change-Id: Ie14afc02ac76d0b5a66b0e52de2aa9e17fd1bec0
The go comment for the blueprint.Context.ParseBlueprintsFiles was
actually place above the blueprint.Context.ParseFileList.
Bug: N/A
Test: N/A
Change-Id: I065d2f6ec034c614e40de745931f4c87cdb73422
Add Context.Singletons and Context.SingletonName to allow primary
builders to query Singletons returned by all registered
SingletonFactories.
Change-Id: Iee85643dd47f7472e6bb5ae8004374bd6ea0419e
1. Extract module type documentation.
2. Support primary builder customization of factory function to use for
documentation for each module type.
3. Change the ModuleType list order so that they are grouped by package.
This is basically minor refactoring + readability improvement done on
top of https://github.com/google/blueprint/pull/232.
Change-Id: If7413e5ac23486b85f18d02fb3ba288a38730c32
Writing the ninja file to a byte buffer causes a significant amount
of time to be spent in memmove when growing the byte slice. Write
the file directly to disk instead.
This also fixes some unhandled error warnings, which become more
likely when doing disk IO instead of byte buffer writes.
Change-Id: I5094e4c45cab4012713037f60c5a4fb00718f92e
Throw an error if run into a module without a name when generating
contexts.
Test: context_test.go
Change-Id: I3976d86d1f15b8ac10a7a38aa42ae277740a8f3b
Instead of sometimes re-running minibp/the primary builder during the
next phase, run bpglob earlier to check dependencies.
We've run into issues where the environment is slightly different
between bootstrapping phase and the main build phase. It's also a
problem because our primary builder (Soong) exports information used by
another tool (Kati) that runs in between the bootstrapping phases and
the main phase. When Soong would run in the main phase, it could get out
of sync, and would require the build to be run again.
To do this, add a "subninja" include a build-globs.ninja file to each
build.ninja file. The first time, this will be an empty file, but we'll
always run minibp / the primary builder anyway. When the builder runs,
in addition to writing a dependency file, write out the
build-globs.ninja file with the rules to run bpglob.
Since bpglob may need to be run very early, before it would normally be
built, build it with microfactory.
Change-Id: I89fcd849a8729e892f163d40060ab90b5d4dfa5d
WalkDeps was following every possible path through the dependency
tree, which can be enormous. Modify it to only call visit for
any particular (child, parent) pair once for each direct dependency
by not recursing into child if visitDown returns true but child
has already been visited.
Test: TestWalkDeps, TestWalkDepsDuplicates
Change-Id: Ieef28399bd10e744417cdeb661dfa04fbeb4ec60
Allow adding multiple dependencies on the same module. Adds a flag
to Context.walkDeps to determine whether dependencies should be
visited multiple times, and clarifies the behavior of
VisitDepsDepthFirst (continues to visit each module once, even if
there are multiple dependencies on it) and VisitDirectDeps (may
visit a module multiple times).
Test: TestWalkDepsDuplicates, TestVisit
Change-Id: I58afbe90490aca102d242d63e185386e1fe55d73
Running with the data race detector enabled hits a 8192 active
goroutine limit. Reduce the maximum number of active goroutines
by limiting parallelVisit to 1000 goroutines. Also rewrite
cloneModule in terms of parallelVisit.
Change-Id: Icdd63648e8e010557b624e12592156490b40adb9
This makes it easy for users of Glob to detect whether the match is a
file or directory. Doing the check at this level means that the filelist
file used as a dependency will be updated if a directory is replaced
with a file of the same name, or vice versa.
Change-Id: I79ebba39327218bcdcf50b393498306119de9d6c
Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
'build/blueprint:work^^^' 'build/blueprint:work'
Test: put a syntax error in a file and see that the
reported error reports the location of the violation
Change-Id: Iaeedb91ea8e816cb8e9ee954f21cd6c6bc4afa48
To allow it to validate that the filename equals Android.bp
Bug: 65683273
Test: m -j nothing # which runs tests
Change-Id: I171dddd102590df558053b615d39c75c00b6ac6e
in facilitate moving name resolution to Soong
Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
--products=aosp_arm \
'build/blueprint:work^' 'build/blueprint:work'
# and see that the only changes were:
# 1. adding the name_interface.go file
# 2. changing some line numbers
Change-Id: Ifa7603cca59b3b3d592f2f146fdafe57012bd4b9
PreSingletons use the same interface as Singletons, but run after
parsing all the blueprint files but before running any mutators
or GenerateBuildActions. They can be used to perform global setup,
including generating rules, that may be referenced by later
mutator or generate passes.
Test: m checkbuild
Change-Id: I4b93425a724e739f5b8843262efb8804b3bf3531
because there some symlinks named Android.bp that point to
files named Android.bp that contain <subdir> entries that
are supposed to be interpreted relative to the original
symlink. It's more convenient to just skip following subdirs.
Bug: 64363847
Test: make -j
Change-Id: Ib15f1d1c7d511191fd841b1a29ceed559dd6cdbb
Bug: 64600838
Test: mkdir errtest \
&& ln -s /tmp/dontexist errtest/Android.bp \
# and add errtest to ./Android.bp \
&& m nothing \
# and check that the error message mentions a symlink
Change-Id: I841ec12d613f61ccc3396538062bee48c8c1ca27
