Now that we have multi-stage bootstrapping, we can make the primary
builder build more dynamic. Add the concept of plugins that will be
linked and loaded into bootstrap_go_binary or bootstrap_go_package
modules. It's expected that the plugin's init() functions will do
whatever registration is necessary.
Example Blueprint definition:
bootstrap_go_binary {
name: "builder",
...
}
bootstrap_go_package {
name: "plugin1",
pluginFor: ["builder"],
}
A package may specify more than one plugin if it will be inserted into
more than one go module.
Change-Id: I109835f444196b66fc4018c3fa36ba0875823184
The Go race detector found a race condition in the parser, which
highlighted a few related bugs. A variable could be defined but
not referenced in a Blueprints file, then appended to in multiple
subdirs= Blueprints files. The race detector caught the multiple
writes to assignment.Referenced from the parsers for the subdirs
Blueprints files, but multiple appends would be much more serious.
To fix this, keep local and inherited variables separate in the
Scope object and export that info to the parser. Disallow
appending to non-local variables, which was already the intended
behavior. Only update the referenced boolean for local variables.
Together, this should prevent all writes to Assignment objects
from parsers other than the one that created them.
Also improves the error handling code and some error messages.
Change-Id: Idb4f7d2e61bbe28d90b93074764e64e60d1eba8f
The workaround no longer works with the new stage selector. We may run
bootstrap.ninja.in twice before running the next stage, but we can't
encode whether to run another ninja instance in the checked in
build.ninja.in.
This can likely be solved, but now that there's an official release with
support for multiple passes, just push up the required version.
Change-Id: I76e321912e323d60e462aabec61bdfcc7118cd5e
Modules may decide not to output any build definitions in some cases.
Clean up the ninja file by not adding headers for empty sections.
One particular usecase is my upcoming multi-stage bootstrapping -
bootstrap_go_* will not output any rules in the first stage unless it's
required to build the primary builder.
Change-Id: I6a6b54da7e1702c63bfa736bcf8daf16956f9449
This change fixes a bug where directories that could contain a Blueprints file
but currently do not were not being added as a dependency. This meant that if
a Blueprints file were added to that directory it would not automatically get
picked up by the build.
Refactor parsing Blueprints and walking the Blueprints tree to
allow reuse for tasks that don't involve creating moduleInfo
structures.
Change-Id: I677857a3462999426c8306432074ea97fdcb86c8
The primary builder will now generate a rule to call itself with
--docs=.bootstrap/docs/<name>.html to produce an automatically
generated documentation file.
The documentation generation process is:
- Call each factory once to get empty property structs associated
with the module type
- Use reflection to determine the names of the type of each property
struct
- Use the bootstrap_go_package modules from reading the Blueprints files
to find the source files for each Go package used to build the primary
builder
- Use the go/parser module to find the type declaration for each
property struct
- Extract comments for the property struct and each property declaration
- Format all the comments into HTML
Change-Id: Icae9307cc10549a30bfc14d6922824099de5a9b0
AddVariationDependencies is used to add dependency modules that
have the same variations as the depending module, but with additional
variataions. It cannot be used to add a dependency that is unrelated
to the depending module, for example a dependency on a code generator
that needs to run on the host to generate a target source file.
Add AddFarVariationDependencies, which adds a dependency on a module
that contains all the passed variations, but ignores the variations
of the depending module, as well as any unspecified variations on
the dependency module.
Change-Id: Ief696ec85cf33ad5fb187227d215c1c2e894f962
Setting the "subname" variable will modify the name of Blueprints
files searched in sub directories. If the specified name is not
found it will fall back to trying "Blueprints". The subname
variable stays in scope for all subdirectories.
Change-Id: If3c8ac54735042076fb264d33719d9ea4b8934b3
build is a magic variable that acts similarly to subdirs, but specifies
files to build instead of subdirs. Using files provides a few advantages:
files can be named something more appropriate for the project than
"Blueprints", whatever you want instead of Blueprints, and multiple
Blueprints files can exist in the same directory.
A new variable is used instead of putting filenames into subdirs to avoid
unexpected behavior when a glob matches both files and directories.
subdirs= and build= entries that don't match any directories or files
are now reported as errors.
Change-Id: Id329504ace251eab4ccea1081a3c8665a4c52f5a
This fixes a bug where a split module could be reached by subsequent
mutator passes, but it is no longer in module group's list of modules.
Change-Id: I74c3d849ed2614dcdb6f54a6760122b933bb84c2
makeUniquePackageNames never assigned to pkgs, so all packages
were using their short names, even if another package had the
same short name.
Change-Id: I5420f7839efd25362d6c5d85265399ce7b29ddb2
The patch "Fix panic when dependency cycle includes the first
module visited" caused cycles to print incorrectly by initializing
the current module to be the last module in the cycle, when it
should be the first module in the cycle.
Change-Id: Iaf939283a48faa4cc6eeb9b19aed57993575a687
Running bpfmt or bpmodify on a Blueprints file that references
variables defined in a parent Blueprints file causes parse errors
on unknown variables. Modify the parser to parse in two modes,
parser.Parse and parser.ParseAndEval. The first parses without
attempting to evaluate variables and expressions, the second
performs a full evaluation.
Change-Id: Ic11948ea77c8e65379d7260591ada15db0e4f5b9
Calling CreateVariations with no variants will corrupt the
modules list with a module with nil logicModule, panic early
with a useful message instead.
Change-Id: Ic5c921efcba70c54efb5bb21a9626b2999376a69
createVariations relies on reusing the logicModule for the first
variant in order to replace the original module in the global module
map, which prevents leaving old split modules around in the global
module map. Add comments to explain the behavior.
Change-Id: Ia06d7fa4bc777e807241996d4e8bf977e641d741
spliceModules was carefully reallocating the slice if necessary, and
then throwing away the reallocated slice and writing to the too-short
original slice. Fix the bug, and add tests that would have caught it.
Change-Id: Ifc13b2025198b270c97097fd7d28cd36e460c98c
"Variant" has been used to mean two slightly concepts in Blueprint.
The first and most obvious is a variant of a module, meaning a list
of all the ways that on version of a module differs from other
versions of the same modules, for example "arch:arm, link:static".
The other is for the specific way that a version of a module differs,
"arch:arm".
Rename all of uses of "variant" as the second meaning to "variation",
and update the documentation to clarify "variant" vs. "variation".
This modifies the build logic api to convert CreateVariants to
CreateVariations.
Change-Id: I789ef209ae6ddd39ec12e0c032f9f47f86698fe6
The mutators that run after dependencies are resolved can be too late
to support build logic that needs to vary the dependencies based on
the mutated axis, for example architecture. This patch provides an
EarlyMutator interface that can be used to mutate modules before
any dependencies have been resolved.
In order for dependencies to be satisifed in a later pass, all
dependencies of a module must either have an identical variant,
must have a single variant, or must be inserted using
DynamicDependencyModuleContext.AddVariantDependency.
Change-Id: Ic6ae57e98edfd6c8c09a7788983128d3e4e992f0
Move actionDefs from moduleGroup to moduleInfo. This will result
in multiple build.ninja headers for a single Blueprint file module
if it has multiple variants, each one specifying the variant
used to generate the following rules.
Change-Id: I414cd4d3266da8c2e92118b295569627ddf48cdd
Instead of iterating through groups and then iterating through the
group's modules, iterate through the modules directly. This will
allow circular dependencies between groups as long as the individual
modules don't have circular dependencies. In order to maintain the
ordering of modules in a group, each module is considered to have
an implicit dependency on any earlier modules in the same group.
Change-Id: Ibce67167c7234db70ede0a6b5c2b43fb8e0bb05d
Replace the array of mutator name/variant name pairs with
a map of mutator name to variant name, and store the string
variant name separately.
Change-Id: I181c2fcb05724c8755b90aaf866fdce9ef720b01
Move most of the contents of moduleGroup into moduleInfo. This will
eventually reduce moduleGroup to a simple list of variants of the
same module.
Change-Id: I4289eb9953509751d7637558cd6db73373ccdf78
Forcing module names to be valid ninja names is an unnecessary
restraint on the project build logic. Allow any string as a
module name, and sanitize and uniquify the module name for use
in module-scoped variables.
Also move the module scope to be per-module instead of per-group
so that modules can use the same local variable name for each variant.
Change-Id: If44cca20712305e2c0b6d6b39daa5eace335c148
The cycle check can panic if the first call to check happens to land
on the first module in a cycle. Print the cycle instead of panicking.
Change-Id: I6fc1c66dcc37b1eb6b11b9e65343452af3c8538d
generateModuleBuildActions was continuing to call GenerateBuildActions
on all modules even after one returned an error. This could cause a
later module to panic because it was missing information that was
supposed to be generated by the module that returned an error, hiding
the earlier error message.
If a module returns an error, stop triggering any new goroutines to
call GenerateBuildActions on later modules.
Make integrating with go tools easier by putting the blueprint package
files in the top level directory of the git project instead of in a
subdirectory called blueprint.
Change-Id: I35c144c5fe7ddf34e478d0c47c50b2f6c92c2a03
