It's difficult for wrapping scripts to handle -b properly. Just pass
BUILDDIR instead, which is easier to handle. This still accepts -b, so
that incremental builds work across this change.
Now that we have multi-stage bootstrapping, we can make the primary
builder build more dynamic. Add the concept of plugins that will be
linked and loaded into bootstrap_go_binary or bootstrap_go_package
modules. It's expected that the plugin's init() functions will do
whatever registration is necessary.
Example Blueprint definition:
bootstrap_go_binary {
name: "builder",
...
}
bootstrap_go_package {
name: "plugin1",
pluginFor: ["builder"],
}
A package may specify more than one plugin if it will be inserted into
more than one go module.
Change-Id: I109835f444196b66fc4018c3fa36ba0875823184
With the introduction of $buildDir in bootstrapDir, the ninja cleanup
step hasn't been able to cleanup during the bootstrap stages. The main
stage was unaffected, as long as you were using "." as your buildDir.
Change-Id: I277dd7864989f9052d96cab9ce377548a1391a80
The full package path is no longer stored in the name variable, pass the
pkg variable into getDocs and putDocs so that types with the same name
in different packages don't collide.
Change-Id: I7ecd98815487467dbaadc8a318fa43b123fc83c1
The go compiler and linker changed in v1.5 -- to 'go tool compile' and
'go tool link' instead of 6g and 6l. Move the selection logic to
bootstrap.bash, and have it use compile/link if 6g/6l are missing. This
way the build.ninja.in will continue working with either go 1.4 or 1.5.
Travis and the test suite will fail under 1.5, since the build.ninja.in
is still generated with 1.4, and the function names in the comments
differ between 1.4 and 1.5.
To provide a consistent __FILE__ behavior with cpp, we want to be able
to run with SRCDIR="." and the outputs be saved elsewhere. Other tools
within android also expect to be run from $TOP.
Change-Id: I572bce5c9086b0c3310b42065ae98cbf5a1c6399
API Change -- will require changes to any code using bootstrap.BinDir
This way we can put references to other variables in BinDir, and still
be used properly by other packages.
Change-Id: I497424cb254b3a170401ac9420fa0adbf8d11d1e
This splits the current bootstrap stage into two stages:
A bootstrap stage, which like today, a reference is checked into the
tree. It just builds the "core" blueprint binaries -- minibp,
gotestmain, and choosestage. Just enough to build the next stage's ninja
file.
A primary builder stage. This builds the primary builder, the main ninja
file, and any other bootstrap binaries (bpfmt, etc).
The main advantage here is that the checked in file really only contains
references to blueprint -- not the primary builder. This will allow us
to make the primary builder more dynamic, by loading more module types
that may or may not exist in all trees.
It's even possible to reuse the build.ninja.in in the blueprint repo
directly now. We don't currently do that, since we still want to turn on
tests.
Change-Id: I18683891ed7348b0d7af93084e3a68a04fbd5dbc
This simplifies the bootstrap process while making it more flexible by
moving the stage selection into a go binary(choosestage). It will now be
possible to have more than two build stages.
Now each stage has a ninja template(main.ninja.in) and a timestamp
file(main.ninja.in.timestamp). The timestamp file may be updated by any
build stage that wishes to regenerate the ninja template. If the
choosestage binaries sees that the timestamp is newer than the template,
it will choose the prior stage.
The main stage no longer writes to the source tree to update the
build.ninja.in file. This was a problem for read-only source trees.
Instead, the choosestage binary first checks to see if that file is
newer than the last bootstrap.ninja.in, copies it in place, and starts
the boostrap stage.
The bootstrap stage regenerates it's own ninja template, but that
required a loop through the main stage to actually run it. The
choosestage binary now detects if the template has changed for the
current stage, and will restart the stage.
One change is that if dependencies do get messed up, instead of silently
failing, there's a higher chance that the bootstrap step will just
continue looping, doing nothing. This can happen if the main stage
has a dependency that triggers the bootstrap stage, but the bootstrap
stage doesn't see anything required to rebuild the main ninja file. A
side effect of this requirement is that changes to test code will now
rebuild the main ninja file.
Change-Id: I9965cfba79dc0dbbd3af05f5944f7653054455a2
This only checks to make sure that for a given updated file, the
bootstrap stage is properly run. It doesn't actually check to make sure
that anything was rebuilt.
Change-Id: I9cb6ff1d483264da30e43d5580361d93b148f42c
The workaround no longer works with the new stage selector. We may run
bootstrap.ninja.in twice before running the next stage, but we can't
encode whether to run another ninja instance in the checked in
build.ninja.in.
This can likely be solved, but now that there's an official release with
support for multiple passes, just push up the required version.
Change-Id: I76e321912e323d60e462aabec61bdfcc7118cd5e
The primary builder will now generate a rule to call itself with
--docs=.bootstrap/docs/<name>.html to produce an automatically
generated documentation file.
The documentation generation process is:
- Call each factory once to get empty property structs associated
with the module type
- Use reflection to determine the names of the type of each property
struct
- Use the bootstrap_go_package modules from reading the Blueprints files
to find the source files for each Go package used to build the primary
builder
- Use the go/parser module to find the type declaration for each
property struct
- Extract comments for the property struct and each property declaration
- Format all the comments into HTML
Change-Id: Icae9307cc10549a30bfc14d6922824099de5a9b0
Users that want to enable this option can use the '-t' option to
bootstrap.bash when passing '-r'. Builders that want to enable this can
set the RUN_TESTS environment variable in their bootstrap.bash.
The gotestmain tools is needed to write the main functions for the test
binaries, since 'go test' doesn't work well in this environment.
Change-Id: Iec5c2b5c9c3f5e3ba0ac8677fb88f5e963f9bd3f
Apparently Go only allows a single thread to be executing by default,
so the increase in performance seen when concurrency was added to
Blueprint was entirely due to scheduling goroutines while the
executing goroutine was blocked in a syscall. Whoops.
Set GOMAXPROCS to the number of CPUs. Reduces wall thime by >2x on
one workload.
Change-Id: Ib5fc7761c09a076fc5a1b7f08f5ccf034304477c
If .ninja_log contains an entry that used to be a file but is now a
non-empty directory the os.Remove call will fail. Silently ignore
this error, if the files in the directory are abandoned the directory
will be cleaned up when the files are removed.
Make integrating with go tools easier by putting the blueprint package
files in the top level directory of the git project instead of in a
subdirectory called blueprint.
Change-Id: I35c144c5fe7ddf34e478d0c47c50b2f6c92c2a03
