This duplicates building common blueprint go packages between minibp and
the primary builder, but drastically simplifies the first stage,
removing the need to check in a generated build.ninja.in.
Change-Id: I639a9637f1ed36d4210823ef276c0f7a064a83bd
tl;dr: Read if you don't use the wrapper or use SKIP_NINJA
Previously, we were relying on the ninja behavior of restarting the
build when the build.ninja file was updated to switch between different
bootstrap stages. But that means that every step that could produce a
build.ninja must pass in order to switch to a different stage. That
wasn't a big problem when we had a two stage build -- there was very
little that could fail in the second stage before we chose to go back to
the first stage. But when we had a three stage build, it was possible to
get into a state (usually during development) where you were in the
second stage, but the build was failing because the first stage needed
to be run. This was fixed in d79f1af742
by adding a wrapper that always started building at the first stage.
But this kept all of the complexity of using ninja restarts without any
of the benefits, so this change removes that complexity and just runs
each stage sequentially in the wrapper. So the wrapper is now required.
Since we're no longer going through choosestage, we can also skip the
template parsing for the later stages that don't need to be templated --
this can save a couple of seconds for large files.
In addition to all of the above, this also lets Soong reduce the number
of times the main ninja file is loaded. We had been running the wrapper
once (3 stages), then running ninja again after combining the
Soong-generated build.ninja with the Kati-generated build.ninja. This
change lets us removing the intermediate parsing of Soong's build.ninja,
so that we only execute ninja 3 times per build. It also lets us have
dependencies on pools or rules from Kati in the primary builder, since
we're never executing the main build.ninja without the Kati build.ninja.
The wrapper has a new option, NINJA to provide the path to ninja. This
used to be hardcoded to `ninja`, and will still default to that. But
we'll be running the first two bootstrap stages with $NINJA even if
SKIP_NINJA is set.
The wrapper passes "-w dupbuild=err" to ninja now -- this really should
always be turned on if you care about reliable builds.
Change-Id: I6f656b74eb3d064b8b9e69d1d6dac1129d72b747
Move these tools from $buildDir/.bootstrap/bin to $buildDir/bin
(configurable by the primary builder).
Also delay building them until the main stage, and give them a phony
target "blueprint_tools".
Build logic can now implement a RemoveAbandonedFiles, and the bootstrap
logic will only remove abandoned files if that method returns true.
Leaving the method unimplemented will result in the existing behavior of
always removing abandoned files.
The go compiler and linker changed in v1.5 -- to 'go tool compile' and
'go tool link' instead of 6g and 6l. Move the selection logic to
bootstrap.bash, and have it use compile/link if 6g/6l are missing. This
way the build.ninja.in will continue working with either go 1.4 or 1.5.
Travis and the test suite will fail under 1.5, since the build.ninja.in
is still generated with 1.4, and the function names in the comments
differ between 1.4 and 1.5.
To provide a consistent __FILE__ behavior with cpp, we want to be able
to run with SRCDIR="." and the outputs be saved elsewhere. Other tools
within android also expect to be run from $TOP.
Change-Id: I572bce5c9086b0c3310b42065ae98cbf5a1c6399
This splits the current bootstrap stage into two stages:
A bootstrap stage, which like today, a reference is checked into the
tree. It just builds the "core" blueprint binaries -- minibp,
gotestmain, and choosestage. Just enough to build the next stage's ninja
file.
A primary builder stage. This builds the primary builder, the main ninja
file, and any other bootstrap binaries (bpfmt, etc).
The main advantage here is that the checked in file really only contains
references to blueprint -- not the primary builder. This will allow us
to make the primary builder more dynamic, by loading more module types
that may or may not exist in all trees.
It's even possible to reuse the build.ninja.in in the blueprint repo
directly now. We don't currently do that, since we still want to turn on
tests.
Change-Id: I18683891ed7348b0d7af93084e3a68a04fbd5dbc
This simplifies the bootstrap process while making it more flexible by
moving the stage selection into a go binary(choosestage). It will now be
possible to have more than two build stages.
Now each stage has a ninja template(main.ninja.in) and a timestamp
file(main.ninja.in.timestamp). The timestamp file may be updated by any
build stage that wishes to regenerate the ninja template. If the
choosestage binaries sees that the timestamp is newer than the template,
it will choose the prior stage.
The main stage no longer writes to the source tree to update the
build.ninja.in file. This was a problem for read-only source trees.
Instead, the choosestage binary first checks to see if that file is
newer than the last bootstrap.ninja.in, copies it in place, and starts
the boostrap stage.
The bootstrap stage regenerates it's own ninja template, but that
required a loop through the main stage to actually run it. The
choosestage binary now detects if the template has changed for the
current stage, and will restart the stage.
One change is that if dependencies do get messed up, instead of silently
failing, there's a higher chance that the bootstrap step will just
continue looping, doing nothing. This can happen if the main stage
has a dependency that triggers the bootstrap stage, but the bootstrap
stage doesn't see anything required to rebuild the main ninja file. A
side effect of this requirement is that changes to test code will now
rebuild the main ninja file.
Change-Id: I9965cfba79dc0dbbd3af05f5944f7653054455a2
Users that want to enable this option can use the '-t' option to
bootstrap.bash when passing '-r'. Builders that want to enable this can
set the RUN_TESTS environment variable in their bootstrap.bash.
The gotestmain tools is needed to write the main functions for the test
binaries, since 'go test' doesn't work well in this environment.
Change-Id: Iec5c2b5c9c3f5e3ba0ac8677fb88f5e963f9bd3f
Make integrating with go tools easier by putting the blueprint package
files in the top level directory of the git project instead of in a
subdirectory called blueprint.
Change-Id: I35c144c5fe7ddf34e478d0c47c50b2f6c92c2a03
2015-01-23 14:23:27 -08:00
Renamed from blueprint/bootstrap/config.go (Browse further)
