We were previously setting GOROOT to "prebuilts/go/linux-x86" during the
ninja executions when we were running Soong. But we can also run Soong
during the main ninja execution, were GOROOT was unset. When the GOROOT
was unset, the default GOROOT in our Go installation is
"./prebuilts/go/linux-x86" (note the extra ./).
This would cause g.bootstrap.goRoot to change between some soong runs,
causing us to rebuild all go programs (and anything depending on them)
more often than necessary.
So instead, keep GOROOT undefined when running Soong. Everything that
matters is using runtime.GOROOT(), which will fall back to the default.
Continue setting $GOROOT for bootstrap.bash, otherwise it fails when
there is no system provided go binary. What we give bootstrap.bash
doesn't really matter, since we don't actually use the blueprint wrapper
in Android.
Test: m blueprint_tools; touch bionic/libc/tzcode/new.c;
m blueprint_tools <doesn't rebuild everything>
Change-Id: I82f30c7c3b5d25e5cbf28fe37a97fdb776c4a164
This way config.mk no longer needs to check which java is in PATH and
fix it. It'll be consistent for all build steps under soong_ui.
Also unify handling of ANDROID_JAVA_HOME / JAVA_HOME with
OVERRIDE_ANDROID_JAVA_HOME / EXPERIMENTAL_USE_OPENJDK9.
Test: m nothing
Test: build/soong/soong_ui.bash --make-mode nothing (w/o envsetup.sh)
Test: aosp_arm ninja files are the same before/after
Test: before/after ninja files match with OVERRIDE_ANDROID_JAVA_HOME
Test: before/after ninja files match with EXPERIMENTAL_USE_OPENJDK9
Change-Id: Icdb65093d9c346524074de239a4f895e4230a24d
We can call directly into the blueprint bootstrap.bash using values that
soong_ui has already calculated.
Instead of calling into blueprint.bash, build minibp with microfactory,
and directly run ninja. This allows us to get individual tracing data
from each component.
Test: m -j blueprint_tools
Test: m clean; m -j blueprint_tools
Change-Id: I2239943c9a8a3ad6e1a40fa0dc914421f4b5202c
This way we only have one way to start a build, which always has logging
/ tracing / etc, even if we don't need Kati.
There's two ways to use this:
As a direct replacement for mkdir out; cd out; ../bootstrap.bash;
./soong -- as long as --skip-make is always passed, we'll never run
Kati, and Soong will run outside of it's "make" mode. This preserves
most of the speed, and allows full user control over the Soong
configuration.
A (experimental, dangerous) way to temporarily bypass the product
variable and kati steps of a build. As long as a user is sure that
nothing has changed from the last build, and they know exactly which
Ninja targets they want to build (which may not be the same as the
arguments normally passed to 'm'), this can lead to shorter build
startup times.
Test: rm -rf out; m --skip-make libc
Test: rm -rf out; m libc; m --skip-make libc
Test: rm -rf out; mkdir out; cd out; ../bootstrap.bash; ./soong libc
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: Ic0f91167b5779dba3f248a379fbaac67a75a946e
Instead of calling SetNinjaBuildDir, pass it to bootstrap.bash, so that
the bootstrap package can set it consistently during bootstrapping and
normal execution.
Bug: 63720725
Test: m -j nothing
Test: mkdir o; ../bootstrap.bash; ./soong
Change-Id: Ica88d2d5f1461b5be49bfe6316c6ec4ef4d89d49
Wrap os/exec.Cmd to use our Context and Config interfaces for automatic
logging and error handling. It also simplifies environment modification
based on the Config's environment.
This also adds sandboxing on Macs using sandbox-exec. A simple profile
is provided that only logs on violations, though multiproduct_kati on
AOSP has no violations. This isn't applied to ninja, only make / soong /
kati to start with. I measured <5% time increase in reading all
makefiles, and no noticable difference when kati doesn't regenerate.
I'd like to spin up a process to dump violation logs into our log file,
but the log reporting changed over the range of Mac versions that we
support, so that's going to be more complicated. Opening Console.app
works in all cases if you're local -- just search/filter for sandbox.
Linux sandboxing will be implemented later -- the sandbox definition is
opaque enough to support a different implementation.
Test: multiproduct_kati on AOSP master on Mac
Change-Id: I7046229333d0dcc8f426a493e0f7380828879f17
For proper ninja smart terminal support, we need to pass stdin to
./soong. Otherwise it starts a new line if the terminal isn't wide
enough.
Test: `rm -rf out/soong/.bootstrap; m -j` in narrow terminal
Change-Id: I643a526001adc2323a420a03fa1df282554c7886
This creates a rotating build.trace.gz in the out directory that can be
loaded with chrome://tracing. It'll include start and end timings for
make/soong/kati/ninja, and it will import and time-correct the ninja log
files.
Test: m -j; load out/build.trace.gz in chrome://tracing
Test: multiproduct_kati -keep; load out/multiproduct*/build.trace.gz
Change-Id: Ic060fa9515eb88d95dbe16712479dae9dffcf626
Right now this mostly just copies what Make is doing in
build/core/ninja.mk and build/core/soong.mk. The only major feature it
adds is a rotating log file with some verbose logging.
There is one major functional difference -- you cannot override random
Make variables during the Make phase anymore. The environment variable
is set, and if Make uses ?= or the equivalent, it can still use those
variables. We already made this change for Kati, which also loads all of
the same code and actually does the build, so it has been half-removed
for a while.
The only "UI" this implements is what I'll call "Make Emulation" mode --
it's expected that current command lines will continue working, and
we'll explore alternate user interfaces later.
We're still using Make as a wrapper, but all it does is call into this
single Go program, it won't even load the product configuration. Once
this is default, we can start moving individual users over to using this
directly (still in Make emulation mode), skipping the Make wrapper.
Ideas for the future:
* Generating trace files showing time spent in Make/Kati/Soong/Ninja
(also importing ninja traces into the same stream). I had this working
in a previous version of this patch, but removed it to keep the size
down and focus on the current features.
* More intelligent SIGALRM handling, once we fully remove the Make
wrapper (which hides the SIGALRM)
* Reading the experimental binary output stream from Ninja, so that we
can always save the verbose log even if we're not printing it out to
the console
Test: USE_SOONG_UI=true m -j blueprint_tools
Change-Id: I884327b9a8ae24499eb6c56f6e1ad26df1cfa4e4
