build/soong/soong_ui.bash --dumpvars-mode \
--vars="..." \
--abs-vars="..." \
--var-prefix="..." \
--abs-var-prefix="..."
is similar to the previous:
CALLED_FROM_SETUP=true BUILD_SYSTEM=build/core \
make -f build/core/config.mk dump-many-vars \
DUMP_MANY_VARS="..." \
DUMP_MANY_ABS_VARS="..." \
DUMP_VAR_PREFIX="..." \
DUMP_ABS_VAR_PREFIX="..."
and
build/soong/soong_ui.bash --dumpvar-mode [--abs] VAR
is similar to the previous:
CALLED_FROM_SETUP=true BUILD_SYSTEM=build/core \
make -f build/core/config.mk dumpvar-[abs-]-VAR
But uses soong_ui and ckati, so that we use a consistent make parser and
sandboxing configurations.
One major output difference between the pure make implementation and
this one is that report_config in Go is implemented using embedded
newlines in single quotes, while the make implementation uses `` with
embedded echo commands. This seems to work fine for both bash and zsh,
and report_config isn't meant to be machine-parsed anyways.
Test: build/soong/soong_ui.bash --dumpvar-mode report_config
Test: build/soong/soong_ui.bash --dumpvar-mode TARGET_DEVICE
Test: build/soong/soong_ui.bash --dumpvar-mode --abs PRODUCT_OUT
Test: build/soong/soong_ui.bash --dumpvar-mode --abs ALL_PRODUCTS
Test: build/soong/soong_ui.bash --dumpvars-mode --vars="report_config TARGET_DEVICE" --abs-vars="ALL_PRODUCTS"
Test: build/soong/soong_ui.bash --dumpvars-mode --vars=TARGET_DEVICE --abs-vars=PRODUCT_OUT --var-prefix=v_ --abs-var-prefix=a_
Change-Id: I0fbd0732bbf6fcfcd24084cf3c830a91a4b6bfc2
This reverts commit a3e6c520ac. I fixed
a few heavy stack users, and we've got better debugging.
Bug: 36182021
Test: m dist
Change-Id: Iefd8589185720e44c40a2e0ceff5fd1438211b86
Pass --dump-stats to ckati, but filter out the '*kati*' lines so that
they only end up in our verbose output. That way we've always got access
to the statistics.
Bug: 36182021
Test: m nothing; cat out/soong.log
Change-Id: Iaf7a814fc67f3e475c913faf69924a7f4e2ae3b3
This speeds up dumping make variables from ~380ms using make to ~220ms
using ckati. It also means that we're consistently using the same parser
for builds (with the same .KATI_READONLY/etc extensions).
envsetup.sh (lunch) / other scripts still use make, changing those to go
through soong_ui will be a future change.
Test: m clean; m nothing
Test: USE_GOMA=true m nothing
Test: m PRODUCT-aosp_x86-sdk
Test: m APP-Calculator
Test: build/soong/build_test.bash -only-config (on AOSP and internal master)
Change-Id: I6ca554de8de4955fb869001d06d29969b75751cc
They don't really affect anything general in android except for the
output path.
Bug: 65453318
Test: TARGET_BUILD_TYPE=debug m
Change-Id: I98c17fefae61f9f7936e184bd1f8441086bb7d23
Instead of pairing the reading of CleanSpec.mk files with the reading of
Android.mk files, split them into separate, individually cachable steps.
This way we only read Android.mk files once after a clean/sync. We'll
still read the CleanSpec.mk files multiple times, but that's
significantly faster than reading all the Android.mk files.
This adds about 50ms if kati doesn't need to reread the CleanSpec.mk
files. Reading all the CleanSpec.mk files takes about a second.
Bug: 35970961
Test: m clean; m nothing; m nothing
Test: Add CleanSpec.mk line, see it executed.
Change-Id: I83bad15c50709510959d5b8b673a907b8aa7de82
Kati sets detect_leaks=0 by default now.
Test: unset ASAN_OPTIONS; SANITIZE_HOST=address m nothing
Change-Id: I0cbf04ae9ed40de520abca3fb35c65e7f543682d
We can call directly into the blueprint bootstrap.bash using values that
soong_ui has already calculated.
Instead of calling into blueprint.bash, build minibp with microfactory,
and directly run ninja. This allows us to get individual tracing data
from each component.
Test: m -j blueprint_tools
Test: m clean; m -j blueprint_tools
Change-Id: I2239943c9a8a3ad6e1a40fa0dc914421f4b5202c
The Finder runs roughly 200ms faster than findleaves.py in aosp,
and runs roughly 400ms faster in internal master.
Bug: 64363847
Test: m -j
Change-Id: I62db8dacc90871e913576fe2443021fb1749a483
This reverts commit b6d161bf16.
Reason for revert: New Build Breakage: aosp-master/sdk_mac @ 4260825
Change-Id: I8bda8c50c5e5c9f84621d11a4c15b168833bcd21
This way we only have one way to start a build, which always has logging
/ tracing / etc, even if we don't need Kati.
There's two ways to use this:
As a direct replacement for mkdir out; cd out; ../bootstrap.bash;
./soong -- as long as --skip-make is always passed, we'll never run
Kati, and Soong will run outside of it's "make" mode. This preserves
most of the speed, and allows full user control over the Soong
configuration.
A (experimental, dangerous) way to temporarily bypass the product
variable and kati steps of a build. As long as a user is sure that
nothing has changed from the last build, and they know exactly which
Ninja targets they want to build (which may not be the same as the
arguments normally passed to 'm'), this can lead to shorter build
startup times.
Test: rm -rf out; m --skip-make libc
Test: rm -rf out; m libc; m --skip-make libc
Test: rm -rf out; mkdir out; cd out; ../bootstrap.bash; ./soong libc
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: Ic0f91167b5779dba3f248a379fbaac67a75a946e
It can find every Android.bp in internal master in
about 2.5 sec the first time and 0.3 sec subsequent times
Bug: 62455338
Test: m -j blueprint_tools # which runs the unit tests
Test: m -j blueprint_tools && \
out/soong/host/linux-x86/bin/finder \
-v
--db /tmp/mydb \
--names Android.mk \
--prune-files .android-out-dir \
--exclude-dirs .git,.repo \
. \
>/tmp/finder-log 2>&1
Change-Id: I5ab2650459a1dae0d5d076faf411ec2d053c743d
Instead of calling SetNinjaBuildDir, pass it to bootstrap.bash, so that
the bootstrap package can set it consistently during bootstrapping and
normal execution.
Bug: 63720725
Test: m -j nothing
Test: mkdir o; ../bootstrap.bash; ./soong
Change-Id: Ica88d2d5f1461b5be49bfe6316c6ec4ef4d89d49
The build now uses a prebuilt javac binary distributed with the
source, checking the host java version is no longer required.
Reapplies Ieef83481d0e6d68371bbd8a422f870ffe2bbb428 after updating
some more manifests to include the prebuilts.
Test: m -j checkbuild
Bug: 62956999
Change-Id: I774dbb75c8ae2ff1a990423a3827c77886e44779
(cherry picked from commit c6f07f00da)
The build now uses a prebuilt javac binary distributed with the
source, checking the host java version is no longer required.
Test: m -j checkbuild
Bug: 62956999
Change-Id: Ieef83481d0e6d68371bbd8a422f870ffe2bbb428
MAKELEVEL and MAKEFLAGS were are both unset by makeparallel when invoked
from Make. In preparation for removing the Make wrapper (and
makeparallel), strip them in soong_ui as well.
Add MFLAGS for completeness.
Test: m -j
Change-Id: I029f9464d11b432e979b32ede360450f4facb038
Make supports specifying all types of variables on the command line
(using =, :=, +=, and other variable references. When running soong_ui
through make/makeparallel these all effectively become environment
variables.
So in preparation to remove the Make wrapper, support a simplified form
of this syntax, roughly equivalent to what the shell supports if
specified before the command (<NAME>=<VALUE>).
Test: m -j blueprint_tools
Change-Id: I08fa2b86710f282e619b0cc324a3e5bbaf62d26e
In preparation to remove Make/makeparallel from soong_ui startup, we
need to preserve compatibility with the different ways that make
supports the -j option.
Nothing changes unless Make/makeparallel is removed from the startup.
Once that is removed, not specifying a -j value will be equivalent to
'-j' instead of '-j1', like Ninja. A value will also be supported when
specifying -k, like Ninja (though specifying it alone will be equivalent
to '-k 0').
Test: m -j blueprint_tools
Change-Id: I9d5d59bedd4f6e5cca76bdb4cd47e0b5b7d523f0
This causes Soong to put the outputs of each genrule into a temporary
location and copy the declared outputs back to the output directory.
This gets the process closer to having an actual sandbox.
Bug: 35562758
Test: make
Change-Id: I8048fbf1a3899a86fb99d71b60669b6633b07b3e
This doesn't catch all the possible causes of timeouts,
(like if Ninja is only partially stuck or if Kati is stuck)
but it should clarify some causes of stuckness
Bug: 62065855
Test: m -j showcommands NINJA_HEARTBEAT_INTERVAL=500ms
Change-Id: I73a792ae91873b19d7b336166a2d47f37c549906
I removed *.txt from the installclean list yesterday, but it turns out
that android-info.txt being removed was the only thing triggering
system.prop to be rebuilt on the incremental build servers. So continue
android-info.txt so that the version information in system.prop is
correct.
I'll follow this up with a more complicated change to make the
system.prop generation depend on the build number/versions more
directly, but that's a more complicated change since we need to change
how the build number file is written.
Bug: 62252890
Test: m ...intermediates/system.prop; m installclean; m ...intermediates/system.prop
Change-Id: I25dd8ddd99f7b2c927a8cbfb7f1aca1dd3e82a82
For *.ini/*.txt files we should be recreating them as the instructions
change, so we shouldn't need to remove them. The biggest reason to
remove them is that the auto-installclean was removing
$PRODUCT_OUT/build_fingerprint.txt just after Kati wrote it out, causing
the fingerprint to be blank on builds where the auto installclean code
ran.
There's no reference to xlb in our tree, and it's been around since the
beginning of git history, so remove it.
Bug: 62224537
Test: lunch aosp_arm-eng; m nothing;
lunch aosp_arm-userdebug; m -j nothing;
check out/target/product/generic/build_fingerprint.txt
Change-Id: Id391af5c7d95ea78e4e68fae294d8295fccb1964
By default, the Android build enforces an OpenJDK 8 toolchain,
whose name contains the strings "openjdk" and "1.8".
After this CL, the check can be changed to enforce a toolchain
name starting with "9" and without the need for "openjdk" having
to occur in the name.
This experimental new check can be enabled by running:
export EXPERIMENTAL_USE_OPENJDK9=true
To switch back to the standard check, run:
unset EXPERIMENTAL_USE_OPENJDK9
Test: make ANDROID_COMPILE_WITH_JACK=false checkbuild tests \
&& make checkbuild tests
(with OpenJDK 8u45 toolchain on the PATH)
Test: make EXPERIMENTAL_USE_OPENJDK9=true \
ANDROID_COMPILE_WITH_JACK=false checkbuild
(with jdk 9-ea+170 toolchain on the PATH)
Bug: 38177295
Change-Id: I75de3e23fe0b7f41eb6dd3f55dadd3fa3c3383bd
This flag allowed for building with an OpenJDK 7 toolchain. It was
used for build bot builds that now work with an OpenJDK 8 toolchain.
Hence, this feature is no longer required. This CL drops it.
Bug: 27583810
Test: Treehugger build succeeds.
Change-Id: Iba9cf12cff2597fa70aa9999e8a6bda190f29f3c
This speeds things up a little bit, but the major win is that we don't
need to recursively call into the build during the auto installclean.
Test: m -j installclean
Test: m -j dataclean
Change-Id: I09ded8dbd8a2015c7848bc0042dfd3adad1b31c9
This way kati won't need to be run as often (either initially, or when
switching products with the same device).
Bug: 35970961
Test: m clean; m -j blueprint_tools; m -j blueprint_tools; m -j blueprint_tools
Test: lunch aosp_arm-eng; m -j blueprint_tools; lunch full-eng; m -j blueprint_tools; <repeat>
Change-Id: Ie9fca3c8f1dd412459ea47c7090c7c5fdb0bcf6e
When kati keeps state around, it has to regenerate the ninja file every
time the state is changed. So move the java version checking into
soong_ui, where we can parallelize it with other operations instead of
only checking it occasionally.
Bug: 35970961
Test: Put java7 in PATH, m -j
Test: Put java8-google in PATH, m -j
Test: Put a space in TOP, m -j
Test: OUT_DIR=<case-preserving fs> m -j
Test: OUT_DIR=<path with space> m -j
Test: DIST_DIR=<path with sapce> m -j
Change-Id: I3245c8dd6d856240d17d54cb05d593dc9df71a27
It now uses the same output style as ninja, overwriting status lines in
smart terminals.
Test: multiproduct_kati
Test: multiproduct_kati | cat
Change-Id: I8db5198ffdc5ebc5503241ac492379753d92978e
So that we don't have to load up all the makefile state just to remove
the output directory.
Starting from a completely empty out directory:
kati: 16s
soong_ui: 2.0s
From a minimal out directory (m -j blueprint_tools):
kati: 3.8s
soong_ui: 0.4s
Test: m -j clean
Test: m -j clobber
Change-Id: Ibeec6fbfa29387750342a752a55336caca4b3992
Otherwise Make and soong_ui disagree about where the soong.variables
file should be. For example, when "OUT_DIR=out/":
Error dumping make vars: Failed to parse make line: "make: Nothing to be done for `out/soong/soong.variables'."
We would need to pass out//soong/soong.variables into make, since it
doesn't attempt to match clean/unclean paths.
Bug: 37553659
Test: OUT_DIR=out/ m -j
Change-Id: Iebeea4868b96b11e8b08735f272e22be5db1d701
We're seeing some rare crashes that appear to be stack overflows. To get
better debugging, use the asan version of ckati when 'dist' is
specified.
Bug: 36182021
Test: m -j
Test: m -j dist
Test: make -j dist (w/o lunch)
Change-Id: Ic3a5590974bfd718bf1929355d344b5933ac1d4f
