Soong UI will generate a new proto file in the out directory for
other systems to read the build completion status during a course
of a build. The proto file contains build action counters where
total_actions: represents the total build actions of a build (will
increase and may decrease during a course of the build)
finished_actions: represents the number of executed build actions.
finished_actions never decreases and finished_actions <= total_actions.
current_actions: represents the number of build actions being executed
and current_actions + finished_actions <= total_actions.
Bug: 150401146
Test: executed regen.sh script.
Change-Id: I04f4c07855723d0684b141d88cb3529ab5d9fccd
If this couldn't read the current user, it would print a generic
message, then proceed to dereference the invalid user struct :(
Provide somewhat reasonable defaults, and better error messages instead.
Test: m nothing
Test: run docker with misconfigured user
Change-Id: I21af77c7d8d1d2d27cb04546667eb1094c62a7a1
We have known problems (docker; distros w/o user namespaces) which mean that we won't be turning this into a fatal error anytime soon. Also remove the bug report link, it's not useful to continue getting the same reports.
Change-Id: I271871d68150417ac938f074d3730cad4518e327
Test: treehugger
Collect the number of modules and variants and some basic statistics
on memory usage inside soong_build by writing out a proto that is
read back in by soong_ui.
Test: examine soong.log
Change-Id: I6926876377a4f6229cf41fdbf166ae03c885ea55
Trying to override the default directory for ccache by using the
CCACHE_DIR environment variable fails unless it is in the allowed
list.
Bug: 149670916
Test: manual
Change-Id: I8e7eea7a5c25d7ea5f0956fafc70d62522f3c4fc
Apparently PackageManager gets confused when the apk disappears, but the
directory still exists.
Test: lunch aosp_arm-eng; m BasicDreams
<remove BasicDreams from PRODUCT_PACKAGES>
m nothing
ls out/target/product/generic/system/app
Change-Id: I486c0ddadde55fad226049d3fb0f09af88117da0
These variables control whether we accept cached-results / not and
whether we should update the cache with locally executed results or not.
I need accpet-cached whitelisted to purge the the RBE cache for the
invalid cache entry we have set for the failure we say yesterday.
Bug: b/148387048
Change-Id: I7344fc083f82e0b7bc11084376a267d19cf30bb8
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.
Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable. Put
metalava into the highmem pool.
Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool. When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools. Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.
Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
Determining the total build time from the existing metrics is
tricky because some of the metrics are overlapping. For example,
the "soong" metric includes "blueprint bootstrap",
"environment check", "minibp", "bpglob", "minibootstrap", and
"bootstrap". Create a new singleton "total" metric that
covers the interesting parts of the build.
Test: m nothing
Change-Id: Iedcf5c9cf0d27252b694d35e6ff66ca1fcf734ac
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.
Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.
Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
This is used by the art apex test.
Bug: 147197813
Test: m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true art-check-debug-apex-gen
Change-Id: Id185cd35f16131f2c9a8afeba2e5b87834b0e620
We're passing in ASAN_OPTIONS=detect_leaks=0 on the command line, and
the build started failing after we limited the environment.
Test: forrest of aosp-master/aosp_x86_64-eng
Change-Id: I42c91897c7677e1a249412e5a8bc7bb1edb2f881
Make now exports (partial) lists of what should be installed. Those
lists are incomplete, but if something is removed from the list, we can
remove it from the filesystem so that incremental builds work better.
Test: built-in tests (m blueprint_tools)
Test: remove a module from PRODUCT_PACKAGES, see the print and file removed
Test: change the name of a cts test, see the old one removed from cts
Change-Id: I6bc14c6a5af63383ea265e97ed8b875fa51af0ed
See the change that added ALLOW_NINJA_ENV for more information.
Test: m nothing; check out/soong.log for small list
Test: ALLOW_NINJA_ENV=true m nothing; check out/soong.log
Change-Id: I7761c6a07a7f8b0acee107e9c27c7739dd4b63ab
Ninja does not track changes in environment variables, so we get
potentially incorrect builds when environment variables change during
incremental builds if some action was using one of them.
Add a variable to limit exposure of these variables to ninja, and thus,
all actions run by ninja. Kati and Soong can still read environment
variables, they explicitly track which ones they read so that we can
re-run them appropriately.
This list is just the beginning, there's no good way to detect which
environment variables are currently being used and to pass them through.
So this initial change won't have a behavioral change, and we'll flip
the switch and see what fails or who complains, flipping it off and on
and adding to the list until we can make this always happen.
Also adds a board-specific `BUILD_BROKEN_NINJA_USES_ENV_VARS := ...`
list so that we can temporarily allow board-specific variables until
they're fixed.
Test: check out/soong.log
Test: ALLOW_NINJA_ENV=false m nothing; check out/soong.log
Test: set BUILD_BROKEN_NINJA_USES_ENV_VARS := OLDPWD
ALLOW_NINJA_ENV=false m nothing; check out/soong.log
Change-Id: I08e4834ce12100a577ef7d6a9a21b9e9d345cb93
Colin originally wrote this for the highmem pool:
https://android-review.googlesource.com/c/platform/build/soong/+/1168271
But while that's a larger change, this is useful to just be in the logs,
and for use in multiproduct_kati to better limit the number of
concurrent jobs.
Test: check soong.log on linux and mac
Change-Id: I0518d303a220d775f8d78dba9f660b2954e68e3e
(Although I was the one who added the gzcat reference, I don't believe
we actually use it. And neither my Debian host nor toybox has gzcat --
as opposed to zcat -- anyway. So just quietly drop that.)
Test: treehugger
Change-Id: I95d94dab1daf72faedfa170770e8ebfaa37b70c6
When LOCAL_COPY_HEADER lines are removed, we just removed the copy rule,
we didn't actually remove the old header, so C/C++ files could still
include the old headers during incremental builds.
While we do consider LOCAL_COPY_HEADERS as near-obsolet, and it'll
disappear as we move everything over to Soong, this could produce some
unfortunate incremental build results while people are attempting to
remove them.
Fix this by ensuring that only the files currently listed in
LOCAL_COPY_HEADERS exist in TARGET_OUT_HEADERS after we run Kati.
Test: Remove a LOCAL_COPY_HEADERS entry; see the header removed
Change-Id: I817305703a6996d50490d552623d7df019b608c9
Add a cuj_tests binary that runs through a sequence of critical
user journey builds and collects metrics.
Bug: 145688720
Test: cuj_tests
Change-Id: I6bb1a3fc4a458fd8cea7b7c924f565d0cf3e6df3
Use Printf instead of Println for formatted strings, and pass
the filename instead of the proto.
Test: cuj_tests
Change-Id: Id261b5a34304a0caa61faa1f3bbc388aacdd25a6
Delay writing the BUILD_DATETIME_FILE until after the out directory
has been created.
Test: cuj_tests
Change-Id: Ice6f34d003f93c26b5d2d0b64f92b11efe16c2d4
This runs an arbitrary shell script for all non-interactive bash sessions, and we've got lots of those.
It was causing problems with Fedora 30, which triggers the `manpath` executable, which isn't available in our $PATH:
https://groups.google.com/d/msg/android-building/WqEyZQuxYtY/zDXl9irOCAAJ
Change-Id: If2f63bbbcc3accb8d239f93fa06823d301d69488
