Previously we ensured the kati marker exists in kati-enabled builds, but
did not ensure the kati marker didn't exist in non-kati-enabled builds,
resulting in soong thinking kati would run after it when it would not.
Also change the condition to only create the kati_enabled marker if we
don't skip kati or kati ninja.
Bug: 189187214
Test: m nothing && soong_ui --make-mode --soong-only && verify
kati_enabled marker is gone
Change-Id: If196b194b6aee38e9d4135889a4a4048ee5add82
The source tree will eventually be made ReadOnly, and recipes that write
directly to the source tree will fail. Use a pattern-match approach on
the results of stdout/stderr to provide hints to the user in such a
scenario.
If multiple patterns are found in raw output, print error hint
corresponding to first pattern match. first pattern match is chosen
since the failing function will be at the top of the stack, and hence
will be logged first
Test: Wrote a unit test to assert errorhint is added to output.
Wrote an integration test that writes to a file in the source tree
1. When source_tree is RO, the recipe fails and an error hint is printed
to stdout
2. When source tree is RW, the recipe succeeds and no error hint is
printed
Bug: 174726238
Change-Id: Id67b48f8094cdf8a571c239ae469d60464a1e89c
This CL adds generateJsonModuleGraph bazelBuildMode and ensures that it
returns Soong early, before Kati and Soong metrics collection begin
(which causes errors, since they're looking for files that Soong in json
dump mode did not write).
Test: TH
Test: rm -rf out && SOONG_DUMP_JSON_MODULE_GRAPH=/tmp/soong.json m nothing
Change-Id: I264eadb3b3b0cd6b6e7a65adc7b39bb1c01ca136
--skip-config can be combined with --soong-only to skip creating
soong.variables from Make, allowing a custom soong.varibles.
Test: manual
Change-Id: Iee9bfa4efeed801339e7c3c1e4807709485e701a
1. Initialize an empty glob file for .bootstrap/build.ninja. Initializing
in soong_ui (and not soong_build) prevents inadvertently creating the
file inside the source tree
2. Remove soong-build-globs.ninja, which is not used during the build
Bug: 187194795
Test: Ran the following command locally for the target
art-target-arm:git_master-art
```
. ./build/envsetup.sh && lunch armv8-eng &&
art/tools/buildbot-build.sh --target
```
Change-Id: Ibe6eeff65ea1ab25136642299e9878d0da1cac42
The previous --skip-kati flag could be interpreted as "do not run kati
to re-generate ninja file". Add a more specific flag for the "soong
only" build use-case, where we do not load the kati-generated ninja
files at all.
Bug: 189187214
Test: build/soong/soong_ui.bash \
--make-mode \
--soong-only
--skip-soong-tests \
TARGET_PRODUCT=mainline_sdk \
SOONG_ALLOW_MISSING_DEPENDENCIES=true \
SOONG_SDK_SNAPSHOT_VERSION=unversioned \
SOONG_SDK_SNAPSHOT_USE_SRCJAR=true \
out/soong/mainline-sdks/art-module-sdk.zip
Change-Id: I91abbd28af517d4b550ebc6d88fd64947caf9545
Merged-In: I91abbd28af517d4b550ebc6d88fd64947caf9545
(cherry picked from commit 546de4a1f3)
- Rename the "BuildX" variables to "RunX"
- Remove redundant comments
- Inline all the "what to do" based on config in build.go
- Inline some constants only used in one place
Bug: 189187214
Test: m nothing
Test: build/soong/build_test.bash
Change-Id: I111a69e642212d7938d4971283545e0d9acbb01a
Merged-In: I111a69e642212d7938d4971283545e0d9acbb01a
(cherry picked from commit d274ea9196)
The following two Make vars control RO/RW access to the source tree
1. BUILD_BROKEN_SRC_DIR_IS_WRITABLE
2. BUILD_BROKEN_SRC_DIR_RW_ALLOWLIST
By default, (1) will be truthy.
- this ensures that this CL is a non breaking change across all products
- different products can opt in to set is as "false"
Bug: 174726238
Test: from build/soong dir, ran go test ./ui/build
Change-Id: I4d55ac74f02b2a73194d31506a9010162620b25a
I6dca478f356f56a8aee1e457d71439272351390b replaced calling
build/blueprint/bootstrap.bash with running minibp directly,
but didn't propagate the EMPTY_NINJA_FILE environment variable
to minibp. Since everything that uses EMPTY_NINJA_FILE is
executed directly from soong_ui now, replace the EMPTY_NINJA_FILE
environment variable with a config bool and pass it to minibp.
Bug: 189148777
Test: build/soong/build_test.bash --dist --incremental --shard-count=300 --shard=1 && du out/multiproduct
Change-Id: I4d64275ce02c5d68948012f71ac4dc3795af9e85
This CL fixes a typo in writeFakeNinjaFile to correctly write
a fake out/soong/build.ninja and its depfile. It also modifies bootstrap
phase to *not* run the main soong build phase (which takes more than a
minute) if GENERATE_BAZEL_FILES=1.
This change has the side effect that `GENERATE_BAZEL_FILES=1 m nothing`
no longer generates the real build.ninja, which is fine because one
shouldn't be using GENERATE_BAZEL_FILES=1 for that anyway (or, use
USE_BAZEL_ANALYSIS=1). This change has no effect on mixed builds.
Time on a change to Soong or any Android.bp files:
Before: bp2build_workspace_marker (~20 seconds) + build.ninja (1 min)
After: bp2build_workspace_marker (~20 seconds)
Time on the second of two consecutive `b build //bionic/...`: 2.070s
Test: TH
Test: Soong integration tests
Change-Id: I43720641815994caba97b8d165d7c3fc254cbd06
SOONG_SDK_SNAPSHOT_VERSION=current will generate unversioned and
versioned prebuilts and a versioned snapshot module. This is the
default behavior. The zip file containing the generated snapshot will
be <sdk name>-current.zip.
SOONG_SDK_SNAPSHOT_VERSION=unversioned will generate unversioned
prebuilts only and the zip file containing the generated snapshot will
be <sdk name>.zip.
SOONG_SDK_SNAPSHOT_VERSION=<number> will generate versioned prebuilts
and a versioned snapshot module only. The zip file containing the
generated snapshot will be <sdk name>-<number>.zip.
Bug: 157884619
Test: m nothing
m SOONG_SDK_SNAPSHOT_VERSION=current art-module-sdk
- check that the generated Android.bp file has not changed
from the default.
m SOONG_SDK_SNAPSHOT_VERSION=none art-module-sdk
- check that the generated Android.bp file does not contain
versioned modules.
m SOONG_SDK_SNAPSHOT_VERSION=2 art-module-sdk
- check that the generated Android.bp file only contains
version 2 of each module.
Change-Id: I087e9d7d3ad110508a3d6a39bca50cbb46b3ce82
Installclean is used in some build server builds when switching between
different TARGET_BUILD_APPS, remove the symbols directory so it doesn't
accumulate old symbols files.
Bug: 186507256
Test: m && m installclean && m
Change-Id: I316703cd0985ee555959a6dcdb39d5a702ab0332
This refactors bazel-build mode determination logic in soong_ui so it's
clearer which of three possible modes are being used in a given
invocation (NO_BAZEL, GENERATE_BUILD_FILES, or MIXED_BUILDS).
Test: bootstrap tests
Change-Id: I41d2baebf8d560c2cc42db8daa8b936101d453e3
It now handles adding .bp files and changing globs.
In order to do this, depfiles are now written separately from RunBlueprint.
This is necessary due to the confluence of a number of seemingly
unrelated factors:
1. The glob filelist dependencies are discovered in globSingleton
2. Singletons need to be registered because otherwise singleton module
types panic
3. Singletons don't work because they require mutators bp2build does not
run
Due to (1), we would need to run the glob singleton. However, due to (2)
and (3), we can't run singletons and have to run Blueprint with
StopBeforeGeneratingBuildActions, which is when the build actions
writing glob files would be generated. So what happens is:
1. When bp2build is run, the glob singleton is disabled
2. At the end of bp2build, the list of glob files is artifically added
to the depfile of the workspace marker file
3. When build.ninja is generated, the Ninja file containing the glob
list file is written by the now-active glob singleton
Test: Presubmits.
Change-Id: I3c5898d8c57c554a93520276c64a952afc912dbe
This is done by setting the INTEGRATED_BP2BUILD environment variable
when invoking the build.
Even though the name of the marker file insinuates that a Bazel
workspace is already created, this is not the case yet.
An issue that remains is that a .d file is not written for the marker
file so it won't be rebuilt if a .bp file changes. Fixing this requires
delicate surgery because writing the .d file is the result of delicate
interplay between Soong and Blueprint.
There are also a number of semi-related fixes:
- The name of soong.environment.{used,available} is now on the command
line of soong_build (soong_docs is still special cased because its
command line in the Ninja file is taken from the os.Args of
soong_build so it's not trivial to remove the --{available,used}_env
from it
- bp2build writes a separate soong.environment.used file
- I had to call SetAllowMissingDependencies() separately when creating
the android.Context for bp2build so that bp2build runs in the
integration tests (it was not obvious how not to do this)
- Fixed a number of integration tests where a command with an expected
exit code of 1 was used as the last one in a test case, thereby
breaking the test suite
Test: Presubmits.
Change-Id: Ibeb61c26022cf801dcb98505b4039151b3409873
This is so that the way soong_build is invoked is simpler, which is in
turn useful so that it can be invoked multiple times within a single
build, which in turn will be used to invoke bp2build routinely (as
opposed to manually when needed)
Test: Presubmits.
Change-Id: Iddaebb05ff7bcedc0db8273192bb31284b739920
The output from bootstrap would contain the a one-line summary of RBE
download / upload stats.
Bug: b/183007125
Change-Id: I90082a8433504e40bbf92992c3c8d1d656c49429
This variable tells reproxy the number of times an action that
mismatches between local and remote-executions should be run remotely to
determine if the action is inherently non-deterministic.
An action is considered non-deterministic if the number of unique output
digests we get from the remote-run is > 1.
Bug: b/178842439
Change-Id: I0d50eed29b801be67a7ede9b25ce7f4535980f05
It's replaced with a flag in bootstrap.Args .
Test: "m nothing" (presubmits take a full workday these days)
Change-Id: Ia0bfa2091013e389890e583c559006077806af99
This requires linking Blueprint into soong_ui. It lets us avoid the
complicated dance of Ninja files and shell scripts: now the information
as to how soong_build is built is passed directly to Blueprint using a
struct that contains all the information the command line arguments used
to contain.
The ability to run Blueprint from the command line is kept (for now).
Some variables in bootstrap/command.go needed public accessor functions
because soong_build reads them. This will be disentangled by moving the
flag parsing to soong_build.
The presence of the flag definitions in Blueprint means that soong_ui
now also accepts them. This is not a problem in practice because they
are ignored and because soong_ui itself is hidden behind a few layers of
shell scripts.
Test: Presubmits + the new bootstrap_test.sh .
Change-Id: I6dca478f356f56a8aee1e457d71439272351390b
This lets us not leak $PATH to soong_build. It was only needed for
Darwin tests anyway.
Test: Will ask jingwen@ who has a Mac.
Change-Id: I4647e41275b323fe6283580f8f92718c6229f23e
- Incorporate BAZEL_* variables in soong.environment.available
- Set SOONG_DELVE when requested
Test: Manual.
Change-Id: I3ac675f4876220b788cc48d288b402ac6016c37b
This is exclusively used by tests so it's awful, but Darwin builds are
currently broken so let's unbreak them first then start thinking.
Test: "m nothing" on Linux (I don't have access to a mac OS machine)
Change-Id: I1ea9ca71a5aff36a3454282a4c87ad368138baaa
This reverts commit 6e49493dac.
FULL_BUILD is not set during the config stage, so this never calls
cleanOldInstalledFiles.
Bug: 168105598
Bug: 182008128
Test: manual
Change-Id: I58a54f167763cdb77b3c1bd4f18f519fcaf8751a
This sandboxes Soong's tests better.
Setting "TOP" is slightly ugly, but doing it properly would conflict
with aosp/1609218.
Test: Manual ("m nothing")
Change-Id: Ie3d3cd17f116a5fe0ad25c670fc5ed9c33f8123b
This works by setting the SOONG_UI_DELVE= environment variable to the
port on which soong_ui should accept a Delve connection on.
This is achieved by reusing the Delve execution logic between soong_ui
and soong_build.
Test: Manual.
Change-Id: Id2c1d4b6faac1a4a3918c91030ce2239f7daf54f
Rename to get consistent ramdisk directory naming in
out/target/product/<name>:
debug_ramdisk
ramdisk
vendor_debug_ramdisk
vendor_ramdisk
Test: build and inspect out/target/product/<name>
Change-Id: Ib86e6509bd910e8c95bcb6e3f7bb639ebbfe3758
This lets one avoid any decisions as to when to chdir there during its
execution and leads to better sandboxing because the pwd doesn't leak to
init() functions anymore.
Test: Manual.
Change-Id: I1560da8ed3a621249426f9e8908aa890c21e13ba
TestSendLog sometimes hangs on the buildbot, although it has never
been reproduced locally. The cause appears to be a unix domain
packet that was sent but never received. Skip the test in short
mode for now, which will leave it manually runnable but not run
it as part of the build.
Fixes: 120596545
Test: m nothing
Change-Id: I3e6bc9b5f1d9c15dc9a7294d98cbd917d0637c44
Also delete the now-unnecessary soong_env binary.
Test: Manually. Also checked what happens when a used environment
variable changes.
Change-Id: Ib393e7f444e94819198c6cce4bcd8719d9fd9441
bp2build is a Soong mode that returns as soon as the BUILD files are generated. This causes it not generate the build.ninja file, which the Ninja process executing soong_build itself expects to produce as an output. If there isn't an existing build.ninja file generated from a previous build, GENERATE_BAZEL_FILES=true m nothing will fail.
This CL generates the expected files as a workaround, and also makes GENERATE_BAZEL_FILES=true skip the Kati/Ninja steps in soong_ui since they aren't needed.
Test: rm -rf out/ && GENERATE_BAZEL_FILES=true m nothing && m libc
Fixes: 178683777
Change-Id: I2515ef7961682d2be5f096ed24831cc185165a67
This error message it currently being
logged to stdout which goes to build.log file . We want it to be logged to
build_error.log file instead so that downstream systems that need to run
based on the error message will continue to work.
Test: Tested by running `m toybox` and ensuring that FAILED: prefix is
at the beginning of the line.
Bug: b/177221705
Change-Id: Ie0c9acee2739acf149384e33804b5fca3273c24f
Therefore, no need to remove vendor-ramdisk-debug.cpio.lz4
for installclean as it will not be installed to $OUT dir.
Bug: None
Test: make vendorbootimage_debug
Change-Id: Ib7fb9931c55a9bb24e5ca8c4a80bd72f7233c57f
There are few build targets that don't generate an installed file list,
e.g. product-graph. Skip the old installed file cleanup step so that the
func doesn't complain about a lack of the list file.
Fixes: 168105598
Test: m product-graph
Change-Id: Ib7dce6b801979bb565b74d6355143bac23b84fe6
To measure the MaxRSS memory correctly, read the proc status file
under /proc/<pid>/status and extract the MaxRSS value from it. The
implementation is only available for Linux based distributions.
Bug: b/169453825
Test: go test
Change-Id: I32e3068fee7447f9ef5dfb5a8d8dcb6934e0af23
The metrics.go had missing comments and also did minor code refactoring
to be more in golang best practices.
Bug: b/172917718
Test: m nothing.
Change-Id: Id63e4af3f27b582fc13ef3f988ac5c8707fb9adf
No needed to define soong_metrics_proto since it is defined in
the Android.bp file.
Bug: b/172917718
Test: m nothing
Change-Id: Ib5f4cd6d06c08660803cf070da544e9c2d1dce1f
The setup_tools metrics has been missing since day one and is
needed to track down slowness of setting up the build system.
Bug: b/175706305
Test: m nothing and ran the printproto command to check that the
setup_tools field has been set.
Change-Id: Idc8a38fa830e0a906170feeede63435715eaa05b
The event.go code was refactored to follow the best practices
of golang code style.
Bug: b/172918681
Test: go test ./ui/metrics
Change-Id: Ic57d7cd576a9b146530ae37f03f4f5679d6cffef
Bazel is executed several times during the execution of soong_build.
For each bazel execution, generate a profile and save under the
BAZEL_METRICS_DIR which is defined in soong_ui.
Bug: b/174479924
Test: * USE_BAZEL_ANALYSIS=1 USE_BAZEL=1 m nothing and checked
if the cquery and graph build bazel profiles were generated.
* Verified that the generated bazel profiles were uploaded
to the local dev metrics pipeline.
Change-Id: I3d20204484dc6c5a1525a5d3eec1d62cfb33535b
Also get Bazel to write real files there (not symlinks) so that the DIST_DIR can be independent.
Test: Manually using e.g. DIST_DIR=/tmp/foo USE_BAZEL=1 m dist
Change-Id: I39d5219500864c9ecc85f356a028e9b5bf2607f4
The upload functionality now supports directories to be uploaded
for metrics purpose.
Bug: b/174479728
Test: Wrote unit test case.
Change-Id: I6906be4c1b7fd76ddf6ff7b94e48fe1c4209c59d
Code refactoring has been done for logs directory logic code since
the bazel metrics directory depends on the log directory. For builds
that did not specify a dist directory, the log directory is under
out directory. With dist, the logs directory is under <dist dir>/logs.
This matters for Android CI builds where the metrics files are
placed under logs directory. With this change, the bazel metrics
directory and corresponding files will be under <dist dir>/logs
directory for Android CI builds.
Bug: b/174479728
Test: * USE_BAZEL=1 m nothing (bazel_metrics dir in out dir)
* m nothing (bazel_metrics dir deleted)
* DIST_DIR=/tmp/build USE_BAZEL=1 m nothing dist (bazel_metrics
is in /tmp/build/logs directory)
Change-Id: Ic9e1ff49a1964fcaaf801bde2c19f33597ca1db4
The existing --skip-make flag disables both the config step and the kati
step in the build. Add support for a --skip-kati flag that skips just
the kati step, and refactor things so that the logic is shared between
these two.
Bug: 174315599
Test: TARGET_PRODUCT=aosp_arm64 soong_ui --make-mode --skip-kati;
(verify soong.variables is regenerated)
Change-Id: I75b1910fc1c12fcda130e37b7bc4c050131c7b33
On top of improving the comments, I also refactored the status checker
to simplify and clarify its use case to be a ninja stuckness checker,
since it doesn't appear to have other purposes.
Test: TH presubmit
Bug: b/173474588
Change-Id: I2cf51a1ebf16071a24a1c13c06c7b1adf60256de
This CL also removes --warn_real_to_phony and --warn_phony_looks_real,
since --werror_real_to_phony and --wrror_phony_looks_real are already
specified.
Test: treehugger
Bug: 173474588
Change-Id: Icbb0f53182d1b912c62f8044f52bb79d985bf868
This removes the need to source bazelenv.sh for USE_BAZEL_ANALYSIS, and
unifies mixed builds to use the checked in tools/bazel and bazelrc.
It also unifies all bazel-related output to be in out/bazel.
Without aosp/1502095, this change still requires toplevel_output_directories to be an empty
list, otherwise there'll be this error:
ERROR: Directories specified with toplevel_output_directories should be
ignored and can not be used as sources.
Test: With aosp/1441774: rm -rf out/ && lunch aosp_cf_x86_auto && USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 results
Change-Id: I69b217ec88da531415792bb6e04b6a194ca4718d
This patch adds information behind the Bazel USE_BAZEL=1 execution
pipeline.
Test: m
Bug: 173474588
Change-Id: I54bdd26fb14c8a371de0922d193419112e8c3ece
Ensures that Bazel and Ninja are using the same binaries from PATH, such
as `sort` from the linked bug.
Test: USE_BAZEL=1 m
Test: rm out/soong/apex/depsinfo/new-allowed-deps.txt && USE_BAZEL=1 m
Bug: 161074854
Fixes: 161074854
Change-Id: Icc3e717e6d3cfcecfe90f0fb3a94a86485a11918
break and default statements are not needed inside of a switch
statement.
Bug: b/173028918
Test: USE_BAZEL m nothing
Change-Id: I14aa8cbae3a9b3c859f147c5134600f1e75c6a89
Added a tracer named "bazel" in runBazel function to capture topline
metrics when running bazel.
Bug: b/173028918
Test: * USE_BAZEL=1 m nothing and checked soong_metrics file using
printproto command to verify that bazel topline metrics
were captured.
Change-Id: I5cb0728a4da48545dc701cc79627c318d5d9a29d
Generate the bazel profile file that contains the metrics of
executing the ninja build phase of Soong.
Bug: b/173028918
Test: * m nothing and check out dir that there is no bazel_metrics dir.
* USE_BAZEL=1 m nothing and checked
out/bazel_metrics/build_bazel_profile.gz existed.
* m nothing and checked out/bazel_metrics does not exist.
Change-Id: I797d1eecf5c2c260ade8b55dc69c237bfd0fcbd4
$OUT/bazel_metrics directory is created in order to dump the
set of bazel profile files being created from a single build
execution. The directory is created from soong_ui.
Bug: b/173028918
Test: * m nothing and check out dir that there is no bazel_metrics dir.
* USE_BAZEL=1 and checked out/bazel_metrics was created.
* Ran m nothing and verified that out/bazel_metrics directory is
deleted to ensure no metrics is uploaded or collected from the
pipeline.
Change-Id: I1d3c2471065fae6931f35cef7f0f35fd84f76bef
The UseBazel() function will be used in several places to perform
different functionality such as setting up the metrics path.
Bug: b/173028918
Test: m nothing
Change-Id: Ie94073b8f0552ec7528e1cc2f3be74b693dc3135
This ensures that the lunch repo rule always reads the right Ninja file.
Test: USE_BAZEL=1 m
Test: m
Change-Id: I7f670dd538f46c4dae5ea537b857ff04520b60e7
This ensures that USE_BAZEL=1 mode has the same directory structure
under out/ as it does without USE_BAZEL=1.
Test: Manually verified building USE_BAZEL=1 aosp_flame with droid and
libc
Change-Id: I27e810772d1bf04bd7930ea122217b93d2fd48b4
Collect the resource information of each executed process in soong_ui
for metrics analysis.
Bug: b/169453825
Test: m nothing; "lunch 1" && m
Change-Id: I8c7fe019111921c4c59174d133b85439ec825885
This was caught while running the soong_build docs action under Bazel as
ninja executor. Since the config file is not an explicit input to the
action, it's not present in Bazel's execution root during action
execution time, leading to a missing file error. To fix it, add the file
as an input to the action, if it's configured in product variables.
New soong_build statement:
build out/soong/docs/soong_build.html: s.writedocs.soongDocs $
out/.module_paths/Android.bp.list out/soong/soong.variables $
out/soong/dexpreopt.config | out/soong/.bootstrap/bin/soong_build
outDir = out/soong/docs
Test: m
Test: USE_BAZEL=1 m
Bug: 161958189
Fixes: 161958189
Change-Id: I2bf264e848bb29ac91f4196fbe9c4ecf3cea8a2c
There is no need for multiproduct_kati to run the tests for
every product, they don't vary by product config. --skip-soong-tests
can also be used for local development to run soong_build even if
the tests don't pass.
Bug: 156428456
Test: m --skip-soong-tests nothing
Change-Id: I9c00e3d1b6e51d17bb290339c3f124d4d1c9e69f
We need to capture the user build command to understand what additional
arguments is passed in to Soong. This data is useful to know which
targets are being executed and have statistics on top build targets
are specified.
Bug: b/169453712
Test: ran "m blueprint_tools", "m nothing", "lunch 1" and checked
in soong_metrics proto.
Change-Id: I7ab94dcf037211f1abffe6afd012b03ffbf99756
Several structs and files are named time which records the total
execution of a traceable event. Soon, resource information of an
executed process is collected during a build event so a more generic
name is applied.
Bug: b/169453825
Test: go test and m nothing. Checked the soong_metrics protobuf.
Change-Id: I65a782603530d51018d6cc1192c18cfeef7566a0
We would like to collect the resource usage of processes such as
kati, soong_build and ninja on the resource usage such as memory
and context switching. This changelist contains the defined fields
in the protobuf file of the resource information to be collected
from interested processes.
Bug: b/169453825
Test: N/A
Change-Id: I4b3cbbc3e7b78900875c10d34d574425f8b11730
Test:
m USE_GOMA=true GOMA_DIR=$(goma_ctl goma_dir) toybox
m FORCE_USE_GOMA=true GOMA_DIR=$(goma_ctl goma_dir) toybox
both result in error message being printed.
Change-Id: I94ccaf217dba71aca990d88d205bad669a49a2f5
This CL adds symlink_outputs to various locations in Soong that creates
actions that creates symlink outputs, and explicitly mark them as such.
Test: m
Bug: 160568334
Change-Id: I322751bada52a9f49011c74731d84761586e03e7
Propagate the subprocess stats from ninja to build.trace.gz.
There is too much data here to put into separate counters for each
statistic, so put them into args on each duration event for now.
We would like to track max RSS for each subprocess, but the Linux
kernel inherits the max RSS of the ninja process in each subprocess,
which sets a lower bound on the measurable max RSS to the size of
the ninja process. The ninja process is large due to the multi-GB
build.ninja files.
Bug: 170701554
Test: examine build.trace.gz
Change-Id: I8aaaafe627a57f1a500af098c097c6381c583ba5
Clean the odm_dlkm and vendor_dlkm staging directories in installclean
to get consistent results in incremental builds on the build servers.
Test: treehugger
Change-Id: Id431d7457d85e0be68437a2034fd544d74cc3709
Collect the total number of CPU cores and available RAM for each build
as part of the system resource info. This is to know what kind of
build machines that the developers are using.
Bug: b/169453825
Test: m nothing and ran printproto on soong_metrics to validate the
data.
Change-Id: I8617bdb9490b37d7689abd27d349af1a651533db
This retriggers soong_build whenever a new bzl, WORKSPACE, or
BUILD.bazel file is changed or added.
Test: Manually verified on bionic/libc genrules with manual changes to
related BUILD/bzl/WORKSPACE files -- these all retrigger builds.
Test: Updated finder_test.go
Change-Id: I634384f88781a6b6db32f5d6bf9c07e179e14c39
Added additional instruction in the regen.sh script to build the
blueprint_tools and aprotoc in case the regen.sh script fails.
Bug: b/169395325
Test: ./regen.sh
Change-Id: Icb38c907fb47febc17eef1067d06d0588dd0b7ba
The hostname is collected during a build to determine where the user
is build from (from office workstation, cloudtop, etc...). This is a
useful metrics for RBE related builds.
Bug: b/169395325
Test: m nothing, ran printproto to verify that soong_metrics has the
hostname.
Change-Id: I3e5d64f948ae43c5dda60de1d4a52b187e36dff8
Tradefed packages are created in out/host/common/obj/PACKAGING
and should be removed during installclean so that they get an
accurate embedded build number.
Fixes: 170308394
Test: treehugger
Change-Id: Ib7a57377b5c9512584bc93f10da316eadf786979
This reverts commit 323dc60712.
Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests
Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
Prints the following warning messsage:
WARNING:
Setting OUT_DIR to a path other than out may result in slow RBE builds.
See http://go/android_rbe_out_dir for a workaround.
Bug: b/169676232
Change-Id: Id1ad06c4dc672c24373642e0b624833eb6a0dbcf
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.
Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
FORCE_USE_GOMA is needed to be collected as a metric for the
deprecation of GOMA on Android builds.
Bug: b/169423400
Test: m nothing, ran printproto command
Change-Id: I12300389f1f27239d8f21875d1b6e4ad069d95d7
We should override log-dir with the environment variables if its set.
This is needed in our integration tests and generally to allow for
user-overrides.
Test: RBE_log_dir=`pwd`/rbelogs mmma external/boringssl
Change-Id: I3d316a2d60122c42f5c3b5dfede4037362a7b5f1
This is an initial CL that collects the environment variables
USE_GOMA and USE_RBE. This is a needed metric to determine how
many users are currently using GOMA so we can help them migrate
to RBE.
Bug: b/163598127
Test: m nothing and checked soong_metrics
Change-Id: I39536eabad01aafbb3bd7182515d72230b36f658
This hasn't worked for a couple years, and continues to bitrot. Just
remove it.
Adds a bpfix rule so that we can eventually remove the
product_variables.pdk definition, which is now always a no-op.
Test: treehugger
Change-Id: I830b54d419b59f6db1d4617b45e61a78234f57a7
Merged-In: I830b54d419b59f6db1d4617b45e61a78234f57a7
There are cases where the developer is not building a product so
the metadata information is not being set. For now, set the OS
name so we can track which OS the user is using it.
Fixes: b/163118212
Test: m nothing and checked the blueprint
Change-Id: I9b521eee2d266b50625aa8e916ddaacb4e2feaea
