Commit graph

592 commits

Author SHA1 Message Date
Patrice Arruda
c97d6dcbb3 Collect FORCE_USE_GOMA environment variable as a metric.
FORCE_USE_GOMA is needed to be collected as a metric for the
deprecation of GOMA on Android builds.

Bug: b/169423400
Test: m nothing, ran printproto command
Change-Id: I12300389f1f27239d8f21875d1b6e4ad069d95d7
2020-09-28 18:25:42 +00:00
Kousik Kumar
0d15a72915 Allow overrides for RBE_log_dir
We should override log-dir with the environment variables if its set.
This is needed in our integration tests and generally to allow for
user-overrides.

Test: RBE_log_dir=`pwd`/rbelogs mmma external/boringssl
Change-Id: I3d316a2d60122c42f5c3b5dfede4037362a7b5f1
2020-09-23 02:59:19 -04:00
Kousik Kumar
ec47864ca0 Deprecate USE_GOMA and replace with FORCE_USE_GOMA
Bug: b/163361527
Test: Ran builds with USE_GOMA / FORCE_USE_GOMA flags
Change-Id: I67b229fbf52e913b0c4f5e7932021132ddc0598f
2020-09-21 15:35:57 -04:00
Ramy Medhat
81b3a83401 Fallback to /tmp when socket address is longer than max length.
Test: Temporary change of base name to cover handled cases.
Change-Id: I1f6953752e7861ccbe7180351386ed65ab5fc259
2020-08-28 23:53:02 -04:00
Ramy Medhat
f668917c25 Add Goma deprecation PSA to soong.
Test: build with Goma
Change-Id: Ie7e146285afd40bc0a1dc17a0b898960f57a118b
2020-08-18 17:29:48 -04:00
Ramy Medhat
8c56bc2726 Add a default value for the RBE_log_dir variable.
Test: built simple target.
Change-Id: Id93dd80119037b594e5f8c370f4ac2fbc12a5144
2020-08-14 14:17:33 -04:00
Ramy Medhat
a958d35247 Do not generate a random socket address for RBE if NOSTART_RBE is set.
Test: A build with NOSTART_RBE=1
Change-Id: I5872632442c754b597901b7729eed597ee45859f
2020-08-13 22:53:42 -04:00
Ramy Medhat
5d942d05b1 Merge "Add reasonable defaults to RBE configuration parameters." 2020-08-12 22:41:43 +00:00
Ramy Medhat
0fc67eb5e9 Add reasonable defaults to RBE configuration parameters.
Test: simple one action build
Change-Id: Ic66ad2b89866a67008950035bc3b559dae4e3a3e
2020-08-12 12:30:05 -04:00
Patrice Arruda
9685036e6e Start collecting build configuration metrics.
This is an initial CL that collects the environment variables
USE_GOMA and USE_RBE. This is a needed metric to determine how
many users are currently using GOMA so we can help them migrate
to RBE.

Bug: b/163598127
Test: m nothing and checked soong_metrics
Change-Id: I39536eabad01aafbb3bd7182515d72230b36f658
2020-08-12 14:25:37 +00:00
Dan Willemsen
9f43597ff7 Remove obsolete PDK build functionality
This hasn't worked for a couple years, and continues to bitrot. Just
remove it.

Adds a bpfix rule so that we can eventually remove the
product_variables.pdk definition, which is now always a no-op.

Test: treehugger
Change-Id: I830b54d419b59f6db1d4617b45e61a78234f57a7
Merged-In: I830b54d419b59f6db1d4617b45e61a78234f57a7
2020-08-11 01:11:44 +00:00
Patrice Arruda
add7ea9cc8 Set the OS name in the metrics protobuf file
There are cases where the developer is not building a product so
the metadata information is not being set. For now, set the OS
name so we can track which OS the user is using it.

Fixes: b/163118212
Test: m nothing and checked the blueprint
Change-Id: I9b521eee2d266b50625aa8e916ddaacb4e2feaea
2020-08-07 17:55:23 +00:00
Elliott Hughes
325d73f260 Merge "Switch to toybox patch(1)." 2020-08-05 23:30:14 +00:00
Patrice Arruda
79dcf732cc Add newline to each generated bash script in rbe_test.go.
"exec format error" showed up on Mac when running the rbe_test.go.
The generated bash scripts did not have a newline at the end of
the file.

Bug: b/162600461
Fixes: b/162600461

Test: "m blueprint_tools" on Android Mac builder.
Change-Id: I425fa5f428733b6569c3fcdaa312023f9761d020
2020-08-01 17:51:52 +00:00
Patrice Arruda
fcb53d9d2a Disable additional RBE metrics upload test errors for now.
Disable the error test cases to allow developers to build
AOSP on Mac.

Bug: b/162600461
Test: m nothing

Change-Id: Ic8c6fe04743a8e4fe197a1bb71259036fb749edc
2020-08-01 01:02:33 +00:00
Patrice Arruda
a8d04fd8ac Disable RBE metrics upload test errors for now.
Disable the error test cases to allow developers to build
AOSP on Mac.

Bug: b/162600461
Test: m nothing
Change-Id: Iac4373dcb10710d8a2325eeb3ee20a6eeca6143f
2020-07-31 22:11:37 +00:00
Treehugger Robot
5fd4f5d470 Merge "Inclusive fix: renaming dumb terminal to simple terminal." 2020-07-29 05:08:40 +00:00
Patrice Arruda
f445ba1788 Inclusive fix: renaming dumb terminal to simple terminal.
Bug: b/161896447
Test: "lunch 1" and "m nothing"
Change-Id: Ifb228c0f0f634477932c9240e57170f7390201f7
2020-07-28 20:25:03 +00:00
Patrice Arruda
90109177f8 Inclusive fix: removed the word Sane in Soong build UI config
Bug: b/161896447
Test: "lunch 1" and "m nothing"
Change-Id: Icd57d339bcc9978c505428730fba33fe291c32d4
2020-07-28 18:10:26 +00:00
Treehugger Robot
f8b85dedde Merge changes Ide9c7a6a,I706c1433
* changes:
  Add Validations to android.BuildParams
  Use validations for blueprint tests
2020-07-17 19:54:49 +00:00
Colin Cross
6e17ad5937 Use validations for blueprint tests
Pass -n to blueprint to make it use validation dependencies for tests,
which will let them run in parallel with soong_build.

Bug: 156428456
Test: m nothing
Change-Id: I706c14330543e2a7728e5dc385976900c67cb2f5
2020-07-16 13:07:06 -07:00
Ramy Medhat
ca1e44cb4b Generate RBE socket address randomly for each invocation of soong.
Test: started soong multiple times and saw different socket files in
out/soong/.temp

Change-Id: Ib0087dd3867bdbf584c6868196070840cff291aa
2020-07-16 12:18:37 -04:00
Patrice Arruda
73c790f59e Use the build start time from Soong main UI for metrics build timestamp.
Currently, the build timestamp saved to the build_date_timestamp
field in the MetricsBase metrics protobuf usually comes from a text
file named build_date.txt. It may not be accurate for metrics purposes.
Instead, use the build start time from Soong main UI for a better
timestamp.

Bug: b/140638454
Test: Ran and checked the build_date_timestamp by running the printproto
      command.
Change-Id: I4d893fbbf9830ba21911e56ae13bc5272be47ae6
2020-07-14 17:23:23 +00:00
Patrice Arruda
958b89c0b5 Use monotonic clock instead of wall time clock for metrics.
From several dashboards, there are high spikes from several
builds where the real time of executing a part of the build
system took many many hours. This is due to using the wall
time clock when calculating the duration. For metrics, the
monotonic clock should be used for more accurate measurement.
Also, cleaned up the time.go file to use the proper proto
functions when marshalling perf protobuf and deleted
unnecessary functions.

Bug: b/143492984
Test: Wrote and ran unit test case.
Change-Id: I1540cfbf37915e3045ecedaa2595c1f9429bcd62
2020-07-13 21:14:34 +00:00
Patrice Arruda
62f1bf2378 Add RBE metrics dump in Soong UI.
From aosp/1329396, the RBE metrics protobuf file is part of the
metrics uploading process. The RBE metrics protobuf file is
generated by running the bootstrap shutdown command. A new function
named DumpRBEMetrics was written in order to generate the RBE metrics
protobuf file before sending to the uploading process.

Bug: b/140638454
Test: * Unit test cases
      * Ran RBE build on my local host and verified the
        metrics protobuf file is created.
      * Ran non-RBE build after RBE build and verified that
        the previous metrics protobuf file was deleted.

Change-Id: I4b8068905cb67c4b8c2d94793917b98974fed707
2020-07-09 21:06:22 +00:00
Colin Cross
96e5e41b88 Dist module_paths/files.db
Help debugging finder issues by disting module_paths/files.db for
later analysis.

Bug: 157656545
Test: treehugger
Change-Id: Idfa4cffe7ef724b399a2667efd487f9c3c76bb6b
2020-07-06 17:15:05 -07:00
Christopher Parsons
0eeff82c98 Merge "Add missing soong_docs inputs" 2020-07-06 22:34:51 +00:00
Chris Parsons
8f232a2702 Add missing soong_docs inputs
Specifically, make Android.bp.list and soong.variables explicit inputs

While this is not a comprehensive list of all inputs
of this action (as the action depends on all blueprint
files in the source tree), this is closer to the truth.

This is a rollforward CL, which was originally rolled
back, as path validation failed for when OUT_DIR was
an absolute path. Validation has now been relaxed.

Test: Manually verified ninja output, checkbuild approved validation
  for aosp-crosshatch, and manually ran frameworks/rs/build_rs.py with
  patch fix
Change-Id: I4eb0d517f57336dd54eaa4bd31f46df9e93e6da2
2020-06-26 10:36:18 -04:00
Colin Cross
8ba7d47bba Dist build.ninja and Android.bp.list
Debugging issues on the build servers can be difficult because the
intermediate files are not visible.  Gzip ninja file and Makefiles
generated by Soong and the ninja files generated by Kati to the dist
directory, and also copy all of the finder output files.

Bug: 157656545
Test: m dist nothing
Change-Id: I48d75305e551ccae81c7a55721981cf58acd838b
2020-06-25 12:50:00 -07:00
Christopher Parsons
cd767eefe5 Revert "Add Android.bp.list and soong.variables to inputs for soong_docs action"
This reverts commit 67eee7cae4.

Reason for revert: Broke renderscript build

Change-Id: Ie26b3c7a78afe44d719f399520e643174109593f
2020-06-18 21:49:48 +00:00
Chris Parsons
67eee7cae4 Add Android.bp.list and soong.variables to inputs for soong_docs action
While this is not a comprehensive list of all inputs of this action (as the action
depends on all blueprint files in the source tree), this closer to the truth.

Test: Manually verified ninja output, checkbuild approved validation for aosp-crosshatch
Change-Id: I77a0f612afb025232bee7d3ac31257b808991829
2020-06-18 12:06:37 -04:00
Elliott Hughes
bdbbfd53ec Switch to toybox patch(1).
This reverts commit e44c412220.

Reason for revert: http://b/122739027

Change-Id: Id228ef6b819a27b4e7c301c529c9ee80717b141b
2020-06-17 15:12:41 -07:00
Patrice Arruda
7cc2074885 Run the metrics uploader in the background.
The metrics uploader was currently running on foreground where it
would copy the metrics files in a separate directory and then forked
into the background for the upload process. As a result, running the
lunch command would take a second longer to run since each metrics
uploader run had an average of half a second.

Bug: 140638454
Test: * Wrote and updated unit test cases.
      * Set ANDROID_ENABLE_METRICS_UPLOAD to point to the latest
        metrics_uploader bash script. Executed the "lunch 1" command
	and measured the running time. Executed "m nothing" command
	and checked that the metrics were uploaded.
      * Ran "lunch 1" and "m nothing" with
        ANDROID_ENABLE_METRICS_UPLOAD=""
      * Removed oauth from metrics_uploader and ran "m nothing" and
        "lunch 1". The oauth Message appeared only to "m nothing"

Change-Id: I13c61e666c8f44613dee291a704cef6a27335188
2020-06-13 00:34:37 +00:00
Martin Stjernholm
fa096325fe Merge changes from topics "target-build-unbundled", "unbundled-preopt"
* changes:
  Don't disable dexpreopting implicitly in unbundled builds.
  Add TARGET_BUILD_UNBUNDLED as a variable to control builds.
2020-06-11 10:01:59 +00:00
Patrice Arruda
ae2694b30d Pass the TMPDIR directly to the command when dumping makefile variables.
External applications such as metrics uploader depends on the TMPDIR.
DumpMakeVars is changing the TMPDIR environment variable value. Once
the work is done, TMPDIR directory is deleted. Additional flow of
the tool or external applications can panic since TMPDIR directory
does not exist. Since this is an isolated case, pass the TMPDIR
directory to the command of dumping the makefile variables.

Bug: b/140638454
Test: * Ran lunch command with ANDROID_ENABLE_METRICS_UPLOAD set
        and metrics_uploader did not crash.
      * m nothing
Change-Id: Ib78a3f8cfb5da48f65bd9bce1511e9abdf3d9cca
2020-06-05 17:28:41 +00:00
Martin Stjernholm
08802338fb Add TARGET_BUILD_UNBUNDLED as a variable to control builds.
Test: m droid
Test: TH, in particular builds green on ub-launcher3-master
Bug: 157549171
Change-Id: Ifc5b2759c10672e4b253ddd208b02e5af17e8dad
2020-06-04 19:01:55 +01:00
Patrice Arruda
219eef3878 Upload build metrics after a build is completed.
Soong now supports the ability to upload metrics to another location
by setting the ANDROID_ENABLE_METRICS_UPLOAD to an uploader that
accepts the upload.proto proto buffer message. When the environment
variable is set, a set of build metrics files (soong_metrics,
rbe_metrics.pb and build_error) is uploaded.

Bug: 140638454
Test: * Wrote unit test cases
      * Setup the uploader, built a succcessful and failed aosp_arm-eng
        target and monitor the uploading of the metrics.
Change-Id: I76a65739c557dc90345e098ca03119a950ece2d2
2020-06-01 17:29:30 +00:00
Dan Willemsen
570a292bab Better handling of low memory situations
Drop down to a single high-mem task when we've got <=16GB, as the system
probably isn't completely free RAM, ninja will be using a couple gigs,
along with whatever "normal" actions will be running concurrently. So
it's unlikely that we can handle two 6-8GB actions along with everything
else.

Also print warnings when we detect <=16GB total RAM, or when we're
running more parallel jobs than than we have GB RAM. These both notify
the user and suggest lowering the `-j` value if they run into problems.

Test: fake totalRAM to [0.5,8,16]GB, checking warning
Test: fake totalRAM to 17GB, `m -j4 nothing` has no warning
Test: `m -j187 nothing` on a 188GB system
Test: `m -j188 nothing` on a 188GB system
Change-Id: Ieb008e9f462d5f40fb65781d94cf116b1caf8446
2020-05-26 23:29:18 -07:00
Dan Willemsen
082182275f Update ninja proto, always enable ninja -d stats
The stats output will now use the new "DEBUG" message type, which we can
always redirect to verbose.log.gz.

Test: m aprotoc  (look in verbose.log.gz)
Change-Id: Ie1b58f12c008ff7d29f11ff7a9807488dba8a504
2020-05-18 17:52:28 -07:00
Daniel Norman
b8e7f81af6 Removes $(PRODUCT_OUT)/misc_info.txt on installclean.
misc_info.txt's build rule does not contain inputs to tell it to
refresh on an incremental build, so incremental builds (even after
installclean) were reusing this file.

With this change, misc_info.txt is removed by installclean so that an
incremental build generates the file fresh.

Test: # misc_info.txt exists; m installclean; # misc_info.txt gone
Bug: 155930200
Change-Id: Ia319925dad26f98049a988ee1ceb1e20db621353
2020-05-07 23:48:30 +00:00
Treehugger Robot
f39f6a4772 Merge changes I2e370952,I811cc8e1
* changes:
  Remove TEMPORARY_DISABLE_PATH_RESTRICTIONS
  nsjail: Always remount /tmp before src/out/dist
2020-05-04 22:11:18 +00:00
Dan Willemsen
347ba75679 Remove TEMPORARY_DISABLE_PATH_RESTRICTIONS
This was a temporary migration tool, and has been broken ever since the
ninja environment started being restricted. Remove it, since it has
lasted two releases.

Test: TEMPORARY_DISABLE_PATH_RESTRICTIONS=true m
Change-Id: I2e3709525598030f5c24d12fa966e531dfed33ae
2020-05-04 19:54:57 +00:00
Dan Willemsen
1612e2641e nsjail: Always remount /tmp before src/out/dist
A user was trying to use OUT_DIR=/tmp/...-out DIST_DIR=/tmp/...-dist

Test: OUT_DIR=/tmp/test-out DIST_DIR=/tmp/test-dist m nothing
Change-Id: I811cc8e1bda8d766406603df47309a8637cef48c
2020-05-01 16:26:56 -07:00
Dan Willemsen
6dfe30a5e2 Disallow OUT_DIR on the command line
By the time we do argument parsing, we've already written to $OUT_DIR or
the default, so don't allow overriding the output directory on the
command line.

Test: m OUT_DIR=newdir
Change-Id: I6f8c07cbef3a71d393b766dc5a01ae9c3df8631c
2020-05-01 02:23:31 +00:00
Ramy Medhat
1dcc27e47c Refactor RBE support for Javac/R8/D8 to use the remoteexec package.
Test: build with and without RBE_JAVAC/RBE_R8/RBE_D8 set.
Change-Id: I1607f8cfb4d2c3cbb3672152bbeb561d9968bc60
2020-04-28 11:37:20 -04:00
Dan Willemsen
c4c33e0e58 Merge "Stop dumping obsolete variables" 2020-04-21 22:35:36 +00:00
Treehugger Robot
9145d847e4 Merge "Turn on ninja phony outputs" 2020-04-21 22:30:24 +00:00
Treehugger Robot
a268d59e8b Merge "Report reverse dependencies for dangling rules." 2020-04-20 13:41:55 +00:00
Dan Willemsen
acafc67399 Add logging for stats on finished commands
Includes the exit code, real time, user time, system time, and maxrss.

Test: m nothing; check out/soong.log
Change-Id: I93dea9fc1c7fb892150e16e11cae09f2372f4429
2020-04-19 12:06:14 -07:00
Dan Willemsen
c615e29b3e Stop dumping obsolete variables
Test: m nothing
Change-Id: I4d7cce3a1d5f7f5f517d74e440059a69de36249d
2020-04-19 03:59:35 +00:00
Dan Willemsen
6587beddfa Turn on ninja phony outputs
This replaces the _kati_always_build_ hack with a ninja feature so that
ninja can actually understand what's happening. This means that we can
turn on some more options and checks around expected output filenames:

* Remove the output file(s) before the command executes
* Error if the output file(s) don't exist after the execution
* Error if the output is a directory

They're turned on immediately during the soong bootstrap runs, as those
run a limited number of rules. The main ninja execution does not remove
the output files, and prints warnings instead of errors for the others.
I'll turn them on as we understand how often those warnings are seen.

Test: m (check for new warnings)
Test: treehugger (check for new warnings)
Change-Id: I7f5c1a361dd1498eb54a2c07a918f3b0aa086e4c
2020-04-18 20:25:59 -07:00
Martin Stjernholm
946fb67272 Report reverse dependencies for dangling rules.
To help find what pulls them in.

Test: m checkbuild on a tree that exhibited the problem in
  https://android-build.googleplex.com/builds/pending/P13607612/aosp_crosshatch-userdebug/latest/view/logs/build_error.log

Change-Id: I59f3c98f21e5fb45dbd0aa814400f734d769a442
2020-04-17 16:00:43 +01:00
Ramy Medhat
383683559c Merge "Allow remote execution of link actions." 2020-04-17 13:59:26 +00:00
Ramy Medhat
9a90fe5e23 Allow remote execution of link actions.
This CL adds a remoteexec package that allows adding a configurable RBE
prefix to the template.

Test: built aosp crosshatch userdebug with and without RBE_CXX_LINKS.
Change-Id: Ica920c3d7f79f2996210b9cbd448126451c1707c
2020-04-16 21:18:10 -04:00
Treehugger Robot
045d75f11a Merge "Add a new protobuf message named Upload to upload build metrics." 2020-04-16 23:13:00 +00:00
Patrice Arruda
adfc1120b9 Add a new protobuf message named Upload to upload build metrics.
A new protobuf message named Upload is defined to send build metrics
to the Android Build Server API after a build is completed. These
metrics are collected from googlers only, as described in
go/local-developers-metrics-wfh.

Bug: b/140638454
Test: m nothing
Change-Id: I5e49b5ace053913be2b75673cb1375f4cef37825
2020-04-16 14:02:56 -07:00
Diego Wilson
10e564a98e Dereference paths to be mounted into soong sandbox
Bind mounts require source paths without symbolic links.

Fixes: 153378837
Test: lunch aosp_cf_x86_phone-userdebug && make dist
Change-Id: I5c128f94233d81c52ec5692241b711b3e9d15a85
2020-04-15 20:29:06 +00:00
Diego Wilson
a5d9653cc5 Only mount dist dir to soong sandbox if it exists
The dist dir is created late in the build process. If a
soong sandbox attempts to mount it before it's created
the sandbox will create an empty file in its place.

Test: lunch aosp_cf_x86_phone-userdebug && make -j dist
Change-Id: Ie6513bf702de6e6322c78753d406d70ea3ccc04d
2020-04-06 22:07:56 +00:00
Diego Wilson
a22240bf22 Mount soong sandbox chroot as read-only
It's safer to disallow writing to the root filesystem of the sandbox.
Still allow write access to the source, tmp, out and dist directories.

Test: build aosp_cf_x86_64_phone-userdebug
Change-Id: Idf0d3a420669fbf6c3aba24e058f09a314ec6d0e
2020-04-02 18:11:28 +00:00
Dan Shi
984c129c84 Rename VTS to VTS10
Bug: 151896491
Test: m -j vts10
Change-Id: I7b70b13401678d70e4b46f9e7eb9d844bcfa315e
2020-03-28 16:42:17 -07:00
Treehugger Robot
8bf9eeb912 Merge "Cleanup vendor_boot artifacts in PRODUCT_OUT on installclean" 2020-03-20 17:23:17 +00:00
Ola Rozenfeld
27818b9919 Merge "Migrating RBE flags to RBE_ variables." 2020-03-20 13:16:45 +00:00
Patrice Arruda
85f227eed7 Generate the build_error protobuf file on first build action error.
buld_error protobuf file is created the moment a build action failed.
This is for external system to detect immediately what caused the
build failure while monitoring the build_progress.pb file.

Bug: b/150401146
Test: * m nothing
      * Built successfully aosp_arm-eng and checked build_error file
        was not created.
      * Modified a cpp file to cause a build failed. Ran m and build
        failed. Checked immediately during the build that the
	build_error file was created.
      * m clean; m and during build, checked if build_error was
        removed.
Change-Id: I778616ae8cf242b49ad263b79c93321959112caa
2020-03-19 18:23:26 -07:00
Ola Rozenfeld
3992e37599 Migrating RBE flags to RBE_ variables.
Test: N/A, this only adds new variables.

Change-Id: Iad09352090ec255af82e77941e6e050137f1e83b
2020-03-19 20:05:14 -04:00
Treehugger Robot
f597301853 Merge "Add build progress in Soong UI" 2020-03-19 22:09:02 +00:00
Treehugger Robot
0835c1b27a Merge "Do not write build_error proto file if there are no build errors." 2020-03-19 21:57:51 +00:00
Patrice Arruda
36b5e31f62 Do not write build_error proto file if there are no build errors.
For successful builds, a blank build_error proto file is created.
This slows down the data processing part where it reads the build_error
file and there are no errors to be processed.

Bug: 142277430
Test: Did a successful build and checked if the build_error file
      was not available. Changed a source file to make the build
      failed and checked if the build_error file was available.
Change-Id: I300dffe32fb7f8bf984fa20ae368bdd02bf12992
2020-03-19 13:20:55 -07:00
Patrice Arruda
74b43998e7 Add build progress in Soong UI
Soong UI generates a proto file named build_progress.pb in $(OUT_DIR)
output directory that contains build action numbers (how many are executing,
finished and total) during the course of a build. This is for external
systems that invokes the Platform Build Systems and would like to know
the completion status.

Bug: b/150401146
Test: Wrote a bash script that continuously read the
build_progress.pb file and computed the build completed percentage
while building the aosp_arm-eng target. Compared the percentage between
the Soong output console and the one reported by the bash script.

Change-Id: I7c7347bc8e41958093892d8e2731c4f4169937dd
2020-03-19 13:15:05 -07:00
Will McVicker
4cee625279 Cleanup vendor_boot artifacts in PRODUCT_OUT on installclean
This makes sure that we don't keep old vendor_boot artifacts around
after a clean.

Signed-off-by: Will McVicker <willmcvicker@google.com>
Fixes: 151957122
Bug: 151949371
Test: touch $OUT/vendor-ramdisk/test-file && make installclean
Change-Id: I357929583f234212a998dceaa31506fce42e0255
2020-03-19 19:58:03 +00:00
Elliott Hughes
cc868de8a7 Merge "Remove unused mips workarounds." 2020-03-16 21:06:13 +00:00
Dan Willemsen
6d3cad93e8 Always keep rsp files
It's painful to debug without these.

Test: do a build; find out -name '*.rsp'
Change-Id: I6fc25095442178c0bcbc1dea444ba51e9259c600
2020-03-12 10:30:47 -07:00
Treehugger Robot
2d80d382fa Merge "Define a proto file that generates the current build completion status." 2020-03-11 16:28:08 +00:00
Patrice Arruda
7d401482f3 Define a proto file that generates the current build completion status.
Soong UI will generate a new proto file in the out directory for
other systems to read the build completion status during a course
of a build. The proto file contains build action counters where

total_actions: represents the total build actions of a build (will
increase and may decrease during a course of the build)

finished_actions: represents the number of executed build actions.
finished_actions never decreases and finished_actions <= total_actions.

current_actions: represents the number of build actions being executed
and current_actions + finished_actions <= total_actions.

Bug: 150401146
Test: executed regen.sh script.
Change-Id: I04f4c07855723d0684b141d88cb3529ab5d9fccd
2020-03-10 16:19:22 -07:00
Dan Willemsen
a17ac9632d Improve error handling around username/hostname
If this couldn't read the current user, it would print a generic
message, then proceed to dereference the invalid user struct :(

Provide somewhat reasonable defaults, and better error messages instead.

Test: m nothing
Test: run docker with misconfigured user
Change-Id: I21af77c7d8d1d2d27cb04546667eb1094c62a7a1
2020-03-10 15:45:47 -07:00
Elliott Hughes
da3a071eaa Remove unused mips workarounds.
This was never really finished, and hasn't been supported for years.

Test: treehugger
Change-Id: I21d4c3112aa8cf0c56e59f0cc19ff8725ef714b9
2020-03-06 18:15:44 -08:00
Dan Willemsen
669f769470 Stop using the host python3
We've now got a prebuilt in prebuilts/build-tools

Test: treehugger
Change-Id: Ic0fb09c8970f3a1c9f3b7b0303168520dcbfc3d8
2020-03-06 10:28:04 -08:00
Dan Willemsen
1871d88e4f Simplify message around nsjail error
We have known problems (docker; distros w/o user namespaces) which mean that we won't be turning this into a fatal error anytime soon. Also remove the bug report link, it's not useful to continue getting the same reports.

Change-Id: I271871d68150417ac938f074d3730cad4518e327
Test: treehugger
2020-03-02 20:36:06 +00:00
Colin Cross
9403e4b704 Merge "Collect metrics from inside soong_build" 2020-02-28 23:23:29 +00:00
Colin Cross
b72c909899 Collect metrics from inside soong_build
Collect the number of modules and variants and some basic statistics
on memory usage inside soong_build by writing out a proto that is
read back in by soong_ui.

Test: examine soong.log
Change-Id: I6926876377a4f6229cf41fdbf166ae03c885ea55
2020-02-25 20:50:00 +00:00
John Eckerdal
974b0e80c7 Add CCACHE_DIR to allowed environment variables
Trying to override the default directory for ccache by using the
CCACHE_DIR environment variable fails unless it is in the allowed
list.

Bug: 149670916
Test: manual
Change-Id: I8e7eea7a5c25d7ea5f0956fafc70d62522f3c4fc
2020-02-18 12:28:53 +01:00
Dan Willemsen
46459b08bb Remove empty directories when removing previously installed files
Apparently PackageManager gets confused when the apk disappears, but the
directory still exists.

Test: lunch aosp_arm-eng; m BasicDreams
      <remove BasicDreams from PRODUCT_PACKAGES>
      m nothing
      ls out/target/product/generic/system/app
Change-Id: I486c0ddadde55fad226049d3fb0f09af88117da0
2020-02-13 18:34:22 -08:00
Treehugger Robot
55f3001447 Merge "Add additional variables to ninja variable whitelist" 2020-01-28 20:40:31 +00:00
Kousik Kumar
0f095e1e65 Add additional variables to ninja variable whitelist
These variables control whether we accept cached-results / not and
whether we should update the cache with locally executed results or not.
I need accpet-cached whitelisted to purge the the RBE cache for the
invalid cache entry we have set for the failure we say yesterday.

Bug: b/148387048

Change-Id: I7344fc083f82e0b7bc11084376a267d19cf30bb8
2020-01-28 10:56:04 -08:00
Ramy Medhat
8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Treehugger Robot
6cc0515e82 Merge "Create a highmem pool and put metalava into it" 2020-01-25 01:09:43 +00:00
Colin Cross
8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Treehugger Robot
5fdf5d4b3e Merge "Add total build time to metrics" 2020-01-17 22:39:11 +00:00
Colin Cross
74cda72f5e Add total build time to metrics
Determining the total build time from the existing metrics is
tricky because some of the metrics are overlapping.  For example,
the "soong" metric includes  "blueprint bootstrap",
"environment check", "minibp", "bpglob", "minibootstrap", and
"bootstrap".  Create a new singleton "total" metric that
covers the interesting parts of the build.

Test: m nothing
Change-Id: Iedcf5c9cf0d27252b694d35e6ff66ca1fcf734ac
2020-01-16 15:25:50 -08:00
Treehugger Robot
0b031cf181 Merge "Disallow system m4" 2020-01-15 19:38:22 +00:00
Elliott Hughes
5ce6dfbd36 Merge "Switch to toybox gzip." 2020-01-13 17:21:02 +00:00
Treehugger Robot
ae7cddd5ad Merge "Sandbox soong_build by changing to root directory" 2020-01-11 06:00:21 +00:00
Colin Cross
988414c2cf Sandbox soong_build by changing to root directory
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.

Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
2020-01-11 01:11:46 +00:00
Colin Cross
e734832d25 Merge "Revert "Sandbox soong_build by changing to root directory"" 2020-01-10 22:00:59 +00:00
Colin Cross
47e4f9e1e8 Revert "Sandbox soong_build by changing to root directory"
This reverts commit 05c25ccb4a.

Reason for revert: broke absolute OUT_DIR
Bug: 146437378

Change-Id: I523ed79d40e1c1ef040212ba794a7a084abea75d
2020-01-10 18:51:04 +00:00
Colin Cross
e35e961aa9 Merge "Sandbox soong_build by changing to root directory" 2020-01-10 17:35:22 +00:00
Colin Cross
05c25ccb4a Sandbox soong_build by changing to root directory
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
2020-01-09 14:19:46 -08:00
Kousik Kumar
ade12e744a Add FLAG_compare to whitelisted environment variables
Bug: b/147419757
Test: Ran a compare build and tested that compare mode works again

Change-Id: I743d5070673f3ec9d0c2b099019f88935f3894fc
2020-01-09 08:53:01 -08:00
Dan Willemsen
5cacfe1e24 Allow EMMA_INSTRUMENT_FRAMEWORK during ninja
This is used by the art apex test.

Bug: 147197813
Test: m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true art-check-debug-apex-gen
Change-Id: Id185cd35f16131f2c9a8afeba2e5b87834b0e620
2020-01-06 12:25:40 -08:00
Dan Willemsen
7da042912b Likely fix SANITIZE_HOST=address builds
We're passing in ASAN_OPTIONS=detect_leaks=0 on the command line, and
the build started failing after we limited the environment.

Test: forrest of aosp-master/aosp_x86_64-eng
Change-Id: I42c91897c7677e1a249412e5a8bc7bb1edb2f881
2020-01-04 13:58:54 -08:00
Dan Willemsen
2de4152e61 Merge "Limit environment during ninja" 2020-01-04 17:51:03 +00:00
Colin Cross
68534a29d6 Don't print critical path parallelism ratio if elapsed time is 0.
Test: m nothing, examine out/soong.log
Change-Id: Ib23902c703d306337b2a5150d6d63b1e3d0f95b3
2020-01-04 07:26:28 +00:00
Treehugger Robot
6e3b471d3d Merge "Remove files that should no longer be installed" 2020-01-04 05:16:24 +00:00
Dan Willemsen
1e775d7619 Remove files that should no longer be installed
Make now exports (partial) lists of what should be installed. Those
lists are incomplete, but if something is removed from the list, we can
remove it from the filesystem so that incremental builds work better.

Test: built-in tests (m blueprint_tools)
Test: remove a module from PRODUCT_PACKAGES, see the print and file removed
Test: change the name of a cts test, see the old one removed from cts
Change-Id: I6bc14c6a5af63383ea265e97ed8b875fa51af0ed
2020-01-03 16:46:15 -08:00
Dan Willemsen
260db53da3 Limit environment during ninja
See the change that added ALLOW_NINJA_ENV for more information.

Test: m nothing; check out/soong.log for small list
Test: ALLOW_NINJA_ENV=true m nothing; check out/soong.log
Change-Id: I7761c6a07a7f8b0acee107e9c27c7739dd4b63ab
2020-01-03 15:36:33 -08:00
Dan Willemsen
e333635983 Add option to limit environment variables given to ninja
Ninja does not track changes in environment variables, so we get
potentially incorrect builds when environment variables change during
incremental builds if some action was using one of them.

Add a variable to limit exposure of these variables to ninja, and thus,
all actions run by ninja. Kati and Soong can still read environment
variables, they explicitly track which ones they read so that we can
re-run them appropriately.

This list is just the beginning, there's no good way to detect which
environment variables are currently being used and to pass them through.
So this initial change won't have a behavioral change, and we'll flip
the switch and see what fails or who complains, flipping it off and on
and adding to the list until we can make this always happen.

Also adds a board-specific `BUILD_BROKEN_NINJA_USES_ENV_VARS := ...`
list so that we can temporarily allow board-specific variables until
they're fixed.

Test: check out/soong.log
Test: ALLOW_NINJA_ENV=false m nothing; check out/soong.log
Test: set BUILD_BROKEN_NINJA_USES_ENV_VARS := OLDPWD
      ALLOW_NINJA_ENV=false m nothing; check out/soong.log
Change-Id: I08e4834ce12100a577ef7d6a9a21b9e9d345cb93
2020-01-03 15:36:29 -08:00
Dan Willemsen
2bb82d0011 Add TotalRAM detection
Colin originally wrote this for the highmem pool:

https://android-review.googlesource.com/c/platform/build/soong/+/1168271

But while that's a larger change, this is useful to just be in the logs,
and for use in multiproduct_kati to better limit the number of
concurrent jobs.

Test: check soong.log on linux and mac
Change-Id: I0518d303a220d775f8d78dba9f660b2954e68e3e
2019-12-27 09:43:49 -08:00
Elliott Hughes
686dd0c3a4 Switch to toybox gzip.
(Although I was the one who added the gzcat reference, I don't believe
we actually use it. And neither my Debian host nor toybox has gzcat --
as opposed to zcat -- anyway. So just quietly drop that.)

Test: treehugger
Change-Id: I95d94dab1daf72faedfa170770e8ebfaa37b70c6
2019-12-20 12:01:45 -08:00
Elliott Hughes
936a18d8da Merge "Switch to toybox realpath." 2019-12-20 00:27:10 +00:00
Elliott Hughes
c5aedc4c49 Switch to toybox realpath.
Change-Id: I9f1d3df08d860e5bfa131a531e274c75b72148e7
2019-12-17 22:03:45 -08:00
Treehugger Robot
0f41daf682 Merge "Switch the mac back to toybox xargs." 2019-12-15 16:35:05 +00:00
Dan Willemsen
7842826f49 Remove obsolete copy headers
When LOCAL_COPY_HEADER lines are removed, we just removed the copy rule,
we didn't actually remove the old header, so C/C++ files could still
include the old headers during incremental builds.

While we do consider LOCAL_COPY_HEADERS as near-obsolet, and it'll
disappear as we move everything over to Soong, this could produce some
unfortunate incremental build results while people are attempting to
remove them.

Fix this by ensuring that only the files currently listed in
LOCAL_COPY_HEADERS exist in TARGET_OUT_HEADERS after we run Kati.

Test: Remove a LOCAL_COPY_HEADERS entry; see the header removed
Change-Id: I817305703a6996d50490d552623d7df019b608c9
2019-12-13 19:38:23 -08:00
Elliott Hughes
d329254328 Switch the mac back to toybox xargs.
Test: make sdk
Change-Id: I29a6b0404e248553ecfdfdeb263aa9cd0d2fdb3e
2019-12-12 16:52:06 -08:00
Elliott Hughes
e2d33d52e1 Switch to our hermetic bc.
Test: treehugger
Change-Id: I12142a7f54190edd21e719bbd8f5df831ada6161
2019-12-10 10:12:18 -08:00
Colin Cross
d0be2109cf Add CUJ tests
Add a cuj_tests binary that runs through a sequence of critical
user journey builds and collects metrics.

Bug: 145688720
Test: cuj_tests
Change-Id: I6bb1a3fc4a458fd8cea7b7c924f565d0cf3e6df3
2019-12-05 11:11:37 -08:00
Treehugger Robot
dc0e8fd131 Merge changes Id261b5a3,Ice6f34d0
* changes:
  Fix errorProtoLog error messages
  Write BUILD_DATETIME_FILE after SetupOutDir
2019-12-05 09:10:13 +00:00
Colin Cross
ff27ce4cc0 Fix errorProtoLog error messages
Use Printf instead of Println for formatted strings, and pass
the filename instead of the proto.

Test: cuj_tests
Change-Id: Id261b5a34304a0caa61faa1f3bbc388aacdd25a6
2019-11-26 16:21:58 -08:00
Colin Cross
28f527c3da Write BUILD_DATETIME_FILE after SetupOutDir
Delay writing the BUILD_DATETIME_FILE until after the out directory
has been created.

Test: cuj_tests
Change-Id: Ice6f34d003f93c26b5d2d0b64f92b11efe16c2d4
2019-11-26 16:21:58 -08:00
Dan Shi
53f1a190ff Clean vts-core in host/out to ensure old package to be removed.
Bug: None
Test: m installclean
Change-Id: I5aa451d87dc377e62dcf1b88baa58539fa661feb
2019-11-26 10:02:53 -08:00
Dan Willemsen
1c504d9c7e Remove BASH_ENV from the build environment
This runs an arbitrary shell script for all non-interactive bash sessions, and we've got lots of those.

It was causing problems with Fedora 30, which triggers the `manpath` executable, which isn't available in our $PATH:
https://groups.google.com/d/msg/android-building/WqEyZQuxYtY/zDXl9irOCAAJ

Change-Id: If2f63bbbcc3accb8d239f93fa06823d301d69488
2019-11-18 19:13:56 +00:00
Elliott Hughes
7e4452511c Merge "Switch to our own zipinfo(1)." 2019-11-16 00:31:45 +00:00
Elliott Hughes
3efacee1fd Switch to our own zipinfo(1).
Test: treehugger
Change-Id: If71e87e45a52959be9a316810fe79250771eb9d4
2019-11-14 20:09:24 -08:00
Colin Cross
9016b91290 Set default pool when RBE or goma is enabled
Pass --default_pool=local_pool to kati when RBE or goma is enabled
to put most rules into the local_pool.  Specific rules that support
RBE or goma will set .KATI_NINJA_POOL := none to remove themselves
from the local_pool.  Passing --default_pool will also disable the
hack in kati that sets the pool based on the presence of the string
"/gomacc" in the command line.

Fixes: 143938974
Test: inspect pools in build-${TARGET-PRODUCT}.ninja for m USE_RBE=true
Test: inspect pools in build-${TARGET-PRODUCT}.ninja for m USE_GOMA=true
Change-Id: I839b2488383fcd63fffd613e25b0b9abcb72b567
2019-11-11 16:08:42 -08:00
Pete Gillin
a7a3d64a36 Remove the EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN option.
Previously, this setting environment variable to false reverted to an
OpenJDK 9 toolchain. This change removes that option.

Test: m core-all-system-modules
Test: zcat out/verbose.log.gz | grep 'prebuilts/jdk/jdk[0-9]*'
Test: EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN=false m core-all-system-modules (fails)
Bug: 131683177
Change-Id: I0d679648e1236925ce3ef7e1652379127e846b00
2019-11-11 11:03:58 +00:00
Pete Gillin
abbcddaaf4 Switch to an OpenJDK 11 toolchain by default.
Prior to this change, the default was to use and OpenJDK 9 toolchain,
with the EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN=true environment
variable available to opt into OpenJDK 11. After this change, the
default is to use OpenJDK 11, with
EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN=false available to opt out.

This change affects:
 - The version of javac used.
 - The version of javadoc used.
 - The version of other tools used during the build process (e.g. jar,
   jmod, and jlink).
 - For Java bytecode executed on the host, the version of the java
   executable used, and so the versions of the JVM and the OpenJDK
   libraries in the system module (or bootclasspath).

This change does not affect:
 - The Java language level, i.e. the version of Java expected in
   source code (the -source option to javac) or the version of Java
   bytecode produced (the -target option to javac).
 - Anything to do with code execution on the device.

Bug: 131683177
Test: make java droid docs cts tests
Test: zcat out/verbose.log.gz | grep 'prebuilts/jdk/jdk[0-9]*'
Test: make RunBluetoothRoboTests RunCarSettingsLibRoboTests
Test: cts-tradefed help
Test: atest CtsHostTzDataTests
Test: atest CtsLibcoreTestCases
Change-Id: I09dc22f1af4d1f2d7d3b85c08cb0ed9a1105aaca
Merged-In: Iaae89ef25c92ec099575c026fc50e41da4e143e8
2019-11-05 11:33:58 +00:00
Colin Cross
8cc19911b1 smart_status: move channel write outside lock to avoid deadlock
The smartStatusOutput.done channel is read from a goroutine that
locks smartStatusOutput.lock.  If the goroutine that writes to
smartStatusOutput.done is holding smartStatusOutput.lock it can
lead to an AB-BA deadlock.  Call stopActionTableTick from outside
the lock.

Fixes: 143558785
Test: none
Change-Id: I93a10ef9ff16c3953a1c5ccb102b024158358fe4
2019-10-29 15:27:32 -07:00
Elliott Hughes
e23c63de21 Merge "Switch to toybox find for darwin and linux." 2019-10-23 14:52:28 +00:00
Inseob Kim
822fdcab1f Add BUILD_BROKEN_TREBLE_SYSPROP_NEVERALLOW
A new sysprop neverallow rules are mandatory only for devices launching
with R or later. For devices already launched, neverallow rules can be
relaxed with adding following line to BoardConfig.mk:

BUILD_BROKEN_TREBLE_SYSPROP_NEVERALLOW := true

Bug: 131162102
Test: Set PRODUCT_SHIPPING_API_LEVEL := 30 and try building with
changing some system_public_prop to system_internal_prop
Test: m cts sepolicy_tests

Change-Id: I5e1640f7b43fd47863bc7bd163c2f43b01fa3326
Merged-In: I5e1640f7b43fd47863bc7bd163c2f43b01fa3326
(cherry picked from commit e36f52754f)
2019-10-15 09:22:42 +09:00
Elliott Hughes
e04b73f100 Merge "Go back to the native mac xargs for now." 2019-10-10 21:18:59 +00:00
Elliott Hughes
27cc0a8f32 Go back to the native mac xargs for now.
Should unbreak the mac build while I work on toybox xargs some more.

Test: suck it and see (since there's no mac presubmit)
Change-Id: I48954f801fc980a1174f93e474d73a24ad12ce66
2019-10-09 18:12:33 -07:00
Treehugger Robot
9026090263 Merge "Add $PRODUCT_OUT/test_harness_ramdisk to installclean" 2019-10-09 23:49:46 +00:00
Pete Gillin
1f52e93db2 Add experimental option to use OpenJDK 11 toolchain.
With this change, setting the environment variable
EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN=true switches from the OpenJDK 9
toolchain to the OpenJDK 11 one (prebuilts/jdk/jdk11).

Since the version of the java.base module has to match the version of
the jlink tool which consumes it, the --module-version argument to the
jmod create call is also switched.

Test: make
Test: EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN make
Bug: 131683177
Change-Id: I606d22538165f309fe5537bd67a26390b1dd5771
2019-10-09 18:38:14 +01:00
Bowgo Tsai
5145c2c11f Add $PRODUCT_OUT/test_harness_ramdisk to installclean
Also removes $PRODUCT_OUT/debug_ramdisk.

Bug: 140036184
Test: `make installclean` removes those directories
Change-Id: I442753817bdda4293b98ac1a7fd0173820dc2a2f
2019-10-09 01:59:51 +00:00
Dan Willemsen
b6699a1414 Restore previous behavior of get_build_var TMPDIR
My previous change makes TMPDIR a temporary directory that's removed
before we exit. So instead of starting ckati to get this value to a
directory that no longer exists, return the semi-stable value that we
use during a normal build.

Fixes: 142277335
Test: get_build_var TMPDIR
Change-Id: I37d1219b1fda09f131dee17ef7b91e475d3e39ad
2019-10-07 15:29:23 -07:00
Dan Willemsen
4e2456bc09 Fix dumpvars $PATH / $TMPDIR
Instead of just using the host $PATH during dumpvars, use our path
configuration from the regular build. But instead of creating a ton of
symlinks to the interposer, just use a small directory of symlinks. This
only takes ~3ms (vs ~300ms), at the expense of error logging. Since we
do just about the same product configuration at the start of the build,
we can just rely on the logging there.

This fixes warnings printed by the Mac build, since we using the host
`date` instead of the toybox version:

usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]

TMPDIR was being shared with the main build, but we're allowed to run in
parallel. If a build is started while we're running, our TMPDIR may be
cleared. Instead, create our own temporary directory, which doesn't need
to be a stable value (as we never do caching here).

Fixes: 141893400
Test: (on mac) source build/envsetup.sh; lunch
Change-Id: I0ff536e71dc5649cae26daa0087eb80861704021
2019-10-04 12:53:51 -07:00
Logan Chien
65ab468452 Merge "Add BUILD_BROKEN_PREBUILT_ELF_FILES to dumpvars.go" 2019-10-03 15:51:09 +00:00
Colin Cross
1f6faeb50f Use ctx.Fatalln instead of log.Fatalln for absolute path errors
log.Fatalln calls os.Exit, which skips deferred functions and
leaves the console with the cursor disabled.  Use ctx.Fatalln
instead, which uses a panic to exit and calls the deferred
function to restore the cursor.

Test: DIST_DIR=/tmp/\test\ foo m dist
Change-Id: Ie92297075e37c171d5ba48848a0ddb19652b051c
2019-09-23 15:55:54 -07:00
Colin Cross
c0b9f6b9d2 Force dumpvars into dumb terminal mode and move log files
Force soong_ui in dumpvars mode to use a dumb terminal so that it
doesn't print an empty status table.  Also move the logs to files
that are prefixed with "dumpvars-" so that they don't rotate the
real build logs.

Fixes: 139307849
Test: lunch
Change-Id: If92421af07731fedc9ca42670f309e95922a2224
2019-09-23 14:24:57 -07:00
Colin Cross
f0b987ecba status table: don't write newlines in non-scrolling terminals
Multiple terminals have had issues with writing newlines into the
non-scrolling region, just set the cursor to the beginning of the
next line instead.

Test: m nothing in JediTerm
Change-Id: I2e434f4cc263ca13b82889a79d6a8bb48d084cb3
2019-09-20 15:03:31 -07:00
Colin Cross
bf8f57e2f6 status table: ANSI code rows and columns are 1-based
Don't pass 0 to setCursor or setScrollingMargins.  Passing 0 hasn't
caused any known issues, but 1 is correct.

Test: m nothing
Change-Id: I8735032eb3f1d46df4c41c4ea9ef95b5fd842824
2019-09-20 15:03:11 -07:00
Logan Chien
f9cf9acd7a Add BUILD_BROKEN_PREBUILT_ELF_FILES to dumpvars.go
This commit adds BUILD_BROKEN_PREBUILT_ELF_FILES to
`ui/build/dumpvars.go` so that its value can be printed to
`out/soong.log`.

Bug: 141176116
Test: lunch aosp_crosshatch-userdebug && make && \
      grep BUILD_BROKEN_PREBUILT_ELF_FILES out/soong.log

Change-Id: I99600a5b540be2bb93647ee8175b9f9afd1a6eb3
2019-09-20 11:40:49 -07:00
Roland Levillain
9c27df7d4d Merge "Add $ANDROID_HOST_OUT/apex to the file set removed by target installclean." 2019-09-18 10:47:06 +00:00
Elliott Hughes
26bce340e3 Track switch to toybox date for macOS.
Bug: http://b/139450866
Test: builds
Change-Id: I5ab0d20d761bcd03f3bee18b8343a2f8aa5e2275
2019-09-12 15:05:13 -07:00
Roland Levillain
e5f9ee56b5 Add $ANDROID_HOST_OUT/apex to the file set removed by target installclean.
Before this change, host APEX (`.zipapex`) packages would still be
there after `m installclean`.

Test: Run `m installclean` and check that host APEX packages are removed.
Change-Id: I446286ce700f12666be5aa90fbe3bbc648abd280
2019-09-11 14:50:08 +01:00
Elliott Hughes
34b49d1402 Use toybox md5sum instead of md5 on macOS.
Test: builds locally
Change-Id: I04aa52de96e0525a44331cb0c057a8e18a95891d
2019-09-06 14:42:24 -07:00
Elliott Hughes
ef19e01f96 Switch to toybox stat on macOS too.
Bug: http://b/139450866
Test: builds
Change-Id: I97af607a311aea5f97544bd3ce1d532d999126ff
2019-09-05 10:01:37 -07:00
Elliott Hughes
5cb822b2cf Switch to toybox find for darwin and linux.
Test: treehugger
Change-Id: Id311b403f50a76e75ed92c2a9e7cc7aa025f2448
2019-08-27 21:47:05 -07:00
Elliott Hughes
f1ff226cd4 Switch most of the darwin tools to toybox.
Bug: http://b/139450866
Test: treehugger
Test: "works for me on my machine" (last week)
Change-Id: I5dd00e6fda134b06967651007afb7565ce4f9e24
2019-08-27 15:17:32 -07:00
Treehugger Robot
2569cc5026 Merge "We have md5sum/sha1sum/sha256sum/sha512sum on Darwin now too." 2019-08-27 03:17:07 +00:00
Steven Moreland
0aabb11dd4 Add $ANDROID_PRODUCT_OUT/apex to installclean.
When removing dependencies from APEXes, the lib files were still there
even after installclean.

Bug: 139016109
Test: run installclean, see apex directories go away :D
Change-Id: I4dad8e3aa0042cdd5bbb76bd6fdd01b025b98495
2019-08-26 18:35:55 +00:00
Elliott Hughes
556a37585a We have md5sum/sha1sum/sha256sum/sha512sum on Darwin now too.
Bug: http://b/139450866
Test: treehugger
Change-Id: I654f706a18949d29c8688b1abe85d574b5315808
2019-08-26 11:15:15 -07:00