Commit graph

215 commits

Author SHA1 Message Date
Elliott Hughes
3a1fd8e0a1 Switch to toybox ln(1).
Test: treehugger
Change-Id: Ia52e73e87a6c4648285d2340908013cd8528d221
2019-01-10 21:43:11 +00:00
Elliott Hughes
0bd4ab5334 Merge "Switch to toybox expr(1)." 2019-01-10 21:40:37 +00:00
Treehugger Robot
20a8856c8f Merge "Add BUILD_USERNAME and BUILD_HOSTNAME" 2019-01-10 21:29:47 +00:00
Elliott Hughes
6491f8df60 Switch to toybox expr(1).
Test: treehugger
Change-Id: Ib015c70841504d74fd0272a8ca6ecb3c617fb759
2019-01-10 17:04:57 +00:00
Elliott Hughes
e1184be0c0 Switch to toybox echo(1).
Test: treehugger
Change-Id: Ie40cce5113f4b9bd17ec28c572952aacf1057804
2019-01-10 06:25:25 +00:00
Elliott Hughes
c33514e3b6 Revert "Switch to toybox cp(1)."
This reverts commit 9886695763.

Reason for revert: http://b/122623408
Bug: http://b/122623408

Change-Id: Ib9ec92f9dd6f9a0db7f598b10241b0474e46aba0
2019-01-10 06:03:25 +00:00
Treehugger Robot
cd21d46df8 Merge "goma_ctl.py ensure_start in Soong UI." 2019-01-10 03:55:22 +00:00
Yoshisato Yanagisawa
2cb0e5df53 goma_ctl.py ensure_start in Soong UI.
As suggested in b/118390303#comment18, let me run
"goma_ctl.py ensure_start" in soong UI.

Bug: 118390303
Test: stop compiler_proxy, and execute following command:
Test: USE_GOMA=true lunch aosp_arm-eng
Test: and confirms compiler_proxy is running.
Change-Id: I859daae6ae2399c5b6bce2fafd874dfdb7c6aae0
Signed-off-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
2019-01-10 10:14:16 +09:00
Elliott Hughes
e0e244130f Switch to toybox du(1).
Test: treehugger
Change-Id: Id66705f35e82ed25af6dbd5a131ce90ceba60912
2019-01-09 09:39:02 -08:00
Elliott Hughes
9886695763 Switch to toybox cp(1).
Test: treehugger
Change-Id: I91eddd95abc03f76d4da7fdc1e70c5cf314e6127
2019-01-08 16:31:54 -08:00
Elliott Hughes
2837d75ffd Merge "Switch to toybox chmod(1)." 2019-01-09 00:27:35 +00:00
Dan Willemsen
71edc8b848 Add BUILD_USERNAME and BUILD_HOSTNAME
As part of a future change to sandbox the build on Linux, the real
username will be switching to "nobody", and the hostname will be
switching to "android-build".

The USER environment variable will reflect the sandboxed value, so for
the build properties that want the external USER, they'll need to use
BUILD_USERNAME.

Similarly, BUILD_HOSTNAME will reflect the real value, while the
`hostname` tool will return "android-build"

Bug: 122270019
Test: check build.prop
Change-Id: I99604b9488732a63690b256dc4dd7894d369a32c
2019-01-07 23:00:54 +00:00
Nan Zhang
17f2767724 Generate build timing metrics to proto format file
Test: Dumped the text formated based metrics file to out dir,
and checked the file.
Bug: b/63815990

Change-Id: Iff476f72a0be74eb53b6b26ef468d11c0f24a404
2019-01-04 15:54:01 -08:00
Dan Willemsen
7f30c076d5 Print a status message when starting ninja
This way we don't appear hung at:

  No need to regenerate ninja file

Change-Id: I8dbdaa2c1b1c5a6a73187d0e6061f363b62e10c9
Fixes: 122251150
Test: m nothing
2019-01-02 12:50:49 -08:00
Elliott Hughes
97295bd391 Switch to toybox chmod(1).
Test: treehugger
Change-Id: Iccb70941906d9cf8b004303a6935b7df1217babf
2018-12-17 14:55:10 -08:00
Elliott Hughes
9add81ed0f Move cut(1) to toybox.
Test: treehugger
Change-Id: I07dd7a45ee56178a72deaffc6c4796c3ec53f564
2018-12-11 09:39:48 -08:00
Elliott Hughes
734a780217 Move touch(1) to toybox.
Test: treehugger
Change-Id: If31fe1fdc0150d07d5421702cbac8e142f522802
2018-12-07 18:30:52 -08:00
Elliott Hughes
cc85770628 Move mkdir(1) to toybox.
Test: treehugger
Change-Id: I5fcbf33acb93cdc279a900586b96e0232c4aea87
2018-12-06 22:30:45 -08:00
Elliott Hughes
ed46164d03 Move head(1) to toybox.
Test: clean build
Change-Id: I264c320b4a63e9b0f440791dc20ff99ec910036a
2018-12-07 02:36:16 +00:00
Elliott Hughes
2ebfd495cf Move dirname(1) to toybox.
Test: treehugger
Change-Id: Ie1ffbfb900803940620dd11a7ed6bfda76bee86c
2018-12-06 08:57:31 -08:00
Elliott Hughes
5172a8b50e Move to toybox unix2dos(1).
This removes the need for the confusingly named "todos".

Test: treehugger
Change-Id: Id8931deb00a06d54262b2803a00d13bd4de88f12
2018-12-05 19:42:43 -08:00
Elliott Hughes
6646904a47 Move to toybox cmp(1).
Test: treehugger
Change-Id: Idf5c9f00d0698aca6ceab2956c7dc49ad7c604e1
2018-12-05 10:03:31 -08:00
Elliott Hughes
6141a7d7ff Move to toybox basename(1).
Test: treehugger
Change-Id: I29f210d86a0dada8a40eca1919e2f6f876d50bcd
2018-12-04 13:42:46 -08:00
Elliott Hughes
5728fb536f Move setsid(1) to toybox.
Test: treehugger
Change-Id: I8caf9dbbb671218379c309e698f23e7c964f1a18
2018-12-03 10:14:52 -08:00
Elliott Hughes
1ff2b5d34c Move tee(1) to toybox.
Test: treehugger
Change-Id: Idf7fc37d6e84c295410f71dfc1ad9e460d7f7461
2018-11-30 12:36:44 -08:00
Elliott Hughes
f3f3b99bb3 Move xxd(1) to toybox.
Test: treehugger
Change-Id: I906b0c94da5b65dc387ae253492356ca78f93fd2
2018-11-29 22:04:16 -08:00
Elliott Hughes
15f2dfe482 Move paste(1) to toybox.
Test: treehugger
Change-Id: Id5e8e841406357d8301198eb433b96d2f1ba02d9
2018-11-29 11:36:51 -08:00
Elliott Hughes
716fa36dd9 Move rmdir(1) to toybox.
Test: treehugger
Change-Id: Ibb43f09c0cfcda631025ef0fea4730519a3b6fe6
2018-11-28 14:14:35 -08:00
Elliott Hughes
10412ec50b Move pwd(1) to toybox.
Test: treehugger
Change-Id: I656e322724d13273bf1099c6cc311a520aa4173e
2018-11-27 18:04:34 -08:00
Elliott Hughes
d50d40fe07 Move tail(1) to toybox.
Test: treehugger
Change-Id: If421513911208db4542c4767168ce50b7b837af8
2018-11-27 16:29:17 -08:00
Elliott Hughes
505b76136b Move od(1) to toybox.
Test: treehugger
Change-Id: I19587d3045992fa8f60743ef358c0fd050b43dc0
2018-11-27 09:00:21 -08:00
Elliott Hughes
bb19518a54 Move env(1) to toybox.
Test: clean build
Change-Id: I52770c022f7d34fb83c990f485b0ca3ea29e97ee
2018-11-15 13:55:56 -08:00
Elliott Hughes
6bdad22f7b Move comm(1) to toybox.
Test: clean build
Change-Id: I2d78388ef7211af6d517702c97f3b95ed2f30a46
2018-11-14 20:49:52 -08:00
Elliott Hughes
2907ceaae2 Move sleep(1) to toybox.
Test: clean build
Change-Id: I724278466c4d1367add79f949d1869e1e50a192d
2018-11-14 14:30:56 -08:00
Elliott Hughes
7a907c9291 Move uniq(1) to toybox.
Test: clean build
Change-Id: I4819098fd298dbc4f3e424e437a448e1fb1baeaf
2018-11-13 22:12:30 -08:00
Elliott Hughes
5d593238e2 Move cat(1) to toybox.
Test: builds
Change-Id: I9a367c3204891518994bfe889072dd1cf9aed490
2018-11-13 13:08:13 -08:00
Elliott Hughes
9b370f5ada Move id(1) and whoami(1) to toybox.
Test: builds
Change-Id: I32347c94438c394eb31f5af9f51cee11ad0ef247
2018-11-11 10:57:58 -08:00
Dan Willemsen
417be1fa9f Start using the toybox prebuilt
Prepend the toybox symlink directory to $PATH, and stop generating
symlinks into out/.path for tools used from toybox.

We don't (yet?) have toybox prebuilts for darwin, so apply the above only to Linux, and preserve existing behavior on Darwin.

Test: check the build banner for uname results
Test: m
Change-Id: I37fc380381e65a628cdc131d462fd4441eacfe9d
2018-10-30 23:25:35 -07:00
Treehugger Robot
a758cda418 Merge "Speed up build_test" 2018-10-30 20:27:42 +00:00
Dan Willemsen
f99915f569 Speed up build_test
Stop writing out ninja files, as they become very large, especially when
multiplied by the number of defined products.

Test: treehugger
Change-Id: Id9529d14040acb72a0188e58b5db2911f142071e
2018-10-29 10:43:37 -07:00
Elliott Hughes
be3cfa514a Disallow "runalarm" and allow the more standard "timeout".
Bug: N/A
Test: builds
Change-Id: Id8fe8ee2569b7410c66f448513cce6ff57458ffb
2018-10-27 08:09:18 -07:00
Dan Willemsen
6f03752422 Allow DIST_DIR to be read via dumpvars
DIST_DIR is controlled by soong_ui, and the make dumpvars code no longer
has access to it. So instead, handle some dumpvar requests inside
soong_ui itself.

Test: get_build_var DIST_DIR
Test: get_build_var OUT_DIR
Test: get_build_var BUILD_SYSTEM
Test: build/soong/soong_ui.bash --dumpvars-mode --vars="DIST_DIR
BUILD_SYSTEM"
Change-Id: Id3bcb8b0748db67c83d0e42d1ae5be564c5eb2f7
2018-10-21 09:26:55 -07:00
Dan Willemsen
2d31a44b8a Stop using DIST_DIR in Soong
We're only using it to distribute files in case of failure, which isn't
well supported currently, but can be handled for now by using the
DIST_DIR environment variable during the command execution.

This was at least one cause that we'd be re-running Soong during every
build server build, as the DIST_DIR values are unique.

Test: m dist
Change-Id: Ibd5e6b6c46695350de80b745bfb6a6aa685033a0
2018-10-20 21:33:41 -07:00
Dan Willemsen
d4ad1da3d7 Merge "Add a Kati-based packaging step" 2018-10-20 22:50:10 +00:00
Dan Willemsen
41cabfbbf4 Remove make and xmllint from the allowed tools list
We've got a prebuilt of make in prebuilts/build-tools, and we build
xmllint from source.

Test: No logs on the build servers in the last 24 hours
Change-Id: Ib60a7b5feaa31097071b67ebf6beb7f0b9a9f81b
2018-10-19 12:29:46 -07:00
Dan Willemsen
fb1271a52b Add a Kati-based packaging step
The idea is that we'd move the installation and packaging tasks over to
it, using data from Soong & the Kati reading Android.mk files.

This would allow us to make more fundamental changes about how we
package things without having to adjust makefiles throughout the tree.
Possible use cases:

* Moving some information from Soong's Android.mk output to a file read
  by the packaging step may allow us to read the Android.mk files less
  often, speeding up builds.

* Refactoring our current two-stage ASAN builds to run the Kati build
  step twice, writing into different object directories, then have a
  single packaging step that reads both outputs. Soong already has the
  capability of writing out a single ninja file with all the asan
  combinations.

* Running two build steps, one building the system-related modules
  using a "generic" device configuration, and one building the vendor
  modules using a specific device configuration. This could enforce a
  GSI/mainline system vs vendor split in a single build invocation.

* If all installation is through this tool, it will be much easier to
  track what should no longer be installed on an incremental build,
  reducing the need for installclean.

* Changing PRODUCT_PACKAGES should be a much faster operation, which
  means we could keep track of local additions to the images. Then
  `mma` would be more persistent, instead of installing something once,
  then never updating it again.

Eventually we plan on switching from Kati to something Go-based, but
this is a more incremental approach while we clean up everything else.

Currently, this just moves the dist-for-goal handling over to the
packaging step, so that we don't need to read Android.mk files when
DIST_DIR changes, or we switch between dist vs not.

Bug: 116968624
Bug: 117463001
Test: m nothing
Change-Id: Idec5ac6f7c7475397ba0fb65bd3785128a7517df
2018-10-19 09:55:00 -07:00
Dan Willemsen
d50e89f389 Always ensure that DIST_DIR is set
Set the default value if one isn't provided.

Test: `get_build_var DIST_DIR` after removing the default setting in envsetup.mk
Change-Id: I0cb310fc65f7747c36de14608b61786ef6863fd1
2018-10-16 17:49:25 -07:00
Dan Willemsen
e9e20dd3e4 Remove perl from the allowed PATH tools list
This is no longer used within Android builds. We don't have any way of
providing extra libraries, or using a hermetic version of perl.

Bug: 117338567
Test: check build logs for the last 48 hours
Change-Id: I06d41a0006a47508a435dbf44bfccc4b385d9f41
2018-10-09 19:51:30 -07:00
Dan Willemsen
6afcd9aab9 Log uses of make and xmllint within the Android Build
Now that we've got prebuilts of make, turn on logging so that
we can find all the users and convert them to the prebuilt.

We also build xmllint as part of the tree, so log uses of that
too.

Test: treehugger
Change-Id: I8a6a5d481dfc709dc0eb3424c51430bc54763c1e
2018-10-09 23:57:45 +00:00
Elliott Hughes
ecdeb1e705 Remove sum(1) from the list of approved binaries.
No-one should be using BSD sum in 2018.

Bug: N/A
Test: treehugger
Change-Id: I9aa313e111217a2d73ba41eaf49035d8293b5785
2018-10-05 19:59:58 +00:00