Commit graph

122 commits

Author SHA1 Message Date
Dan Willemsen
417be1fa9f Start using the toybox prebuilt
Prepend the toybox symlink directory to $PATH, and stop generating
symlinks into out/.path for tools used from toybox.

We don't (yet?) have toybox prebuilts for darwin, so apply the above only to Linux, and preserve existing behavior on Darwin.

Test: check the build banner for uname results
Test: m
Change-Id: I37fc380381e65a628cdc131d462fd4441eacfe9d
2018-10-30 23:25:35 -07:00
Elliott Hughes
be3cfa514a Disallow "runalarm" and allow the more standard "timeout".
Bug: N/A
Test: builds
Change-Id: Id8fe8ee2569b7410c66f448513cce6ff57458ffb
2018-10-27 08:09:18 -07:00
Dan Willemsen
41cabfbbf4 Remove make and xmllint from the allowed tools list
We've got a prebuilt of make in prebuilts/build-tools, and we build
xmllint from source.

Test: No logs on the build servers in the last 24 hours
Change-Id: Ib60a7b5feaa31097071b67ebf6beb7f0b9a9f81b
2018-10-19 12:29:46 -07:00
Dan Willemsen
e9e20dd3e4 Remove perl from the allowed PATH tools list
This is no longer used within Android builds. We don't have any way of
providing extra libraries, or using a hermetic version of perl.

Bug: 117338567
Test: check build logs for the last 48 hours
Change-Id: I06d41a0006a47508a435dbf44bfccc4b385d9f41
2018-10-09 19:51:30 -07:00
Dan Willemsen
6afcd9aab9 Log uses of make and xmllint within the Android Build
Now that we've got prebuilts of make, turn on logging so that
we can find all the users and convert them to the prebuilt.

We also build xmllint as part of the tree, so log uses of that
too.

Test: treehugger
Change-Id: I8a6a5d481dfc709dc0eb3424c51430bc54763c1e
2018-10-09 23:57:45 +00:00
Elliott Hughes
ecdeb1e705 Remove sum(1) from the list of approved binaries.
No-one should be using BSD sum in 2018.

Bug: N/A
Test: treehugger
Change-Id: I9aa313e111217a2d73ba41eaf49035d8293b5785
2018-10-05 19:59:58 +00:00
Dan Willemsen
3eec9c57ca Switch perl to logging
To find the users and see if we can get rid of it.

Test: treehugger
Change-Id: I149f8daee6e21b35840ea1788de7f3e310eb5fba
2018-10-04 23:26:12 +00:00
Dan Willemsen
090ec58af5 Add tee to the allowed PATH tools list
tee is used by art/build/Android.gtest.mk during testing

Change-Id: I45993366622694edef5494d9e74b8465a15a8795
2018-08-31 08:13:13 +00:00
Dan Willemsen
2d31ace93e Add od to allowed PATH tools
od is used by ninja's inline.sh tool to encode a python script in a C
header.

Test: forrest run of build-tools branch
Change-Id: I87001a02b168ebe183b8b7dc495c67bb43f1b86d
2018-08-30 16:53:02 -07:00
Dan Willemsen
8125d2a2a2 Start enforcing the $PATH restrictions
Anything that isn't explicitly marked as Allowed in this list will now
return an error (and log a message) when it is executed.

Test: search all logs from the build server over the last day
Change-Id: I3ceacd9a140097809dde81a8d8979dd2c45f234c
2018-08-30 06:20:26 +00:00
Dan Willemsen
6fa18f4e10 Add unix2dos as an allowed PATH tool
unix2dos is used instead of todos by some build servers in
development/build/tools/patch_windows_sdk.sh

Test: none
Change-Id: I04cb2258f71bebfc8792b7aa95643a45ffe3ec5c
2018-08-29 10:26:47 -07:00
Dan Willemsen
7a08bc5467 Add todos to the allowed PATH list
todos is used by development/build/tools/patch_windows_sdk.sh

Test: m win_sdk; check out/soong.log
Change-Id: I5e1a290838cc34261d7f5fc321afe79a8b1a6fbb
2018-08-28 00:18:02 -07:00
Dan Willemsen
b4d0144a6f Add fuser to allowed PATH tools
fuser is used by goma

Test: none
Change-Id: I9acf4dd062c3b767a1688e4f236a96d5f2b8a7f6
2018-08-27 16:45:23 -07:00
Dan Willemsen
42740f2c78 Add lsof and ps to the allowed tool lists
lsof is used by goma to find its proxy daemon
ps is used on Mac instead of pstree in soong_ui

Test: none
Change-Id: Ifed7d09d604c00b7eebac1488a6f4658788d6e33
2018-08-15 10:14:40 -07:00
Dan Willemsen
14eae19b1c Add du, paste, xxd to allowed PATH tools
du is used by build/make/tools/releasetools/build_image.py
paste is used by system/core/rootdir/update_and_install_ld_config.mk
xxd is used by device/generic/goldfish/tools/mk_qemu_image.sh

Test: none
Change-Id: I707668578d1babe1b149c2c3f989e5f8be37d897
2018-08-14 23:08:56 -07:00
Dan Willemsen
16dbb39afd PATH: Allow more tools
pkill and sleep are used in
device/generic/goldfish/tools/emulator_boot_test.sh

pgrep is used by GOMA

bc and pwd are used by kernel builds

Test: none
Change-Id: If7004255b4776fa4409fdd4b0aa6d6617b471317
2018-06-06 11:33:59 -07:00
Dan Willemsen
f6d300662e Allow more PATH tools, expose HOST_STRIP
atree was using the host's strip tool during sdk generation, so expose
the prebuilt for use instead.

`id` was used with a test running minijail0 during the build

`sha1sum` is used by development/build/tools/mk_sdk_repo_xml.sh

`rmdir` was being used to remove some tmp files

Test: m PRODUCT-sdk_arm64-sdk dist sdk_repo
Test: Run other builds through forrest
Change-Id: I49c756a3e9497c0bbd9030754e6aa193a2d989a4
2018-06-01 12:48:41 -07:00
Dan Willemsen
184901135c Revert "Revert "Revert "Revert "Add path interposer""""
This reverts commit 09f4540d66.

Fixes the raw call to net.Listen in the tests to go through the listen()
helper and use the long socket path fallbacks.

Removes the use of timeouts from the tests -- the behaviors being tested
did not rely on timeouts, so removing them will reduce the flakiness if
the build is heavily loading the machine at the same time the test is
running.

Also fixes some potential nil pointer dereferences.

Test: OUT_DIR=<really long> m blueprint_tools
Test: `while .../soong-ui-build-paths/test/test; do sleep 0.01; done` with a build running
Change-Id: I16d44be7517bc415f1c808284088f4ba40df3bfa
2018-05-31 14:59:33 -07:00
Dan Willemsen
09f4540d66 Revert "Revert "Revert "Add path interposer"""
This reverts commit c59a92cb1a.

Reason for revert: tests are broken with long OUT_DIRs
They're directly calling net.Listen, and not using the fallback
for long socket names.

Change-Id: Id14cbd499fd9b36c6926b7552d3554340cb0916c
2018-05-25 23:26:00 +00:00
Dan Willemsen
c59a92cb1a Revert "Revert "Add path interposer""
This reverts commit 96c957ae20.

Fixes issues on some machines where the socket in TMPDIR ended up with a
unix domain socket pathname over 107 characters long, which Go will
reject due to underlying limitations in the system calls. If this
happens, we'll fall back to opening the directory, then using
/proc/self/fd/#/<file>, or manually creating a similar symlink in /tmp.

Also fixes some issues on Mac where os.Executable returns the symlink
instead of the underlying file, sending a message over a unix domain
socket will block if the reader isn't reading, and sandboxing was
preventing us from running `ps`.

Test: m blueprint_tools
Test: m blueprint_tools on mac
Change-Id: Ib19ccfe10cb0a79f1476fb1d5cd20ed0495be367
2018-05-25 14:07:00 -07:00
Dan Willemsen
96c957ae20 Revert "Add path interposer"
This reverts commit a14704c12b.

Reason for revert: breaking builds

Change-Id: I920b0ff41823428a0baf59b3fd82cbcc766babca
2018-05-19 00:54:13 +00:00
Dan Willemsen
a14704c12b Add path interposer
This will allow us to track (and eventually limit) the commands that the
build references via $PATH. These are mostly implicit dependencies on
the host system -- for Linux, we assume something similar to Ubuntu
14.04 with a few extra packages, but this will let us better define
that.

This will not catch uses of tools with absolute paths (/bin/bash, etc),
but most uses shouldn't be relying on absolute path names anyways.

Adds ~400ms on the first startup, ~140ms on subsequent runs, and
overhead of a few ms for every forwarded execution.

Test: m
Test: build/soong/build_test.bash
Test: Add `gcc --version`, TEMPORARY_DISABLE_PATH_RESTRICTIONS=true m
Change-Id: Id68cbb1c8ceef65bbbb10751e83722c7662d2351
2018-05-18 13:24:36 -07:00