Commit graph

13 commits

Author SHA1 Message Date
Jingwen Chen
6a9bfa122a soong_ui path.go: improve comments.
Test: th presubmit
Bug: 173587901
Change-Id: Ic30fa88ee84cbc48860a7500223432c720039810
2020-11-18 22:13:23 -05:00
Patrice Arruda
ae2694b30d Pass the TMPDIR directly to the command when dumping makefile variables.
External applications such as metrics uploader depends on the TMPDIR.
DumpMakeVars is changing the TMPDIR environment variable value. Once
the work is done, TMPDIR directory is deleted. Additional flow of
the tool or external applications can panic since TMPDIR directory
does not exist. Since this is an isolated case, pass the TMPDIR
directory to the command of dumping the makefile variables.

Bug: b/140638454
Test: * Ran lunch command with ANDROID_ENABLE_METRICS_UPLOAD set
        and metrics_uploader did not crash.
      * m nothing
Change-Id: Ib78a3f8cfb5da48f65bd9bce1511e9abdf3d9cca
2020-06-05 17:28:41 +00:00
Dan Willemsen
347ba75679 Remove TEMPORARY_DISABLE_PATH_RESTRICTIONS
This was a temporary migration tool, and has been broken ever since the
ninja environment started being restricted. Remove it, since it has
lasted two releases.

Test: TEMPORARY_DISABLE_PATH_RESTRICTIONS=true m
Change-Id: I2e3709525598030f5c24d12fa966e531dfed33ae
2020-05-04 19:54:57 +00:00
Dan Willemsen
4e2456bc09 Fix dumpvars $PATH / $TMPDIR
Instead of just using the host $PATH during dumpvars, use our path
configuration from the regular build. But instead of creating a ton of
symlinks to the interposer, just use a small directory of symlinks. This
only takes ~3ms (vs ~300ms), at the expense of error logging. Since we
do just about the same product configuration at the start of the build,
we can just rely on the logging there.

This fixes warnings printed by the Mac build, since we using the host
`date` instead of the toybox version:

usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]

TMPDIR was being shared with the main build, but we're allowed to run in
parallel. If a build is started while we're running, our TMPDIR may be
cleared. Instead, create our own temporary directory, which doesn't need
to be a stable value (as we never do caching here).

Fixes: 141893400
Test: (on mac) source build/envsetup.sh; lunch
Change-Id: I0ff536e71dc5649cae26daa0087eb80861704021
2019-10-04 12:53:51 -07:00
Dan Willemsen
733547d522 Use prebuilt awk on Darwin too
Test: build on mac
Change-Id: I25d21c0df0b9023d6c43b11dde60e6066f107576
2019-02-14 20:11:26 -08:00
Dan Willemsen
9121973130 Stop calling our host $PATH prebuilts toybox
There's more than just toybox in this group now, so let's rename our
variables to something closer to the desired behavior, rather than the
first user.

Test: treehugger
Change-Id: I76d4407792061c8110b194cfe73f1ddc84dbc22f
2019-02-14 20:07:02 -08:00
Nan Zhang
17f2767724 Generate build timing metrics to proto format file
Test: Dumped the text formated based metrics file to out dir,
and checked the file.
Bug: b/63815990

Change-Id: Iff476f72a0be74eb53b6b26ef468d11c0f24a404
2019-01-04 15:54:01 -08:00
Dan Willemsen
417be1fa9f Start using the toybox prebuilt
Prepend the toybox symlink directory to $PATH, and stop generating
symlinks into out/.path for tools used from toybox.

We don't (yet?) have toybox prebuilts for darwin, so apply the above only to Linux, and preserve existing behavior on Darwin.

Test: check the build banner for uname results
Test: m
Change-Id: I37fc380381e65a628cdc131d462fd4441eacfe9d
2018-10-30 23:25:35 -07:00
Dan Willemsen
184901135c Revert "Revert "Revert "Revert "Add path interposer""""
This reverts commit 09f4540d66.

Fixes the raw call to net.Listen in the tests to go through the listen()
helper and use the long socket path fallbacks.

Removes the use of timeouts from the tests -- the behaviors being tested
did not rely on timeouts, so removing them will reduce the flakiness if
the build is heavily loading the machine at the same time the test is
running.

Also fixes some potential nil pointer dereferences.

Test: OUT_DIR=<really long> m blueprint_tools
Test: `while .../soong-ui-build-paths/test/test; do sleep 0.01; done` with a build running
Change-Id: I16d44be7517bc415f1c808284088f4ba40df3bfa
2018-05-31 14:59:33 -07:00
Dan Willemsen
09f4540d66 Revert "Revert "Revert "Add path interposer"""
This reverts commit c59a92cb1a.

Reason for revert: tests are broken with long OUT_DIRs
They're directly calling net.Listen, and not using the fallback
for long socket names.

Change-Id: Id14cbd499fd9b36c6926b7552d3554340cb0916c
2018-05-25 23:26:00 +00:00
Dan Willemsen
c59a92cb1a Revert "Revert "Add path interposer""
This reverts commit 96c957ae20.

Fixes issues on some machines where the socket in TMPDIR ended up with a
unix domain socket pathname over 107 characters long, which Go will
reject due to underlying limitations in the system calls. If this
happens, we'll fall back to opening the directory, then using
/proc/self/fd/#/<file>, or manually creating a similar symlink in /tmp.

Also fixes some issues on Mac where os.Executable returns the symlink
instead of the underlying file, sending a message over a unix domain
socket will block if the reader isn't reading, and sandboxing was
preventing us from running `ps`.

Test: m blueprint_tools
Test: m blueprint_tools on mac
Change-Id: Ib19ccfe10cb0a79f1476fb1d5cd20ed0495be367
2018-05-25 14:07:00 -07:00
Dan Willemsen
96c957ae20 Revert "Add path interposer"
This reverts commit a14704c12b.

Reason for revert: breaking builds

Change-Id: I920b0ff41823428a0baf59b3fd82cbcc766babca
2018-05-19 00:54:13 +00:00
Dan Willemsen
a14704c12b Add path interposer
This will allow us to track (and eventually limit) the commands that the
build references via $PATH. These are mostly implicit dependencies on
the host system -- for Linux, we assume something similar to Ubuntu
14.04 with a few extra packages, but this will let us better define
that.

This will not catch uses of tools with absolute paths (/bin/bash, etc),
but most uses shouldn't be relying on absolute path names anyways.

Adds ~400ms on the first startup, ~140ms on subsequent runs, and
overhead of a few ms for every forwarded execution.

Test: m
Test: build/soong/build_test.bash
Test: Add `gcc --version`, TEMPORARY_DISABLE_PATH_RESTRICTIONS=true m
Change-Id: Id68cbb1c8ceef65bbbb10751e83722c7662d2351
2018-05-18 13:24:36 -07:00