Commit graph

263 commits

Author SHA1 Message Date
Dan Willemsen
0f0214699e Use python / python2.7 prebuilts
Bug: 117811537
Test: treehugger
Change-Id: I35b335edcfa202a45f55f588a643c62c2ced0f26
2019-02-18 19:34:00 +00:00
Elliott Hughes
47de2a2f52 Switch sed(1) to toybox.
Test: treehugger
Change-Id: I575ae9db8d94cbec811d0029691df98373f8fe35
2019-02-15 15:49:12 -08:00
Dan Willemsen
733547d522 Use prebuilt awk on Darwin too
Test: build on mac
Change-Id: I25d21c0df0b9023d6c43b11dde60e6066f107576
2019-02-14 20:11:26 -08:00
Dan Willemsen
9121973130 Stop calling our host $PATH prebuilts toybox
There's more than just toybox in this group now, so let's rename our
variables to something closer to the desired behavior, rather than the
first user.

Test: treehugger
Change-Id: I76d4407792061c8110b194cfe73f1ddc84dbc22f
2019-02-14 20:07:02 -08:00
Yoshisato Yanagisawa
02eaccafbd More verbose message on goma_ctl.py ensure_start failure.
For ease of investigating the issue with error reports, let me make
the error message more verbose.

Test: make goma_ctl.py raise Exception and run.
Test: I confirmed verbose error message shown.
Change-Id: I04f9a5ca4808746994c41ddce8a5b22a1975fcc2
Signed-off-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Bug: 124218940
2019-02-13 10:49:10 +09:00
Elliott Hughes
a0f23ad233 Merge "Switch to one-true-awk." 2019-02-13 00:56:35 +00:00
Sasha Smundak
c0c9ef9964 Provide an option to reduce build's verbosity.
If ANDROID_QUIET_BUILD environment variable is set to '1', do not show 20
lines of the environment variables. In addition, for the failing step show
only the step name and output, omitting the command proper (the verbose log
as well as error.log still contains it).
If build succeeds, the output of the build itself consists of a single
line:
```
> m androidmk
[100% NN/NN] <last command>
```

When it fails, the output does not contain sometimes very long command
line:
```
> m androidmk
[ 97% NN/MM] test androidmk
FAILED: <step>
--- FAIL: TestEndToEnd (0.01s)
    androidmk_test.go:1025: failed testcase 'prebuilt_etc_TARGET_OUT_ETC'
        input:
        include $(CLEAR_VARS)
        LOCAL_MODULE := etc.test1
        LOCAL_MODULE_CLASS := ETC
        LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/foo/bar
        include $(BUILD_PREBUILT)

        expected:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",
        }

        got:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",

        }

FAIL
17:50:53 ninja failed with: exit status 1
```
[The related change in build/make/envsetup.sh suppresses timing display
when the same variable is set.]

Change-Id: I4d3c72457de031ff58a324c2fe98f4c1d10f8239
Test: treehugger
2019-02-07 12:21:45 -08:00
Elliott Hughes
cc74d59a80 Switch to one-true-awk.
Test: treehugger
Change-Id: I297d21f5a463c4d3070335c7533972397014f171
2019-02-06 18:06:48 +00:00
Dan Willemsen
01f0a05361 Add BUILD_BROKEN_ENG_DEBUG_TAGS to soong.log
And add a helper script that can help parse the output from
build_test.bash

Test: check for BUILD_BROKEN_ENG_DEBUG_TAGS in soong.log
Test: go run ../build/soong/scripts/build_broken_logs.go *
Change-Id: Idd0fc8b59770dcdbe44eeba262558708a9497f96
2019-02-05 15:15:16 -08:00
Dan Willemsen
dcac078c91 Turn on more warnings during CleanSpec processing
We don't want these to be warnings, may as well be errors instead of
turning off the warnings.

Bug: 123583617
Test: treehugger
Change-Id: I6ca518c9647e712426952cb88bdb044d933b23f2
2019-02-04 20:58:25 -08:00
Treehugger Robot
de53d15b0f Merge "Increase max line length for KatiReader bufio.Scanner" 2019-01-31 01:46:07 +00:00
Colin Cross
75ec1ec13f Increase max line length for KatiReader bufio.Scanner
bufio.Scanner defaults to a relatively small max line length of
64kB.  Kati output may have long lines, use a max length of 2MB
instead.

Fixes: 123590367
Test: m checkbuild
Change-Id: Ia869c5c2526621244f0065736e0d5fd9a088df06
2019-01-30 23:19:23 +00:00
Dan Willemsen
ef661b7f18 Fix runtime panics being suppressed
fatalLog was matching runtime errors, and essentially hiding them.

Test: m blueprint_tools
Change-Id: Ib48e7e142fc096998bc14b21fb717392adcff0ec
2019-01-28 20:00:01 -08:00
Treehugger Robot
ee04139154 Merge "Add --top_level_phony to catch more real-to-phony problems" 2019-01-26 05:08:15 +00:00
Elliott Hughes
48d716c367 Switch cp(1) to toybox.
Test: treehugger
Change-Id: I908e1d6aa2aa7016432ac7ff4abc56e8163bad8a
2019-01-25 22:29:58 +00:00
Dan Willemsen
cc62890866 Add --top_level_phony to catch more real-to-phony problems
As more phony rules move either to the packaging kati run, or soong, the
existing real-to-phony check failed to notice that they were phony
rules.

So I added --top_level_phony to Make that assumes that all leaf nodes
(source files, or nodes generated by another ninja file generator) that
do not have a '/' in them are phony targets.

The existing phony-looks-real check in Kati should prevent these from
being real generated files, and our source tree shouldn't have any
source files used at the root level.

Test: build_test on downstream branches
Change-Id: I77cff84b536c6cd6402307a9aa5a9c273e72c71f
2019-01-24 15:53:06 -08:00
Elliott Hughes
d5fdd3a134 Switch hostname(1) to toybox.
Test: treehugger
Change-Id: I52908a14da86ed8f162150bb48f72e141e6f060c
2019-01-24 15:18:06 -08:00
Elliott Hughes
d2c476cd2d Revert "Switch to toybox hostname(1)."
This reverts commit b115b81095.

Reason for revert: http://b/123123255

Change-Id: I0f2bfebf1189fdc71e093538a8b228dd0e0b6946
2019-01-19 18:55:54 +00:00
Elliott Hughes
44f0d4e1fc Switch to toybox which(1).
Test: treehugger
Change-Id: Ic14692b90c2b1246c1c8f72d00d6e0a5223aaf70
2019-01-19 17:43:06 +00:00
Elliott Hughes
db96fc8744 Switch to toybox pgrep(1), pkill(1), and ps(1).
Test: treehugger
Change-Id: Ic357b317e0233a626c89c38df152c3361ee8ea82
2019-01-19 17:41:53 +00:00
Elliott Hughes
b115b81095 Switch to toybox hostname(1).
Test: treehugger
Change-Id: Ibf89209befc8564b308c4b26bf08cef721938099
2019-01-18 15:11:22 -08:00
Elliott Hughes
46e68636f8 Switch to toybox wc(1).
Test: treehugger
Change-Id: I14d22406c1a462d4075a8538e549a8d0f5b88e8d
2019-01-18 08:28:54 -08:00
Elliott Hughes
c97a91183d Switch to toybox md5sum/sha1sum/sha256sum/sha512sum.
Test: treehugger
Change-Id: Icc8c3d5765d73f191e47adfe0a9d68727d075e50
2019-01-17 12:37:40 -08:00
Elliott Hughes
3aa6533f39 Merge "Switch to toybox mv(1)." 2019-01-17 18:32:08 +00:00
Dan Willemsen
3a4dbd651f Fix nested minijail0 execution
We've got an internal testcase that uses minijail0 inside an Android.mk
rule. That was failing since we turned on the linux sandbox, as /proc
was mounted read-only, which prevented setting up the uid/gid mappings
for a child namespace.

Fixes: 122985455
Test: treehugger & forrest of breaking build
Change-Id: Ia77a91a7f4eeeb8a24e84075d8272287f5087587
2019-01-16 23:04:16 -08:00
Elliott Hughes
6064792393 Switch to toybox mv(1).
Test: treehugger
Change-Id: If4e53cc8d088c7b79399ae8e10601c9b22bd118d
2019-01-16 21:54:30 +00:00
Elliott Hughes
c0cf454a63 Switch to toybox rm(1).
Test: treehugger
Change-Id: Ib31182ca7ce827396476699ac9d1c636cdd324fc
2019-01-16 17:44:29 +00:00
Dan Willemsen
24024eafee Effectively disable network access during the build
This starts a new network namespace without any connections to the
outside.

Bug: 122270019
Test: USE_GOMA=true m libc
Test: treehugger
Test: add rule to use /usr/bin/wget, fails after this change
Change-Id: Iba262025ce0e4e3bef5c34c817cc678d6c61403b
2019-01-15 16:58:27 -08:00
Dan Willemsen
63663c6bc9 Implement linux sandboxing with nsjail
This really only initializes the sandbox, it does not attempt to change
the view of the filesystem, nor does it turn off networking.

Bug: 122270019
Test: m
Test: trigger nsjail check failure; lunch; m; cat out/soong.log
Test: USE_GOMA=true m libc
Change-Id: Ib291072dcee8247c7a15f5b6831295ead6e4fc22
2019-01-15 13:47:31 -08:00
Elliott Hughes
713ae39f83 Move to toybox sort(1).
Test: treehugger
Change-Id: Ide649bbe9a73ab0e51d9bf4242d5c3c9172a85d6
2019-01-14 21:56:58 +00:00
Elliott Hughes
61456d1515 Switch to toybox readlink(1).
Test: treehugger
Change-Id: Ie1f6be5c2f273809a2691e506fc34c6840c942dd
2019-01-14 20:23:10 +00:00
Elliott Hughes
6712c0eb87 Switch to toybox xargs(1).
Test: treehugger
Change-Id: I8db68577498c1f6c0e1fdff951bc8972184b2754
2019-01-14 17:25:29 +00:00
Elliott Hughes
f1f016eec1 Move mktemp(1) to toybox.
Test: clean build
Change-Id: I7b02c7ee8dafb504c90d3af63698a09627556947
2019-01-12 17:51:43 +00:00
Elliott Hughes
c8a454c620 Switch to toybox getconf(1).
Test: treehugger
Change-Id: I7b43673b6a349116dd3e100f5e7eef704244e7ef
2019-01-12 17:40:18 +00:00
Elliott Hughes
fa6c8a1426 Switch to toybox stat(1).
Test: treehugger
Change-Id: Iebda993046935c17b1e37129a530f6eab9166018
2019-01-12 01:25:07 +00:00
Elliott Hughes
84af3624bb Merge "Revert "Move sed(1) to toybox."" 2019-01-12 00:09:12 +00:00
Elliott Hughes
de62ce15bc Revert "Move sed(1) to toybox."
This reverts commit e259a66d12.

Reason for revert: http://b/122744241

Change-Id: Id0e86ab79df2356fe12a2659fe39d02469ed9280
2019-01-12 00:07:40 +00:00
Treehugger Robot
e61fca299f Merge "Move sed(1) to toybox." 2019-01-11 22:53:10 +00:00
Elliott Hughes
cfc095251e Merge "Revert "Switch to toybox patch(1)."" 2019-01-11 22:36:12 +00:00
Elliott Hughes
e44c412220 Revert "Switch to toybox patch(1)."
This reverts commit e6df3b80a6.

Reason for revert: http://b/122739027

Change-Id: I5a55835fbb068e3129a69caf887611cd705613f7
2019-01-11 22:20:42 +00:00
Elliott Hughes
99363f2567 Merge "Switch to toybox patch(1)." 2019-01-11 21:14:17 +00:00
Treehugger Robot
b3fd876d81 Merge "Turn on ninja error for missing dep files" 2019-01-11 20:50:37 +00:00
Elliott Hughes
e259a66d12 Move sed(1) to toybox.
Test: treehugger
Change-Id: I338cffd5de27af0a93d0f379d021da5ccb2e5721
2019-01-11 17:15:36 +00:00
Elliott Hughes
e6df3b80a6 Switch to toybox patch(1).
Test: treehugger
Change-Id: Ia20e4132d5cb0e374ae6d97c57d94dfb5caa2c06
2019-01-11 16:35:40 +00:00
Elliott Hughes
81dd1686aa Merge "Switch to toybox ls(1)." 2019-01-11 16:34:38 +00:00
Yoshisato Yanagisawa
0c517bd9e4 Fatal error on insufficient resource to use Goma in Soong UI.
As suggested in
https://android-review.googlesource.com/c/platform/build/soong/+/839293
I am moving some features in goma.mk to goma.go in Soong UI.

With this CL, let me implement ulimit check to Soong UI.

Test: export USE_GOMA=true
Test: launch aosp_arm-eng
Test: make
Test: the command succeeds if "ulimit -n" and "ulimit -u" are large enough.
Test: Otherwise, it shows error.  I confirmed both cases.
Change-Id: I5d7d5ed71f620302a0d635770d1a51a2baab51fd
Signed-off-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
2019-01-11 13:29:10 +09:00
Elliott Hughes
0b11f6590c Switch to toybox ls(1).
Test: treehugger
Change-Id: I7abb1f657e02ccf2eaaff465adfb28114f66875e
2019-01-10 16:55:58 -08:00
Elliott Hughes
3a1fd8e0a1 Switch to toybox ln(1).
Test: treehugger
Change-Id: Ia52e73e87a6c4648285d2340908013cd8528d221
2019-01-10 21:43:11 +00:00
Elliott Hughes
0bd4ab5334 Merge "Switch to toybox expr(1)." 2019-01-10 21:40:37 +00:00
Treehugger Robot
20a8856c8f Merge "Add BUILD_USERNAME and BUILD_HOSTNAME" 2019-01-10 21:29:47 +00:00