Commit graph

298 commits

Author SHA1 Message Date
Elliott Hughes
2d10e3f54e Merge "Revert "Revert "Switch to toybox timeout.""" 2019-06-17 23:35:16 +00:00
Elliott Hughes
0e868d0391 Revert "Revert "Switch to toybox timeout.""
This reverts commit 50e91c1a7a.

Reason for revert: SIGRTMIN+2 support added.

Change-Id: I3cb251a102e427335315c4931e23a69b95253ee7
2019-06-17 16:45:15 +00:00
Treehugger Robot
49cd4954a0 Merge "soong_ui: Add build actions commands in soong_ui." 2019-06-13 23:14:56 +00:00
Patrice Arruda
1384822a63 soong_ui: Add build actions commands in soong_ui.
Add the following build actions {BUILD_MODULES_IN_A_DIRECTORY,
BUILD_MODULES_IN_DIRECTORIES} in soong_ui config so the bash code version of
build commands (m, mm, mma, mmm, mmma) in build/make/envsetup.sh can be deprecated.
This is to allow up to date bug fixes on the build commands.

Bug: b/130049705
Test: Unit test cases
Change-Id: I772db1d4e9c1da5273374d1994eb5e8f17cd52f2
2019-06-13 13:15:12 -07:00
Colin Cross
4bc5e802db Revert "Use SIGWINCH to update terminal size"
This reverts commit 49036be407.

Reason for revert: TestSmartStatusOutputWidthChange is flaky

Change-Id: Ie5231dbbab2887ce4c4697f3fe6e52cfd6f4dd17
2019-06-13 18:10:45 +00:00
Colin Cross
49036be407 Use SIGWINCH to update terminal size
Instead of reading the terminal size on every status update, register
for SIGWINCH to read and store the size when it changes.

Test: status_test.go
Change-Id: I555ad21a31a2c924ab0ca681e0c8f00df42a370a
2019-06-12 21:30:56 -07:00
Colin Cross
00bdfd8476 Make status line bold
Bolding the status line provides differentiation between output
of each command.

Test: status_test.go
Change-Id: I9d46761e69c5af0a0aa86c7921e121cfd2a3fc82
2019-06-12 21:30:56 -07:00
Colin Cross
e0df1a36b2 Move all output through StatusOutput
Write log output through StatusOutput so that the status implementation
can synchronize it with its own output.

Test: status_test.go
Change-Id: I917bdeeea4759a12b6b4aa6d6d86ee18a2771723
2019-06-12 21:30:56 -07:00
Colin Cross
097ed2a37c Remove terminal.Writer
terminal.Writer is now just a wrapper around stdio.Stdout() without
any useful functionality.  Replace it with stdio.Stdout() as an
io.Writer.

Test: ui/terminal/status_test.go
Change-Id: I5bc5476afdca950b505642f0135a3af9d37fbe24
2019-06-12 21:30:56 -07:00
Colin Cross
ce525350f4 Move smart and dumb terminals into separate implementations
Support for smart and dumb terminals are implemented in writer.go,
which makes dumb terminals much more complicated than necessary.
Move smart and dumb terminals into two separate implementations
of StatusOutput, with common code moved into a shared formatter
class.

Test: not yet
Change-Id: I59bbdae479f138b46cd0f03092720a3303e8f0fe
2019-06-12 21:30:56 -07:00
Colin Cross
dde49cb687 Add tests for status output
Test: status_test.go
Change-Id: If3febd8fdacb0e70716d0520a41c982bd6474720
2019-06-12 21:30:56 -07:00
Elliott Hughes
bc31b9190a Switch to toybox date.
Test: treehugger
Change-Id: I558870092c29946419b9e15c33a6ee8ee7c6088f
2019-06-11 10:09:08 -07:00
Lokesh Gidra
50e91c1a7a Revert "Switch to toybox timeout."
This reverts commit e9f471635f.

Reason for revert: Multiple art-test failures

Change-Id: I550864c23485a438889ab73c7e4a877cd730cabb
2019-06-08 00:46:09 +00:00
Elliott Hughes
c1bfeed81b Switch to toybox sed (again).
The pathological case has been fixed, so let's see if it sticks this
time.

Test: treehugger
Change-Id: I043d823b05dd8796764e008c5e16e3563b80cec8
2019-06-07 13:10:04 -07:00
Elliott Hughes
e9f471635f Switch to toybox timeout.
Test: treehugger
Change-Id: Ieea84deac5d0ccef0f3d4949aa1ba870d406e482
2019-06-06 21:41:48 -07:00
Dan Willemsen
edff034366 Remove BUILD_BROKEN_ANDROIDMK_EXPORTS
Bug: 73959648
Test: treehugger
Change-Id: I932cc3acc6345f8b1ad8e395235489ae8275e2db
2019-05-25 17:13:55 -07:00
Dan Willemsen
7101d44442 Remove BUILD_BROKEN_ENG_DEBUG_TAGS
All users have been removed.

Test: treehugger
Change-Id: I35e55f3ce79c8e68585be120b5dec810563c7a83
Merged-In: I35e55f3ce79c8e68585be120b5dec810563c7a83
2019-05-25 22:48:12 +00:00
Amin Shaikh
de730781d9 Make soong lock file sync time configurable..
- Add SOONG_LOCK_TIMEOUT environmental variable for users to set a
duration (e.g. 1h), which defaults to the current 10 seconds.
- Instead of logging once per second waiting to acquire the lock,
only log once before the first wait

Fixes: 131854539
Test: make; open another tab and make
Change-Id: I0a02a1b071964f74473a5f55034f916526e1b309
2019-05-24 17:10:48 -04:00
Elliott Hughes
18e0d5e0f0 Go back to the host sed until we have a faster prebuilt.
Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I120fe1200216afdb492bd1353848d12a4be1f496
2019-05-09 13:59:01 -07:00
Dan Willemsen
7e52fa9f33 Don't dump BUILD_BROKEN_USES_BUILD_[HOST_]SHARED_TEST_LIBRARY
These have been errors for years, and are now marked as obsolete.

Test: treehugger
Change-Id: I8055ab689ce5085df763378ad40d9b9aaa7ca85b
2019-05-08 12:38:49 -07:00
Treehugger Robot
6ff39230e8 Merge "Support install-clean and data-clean" 2019-05-03 23:38:40 +00:00
Colin Cross
806fd94bed Support install-clean and data-clean
Allow install-clean as an alias for installclean and data-clean as
an alias for dataclean.

Test: m install-clean
Change-Id: I9c97d60572a524ad68caecd26d52bfb987468075
2019-05-03 13:38:57 -07:00
Elliott Hughes
b0e7320fbc Merge "Allow the host expr." 2019-05-03 17:24:00 +00:00
Elliott Hughes
3a653f419b Allow the host expr.
(Kernel builds require GNU unary + extension.)

Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I404db44d2ad0503d571d6f554a20445ab36aa7de
2019-05-01 12:52:25 -07:00
Elliott Hughes
6b3a0ddf0b xz: force use of the prebuilt.
Also comment why we can't yet do the same for bzip2.

Test: treehugger
Change-Id: I34d5bc0076f5790fcb01c185ef6d08824a4f6142
2019-04-30 22:20:43 -07:00
Elliott Hughes
3d87302a6d Revert "bzip2/xz: use the prebuilts."
This reverts commit e114b3f0a9.

Build broke as follows:

out/soong/host/linux-x86/bin/sbox --sandbox-path out/soong/.temp --output-root out/soong/.intermediates/system/update_engine/ue_unittest_disk_imgs/gen -c 'tar -jxf system/update_engine/sample_images/sample_images.tar.bz2 -C __SBOX_OUT_DIR__/gen disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img' __SBOX_OUT_DIR__/gen/disk_ext2_1k.img __SBOX_OUT_DIR__/gen/disk_ext2_4k.img __SBOX_OUT_DIR__/gen/disk_ext2_4k_empty.img __SBOX_OUT_DIR__/gen/disk_ext2_unittest.img 
"bzip2" is not allowed to be used. See https://android.googlesource.com/platform/build/+/master/Changes.md#PATH_Tools for more information. 
tar: Child returned status 1 
tar: Error is not recoverable: exiting now 
sbox command (tar -jxf system/update_engine/sample_images/sample_images.tar.bz2 -C out/soong/.temp/sbox849645882/gen disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img) failed with err "exit status 2" 

Change-Id: I3fe7100886ea3b748f5a6b5fc6d2811391bb72a8
2019-04-30 21:55:23 +00:00
Elliott Hughes
e114b3f0a9 bzip2/xz: use the prebuilts.
We added the prebuilts a while ago, but forgot to clean this up.

Test: treehugger
Change-Id: I8920f0c9d77951332a663750102ceeabcb565970
2019-04-29 16:23:08 -07:00
Dan Willemsen
2fbb7571bf Obsolete BUILD_BROKEN_DUP_COPY_HEADERS
There are no remaining users.

Test: treehugger
Change-Id: If8719480f73a8fa009d9733479390d2abd4021b1
2019-04-19 11:22:27 -07:00
Dan Willemsen
6097746769 Obsolete BUILD_BROKEN_PHONY_TARGETS
There are no remaining users.

Also mark BUILD_BROKEN_ANDROIDMK_EXPORTS and BUILD_BROKEN_ENG_DEBUG_TAGS
as deprecated in scripts/build_broken_logs.go

Test: treehugger
Change-Id: If7892bef1b9001f12a99565f886b395cf1985e70
2019-04-19 11:16:52 -07:00
Dan Willemsen
368e55671b Dump make deprecation status to logs
So that the output of build_test can be parsed to understand which
devices need changes.

Test: read out/soong.log
Change-Id: I40b25527ef72e3f646c0cfe62114b452fa3f27f6
2019-04-17 14:44:33 -07:00
Dan Willemsen
25e6f09c06 Add BUILD_BROKEN_USES_NETWORK
Some people apparently still talk to the network during their build.
Allow this temporarily with a BUILD_BROKEN_USES_NETWORK check.

Bug: 129992021
Test: attempt to talk to the network during the build with and without
      this flag.
Change-Id: Ifb967c656aa24c4599e7232d0f1b5a303b5bac52
2019-04-09 10:23:49 -07:00
Elliott Hughes
c2dbadc363 Add toybox seq.
Bug: https://issuetracker.google.com/129992021
Test: builds
Change-Id: I63d5b3fd1c71b771e995638094ee4310435105ee
2019-04-08 09:19:45 -07:00
Colin Cross
1aeb049a54 Panic if logging is attempted after Close
Attempt to catch places where logs are truncated by panicing if
logging is attempted after Close.

Test: m nothing
Change-Id: If670f20d08832ed65b63af5589b548e9815f2f0d
2019-03-21 21:17:04 -07:00
Colin Cross
b98d3bcf4e Wait for ninja proto processing to finish when exiting
Wait for the ninja proto processing goroutine to notice the fifo
has closed and exit before continuing.

Test: m nothing
Change-Id: I8cf5f3b8bf6a91496c6d2bbbd3e811eb7f0c9d21
2019-03-21 21:17:04 -07:00
Steven Moreland
0002028c09 dumpvars += BOARD_VNDK_VERSION
Dump variable in log to track remaining targets which don't set it.

Bug: 122954981
Test: $ grep VNDK out/soong.log
2019/03/07 09:22:04.235604 build/soong/ui/build/dumpvars.go:109: BOARD_VNDK_VERSION current

Change-Id: I34146c752a4bf0d799374bb93cc0d75b0f1b048c
2019-03-07 09:28:35 -08:00
Dan Willemsen
0f0214699e Use python / python2.7 prebuilts
Bug: 117811537
Test: treehugger
Change-Id: I35b335edcfa202a45f55f588a643c62c2ced0f26
2019-02-18 19:34:00 +00:00
Elliott Hughes
47de2a2f52 Switch sed(1) to toybox.
Test: treehugger
Change-Id: I575ae9db8d94cbec811d0029691df98373f8fe35
2019-02-15 15:49:12 -08:00
Dan Willemsen
733547d522 Use prebuilt awk on Darwin too
Test: build on mac
Change-Id: I25d21c0df0b9023d6c43b11dde60e6066f107576
2019-02-14 20:11:26 -08:00
Dan Willemsen
9121973130 Stop calling our host $PATH prebuilts toybox
There's more than just toybox in this group now, so let's rename our
variables to something closer to the desired behavior, rather than the
first user.

Test: treehugger
Change-Id: I76d4407792061c8110b194cfe73f1ddc84dbc22f
2019-02-14 20:07:02 -08:00
Yoshisato Yanagisawa
02eaccafbd More verbose message on goma_ctl.py ensure_start failure.
For ease of investigating the issue with error reports, let me make
the error message more verbose.

Test: make goma_ctl.py raise Exception and run.
Test: I confirmed verbose error message shown.
Change-Id: I04f9a5ca4808746994c41ddce8a5b22a1975fcc2
Signed-off-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Bug: 124218940
2019-02-13 10:49:10 +09:00
Elliott Hughes
a0f23ad233 Merge "Switch to one-true-awk." 2019-02-13 00:56:35 +00:00
Sasha Smundak
c0c9ef9964 Provide an option to reduce build's verbosity.
If ANDROID_QUIET_BUILD environment variable is set to '1', do not show 20
lines of the environment variables. In addition, for the failing step show
only the step name and output, omitting the command proper (the verbose log
as well as error.log still contains it).
If build succeeds, the output of the build itself consists of a single
line:
```
> m androidmk
[100% NN/NN] <last command>
```

When it fails, the output does not contain sometimes very long command
line:
```
> m androidmk
[ 97% NN/MM] test androidmk
FAILED: <step>
--- FAIL: TestEndToEnd (0.01s)
    androidmk_test.go:1025: failed testcase 'prebuilt_etc_TARGET_OUT_ETC'
        input:
        include $(CLEAR_VARS)
        LOCAL_MODULE := etc.test1
        LOCAL_MODULE_CLASS := ETC
        LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/foo/bar
        include $(BUILD_PREBUILT)

        expected:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",
        }

        got:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",

        }

FAIL
17:50:53 ninja failed with: exit status 1
```
[The related change in build/make/envsetup.sh suppresses timing display
when the same variable is set.]

Change-Id: I4d3c72457de031ff58a324c2fe98f4c1d10f8239
Test: treehugger
2019-02-07 12:21:45 -08:00
Elliott Hughes
cc74d59a80 Switch to one-true-awk.
Test: treehugger
Change-Id: I297d21f5a463c4d3070335c7533972397014f171
2019-02-06 18:06:48 +00:00
Dan Willemsen
01f0a05361 Add BUILD_BROKEN_ENG_DEBUG_TAGS to soong.log
And add a helper script that can help parse the output from
build_test.bash

Test: check for BUILD_BROKEN_ENG_DEBUG_TAGS in soong.log
Test: go run ../build/soong/scripts/build_broken_logs.go *
Change-Id: Idd0fc8b59770dcdbe44eeba262558708a9497f96
2019-02-05 15:15:16 -08:00
Dan Willemsen
dcac078c91 Turn on more warnings during CleanSpec processing
We don't want these to be warnings, may as well be errors instead of
turning off the warnings.

Bug: 123583617
Test: treehugger
Change-Id: I6ca518c9647e712426952cb88bdb044d933b23f2
2019-02-04 20:58:25 -08:00
Treehugger Robot
de53d15b0f Merge "Increase max line length for KatiReader bufio.Scanner" 2019-01-31 01:46:07 +00:00
Colin Cross
75ec1ec13f Increase max line length for KatiReader bufio.Scanner
bufio.Scanner defaults to a relatively small max line length of
64kB.  Kati output may have long lines, use a max length of 2MB
instead.

Fixes: 123590367
Test: m checkbuild
Change-Id: Ia869c5c2526621244f0065736e0d5fd9a088df06
2019-01-30 23:19:23 +00:00
Dan Willemsen
ef661b7f18 Fix runtime panics being suppressed
fatalLog was matching runtime errors, and essentially hiding them.

Test: m blueprint_tools
Change-Id: Ib48e7e142fc096998bc14b21fb717392adcff0ec
2019-01-28 20:00:01 -08:00
Treehugger Robot
ee04139154 Merge "Add --top_level_phony to catch more real-to-phony problems" 2019-01-26 05:08:15 +00:00
Elliott Hughes
48d716c367 Switch cp(1) to toybox.
Test: treehugger
Change-Id: I908e1d6aa2aa7016432ac7ff4abc56e8163bad8a
2019-01-25 22:29:58 +00:00