Commit graph

285 commits

Author SHA1 Message Date
Elliott Hughes
9b74dfe96c Merge "Switch to toybox tar." 2019-06-24 21:24:53 +00:00
Treehugger Robot
45cdb71290 Merge "Soong: Add BUILD_MODULES build action in soong_ui." 2019-06-21 03:11:34 +00:00
Patrice Arruda
3928206ecc Soong: Add BUILD_MODULES build action in soong_ui.
There was one case that did not work:
  1) Create dir in root source tree
  2) Run "m libc"

The build would fail as it was using the mma logic. A separate
one named BUILD_MODULES was added to allow building specific
modules in any directories using the "m" command.

Bug: b/130049705
Test: unit test cases, ran "m libc" in a tmp directory inside
      of the source tree.

Change-Id: I8d23e685a673a311001ee8edd89bd73b662392dd
2019-06-20 16:35:12 -07:00
Elliott Hughes
63e70530d3 Merge "Allow gzcat as an alias for gzip's -dc mode." 2019-06-20 21:03:11 +00:00
Elliott Hughes
e671f5a397 Allow gzcat as an alias for gzip's -dc mode.
This gets us closer to toybox tar (and toybox gzip/gzcat).

Test: treehugger
Change-Id: If934270439b258eae55cdd3e97ac0e83b1c79c70
2019-06-19 14:02:02 -07:00
Dan Willemsen
c69d715d72 Use prebuilt m4 instead of system m4
Bug: 117561006
Test: treehugger
Change-Id: I588d867d95cc52963ce764018a9c09b26973c8c3
2019-06-19 12:50:32 -07:00
Treehugger Robot
eec8d3aee3 Merge "Soong: Fix the package name in metrics.proto file" 2019-06-17 23:42:33 +00:00
Elliott Hughes
3b584b7861 Switch to toybox tar.
Test: treehugger
Change-Id: I4c561186c14cc44782a9e5c83d4d9c002addb3f6
2019-06-17 16:39:35 -07:00
Elliott Hughes
2d10e3f54e Merge "Revert "Revert "Switch to toybox timeout.""" 2019-06-17 23:35:16 +00:00
Elliott Hughes
0e868d0391 Revert "Revert "Switch to toybox timeout.""
This reverts commit 50e91c1a7a.

Reason for revert: SIGRTMIN+2 support added.

Change-Id: I3cb251a102e427335315c4931e23a69b95253ee7
2019-06-17 16:45:15 +00:00
Patrice Arruda
0cc5b21516 Soong: Fix the package name in metrics.proto file
The metrics.proto needs to be imported to Google3 in order to unmarshal
the build metrics data from bigstore. Cleaned up the enum names to use
the Camel naming convention and renamed the build_metrics to
soong_metrics to be more specific to soong and to allow quering
soong_metrics files from bigstore.

Bug: b/135280521
Test: lunch andf m -j. Checked if soong_metrics was generated correctly
      by using printproto command.

Change-Id: I998c8d05db592e94a653d6ca32250b80df3c9b21
2019-06-14 15:27:46 -07:00
Treehugger Robot
49cd4954a0 Merge "soong_ui: Add build actions commands in soong_ui." 2019-06-13 23:14:56 +00:00
Patrice Arruda
1384822a63 soong_ui: Add build actions commands in soong_ui.
Add the following build actions {BUILD_MODULES_IN_A_DIRECTORY,
BUILD_MODULES_IN_DIRECTORIES} in soong_ui config so the bash code version of
build commands (m, mm, mma, mmm, mmma) in build/make/envsetup.sh can be deprecated.
This is to allow up to date bug fixes on the build commands.

Bug: b/130049705
Test: Unit test cases
Change-Id: I772db1d4e9c1da5273374d1994eb5e8f17cd52f2
2019-06-13 13:15:12 -07:00
Colin Cross
097ed2a37c Remove terminal.Writer
terminal.Writer is now just a wrapper around stdio.Stdout() without
any useful functionality.  Replace it with stdio.Stdout() as an
io.Writer.

Test: ui/terminal/status_test.go
Change-Id: I5bc5476afdca950b505642f0135a3af9d37fbe24
2019-06-12 21:30:56 -07:00
Elliott Hughes
bc31b9190a Switch to toybox date.
Test: treehugger
Change-Id: I558870092c29946419b9e15c33a6ee8ee7c6088f
2019-06-11 10:09:08 -07:00
Lokesh Gidra
50e91c1a7a Revert "Switch to toybox timeout."
This reverts commit e9f471635f.

Reason for revert: Multiple art-test failures

Change-Id: I550864c23485a438889ab73c7e4a877cd730cabb
2019-06-08 00:46:09 +00:00
Elliott Hughes
c1bfeed81b Switch to toybox sed (again).
The pathological case has been fixed, so let's see if it sticks this
time.

Test: treehugger
Change-Id: I043d823b05dd8796764e008c5e16e3563b80cec8
2019-06-07 13:10:04 -07:00
Elliott Hughes
e9f471635f Switch to toybox timeout.
Test: treehugger
Change-Id: Ieea84deac5d0ccef0f3d4949aa1ba870d406e482
2019-06-06 21:41:48 -07:00
Dan Willemsen
edff034366 Remove BUILD_BROKEN_ANDROIDMK_EXPORTS
Bug: 73959648
Test: treehugger
Change-Id: I932cc3acc6345f8b1ad8e395235489ae8275e2db
2019-05-25 17:13:55 -07:00
Dan Willemsen
7101d44442 Remove BUILD_BROKEN_ENG_DEBUG_TAGS
All users have been removed.

Test: treehugger
Change-Id: I35e55f3ce79c8e68585be120b5dec810563c7a83
Merged-In: I35e55f3ce79c8e68585be120b5dec810563c7a83
2019-05-25 22:48:12 +00:00
Amin Shaikh
de730781d9 Make soong lock file sync time configurable..
- Add SOONG_LOCK_TIMEOUT environmental variable for users to set a
duration (e.g. 1h), which defaults to the current 10 seconds.
- Instead of logging once per second waiting to acquire the lock,
only log once before the first wait

Fixes: 131854539
Test: make; open another tab and make
Change-Id: I0a02a1b071964f74473a5f55034f916526e1b309
2019-05-24 17:10:48 -04:00
Elliott Hughes
18e0d5e0f0 Go back to the host sed until we have a faster prebuilt.
Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I120fe1200216afdb492bd1353848d12a4be1f496
2019-05-09 13:59:01 -07:00
Dan Willemsen
7e52fa9f33 Don't dump BUILD_BROKEN_USES_BUILD_[HOST_]SHARED_TEST_LIBRARY
These have been errors for years, and are now marked as obsolete.

Test: treehugger
Change-Id: I8055ab689ce5085df763378ad40d9b9aaa7ca85b
2019-05-08 12:38:49 -07:00
Treehugger Robot
6ff39230e8 Merge "Support install-clean and data-clean" 2019-05-03 23:38:40 +00:00
Colin Cross
806fd94bed Support install-clean and data-clean
Allow install-clean as an alias for installclean and data-clean as
an alias for dataclean.

Test: m install-clean
Change-Id: I9c97d60572a524ad68caecd26d52bfb987468075
2019-05-03 13:38:57 -07:00
Elliott Hughes
b0e7320fbc Merge "Allow the host expr." 2019-05-03 17:24:00 +00:00
Elliott Hughes
3a653f419b Allow the host expr.
(Kernel builds require GNU unary + extension.)

Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I404db44d2ad0503d571d6f554a20445ab36aa7de
2019-05-01 12:52:25 -07:00
Elliott Hughes
6b3a0ddf0b xz: force use of the prebuilt.
Also comment why we can't yet do the same for bzip2.

Test: treehugger
Change-Id: I34d5bc0076f5790fcb01c185ef6d08824a4f6142
2019-04-30 22:20:43 -07:00
Elliott Hughes
3d87302a6d Revert "bzip2/xz: use the prebuilts."
This reverts commit e114b3f0a9.

Build broke as follows:

out/soong/host/linux-x86/bin/sbox --sandbox-path out/soong/.temp --output-root out/soong/.intermediates/system/update_engine/ue_unittest_disk_imgs/gen -c 'tar -jxf system/update_engine/sample_images/sample_images.tar.bz2 -C __SBOX_OUT_DIR__/gen disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img' __SBOX_OUT_DIR__/gen/disk_ext2_1k.img __SBOX_OUT_DIR__/gen/disk_ext2_4k.img __SBOX_OUT_DIR__/gen/disk_ext2_4k_empty.img __SBOX_OUT_DIR__/gen/disk_ext2_unittest.img 
"bzip2" is not allowed to be used. See https://android.googlesource.com/platform/build/+/master/Changes.md#PATH_Tools for more information. 
tar: Child returned status 1 
tar: Error is not recoverable: exiting now 
sbox command (tar -jxf system/update_engine/sample_images/sample_images.tar.bz2 -C out/soong/.temp/sbox849645882/gen disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img) failed with err "exit status 2" 

Change-Id: I3fe7100886ea3b748f5a6b5fc6d2811391bb72a8
2019-04-30 21:55:23 +00:00
Elliott Hughes
e114b3f0a9 bzip2/xz: use the prebuilts.
We added the prebuilts a while ago, but forgot to clean this up.

Test: treehugger
Change-Id: I8920f0c9d77951332a663750102ceeabcb565970
2019-04-29 16:23:08 -07:00
Dan Willemsen
2fbb7571bf Obsolete BUILD_BROKEN_DUP_COPY_HEADERS
There are no remaining users.

Test: treehugger
Change-Id: If8719480f73a8fa009d9733479390d2abd4021b1
2019-04-19 11:22:27 -07:00
Dan Willemsen
6097746769 Obsolete BUILD_BROKEN_PHONY_TARGETS
There are no remaining users.

Also mark BUILD_BROKEN_ANDROIDMK_EXPORTS and BUILD_BROKEN_ENG_DEBUG_TAGS
as deprecated in scripts/build_broken_logs.go

Test: treehugger
Change-Id: If7892bef1b9001f12a99565f886b395cf1985e70
2019-04-19 11:16:52 -07:00
Dan Willemsen
368e55671b Dump make deprecation status to logs
So that the output of build_test can be parsed to understand which
devices need changes.

Test: read out/soong.log
Change-Id: I40b25527ef72e3f646c0cfe62114b452fa3f27f6
2019-04-17 14:44:33 -07:00
Dan Willemsen
25e6f09c06 Add BUILD_BROKEN_USES_NETWORK
Some people apparently still talk to the network during their build.
Allow this temporarily with a BUILD_BROKEN_USES_NETWORK check.

Bug: 129992021
Test: attempt to talk to the network during the build with and without
      this flag.
Change-Id: Ifb967c656aa24c4599e7232d0f1b5a303b5bac52
2019-04-09 10:23:49 -07:00
Elliott Hughes
c2dbadc363 Add toybox seq.
Bug: https://issuetracker.google.com/129992021
Test: builds
Change-Id: I63d5b3fd1c71b771e995638094ee4310435105ee
2019-04-08 09:19:45 -07:00
Colin Cross
b98d3bcf4e Wait for ninja proto processing to finish when exiting
Wait for the ninja proto processing goroutine to notice the fifo
has closed and exit before continuing.

Test: m nothing
Change-Id: I8cf5f3b8bf6a91496c6d2bbbd3e811eb7f0c9d21
2019-03-21 21:17:04 -07:00
Steven Moreland
0002028c09 dumpvars += BOARD_VNDK_VERSION
Dump variable in log to track remaining targets which don't set it.

Bug: 122954981
Test: $ grep VNDK out/soong.log
2019/03/07 09:22:04.235604 build/soong/ui/build/dumpvars.go:109: BOARD_VNDK_VERSION current

Change-Id: I34146c752a4bf0d799374bb93cc0d75b0f1b048c
2019-03-07 09:28:35 -08:00
Dan Willemsen
0f0214699e Use python / python2.7 prebuilts
Bug: 117811537
Test: treehugger
Change-Id: I35b335edcfa202a45f55f588a643c62c2ced0f26
2019-02-18 19:34:00 +00:00
Elliott Hughes
47de2a2f52 Switch sed(1) to toybox.
Test: treehugger
Change-Id: I575ae9db8d94cbec811d0029691df98373f8fe35
2019-02-15 15:49:12 -08:00
Dan Willemsen
733547d522 Use prebuilt awk on Darwin too
Test: build on mac
Change-Id: I25d21c0df0b9023d6c43b11dde60e6066f107576
2019-02-14 20:11:26 -08:00
Dan Willemsen
9121973130 Stop calling our host $PATH prebuilts toybox
There's more than just toybox in this group now, so let's rename our
variables to something closer to the desired behavior, rather than the
first user.

Test: treehugger
Change-Id: I76d4407792061c8110b194cfe73f1ddc84dbc22f
2019-02-14 20:07:02 -08:00
Yoshisato Yanagisawa
02eaccafbd More verbose message on goma_ctl.py ensure_start failure.
For ease of investigating the issue with error reports, let me make
the error message more verbose.

Test: make goma_ctl.py raise Exception and run.
Test: I confirmed verbose error message shown.
Change-Id: I04f9a5ca4808746994c41ddce8a5b22a1975fcc2
Signed-off-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Bug: 124218940
2019-02-13 10:49:10 +09:00
Elliott Hughes
a0f23ad233 Merge "Switch to one-true-awk." 2019-02-13 00:56:35 +00:00
Sasha Smundak
c0c9ef9964 Provide an option to reduce build's verbosity.
If ANDROID_QUIET_BUILD environment variable is set to '1', do not show 20
lines of the environment variables. In addition, for the failing step show
only the step name and output, omitting the command proper (the verbose log
as well as error.log still contains it).
If build succeeds, the output of the build itself consists of a single
line:
```
> m androidmk
[100% NN/NN] <last command>
```

When it fails, the output does not contain sometimes very long command
line:
```
> m androidmk
[ 97% NN/MM] test androidmk
FAILED: <step>
--- FAIL: TestEndToEnd (0.01s)
    androidmk_test.go:1025: failed testcase 'prebuilt_etc_TARGET_OUT_ETC'
        input:
        include $(CLEAR_VARS)
        LOCAL_MODULE := etc.test1
        LOCAL_MODULE_CLASS := ETC
        LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/foo/bar
        include $(BUILD_PREBUILT)

        expected:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",
        }

        got:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",

        }

FAIL
17:50:53 ninja failed with: exit status 1
```
[The related change in build/make/envsetup.sh suppresses timing display
when the same variable is set.]

Change-Id: I4d3c72457de031ff58a324c2fe98f4c1d10f8239
Test: treehugger
2019-02-07 12:21:45 -08:00
Elliott Hughes
cc74d59a80 Switch to one-true-awk.
Test: treehugger
Change-Id: I297d21f5a463c4d3070335c7533972397014f171
2019-02-06 18:06:48 +00:00
Dan Willemsen
01f0a05361 Add BUILD_BROKEN_ENG_DEBUG_TAGS to soong.log
And add a helper script that can help parse the output from
build_test.bash

Test: check for BUILD_BROKEN_ENG_DEBUG_TAGS in soong.log
Test: go run ../build/soong/scripts/build_broken_logs.go *
Change-Id: Idd0fc8b59770dcdbe44eeba262558708a9497f96
2019-02-05 15:15:16 -08:00
Dan Willemsen
dcac078c91 Turn on more warnings during CleanSpec processing
We don't want these to be warnings, may as well be errors instead of
turning off the warnings.

Bug: 123583617
Test: treehugger
Change-Id: I6ca518c9647e712426952cb88bdb044d933b23f2
2019-02-04 20:58:25 -08:00
Treehugger Robot
ee04139154 Merge "Add --top_level_phony to catch more real-to-phony problems" 2019-01-26 05:08:15 +00:00
Elliott Hughes
48d716c367 Switch cp(1) to toybox.
Test: treehugger
Change-Id: I908e1d6aa2aa7016432ac7ff4abc56e8163bad8a
2019-01-25 22:29:58 +00:00
Dan Willemsen
cc62890866 Add --top_level_phony to catch more real-to-phony problems
As more phony rules move either to the packaging kati run, or soong, the
existing real-to-phony check failed to notice that they were phony
rules.

So I added --top_level_phony to Make that assumes that all leaf nodes
(source files, or nodes generated by another ninja file generator) that
do not have a '/' in them are phony targets.

The existing phony-looks-real check in Kati should prevent these from
being real generated files, and our source tree shouldn't have any
source files used at the root level.

Test: build_test on downstream branches
Change-Id: I77cff84b536c6cd6402307a9aa5a9c273e72c71f
2019-01-24 15:53:06 -08:00