Commit graph

325 commits

Author SHA1 Message Date
Justin Yun
d5f6c8261a Rename product_services to system_ext
Bug: 134359158
Test: build and check if system_ext.img is created
Change-Id: Ice3229baf97a94c24a6eea22e7d4571941d7f843
2019-07-09 08:57:05 +00:00
Treehugger Robot
f67e1bee4a Merge changes Icfc893c8,I40f03fc0
* changes:
  Allow debugging with SOONG_DELVE=<listen addr>
  Print ninja stdout live during the build
2019-07-04 03:34:22 +00:00
Colin Cross
aa812d122c Allow debugging with SOONG_DELVE=<listen addr>
Allow running Soong in a headless delve debugger by passing
SOONG_DELVE=<listen addr> in the environment.

Bug: 80165685
Test: SOONG_DELVE=:1234 m nothing
Change-Id: Icfc893c8a8354a9bbc99112d9c83259cb41906d1
2019-07-03 16:33:31 -07:00
Colin Cross
7b97ecd1f5 Print ninja stdout live during the build
When running ninja stream stdout to the status writer.  This improves
the ninja -d explain behavior, and will also allow Soong to be put
into the ninja console pool letting it print timely output to the
console.

Bug:  80165685
Test: NINJA_ARGS="-d explain" m
Change-Id: I40f03fc01d837ad91d400eae378007a7ce807705
2019-07-03 16:33:10 -07:00
Patrice Arruda
baba9a9be6 soong_ui: Build under a path that contains a symbolic link.
The build directory passed in to soong_ui can contain a
symbolic link. soong_ui was not evaluating the build
directory to retrieve the true path, hence failing to
execute the internal soong.

Fixes: b/135995632
Test: Unit test case and executed the scenario in the bug
      description.

Change-Id: I5779c6aa3f3183810437dbe2b2d4e40acbafb205
2019-07-03 13:03:23 -07:00
Mikhail Naganov
9d42425f25 Perform TERM check for file writers only
Tests use fakeSmartTerminal, this case must not be affected
by the TERM setting.

Test: TERM=dumb make
Change-Id: Ib926a1c6ba2d8c4117d412fe351d872ccfc1c799
2019-07-02 09:51:42 -07:00
Mikhail Naganov
b073581441 Improve dumb terminal detection
When figuring out terminal type, check whether TERM=dumb is set.

Test: m nothing
Test: TERM=dumb m nothing
Test: m nothing | tee log
Change-Id: Ie13ca99eb84b798644d618ad1c8c1fb8ec8bb049
2019-06-27 10:54:53 -07:00
Colin Cross
7b6245388c Compute critical path when ninja finishes
Keep a running map of the critical path to each edge as it
finishes.  At the end of the build, find the edge with the
longest critical path and print the path to out/soong.log.

Test: critical_path_test.go
Change-Id: Ie01d26b068b768156b166bff00cdc3273e8124ca
2019-06-26 11:31:17 -07:00
Treehugger Robot
ba9a51979d Merge "Revert "Switch to toybox tar."" 2019-06-25 05:24:44 +00:00
Elliott Hughes
afeb6cb817 Revert "Switch to toybox tar."
This reverts commit 3b584b7861.

Reason for revert: missing --mode.

Change-Id: I0cde94c38905e26c79c31a03fd0177d7d57daa32
2019-06-25 01:40:42 +00:00
Treehugger Robot
7d7cae85ab Merge "Soong: Add build_error proto raw file to $(OUT_DIR)." 2019-06-24 22:59:33 +00:00
Colin Cross
bbeb2c1a50 Merge "Color long running durations when using action table output" 2019-06-24 21:38:51 +00:00
Elliott Hughes
9b74dfe96c Merge "Switch to toybox tar." 2019-06-24 21:24:53 +00:00
Patrice Arruda
297cebad26 Soong: Add build_error proto raw file to $(OUT_DIR).
Soong_ui produces a build_error raw file to $(OUT_DIR) where
the file contains a list of build action errors. Each build action
error represents an error raised by a command. The build_error file
is populated if there was a build error.

Bug: b/132969697
Test: Ran m for successful build. Introduced a broken build change,
      ran m and verified using printproto that build_error was
      generated successfully.

Change-Id: I690ca1778b4e56f144a3173ba1d16d8494c32c15
2019-06-24 13:02:35 -07:00
Colin Cross
5137de0bdd Color long running durations when using action table output
When printing the action table, color the duration of commands that
have been running for 30 seconds yellow, and commands that have been
running for 60 seconds red.

Test: manual
Change-Id: I61cb21b0dae10811d345cb9f62cd07915cfc69ee
2019-06-24 10:49:11 -07:00
Treehugger Robot
45cdb71290 Merge "Soong: Add BUILD_MODULES build action in soong_ui." 2019-06-21 03:11:34 +00:00
Patrice Arruda
3928206ecc Soong: Add BUILD_MODULES build action in soong_ui.
There was one case that did not work:
  1) Create dir in root source tree
  2) Run "m libc"

The build would fail as it was using the mma logic. A separate
one named BUILD_MODULES was added to allow building specific
modules in any directories using the "m" command.

Bug: b/130049705
Test: unit test cases, ran "m libc" in a tmp directory inside
      of the source tree.

Change-Id: I8d23e685a673a311001ee8edd89bd73b662392dd
2019-06-20 16:35:12 -07:00
Elliott Hughes
63e70530d3 Merge "Allow gzcat as an alias for gzip's -dc mode." 2019-06-20 21:03:11 +00:00
Elliott Hughes
e671f5a397 Allow gzcat as an alias for gzip's -dc mode.
This gets us closer to toybox tar (and toybox gzip/gzcat).

Test: treehugger
Change-Id: If934270439b258eae55cdd3e97ac0e83b1c79c70
2019-06-19 14:02:02 -07:00
Dan Willemsen
c69d715d72 Use prebuilt m4 instead of system m4
Bug: 117561006
Test: treehugger
Change-Id: I588d867d95cc52963ce764018a9c09b26973c8c3
2019-06-19 12:50:32 -07:00
Treehugger Robot
fedc47141e Merge "Support an action table that shows longest running actions" 2019-06-19 06:41:28 +00:00
Treehugger Robot
4b1f4fe138 Merge "Use SIGWINCH to update terminal size" 2019-06-18 21:23:43 +00:00
Treehugger Robot
eec8d3aee3 Merge "Soong: Fix the package name in metrics.proto file" 2019-06-17 23:42:33 +00:00
Elliott Hughes
3b584b7861 Switch to toybox tar.
Test: treehugger
Change-Id: I4c561186c14cc44782a9e5c83d4d9c002addb3f6
2019-06-17 16:39:35 -07:00
Elliott Hughes
2d10e3f54e Merge "Revert "Revert "Switch to toybox timeout.""" 2019-06-17 23:35:16 +00:00
Elliott Hughes
0e868d0391 Revert "Revert "Switch to toybox timeout.""
This reverts commit 50e91c1a7a.

Reason for revert: SIGRTMIN+2 support added.

Change-Id: I3cb251a102e427335315c4931e23a69b95253ee7
2019-06-17 16:45:15 +00:00
Patrice Arruda
0cc5b21516 Soong: Fix the package name in metrics.proto file
The metrics.proto needs to be imported to Google3 in order to unmarshal
the build metrics data from bigstore. Cleaned up the enum names to use
the Camel naming convention and renamed the build_metrics to
soong_metrics to be more specific to soong and to allow quering
soong_metrics files from bigstore.

Bug: b/135280521
Test: lunch andf m -j. Checked if soong_metrics was generated correctly
      by using printproto command.

Change-Id: I998c8d05db592e94a653d6ca32250b80df3c9b21
2019-06-14 15:27:46 -07:00
Treehugger Robot
49cd4954a0 Merge "soong_ui: Add build actions commands in soong_ui." 2019-06-13 23:14:56 +00:00
Patrice Arruda
1384822a63 soong_ui: Add build actions commands in soong_ui.
Add the following build actions {BUILD_MODULES_IN_A_DIRECTORY,
BUILD_MODULES_IN_DIRECTORIES} in soong_ui config so the bash code version of
build commands (m, mm, mma, mmm, mmma) in build/make/envsetup.sh can be deprecated.
This is to allow up to date bug fixes on the build commands.

Bug: b/130049705
Test: Unit test cases
Change-Id: I772db1d4e9c1da5273374d1994eb5e8f17cd52f2
2019-06-13 13:15:12 -07:00
Colin Cross
3dac80e519 Support an action table that shows longest running actions
If SOONG_UI_TABLE_HEIGHT is set, enable a new smart terminal display
that prints the normal scrolling build history in the top region of
the screen and an action table of the longest currently running
actions in the bottom region of the screen.  This provides better
visibility into which are the longest running actions and when the
build parallelism is very low.

Test: manual
Change-Id: I677d7b6b008699febd259110d7f9e0f98d80c535
2019-06-13 11:26:06 -07:00
Colin Cross
4355ee64a9 Use SIGWINCH to update terminal size
Instead of reading the terminal size on every status update, register
for SIGWINCH to read and store the size when it changes.

This reapplies I555ad21a31a2c924ab0ca681e0c8f00df42a370a with a fix
for a race condition in TestSmartStatusOutputWidthChange.

Test: status_test.go
Change-Id: If342cb4cc8e4ed57af9e3bb417758348c9c41247
2019-06-13 11:26:06 -07:00
Colin Cross
4bc5e802db Revert "Use SIGWINCH to update terminal size"
This reverts commit 49036be407.

Reason for revert: TestSmartStatusOutputWidthChange is flaky

Change-Id: Ie5231dbbab2887ce4c4697f3fe6e52cfd6f4dd17
2019-06-13 18:10:45 +00:00
Colin Cross
49036be407 Use SIGWINCH to update terminal size
Instead of reading the terminal size on every status update, register
for SIGWINCH to read and store the size when it changes.

Test: status_test.go
Change-Id: I555ad21a31a2c924ab0ca681e0c8f00df42a370a
2019-06-12 21:30:56 -07:00
Colin Cross
00bdfd8476 Make status line bold
Bolding the status line provides differentiation between output
of each command.

Test: status_test.go
Change-Id: I9d46761e69c5af0a0aa86c7921e121cfd2a3fc82
2019-06-12 21:30:56 -07:00
Colin Cross
e0df1a36b2 Move all output through StatusOutput
Write log output through StatusOutput so that the status implementation
can synchronize it with its own output.

Test: status_test.go
Change-Id: I917bdeeea4759a12b6b4aa6d6d86ee18a2771723
2019-06-12 21:30:56 -07:00
Colin Cross
097ed2a37c Remove terminal.Writer
terminal.Writer is now just a wrapper around stdio.Stdout() without
any useful functionality.  Replace it with stdio.Stdout() as an
io.Writer.

Test: ui/terminal/status_test.go
Change-Id: I5bc5476afdca950b505642f0135a3af9d37fbe24
2019-06-12 21:30:56 -07:00
Colin Cross
ce525350f4 Move smart and dumb terminals into separate implementations
Support for smart and dumb terminals are implemented in writer.go,
which makes dumb terminals much more complicated than necessary.
Move smart and dumb terminals into two separate implementations
of StatusOutput, with common code moved into a shared formatter
class.

Test: not yet
Change-Id: I59bbdae479f138b46cd0f03092720a3303e8f0fe
2019-06-12 21:30:56 -07:00
Colin Cross
dde49cb687 Add tests for status output
Test: status_test.go
Change-Id: If3febd8fdacb0e70716d0520a41c982bd6474720
2019-06-12 21:30:56 -07:00
Elliott Hughes
bc31b9190a Switch to toybox date.
Test: treehugger
Change-Id: I558870092c29946419b9e15c33a6ee8ee7c6088f
2019-06-11 10:09:08 -07:00
Lokesh Gidra
50e91c1a7a Revert "Switch to toybox timeout."
This reverts commit e9f471635f.

Reason for revert: Multiple art-test failures

Change-Id: I550864c23485a438889ab73c7e4a877cd730cabb
2019-06-08 00:46:09 +00:00
Elliott Hughes
c1bfeed81b Switch to toybox sed (again).
The pathological case has been fixed, so let's see if it sticks this
time.

Test: treehugger
Change-Id: I043d823b05dd8796764e008c5e16e3563b80cec8
2019-06-07 13:10:04 -07:00
Elliott Hughes
e9f471635f Switch to toybox timeout.
Test: treehugger
Change-Id: Ieea84deac5d0ccef0f3d4949aa1ba870d406e482
2019-06-06 21:41:48 -07:00
Dan Willemsen
edff034366 Remove BUILD_BROKEN_ANDROIDMK_EXPORTS
Bug: 73959648
Test: treehugger
Change-Id: I932cc3acc6345f8b1ad8e395235489ae8275e2db
2019-05-25 17:13:55 -07:00
Dan Willemsen
7101d44442 Remove BUILD_BROKEN_ENG_DEBUG_TAGS
All users have been removed.

Test: treehugger
Change-Id: I35e55f3ce79c8e68585be120b5dec810563c7a83
Merged-In: I35e55f3ce79c8e68585be120b5dec810563c7a83
2019-05-25 22:48:12 +00:00
Amin Shaikh
de730781d9 Make soong lock file sync time configurable..
- Add SOONG_LOCK_TIMEOUT environmental variable for users to set a
duration (e.g. 1h), which defaults to the current 10 seconds.
- Instead of logging once per second waiting to acquire the lock,
only log once before the first wait

Fixes: 131854539
Test: make; open another tab and make
Change-Id: I0a02a1b071964f74473a5f55034f916526e1b309
2019-05-24 17:10:48 -04:00
Elliott Hughes
18e0d5e0f0 Go back to the host sed until we have a faster prebuilt.
Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I120fe1200216afdb492bd1353848d12a4be1f496
2019-05-09 13:59:01 -07:00
Dan Willemsen
7e52fa9f33 Don't dump BUILD_BROKEN_USES_BUILD_[HOST_]SHARED_TEST_LIBRARY
These have been errors for years, and are now marked as obsolete.

Test: treehugger
Change-Id: I8055ab689ce5085df763378ad40d9b9aaa7ca85b
2019-05-08 12:38:49 -07:00
Treehugger Robot
6ff39230e8 Merge "Support install-clean and data-clean" 2019-05-03 23:38:40 +00:00
Colin Cross
806fd94bed Support install-clean and data-clean
Allow install-clean as an alias for installclean and data-clean as
an alias for dataclean.

Test: m install-clean
Change-Id: I9c97d60572a524ad68caecd26d52bfb987468075
2019-05-03 13:38:57 -07:00
Elliott Hughes
b0e7320fbc Merge "Allow the host expr." 2019-05-03 17:24:00 +00:00