Commit graph

32 commits

Author SHA1 Message Date
Joe Onorato
eadb0fbee0 After a build action fails, stop printing more output.
Often the slow commands (errorprone happens to be particularly bad)
print a lot, so this should make it easier to find the error without
lots of scrolling.

This doesn't attempt to parse the output and re-display errors, so
if a command prints a thousand warnings with one error in the middle,
it'll still be hard to find the error.

Bug: 277114612
Test: cd build/soong/ui/terminal ; go test
Change-Id: I6c8285fc2c6e4fc345de57b2c15bc5e7d46b1d1f
2023-06-24 15:13:49 -07:00
Sasha Smundak
2cec8df3a8 Revert "Suppress all progress messages from Ninja if ANDROID_QUIET_BUILD is set."
Reverts commit 827aead340
The change being reverted is the result of misunderstanding of how the
status system works. It has to be reverted because it would suppress
_all_ error messages from Make. Achieving what this change purports is
is more involved and requires changing the code to separate progress message
stream from application output stream.

Test: run failing build with ANDROID_QUIET_BUILD=tree and observe errors being output
Change-Id: If9148a7fa773ae32fb0870a448e9470560e53900
2022-07-19 08:02:42 -07:00
Dan Willemsen
e2cdecf7ba Strip ANSI codes for messages on simple terminals
We had been doing this for actions, but not messages.

Test: m nothing && TERM=dumb m nothing
Bug: 235414673
Change-Id: I5c6c009490abe1d02a4ae2272bd1831bd7ca9f7f
2022-06-09 03:40:14 +00:00
Sasha Smundak
827aead340 Suppress all progress messages from Ninja if ANDROID_QUIET_BUILD is set.
Test: build with ANDROID_QUIET_BUILD=true, observe no progress messages
Change-Id: I98b9a657affa2ebbf86c9c081d51c31698072fed
2021-12-10 16:42:34 -08:00
Colin Cross
3c0fe0edc0 Add environment variable to force keeping ANSI codes
If SOONG_UI_ANSI_OUTPUT is set to a true value force the simple status
output to keep ANSI codes.  This will allow buildbots to collect logs
with ANSI codes in them.  Smart status output is not affected as it
always keeps ANSI codes.

Bug: 147310922
Test: manual
Change-Id: I0b78ceebb65125b8e8dafb4787816fb679d3eb3e
2021-09-21 12:53:30 -07:00
Colin Cross
72a28a6a13 Disable table mode in smart status if window size is not available
Make smart status output automatically disable table mode when the
window size is not available.

Bug: 147310922
Test: status_test.go
Change-Id: Ic57e1efd742e869bb84b6a86b3014335f5a65d39
2021-09-21 12:36:22 -07:00
Bob Badour
02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
Jaewoong Jung
18aefc1977 Remove unnecessary snake case variables.
Test: m nothing + TreeHugger
Change-Id: I99f7162944daa6c57c6ae4763261e108bb5cb6b1
2020-12-22 12:38:35 -08:00
Colin Cross
405af07859 Revert "Make lots of tests run in parallel"
This reverts commit 323dc60712.

Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests

Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
2020-10-09 18:34:24 -07:00
Colin Cross
323dc60712 Make lots of tests run in parallel
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.

Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
2020-10-06 15:12:22 -07:00
Patrice Arruda
f445ba1788 Inclusive fix: renaming dumb terminal to simple terminal.
Bug: b/161896447
Test: "lunch 1" and "m nothing"
Change-Id: Ifb228c0f0f634477932c9240e57170f7390201f7
2020-07-28 20:25:03 +00:00
Colin Cross
8cc19911b1 smart_status: move channel write outside lock to avoid deadlock
The smartStatusOutput.done channel is read from a goroutine that
locks smartStatusOutput.lock.  If the goroutine that writes to
smartStatusOutput.done is holding smartStatusOutput.lock it can
lead to an AB-BA deadlock.  Call stopActionTableTick from outside
the lock.

Fixes: 143558785
Test: none
Change-Id: I93a10ef9ff16c3953a1c5ccb102b024158358fe4
2019-10-29 15:27:32 -07:00
Colin Cross
c0b9f6b9d2 Force dumpvars into dumb terminal mode and move log files
Force soong_ui in dumpvars mode to use a dumb terminal so that it
doesn't print an empty status table.  Also move the logs to files
that are prefixed with "dumpvars-" so that they don't rotate the
real build logs.

Fixes: 139307849
Test: lunch
Change-Id: If92421af07731fedc9ca42670f309e95922a2224
2019-09-23 14:24:57 -07:00
Colin Cross
f0b987ecba status table: don't write newlines in non-scrolling terminals
Multiple terminals have had issues with writing newlines into the
non-scrolling region, just set the cursor to the beginning of the
next line instead.

Test: m nothing in JediTerm
Change-Id: I2e434f4cc263ca13b82889a79d6a8bb48d084cb3
2019-09-20 15:03:31 -07:00
Colin Cross
bf8f57e2f6 status table: ANSI code rows and columns are 1-based
Don't pass 0 to setCursor or setScrollingMargins.  Passing 0 hasn't
caused any known issues, but 1 is correct.

Test: m nothing
Change-Id: I8735032eb3f1d46df4c41c4ea9ef95b5fd842824
2019-09-20 15:03:11 -07:00
Dan Willemsen
1eee154c3b Default to the status table output
For smart terminals, default to using the status table, using 1/4 of the
terminal height (with a min of 1 and a max of 10).

This behavior can still be overriden to a specific height with
SOONG_UI_TABLE_HEIGHT, and turned off by setting SOONG_UI_TABLE_HEIGHT
to 0.

Test: m   <adjust terminal height, see it shrink when it gets too short>
Test: SOONG_UI_TABLE_HEIGHT=20 m
Test: SOONG_UI_TABLE_HEIGHT=0 m
Change-Id: I224348a29a6e07f168c92cf5514a94bd27e32618
2019-07-30 13:44:03 -07:00
Mikhail Naganov
9d42425f25 Perform TERM check for file writers only
Tests use fakeSmartTerminal, this case must not be affected
by the TERM setting.

Test: TERM=dumb make
Change-Id: Ib926a1c6ba2d8c4117d412fe351d872ccfc1c799
2019-07-02 09:51:42 -07:00
Mikhail Naganov
b073581441 Improve dumb terminal detection
When figuring out terminal type, check whether TERM=dumb is set.

Test: m nothing
Test: TERM=dumb m nothing
Test: m nothing | tee log
Change-Id: Ie13ca99eb84b798644d618ad1c8c1fb8ec8bb049
2019-06-27 10:54:53 -07:00
Colin Cross
5137de0bdd Color long running durations when using action table output
When printing the action table, color the duration of commands that
have been running for 30 seconds yellow, and commands that have been
running for 60 seconds red.

Test: manual
Change-Id: I61cb21b0dae10811d345cb9f62cd07915cfc69ee
2019-06-24 10:49:11 -07:00
Colin Cross
3dac80e519 Support an action table that shows longest running actions
If SOONG_UI_TABLE_HEIGHT is set, enable a new smart terminal display
that prints the normal scrolling build history in the top region of
the screen and an action table of the longest currently running
actions in the bottom region of the screen.  This provides better
visibility into which are the longest running actions and when the
build parallelism is very low.

Test: manual
Change-Id: I677d7b6b008699febd259110d7f9e0f98d80c535
2019-06-13 11:26:06 -07:00
Colin Cross
4355ee64a9 Use SIGWINCH to update terminal size
Instead of reading the terminal size on every status update, register
for SIGWINCH to read and store the size when it changes.

This reapplies I555ad21a31a2c924ab0ca681e0c8f00df42a370a with a fix
for a race condition in TestSmartStatusOutputWidthChange.

Test: status_test.go
Change-Id: If342cb4cc8e4ed57af9e3bb417758348c9c41247
2019-06-13 11:26:06 -07:00
Colin Cross
4bc5e802db Revert "Use SIGWINCH to update terminal size"
This reverts commit 49036be407.

Reason for revert: TestSmartStatusOutputWidthChange is flaky

Change-Id: Ie5231dbbab2887ce4c4697f3fe6e52cfd6f4dd17
2019-06-13 18:10:45 +00:00
Colin Cross
49036be407 Use SIGWINCH to update terminal size
Instead of reading the terminal size on every status update, register
for SIGWINCH to read and store the size when it changes.

Test: status_test.go
Change-Id: I555ad21a31a2c924ab0ca681e0c8f00df42a370a
2019-06-12 21:30:56 -07:00
Colin Cross
00bdfd8476 Make status line bold
Bolding the status line provides differentiation between output
of each command.

Test: status_test.go
Change-Id: I9d46761e69c5af0a0aa86c7921e121cfd2a3fc82
2019-06-12 21:30:56 -07:00
Colin Cross
e0df1a36b2 Move all output through StatusOutput
Write log output through StatusOutput so that the status implementation
can synchronize it with its own output.

Test: status_test.go
Change-Id: I917bdeeea4759a12b6b4aa6d6d86ee18a2771723
2019-06-12 21:30:56 -07:00
Colin Cross
097ed2a37c Remove terminal.Writer
terminal.Writer is now just a wrapper around stdio.Stdout() without
any useful functionality.  Replace it with stdio.Stdout() as an
io.Writer.

Test: ui/terminal/status_test.go
Change-Id: I5bc5476afdca950b505642f0135a3af9d37fbe24
2019-06-12 21:30:56 -07:00
Colin Cross
ce525350f4 Move smart and dumb terminals into separate implementations
Support for smart and dumb terminals are implemented in writer.go,
which makes dumb terminals much more complicated than necessary.
Move smart and dumb terminals into two separate implementations
of StatusOutput, with common code moved into a shared formatter
class.

Test: not yet
Change-Id: I59bbdae479f138b46cd0f03092720a3303e8f0fe
2019-06-12 21:30:56 -07:00
Colin Cross
dde49cb687 Add tests for status output
Test: status_test.go
Change-Id: If3febd8fdacb0e70716d0520a41c982bd6474720
2019-06-12 21:30:56 -07:00
Sasha Smundak
c0c9ef9964 Provide an option to reduce build's verbosity.
If ANDROID_QUIET_BUILD environment variable is set to '1', do not show 20
lines of the environment variables. In addition, for the failing step show
only the step name and output, omitting the command proper (the verbose log
as well as error.log still contains it).
If build succeeds, the output of the build itself consists of a single
line:
```
> m androidmk
[100% NN/NN] <last command>
```

When it fails, the output does not contain sometimes very long command
line:
```
> m androidmk
[ 97% NN/MM] test androidmk
FAILED: <step>
--- FAIL: TestEndToEnd (0.01s)
    androidmk_test.go:1025: failed testcase 'prebuilt_etc_TARGET_OUT_ETC'
        input:
        include $(CLEAR_VARS)
        LOCAL_MODULE := etc.test1
        LOCAL_MODULE_CLASS := ETC
        LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/foo/bar
        include $(BUILD_PREBUILT)

        expected:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",
        }

        got:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",

        }

FAIL
17:50:53 ninja failed with: exit status 1
```
[The related change in build/make/envsetup.sh suppresses timing display
when the same variable is set.]

Change-Id: I4d3c72457de031ff58a324c2fe98f4c1d10f8239
Test: treehugger
2019-02-07 12:21:45 -08:00
Dan Willemsen
77fe361175 Strip escape characters when TERM=dumb
Bug: 111447427
Test: m nothing
Test: TERM=dumb m nothing
Test: m nothing | tee log
Change-Id: I041ec8a60e633393a145996d9ba7b5dae2880b06
2018-07-16 10:56:10 -07:00
Dan Willemsen
f78a73444e Move all status output to stdout
I've noticed a few instances of interleaved status messages in between
lines in a terminal/Writer.Print call on our build servers. Since
there's a lock protecting everything we write, I've got to assume this
is a stdout vs stderr problem. Ninja had always been outputing to
stdout, except for error messages, which are now marked with FAILED:
like failed actions.

Test: m blueprint_tools
Test: m missing
Change-Id: Idf8320d40694abf212c902c63a9703e4440ffb7a
2018-07-12 21:59:49 -07:00
Dan Willemsen
b82471ad6d Add a unified status reporting UI
This adds a new status package that merges the running of "actions"
(ninja calls them edges) of multiple tools into one view of the current
state, and gives that to a number of different outputs.

For inputs:

Kati's output parser has been rewritten (and moved) to map onto the
StartAction/FinishAction API. A byproduct of this is that the build
servers should be able to extract errors from Kati better, since they
look like the errors that Ninja used to write.

Ninja is no longer directly connected to the terminal, but its output is
read via the protobuf frontend API, so it's just another tool whose
output becomes merged together.

multiproduct_kati loses its custom status routines, and uses the common
one instead.

For outputs:

The primary output is the ui/terminal.Status type, which along with
ui/terminal.Writer now controls everything about the terminal output.
Today, this doesn't really change any behaviors, but having all terminal
output going through here allows a more complicated (multi-line / full
window) status display in the future.

The tracer acts as an output of the status package, tracing all the
action start / finish events. This replaces reading the .ninja_log file,
so it now properly handles multiple output files from a single action.

A new rotated log file (out/error.log, or out/dist/logs/error.log) just
contains a description of all of the errors that happened during the
current build.

Another new compressed and rotated log file (out/verbose.log.gz, or
out/dist/logs/verbose.log.gz) contains the full verbose (showcommands)
log of every execution run by the build. Since this is now written on
every build, the showcommands argument is now ignored -- if you want to
get the commands run, look at the log file after the build.

Test: m
Test: <built-in tests>
Test: NINJA_ARGS="-t list" m
Test: check the build.trace.gz
Test: check the new log files
Change-Id: If1d8994890d43ef68f65aa10ddd8e6e06dc7013a
2018-07-12 14:15:31 -07:00