Commit graph

15 commits

Author SHA1 Message Date
Liz Kammer
ca9cb2e5bd Include bazel-status of build in metrics
Currently within metrics we cannot distinguish between Bazel-involved
configurations:
  Bazel as Ninja vs Ninja as Ninja builds
  Bazel mixed builds vs non-mixed builds

Add these to the build configuration in order to distinguish between and
track them independently.

Test: go soong tests
Test: USE_BAZEL_ANALYSIS=1 m nothing and verify out/soong_metrics
Bug: 193672397
Change-Id: I94ce56c1a98a76d929598d9fe3070df16ffa1376
2021-07-14 17:48:04 -04:00
Patrice Arruda
b7cf9ba175 Add tracer to runBazel function.
Added a tracer named "bazel" in runBazel function to capture topline
metrics when running bazel.

Bug: b/173028918
Test: * USE_BAZEL=1 m nothing and checked soong_metrics file using
        printproto command to verify that bazel topline metrics
	were captured.
Change-Id: I5cb0728a4da48545dc701cc79627c318d5d9a29d
2020-11-13 13:14:51 -08:00
Patrice Arruda
e92c30dff1 Capture the build command that executed the build system.
We need to capture the user build command to understand what additional
arguments is passed in to Soong. This data is useful to know which
targets are being executed and have statistics on top build targets
are specified.

Bug: b/169453712
Test: ran "m blueprint_tools", "m nothing", "lunch 1" and checked
      in soong_metrics proto.
Change-Id: I7ab94dcf037211f1abffe6afd012b03ffbf99756
2020-10-29 11:01:32 -07:00
Patrice Arruda
fec3bf2e2c Add resource usage metrics in Soong metrics.proto.
We would like to collect the resource usage of processes such as
kati, soong_build and ninja on the resource usage such as memory
and context switching. This changelist contains the defined fields
in the protobuf file of the resource information to be collected
from interested processes.

Bug: b/169453825
Test: N/A
Change-Id: I4b3cbbc3e7b78900875c10d34d574425f8b11730
2020-10-21 08:54:42 -07:00
Patrice Arruda
3edfd48ef4 Collect total number of CPU cores and available RAM for each build.
Collect the total number of CPU cores and available RAM for each build
as part of the system resource info. This is to know what kind of
build machines that the developers are using.

Bug: b/169453825
Test: m nothing and ran printproto on soong_metrics to validate the
      data.
Change-Id: I8617bdb9490b37d7689abd27d349af1a651533db
2020-10-13 23:58:41 +00:00
Treehugger Robot
585d7d4c2e Merge "Collect the hostname of the machine as a metric." 2020-10-13 18:07:51 +00:00
Patrice Arruda
1e2cad95c8 Add additional documentation in metrics proto regen.sh script.
Added additional instruction in the regen.sh script to build the
blueprint_tools and aprotoc in case the regen.sh script fails.

Bug: b/169395325
Test: ./regen.sh
Change-Id: Icb38c907fb47febc17eef1067d06d0588dd0b7ba
2020-10-12 22:58:27 +00:00
Patrice Arruda
4fb8adcc4c Collect the hostname of the machine as a metric.
The hostname is collected during a build to determine where the user
is build from (from office workstation, cloudtop, etc...). This is a
useful metrics for RBE related builds.

Bug: b/169395325
Test: m nothing, ran printproto to verify that soong_metrics has the
      hostname.
Change-Id: I3e5d64f948ae43c5dda60de1d4a52b187e36dff8
2020-10-12 22:44:27 +00:00
Patrice Arruda
c97d6dcbb3 Collect FORCE_USE_GOMA environment variable as a metric.
FORCE_USE_GOMA is needed to be collected as a metric for the
deprecation of GOMA on Android builds.

Bug: b/169423400
Test: m nothing, ran printproto command
Change-Id: I12300389f1f27239d8f21875d1b6e4ad069d95d7
2020-09-28 18:25:42 +00:00
Patrice Arruda
9685036e6e Start collecting build configuration metrics.
This is an initial CL that collects the environment variables
USE_GOMA and USE_RBE. This is a needed metric to determine how
many users are currently using GOMA so we can help them migrate
to RBE.

Bug: b/163598127
Test: m nothing and checked soong_metrics
Change-Id: I39536eabad01aafbb3bd7182515d72230b36f658
2020-08-12 14:25:37 +00:00
Colin Cross
b72c909899 Collect metrics from inside soong_build
Collect the number of modules and variants and some basic statistics
on memory usage inside soong_build by writing out a proto that is
read back in by soong_ui.

Test: examine soong.log
Change-Id: I6926876377a4f6229cf41fdbf166ae03c885ea55
2020-02-25 20:50:00 +00:00
Colin Cross
74cda72f5e Add total build time to metrics
Determining the total build time from the existing metrics is
tricky because some of the metrics are overlapping.  For example,
the "soong" metric includes  "blueprint bootstrap",
"environment check", "minibp", "bpglob", "minibootstrap", and
"bootstrap".  Create a new singleton "total" metric that
covers the interesting parts of the build.

Test: m nothing
Change-Id: Iedcf5c9cf0d27252b694d35e6ff66ca1fcf734ac
2020-01-16 15:25:50 -08:00
Colin Cross
d0be2109cf Add CUJ tests
Add a cuj_tests binary that runs through a sequence of critical
user journey builds and collects metrics.

Bug: 145688720
Test: cuj_tests
Change-Id: I6bb1a3fc4a458fd8cea7b7c924f565d0cf3e6df3
2019-12-05 11:11:37 -08:00
Patrice Arruda
0cc5b21516 Soong: Fix the package name in metrics.proto file
The metrics.proto needs to be imported to Google3 in order to unmarshal
the build metrics data from bigstore. Cleaned up the enum names to use
the Camel naming convention and renamed the build_metrics to
soong_metrics to be more specific to soong and to allow quering
soong_metrics files from bigstore.

Bug: b/135280521
Test: lunch andf m -j. Checked if soong_metrics was generated correctly
      by using printproto command.

Change-Id: I998c8d05db592e94a653d6ca32250b80df3c9b21
2019-06-14 15:27:46 -07:00
Nan Zhang
17f2767724 Generate build timing metrics to proto format file
Test: Dumped the text formated based metrics file to out dir,
and checked the file.
Bug: b/63815990

Change-Id: Iff476f72a0be74eb53b6b26ef468d11c0f24a404
2019-01-04 15:54:01 -08:00