Commit graph

270 commits

Author SHA1 Message Date
Justin Yun
9321804a11 Rename product_services to system_ext
am: d5f6c8261a

Change-Id: Ifd77d7d729addae59b2b26a8cbdd2ebefffcabae
2019-07-09 02:14:59 -07:00
Justin Yun
d5f6c8261a Rename product_services to system_ext
Bug: 134359158
Test: build and check if system_ext.img is created
Change-Id: Ice3229baf97a94c24a6eea22e7d4571941d7f843
2019-07-09 08:57:05 +00:00
Colin Cross
cd48600321 Merge changes Icfc893c8,I40f03fc0
am: f67e1bee4a

Change-Id: I0b9cf1e3df4ef7ef46e73a5dc6d363fe918c29b7
2019-07-03 22:55:03 -07:00
Colin Cross
aa812d122c Allow debugging with SOONG_DELVE=<listen addr>
Allow running Soong in a headless delve debugger by passing
SOONG_DELVE=<listen addr> in the environment.

Bug: 80165685
Test: SOONG_DELVE=:1234 m nothing
Change-Id: Icfc893c8a8354a9bbc99112d9c83259cb41906d1
2019-07-03 16:33:31 -07:00
Colin Cross
a73c8a4e61 Merge "Compute critical path when ninja finishes" am: 7c9b4cf144
am: 7b4a190539

Change-Id: I1f3869fa3d5afb437bab8a4bf77ea03934c1d4ab
2019-06-26 16:53:05 -07:00
Colin Cross
7b6245388c Compute critical path when ninja finishes
Keep a running map of the critical path to each edge as it
finishes.  At the end of the build, find the edge with the
longest critical path and print the path to out/soong.log.

Test: critical_path_test.go
Change-Id: Ie01d26b068b768156b166bff00cdc3273e8124ca
2019-06-26 11:31:17 -07:00
Patrice Arruda
55ac3ddb21 Merge "Soong: Add build_error proto raw file to $(OUT_DIR)." am: 7d7cae85ab
am: 9f9778cff0

Change-Id: I7d2ee16ad31446360142295c61562cff467f9d19
2019-06-24 16:43:43 -07:00
Patrice Arruda
297cebad26 Soong: Add build_error proto raw file to $(OUT_DIR).
Soong_ui produces a build_error raw file to $(OUT_DIR) where
the file contains a list of build action errors. Each build action
error represents an error raised by a command. The build_error file
is populated if there was a build error.

Bug: b/132969697
Test: Ran m for successful build. Introduced a broken build change,
      ran m and verified using printproto that build_error was
      generated successfully.

Change-Id: I690ca1778b4e56f144a3173ba1d16d8494c32c15
2019-06-24 13:02:35 -07:00
Patrice Arruda
4b557d8f86 Merge "Soong: Add BUILD_MODULES build action in soong_ui." am: 45cdb71290
am: d009ef35b8

Change-Id: I0216045ba79f0cc26a4de5883589f52a65d9919c
2019-06-20 20:59:32 -07:00
Patrice Arruda
3928206ecc Soong: Add BUILD_MODULES build action in soong_ui.
There was one case that did not work:
  1) Create dir in root source tree
  2) Run "m libc"

The build would fail as it was using the mma logic. A separate
one named BUILD_MODULES was added to allow building specific
modules in any directories using the "m" command.

Bug: b/130049705
Test: unit test cases, ran "m libc" in a tmp directory inside
      of the source tree.

Change-Id: I8d23e685a673a311001ee8edd89bd73b662392dd
2019-06-20 16:35:12 -07:00
Patrice Arruda
5dbf869760 Soong: Use ctx.Writer in flag usage for dumpVar and dumpVars in soong_ui. am: db4c2f115e
am: e6fe9edbef

Change-Id: Ifcd0332f5f055633e4284c3387beea84cedf86a7
2019-06-18 15:52:12 -07:00
Patrice Arruda
db4c2f115e Soong: Use ctx.Writer in flag usage for dumpVar and dumpVars in soong_ui.
ctx.Writer is recommended stream to use for standard io.

Fixes: b/135487952
Test: Lunch and executed ./out/soong_ui --dumpvar-mode and
--dumpvars-mode to check the usage is output to the console.

Change-Id: I38351ebdb05f49f92d6b4bfb0423092299d1c5ea
2019-06-18 20:32:01 +00:00
Patrice Arruda
268a702f06 Merge "soong_ui: Add --build-mode flag in soong_ui" am: 2f93a2e07b
am: 9123c55392

Change-Id: Ib7ca530857ad469da3e634b7115a308b6d0d1972
2019-06-17 21:25:36 -07:00
Patrice Arruda
b7b2282fd1 soong_ui: Add --build-mode flag in soong_ui
The --build-mode flag is a new flag in soong_ui that accepts a build
action (m, mm, mmm, mma, mmma), the directory where the build action
is occuring and an optional list of build arguments and invokes the
build option. This is to deprecate the build actions behavior in
envsetup.sh and start using the one defined in soong_ui.

Bug: b/130049705
Test: Ran lunch and ran all the build action commands in soong_ui.
      Ran a bunch of invalid commands such as no Android.mk file in
      a specific directory, invalid target name, etc...

Change-Id: I88e9e59f47f8c5cdc872fb83a0f3f0a0919885b6
2019-06-17 17:21:32 -07:00
Patrice Arruda
10b659a60f Merge "Soong: Fix the package name in metrics.proto file" am: eec8d3aee3
am: 33989c88c3

Change-Id: Ife7e9c2ddeeeda40976c23f8ba07ba4bee53935c
2019-06-17 17:12:42 -07:00
Patrice Arruda
0cc5b21516 Soong: Fix the package name in metrics.proto file
The metrics.proto needs to be imported to Google3 in order to unmarshal
the build metrics data from bigstore. Cleaned up the enum names to use
the Camel naming convention and renamed the build_metrics to
soong_metrics to be more specific to soong and to allow quering
soong_metrics files from bigstore.

Bug: b/135280521
Test: lunch andf m -j. Checked if soong_metrics was generated correctly
      by using printproto command.

Change-Id: I998c8d05db592e94a653d6ca32250b80df3c9b21
2019-06-14 15:27:46 -07:00
Colin Cross
ceec6a2295 Merge changes I555ad21a,I9d46761e,I917bdeee,I5bc5476a,I59bbdae4, ... am: 6ebe07def7
am: 1337decdff

Change-Id: I2f27851391ad68137cc66133de575d75a3c83a74
2019-06-14 01:30:19 -07:00
Colin Cross
e0df1a36b2 Move all output through StatusOutput
Write log output through StatusOutput so that the status implementation
can synchronize it with its own output.

Test: status_test.go
Change-Id: I917bdeeea4759a12b6b4aa6d6d86ee18a2771723
2019-06-12 21:30:56 -07:00
Colin Cross
097ed2a37c Remove terminal.Writer
terminal.Writer is now just a wrapper around stdio.Stdout() without
any useful functionality.  Replace it with stdio.Stdout() as an
io.Writer.

Test: ui/terminal/status_test.go
Change-Id: I5bc5476afdca950b505642f0135a3af9d37fbe24
2019-06-12 21:30:56 -07:00
Dan Willemsen
7488174f2c Merge "Refactor .aar resource compilation" am: 588857b418
am: 2758050dc4

Change-Id: I98dfbdbef71972a040fd53f9e77eb321fb6f5e0e
2019-06-10 21:16:55 -07:00
Dan Willemsen
304cfec778 Refactor .aar resource compilation
Instead of extracting the aar, and passing the res directory to another rule
(along with a dependency to a different known file), don't keep the
extracted res directory, and pass the aar directly to the resource
compliation rule, which will extract just the res directory.

I need this for my RBE experiments, where non-listed output files won't
exist in other rules.

Test: m
Change-Id: I99074381052cbcebb6a402484abae9ab2e40284f
2019-05-29 16:54:43 -07:00
Patrice Arruda
d2e1149cc5 resolve merge conflicts of a3898089ba to qt-dev-plus-aosp
Bug: None
Test: Ran soong_ui.bash which compiles the soong_ui.
Change-Id: I0b8083e3e5ee69c8cbf17443fbc038ec7b5a0b10
2019-05-06 19:24:32 -07:00
Treehugger Robot
565ee6ed87 Merge "Soong: Refactor the soong_ui arguments processing to be more like bazel." 2019-05-07 00:49:21 +00:00
Patrice Arruda
a5c2542f68 Soong: Refactor the soong_ui arguments processing to be more like bazel.
Currently, the command line argments is being processed in multiple
places. In the main soong_ui arguments, there are several if statements
that checks if a specific argument was specified in order to execute
the requested operation. This does not scale well when adding or
removing a command in the near future. In order to support the build
commands (m, mma, etc...) from the envsetup.sh in soong_ui, a refactor
was required in order to add a command rather quickly and to have the
flexibiity to unit test the command. The soong_ui arguments format is
as follows:

soong_ui <command> [<arg 1> <arg 2> ... <arg n>]

The <command> is a specific operation to be executed. The arguments
after the command are processed by the command itself.

Below is the list of changes made in this commit:

 * Created a new command infrastructure that allows adding
   or deprecating a command easily.

 * Fixed a bug when running ./soong_ui.bash directly would cause
   a panic due to index being out of range for args list.

Bug: b/130049705
Test: Below is the list of testing done on this commit:
    * Ran lunch and verified that the output is the same as the the output
      without the modifications. lunch indirectly runs soong_ui passing
      in the --dumpvar-mode (to read makefile variables such as
      TARGET_PRODUCT) and --dumpvars-mode (to build the build cache).
    * Ran ./soong_ui.bash directly (with unsupported flags and no flags)
      and the proper message appeared that soong native UI is not yet
      available.
    * Ran m, mm, mmm, mma, mmma commands.
    * Ran the make installclean command:
         make -j80 TARGET_PRODUCT=aosp_arm64 TARGET_BUILD_VARIANT=eng dist DIST_DIR=/tmp/helloworld installclean
    * Ran ./out/soong_ui -j80 --make-mode PRODUCT-aosp_arm64-eng dist checkbuild tests

Change-Id: Iee4de7ec0fa4661206fda8ae1fe6fa4487d9bb22
Merged-In: Iee4de7ec0fa4661206fda8ae1fe6fa4487d9bb22
2019-05-06 14:05:52 -07:00
Colin Cross
f41ed9ae63 Merge "Remove services.art from known_nondeterminism.whitelist" am: 42603c4ab5
am: 828a488720

Change-Id: Idbdeec76c0d8207890afbf94ea20ebac4c3d5379
2019-04-30 15:24:25 -07:00
Colin Cross
497d4c24b1 Remove services.art from known_nondeterminism.whitelist
Bug: 120039850
Test: none
Change-Id: I70d5243f427a4da5c8126be6b8ebc90221133ad3
2019-04-29 17:36:19 +00:00
Dan Willemsen
7d155094be Merge "pom2bp: Implement -static-deps to match pom2mk" am: 80ae6816ae
am: 9181e08738

Change-Id: I331a7bb1cd82315f673d9007c2240da423864034
2019-04-22 20:17:35 -07:00
Dan Willemsen
52c90d868e pom2bp: Implement -static-deps to match pom2mk
This changes the default behavior of pom2bp to match pom2mk, and not
require pre-extraction of all of the manifests.

Test: convert an internal pom2mk user to pom2bp
Change-Id: I08120cce1f923ef28519563ffd772032b2e500ed
2019-04-21 21:46:42 -07:00
Dan Willemsen
8a4c0da74d Add -jetifier to pom2mk and pom2bp am: 7fdab6eeec
am: fd9fa1a56e

Change-Id: Ifaa39849ef2d858dd63745e2a8a0b9e5587bbd66
2019-04-21 02:47:34 -07:00
Dan Willemsen
7fdab6eeec Add -jetifier to pom2mk and pom2bp
Test: use pom2mk -jetifier to replace some hand-modified makefiles
Change-Id: I2a1988d0120fee1edf2d85c8b3294041840efe7c
2019-04-20 21:47:14 -07:00
Paul Duffin
0018e7f48b Merge "Rename -extra-deps to --extra-static-libs and add --extra-libs" am: 38b122021a
am: 2b9f534fba

Change-Id: I1a607c72ad7f1a71b7dcc0aa216beee2b868776f
2019-04-17 07:32:57 -07:00
Paul Duffin
38b122021a Merge "Rename -extra-deps to --extra-static-libs and add --extra-libs" 2019-04-17 14:06:48 +00:00
Paul Duffin
babaf07b13 Rename -extra-deps to --extra-static-libs and add --extra-libs
Allow callers of pom2bp to add dependencies on runtime libraries as
well as static libraries. Renames -extra-deps to --extra-static-libs
to make it clearer what it does.

Bug: 130287656
Test: prebuilts/sdk/update_prebuilts.py --stx 5280039
      prebuilts/misc/common/androidx-test/update-from-gmaven.py

Change-Id: I8a3ee8056aa4ed9935a4f47d563ec423672cf38b
2019-04-16 11:55:48 +01:00
Jaewoong Jung
a7d23ae804 Merge "Embed CSS entries in soong_build.html." am: 0a0b3fa605
am: 6d48aa4186

Change-Id: I0e162fdc0eabafc152686f637436b41295540935
2019-04-15 18:17:14 -07:00
Dan Willemsen
891af48379 Support RuleBuilder.Sbox to wrap commands in sbox am: 633c502295
am: b933c57635

Change-Id: I07cb4998e8d860f97c4ae00f9f776744cc8a5adf
2019-04-15 17:50:48 -07:00
Treehugger Robot
0a0b3fa605 Merge "Embed CSS entries in soong_build.html." 2019-04-16 00:49:18 +00:00
Jaewoong Jung
5f867c0580 Embed CSS entries in soong_build.html.
This will prevent it from being affected by future css location changes.

Test: m soong_docs
Fixes: 130567020
Change-Id: I80fd7b8a12a32bcea7b270423b8d8f8dc0976177
2019-04-15 15:21:55 -07:00
Dan Willemsen
633c502295 Support RuleBuilder.Sbox to wrap commands in sbox
This essentially allows you to declare that everything in a directory
will be created by the rule, and we'll ensure that your command actually
writes out all of the claimed outputs, and remove any other files that
previously existed in that directory.

Test: built-in tests
Change-Id: I990dce2b3a0d89ebd2736ac1a0cadfb5864c6e73
2019-04-15 14:52:05 -07:00
Colin Cross
537cfd2477 Merge "pom2bp: add jar deps to -nodeps modules" am: 34df71446f
am: 694b4629e1

Change-Id: I519921487ae1813091e08074df70376fa39efb61
2019-04-11 16:23:49 -07:00
Colin Cross
1aa7f26a22 pom2bp: add jar deps to -nodeps modules
The dependency type in the pom file is not always accurate, just
add all jar and aar dependencies to both the -nodeps and final
module.

Test: cd prebuilts/sdk/current/androidx && pom2bp -regen Android.bp && mma
Change-Id: I1784b8ebf3485db19405cb353c9bb5fb3ae837d8
2019-04-10 18:31:55 +00:00
Colin Cross
11fbb890c9 Fix package path of android/soong/android pctx am: cc0ce80ed5
am: a1c57b4664

Change-Id: Ifba10892f902e70bd0a52a82a20bd5f1b64a61de
2019-04-04 01:39:57 -07:00
Colin Cross
cc0ce80ed5 Fix package path of android/soong/android pctx
android/soong/common was renamed to android/soong/android long
ago, but the pctx package path was still "android/soong/common".
This required all users of rules defined in android/soong/android
to import "android/soong/android" and then
pctx.Import("android/soong/common").

Test: m checkbuild
Change-Id: I20d096522760538f7cfc2bec3d4bfeba99b275d4
2019-04-04 03:59:34 +00:00
Colin Cross
a50a2464c6 Merge changes I7dd4ba7f,I7887a67a am: 92cc71606c am: 900574f0cd
am: 6c99a30bb7

Change-Id: If557897ed64ecdf8967299b5512392e5485767ae
2019-04-02 08:21:42 -07:00
Colin Cross
92cc71606c Merge changes I7dd4ba7f,I7887a67a
* changes:
  Add depfile support to RuleBuilder
  Add RuleBuilderCommand.Flags
2019-04-02 15:04:06 +00:00
Colin Cross
f37f6b064c Merge changes Ie7f776a4,I3ca5dd1f,I2e911262 am: 11581cf6b3 am: 89d99337fb
am: 92270109eb

Change-Id: Ic7ac18a1d60138e909c6045fd5313620256989d4
2019-04-01 17:48:29 -07:00
Colin Cross
11581cf6b3 Merge changes Ie7f776a4,I3ca5dd1f,I2e911262
* changes:
  Add a whitelist for services.art changing between builds
  Add diff_target_files tool
  Import files from compare_target_files for use in diff_target_files
2019-04-02 00:03:05 +00:00
Colin Cross
93599cdb93 Merge "Touch output files in sbox" am: 80eb6907ac am: 9457c97c40
am: 5cb90e6517

Change-Id: I07587e71632567f553f417db40b791e4d335aaae
2019-04-01 11:15:17 -07:00
Colin Cross
d1c1e6fd6c Touch output files in sbox
Some tools used by genrules set the timestamp of the generated file
to an old value.  For example, when using tar to extract files it
will set the timestamp of the extracted file to the value stored in
the tar file.  This can cause unnecessary rebuilds due to output files
that are older than their inputs.  Touch output files in sbox before
moving them into the output location.

Test: m checkbuild && m checkbuild
Change-Id: Idb0e71d2bf0ac3618338371132cc71be74bf9d9d
2019-04-01 10:43:20 -07:00
Colin Cross
d93112ae26 Add a whitelist for services.art changing between builds
Bug: 120039850
Test: manual
Change-Id: Ie7f776a4e2743883ea006c66148fc4f40f90ce07
2019-04-01 10:07:25 -07:00
Colin Cross
91ee4c793d Add diff_target_files tool
Add a tool that lists files that have changed between two builds by
comparing the target-files.zip artifacts.

Bug: 121158314
Test: unit tests
Change-Id: I3ca5dd1fdf652dffef923ad7307fd71e49e2e95a
2019-04-01 10:07:21 -07:00