Commit graph

1291 commits

Author SHA1 Message Date
Dan Willemsen
ec5085e613 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Actually write an empty ninja file with --empty-ninja-file

Test: treehugger (build_test)
Change-Id: Iff7851f2aa39eb14b7ae72ed6ce94fa0ce8d2621
2020-04-19 15:20:02 -07:00
Dan Willemsen
2a062a2ef0
Merge pull request #296 from danw/emptyfile
Actually write an empty ninja file with --empty-ninja-file
2020-04-19 13:32:06 -07:00
Dan Willemsen
6c6c103612 Actually write an empty ninja file with --empty-ninja-file
Otherwise we'll re-run the primary builder every time this is used. This
triggered my new ninja checks that verify that all commands write their
outputs.

Change-Id: Ic92d1ea57203b953da8025a170b24e6420c5436c
2020-04-19 12:50:48 -07:00
Colin Cross
5ef2a1eeaa Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  WalkDeps - only record module visited when it has been recursed into
  Allow missing variants when allowMissingDependencies=true
  bpmodify: fix os.Exit() shouldn't shadow panic()
  bpmodify: respect exitCode
  Set bpmodify usage function

Test: treehugger
Change-Id: Ida99d7412ea3c034e1ddce228b1e851c1dc40cc2
2020-04-02 16:39:51 -07:00
colincross
6957a46d38
Merge pull request #293 from paulduffin/master
WalkDeps - only record module visited when it has been recursed into
2020-04-02 12:58:05 -07:00
Paul Duffin
72bab1707e WalkDeps - only record module visited when it has been recursed into
Previously, WalkDeps() would record that a module was visited after the
first time it encountered the module irrespective of whether it recursed
into or not. This change moves the recording so it happens only after it
has been recursed into.

Added TestWalkDepsDuplicates_IgnoreFirstPath to test the change. Without
the change the test fails because it does not visit E.

Test refactoring:
* A depsMutator was added instead of relying on blueprintDepsMutator to
  allow different tags to be used for different dependency types.
* Modified barModule and fooModule to support the new depsMutator and
  add support for another type of dependency that is ignored by the
  walking code.
* Extracted walkDependencyGraph() function to reuse common code.
2020-04-02 10:56:13 +01:00
colincross
ef33c90fe3
Merge pull request #292 from paulduffin/master
Allow missing variants when allowMissingDependencies=true
2020-04-01 15:13:27 -07:00
Paul Duffin
b77556bdca Allow missing variants when allowMissingDependencies=true 2020-03-24 19:17:27 +00:00
Dan Willemsen
215230a3e8
Merge pull request #291 from silverneko/bpmodify-exit
bpmodify: fix os.Exit() shouldn't shadow panic()
2020-03-05 10:29:27 -08:00
Yo Chiang
9342b43c95 bpmodify: fix os.Exit() shouldn't shadow panic()
The `defer func() { os.Exit() }()` in main() method shadows panic().
Make the exit handler recover() from panic(), log the panic(), and then
gracefully exit.

Test: m bpmodify
Change-Id: Icc89f8fce0b6096489baa0ba0f08c21d1ef623bc
2020-03-05 19:45:49 +08:00
Dan Willemsen
aa919ddcbb
Merge pull request #290 from silverneko/bpmodify-exitcode
bpmodify: respect exitCode
2020-03-04 20:59:31 -08:00
Dan Willemsen
571420830a
Merge pull request #289 from silverneko/bpmodify-usage
Set bpmodify usage function
2020-03-04 20:59:23 -08:00
Yo Chiang
e5a91f55e4 bpmodify: respect exitCode
main() method calls os.Exit(exitCode) upon return, respecting any
exitCode set by the report() method.

Test: bpmodify -w; echo "$?"
Test: bpmodify Android.bp; echo "$?"
Test: # echo command should output "2"
Change-Id: Iaf056301eaba3f249b256ecf0f0d87f8a4df1c58
2020-03-05 11:24:01 +08:00
Treehugger Robot
7a56608b91 Merge changes Id1b9ae00,I48723e00
* changes:
  Merge remote-tracking branch 'upstream/master' into upstream_merge
  Merge remote-tracking branch 'upstream/master' into upstream_merge
2020-03-05 00:51:07 +00:00
Sasha Smundak
27f101a285 Merge remote-tracking branch 'upstream/master' into upstream_merge
* upstream/master:
  Fix bug in buildPropertyMap in previous commit.
  Fix missing dependencies from mutators

Test: 'm nothing' runs internal tests
Change-Id: Id1b9ae002e62351a8ec0d27c249ca25ef949ff5f
2020-03-04 15:05:35 -08:00
asmundak
ca6940754e
Merge pull request #288 from asmundak/master
Fix the bug in buildPropertyMap in the previous commit.
2020-03-04 14:55:25 -08:00
Yo Chiang
3edfbc214a Set bpmodify usage function
golang `flag` package's default FlagSet `flag.CommandLine` calls
`flag.Usage` and `os.Exit(2)` on error to print the usage string.
Set `flag.Usage` to our custom usage function.

Test: m bpmodify; bpmodify -h; bpmodify --help
Change-Id: Ida107b0dbb07c291c3d7ea90eda9147d04a7cd51
2020-03-04 09:46:17 +08:00
Sasha Smundak
de4d9f94ac Fix bug in buildPropertyMap in previous commit.
Property value should be evaluated before proeprty map is populated with
it. Added the test for this case.
2020-03-03 17:42:38 -08:00
colincross
03d3ee51f6
Merge pull request #287 from colincross/mutator_ninja_file_deps
Fix missing dependencies from mutators
2020-03-03 14:31:17 -08:00
Colin Cross
ab0a83f09c Fix missing dependencies from mutators
Mutators were not propagating the results of ctx.AddNinjaFileDeps.

Test: examine out/soong/build.ninja.d
Fixes: 150689149
Change-Id: Ia1e69ebc9dfa94a05f4ecd9cc2a8691ee63c9dd5
2020-03-03 14:24:24 -08:00
Sasha Smundak
1b4e8b7263 Merge remote-tracking branch 'upstream/master' into upstream_merge
* upstream/master:
  Implement list of maps

Test: internal tests are run during 'm nothing'
Change-Id: I48723e00501de59c3d0946b6031de50957e313f7
2020-03-02 19:48:12 -08:00
asmundak
86497ca795
Merge pull request #286 from asmundak/master
Implement list of maps
2020-03-02 17:55:09 -08:00
Sasha Smundak
29fdcad56c Implement list of maps
Allow property value to be a list of maps, e.g.
my_module {
  my_list: [
    { name: "foo", value: 42, something: true, },
    { name: "bar", value: 34, something: false, },
  ],
}

Test: internal
Change-Id: I2fc37d692aac39f23c9aa7bda2859ab49f3bc672
2020-03-02 17:26:20 -08:00
Automerger Merge Worker
c528a0d923 In kzip files, compilation unit paths should be relative to top. am: 59a3a23e92
Change-Id: I672f992dacbcbf0f0e83d5f111996f6f5486181d
2020-02-26 21:39:13 +00:00
Sasha Smundak
59a3a23e92 In kzip files, compilation unit paths should be relative to top.
Fixes inter-module cross-references.

Bug: 137798757
Test: build kzips, verify paths
Change-Id: I98160d48d3fb18cb122bb8a7d1a18fd0f6f9551b
2020-02-25 11:22:41 -08:00
Automerger Merge Worker
99ea787237 Merge remote-tracking branch 'aosp/upstream' into uppercase_properties am: f0e6d47072
Change-Id: Ic83f50f2f76b8842757165ccb297d81d5af04245
2020-02-06 04:32:48 +00:00
Colin Cross
f0e6d47072 Merge remote-tracking branch 'aosp/upstream' into uppercase_properties
* aosp/upstream:
  Fix PropertyNameForField for X86.
  Support unpacking capitalized property names
  Make ninjaString an interface

Fixes: 148865218
Test: m checkbuild
Change-Id: I3680cd261bf420601a7b943e21acde6837ae8619
2020-02-05 17:41:15 -08:00
colincross
c42ef965a0
Merge pull request #284 from colincross/uppercase_properties
Fix PropertyNameForField for X86.
2020-02-05 17:26:16 -08:00
Colin Cross
fc6efcb4a4 Fix PropertyNameForField for X86.
Field "X86" has no lowercase runes and was being left uppercase.
Change the new PropertyNameForField rules to lowercase the name unless
it has any uppercase rune after the first rune (which is always
uppercase) and no lowercase runes.

Bug: 148865218
Test: proptools_test.go
Change-Id: Ifd1c10fc03f5ae1765d25b3f73dba8fd61c5c956
2020-02-05 17:13:08 -08:00
colincross
d851df9ada
Merge pull request #283 from colincross/uppercase_properties
Support unpacking capitalized property names
2020-02-05 14:02:57 -08:00
Colin Cross
3bbbdf31e3 Support unpacking capitalized property names
Soong config variables may propagate an uppercase name from Make.
Blueprint properties have traditionally been all lowercase, and
using an uppercase property struct field name resulted in a strange
Blueprint property name with the first rune lowercase and the
remaining runes uppercase.

Update the rules for proptools.PropertyNameForField to not lowercase
the first rune if the field name has mulitple runes and is not all
uppercase.

Fixes: 148865218
Test: proptools_test.go
Change-Id: I8de2f65ffb00e5a8ce0aea0caf09f5859315f6b8
2020-02-05 13:50:28 -08:00
colincross
a5eeec2a89
Merge pull request #282 from colincross/ninjastring
Make ninjaString an interface
2020-01-29 16:45:27 -08:00
Colin Cross
2ce594e446 Make ninjaString an interface
There are 8935901 *ninjaString objects generated in an AOSP
aosp_blueline-userdebug build, and 7865180 of those are a literal
string with no ninja variables.
Each of those *ninjaString objects takes a minimum of 48 bytes for
2 slices, plus 8 bytes for the pointer to the ninjaString.  For
the literal string case, one of those slices has a single element,
(costing another 16 bytes for the backing array), and the other
slice is empty, for a total of 72 bytes.

Replace *ninjaString with a ninjaString interface.  This increases
the size of the reference from 8 bytes to 16 bytes, but using
a type alias of a string for the literal string implementation uses
only 16 bytes, saving 40 bytes per literal string or 314 MB.

Test: ninja_strings_test
Change-Id: Ic5fe16ed1f2a244fe6a8ccdf762919634d825cbe
2020-01-29 16:23:40 -08:00
Colin Cross
f46eed3e5d Merge remote-tracking branch 'aosp/upstream' into master
am: 5b1a877f0a

Change-Id: I10ba0c1a4aa766b65d39a384b900924cad6fa437
2020-01-28 17:20:33 -08:00
Colin Cross
5b1a877f0a Merge remote-tracking branch 'aosp/upstream' into master
* aosp/upstream:
  Make proptools functions consistently take *struct types
  Add isStruct and isStructPtr helpers
  Fix null pointer dereference printing an expression.

Test: m checkbuild
Change-Id: I547f3cd5f286edd32efee60a7300c7eabe293e7b
2020-01-28 11:20:41 -08:00
colincross
38e095ae8a
Merge pull request #281 from colincross/proptools_consistency
Proptools consistency
2020-01-28 10:02:20 -08:00
Colin Cross
5d57b2d347 Make proptools functions consistently take *struct types
The proptools functions took an inconsistent variety of
struct and *struct types.  Some methods even took a struct
but returned a *struct.  Make all the exported methods
take a *struct, with internal helpers for the ones that need
to take a struct.

Test: proptools tests
Change-Id: I60ce212606e96adcef66c531d57f69c39e1a1638
2020-01-28 09:51:19 -08:00
Colin Cross
6898d26054 Add isStruct and isStructPtr helpers
Test: proptools tests
Change-Id: I7814b2138cd19b538a3a33036a15119e118d7644
2020-01-28 09:51:19 -08:00
asmundak
30f225ba8a
Merge pull request #280 from asmundak/master
Fix null pointer dereference printing an expression.
2020-01-27 10:17:26 -08:00
Automerger Merge Worker
2e9dbb516b Merge remote-tracking branch 'aosp/upstream' into master am: a70b3381cb
Change-Id: Id96124baa24e4112b262fe2b9482c82450240a06
2020-01-23 21:46:24 +00:00
Sasha Smundak
77418b70b4 Fix null pointer dereference printing an expression.
Parser.parseVariable method should always set the value of the variable
it creates. Failure to do so may end up in the following:

```
$ androidmk  <(printf "FOO:=(X)\nFOO:=bar\n")
parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = %!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference) (%!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference)) false
```

The cause is that calling Parser.Parse to parse `FOO=abc` created
a Variable instance with nil value, causing panic on print attempt.

Test: m androidmk && androidmk  <(printf "FOO:=(X)\nFOO:=bar\n")
(should print:
ERROR:  parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = X = Not Evaluated (X = Not Evaluated) false)

Change-Id: I296d7984df6d8796e0075f9eb692b234f8c94f08
2020-01-23 13:32:43 -08:00
Colin Cross
a70b3381cb Merge remote-tracking branch 'aosp/upstream' into master
* aosp/upstream:
  Remove blueprint:"filter(*)" tag support
  Make FilterPropertyStructSharded smarter

Bug: 146234651
Test: m checkbuild
Change-Id: Ib3de8d8dd43e6354c17f1734705a9feb2ca7f701
2020-01-22 18:09:56 +00:00
colincross
0c4d1db0a0
Merge pull request #278 from colincross/filtershard
Make FilterPropertyStructSharded smarter
2020-01-21 12:12:08 -08:00
Colin Cross
571f77a60d Remove blueprint:"filter(*)" tag support
The filter tag is unused, replaced with FilterPropertyStruct to
generate a new type at runtime that only contains the filtered
fields.

Test: unpack_test.go
Change-Id: Id91cf99290832094d05426f3263279836f0fea73
2020-01-21 11:49:27 -08:00
Colin Cross
b89d91c67c Make FilterPropertyStructSharded smarter
FilterPropertyStructSharded was just sharding the top level
properties into groups of 10.  For nested property structs
this can be insufficient - there could be a single top level
property with many properties below it.

Take a maximum name size, and track the size used by parent
structs to determine when sharding a nested struct is necessary.

Bug: 146234651
Test: filter_test.go
Change-Id: I5b5ed11ea27a0325b2fd6c2c3fb427ea1e2af0c2
2020-01-21 11:49:27 -08:00
Automerger Merge Worker
6c9bd9ab34 Merge "Add mappings for Go extractor." am: 450c4f565a
Change-Id: I33befabb4d828c9b2a530c0b0e60fde53bc019aa
2020-01-17 08:08:26 +00:00
Treehugger Robot
450c4f565a Merge "Add mappings for Go extractor." 2020-01-17 07:58:43 +00:00
Colin Cross
8623fab804 Merge remote-tracking branch 'aosp/upstream' into master
am: 76a390c22e

Change-Id: Ibf73b384760e3f5fdb729be7e86032a9fabdca27
2020-01-16 13:32:43 -08:00
Colin Cross
76a390c22e Merge remote-tracking branch 'aosp/upstream' into master
* aosp/upstream:
  Fix missing glob filelists when sandboxed
  Fix primary builder reruns caused by globs
  Add EarlyModuleContext.BlueprintsFile

Test: m checkbuild
Change-Id: I239f16f6f9fe9ba0e83da28bdc4f32d8ee807351
2020-01-16 11:14:58 -08:00
colincross
6ea3f9ddd2
Merge pull request #277 from colincross/glob_incremental
Speed up incremental builds after globs
2020-01-16 11:01:00 -08:00