Commit graph

1277 commits

Author SHA1 Message Date
Treehugger Robot
7a56608b91 Merge changes Id1b9ae00,I48723e00
* changes:
  Merge remote-tracking branch 'upstream/master' into upstream_merge
  Merge remote-tracking branch 'upstream/master' into upstream_merge
2020-03-05 00:51:07 +00:00
Sasha Smundak
27f101a285 Merge remote-tracking branch 'upstream/master' into upstream_merge
* upstream/master:
  Fix bug in buildPropertyMap in previous commit.
  Fix missing dependencies from mutators

Test: 'm nothing' runs internal tests
Change-Id: Id1b9ae002e62351a8ec0d27c249ca25ef949ff5f
2020-03-04 15:05:35 -08:00
asmundak
ca6940754e
Merge pull request #288 from asmundak/master
Fix the bug in buildPropertyMap in the previous commit.
2020-03-04 14:55:25 -08:00
Sasha Smundak
de4d9f94ac Fix bug in buildPropertyMap in previous commit.
Property value should be evaluated before proeprty map is populated with
it. Added the test for this case.
2020-03-03 17:42:38 -08:00
colincross
03d3ee51f6
Merge pull request #287 from colincross/mutator_ninja_file_deps
Fix missing dependencies from mutators
2020-03-03 14:31:17 -08:00
Colin Cross
ab0a83f09c Fix missing dependencies from mutators
Mutators were not propagating the results of ctx.AddNinjaFileDeps.

Test: examine out/soong/build.ninja.d
Fixes: 150689149
Change-Id: Ia1e69ebc9dfa94a05f4ecd9cc2a8691ee63c9dd5
2020-03-03 14:24:24 -08:00
Sasha Smundak
1b4e8b7263 Merge remote-tracking branch 'upstream/master' into upstream_merge
* upstream/master:
  Implement list of maps

Test: internal tests are run during 'm nothing'
Change-Id: I48723e00501de59c3d0946b6031de50957e313f7
2020-03-02 19:48:12 -08:00
asmundak
86497ca795
Merge pull request #286 from asmundak/master
Implement list of maps
2020-03-02 17:55:09 -08:00
Sasha Smundak
29fdcad56c Implement list of maps
Allow property value to be a list of maps, e.g.
my_module {
  my_list: [
    { name: "foo", value: 42, something: true, },
    { name: "bar", value: 34, something: false, },
  ],
}

Test: internal
Change-Id: I2fc37d692aac39f23c9aa7bda2859ab49f3bc672
2020-03-02 17:26:20 -08:00
Automerger Merge Worker
c528a0d923 In kzip files, compilation unit paths should be relative to top. am: 59a3a23e92
Change-Id: I672f992dacbcbf0f0e83d5f111996f6f5486181d
2020-02-26 21:39:13 +00:00
Sasha Smundak
59a3a23e92 In kzip files, compilation unit paths should be relative to top.
Fixes inter-module cross-references.

Bug: 137798757
Test: build kzips, verify paths
Change-Id: I98160d48d3fb18cb122bb8a7d1a18fd0f6f9551b
2020-02-25 11:22:41 -08:00
Automerger Merge Worker
99ea787237 Merge remote-tracking branch 'aosp/upstream' into uppercase_properties am: f0e6d47072
Change-Id: Ic83f50f2f76b8842757165ccb297d81d5af04245
2020-02-06 04:32:48 +00:00
Colin Cross
f0e6d47072 Merge remote-tracking branch 'aosp/upstream' into uppercase_properties
* aosp/upstream:
  Fix PropertyNameForField for X86.
  Support unpacking capitalized property names
  Make ninjaString an interface

Fixes: 148865218
Test: m checkbuild
Change-Id: I3680cd261bf420601a7b943e21acde6837ae8619
2020-02-05 17:41:15 -08:00
colincross
c42ef965a0
Merge pull request #284 from colincross/uppercase_properties
Fix PropertyNameForField for X86.
2020-02-05 17:26:16 -08:00
Colin Cross
fc6efcb4a4 Fix PropertyNameForField for X86.
Field "X86" has no lowercase runes and was being left uppercase.
Change the new PropertyNameForField rules to lowercase the name unless
it has any uppercase rune after the first rune (which is always
uppercase) and no lowercase runes.

Bug: 148865218
Test: proptools_test.go
Change-Id: Ifd1c10fc03f5ae1765d25b3f73dba8fd61c5c956
2020-02-05 17:13:08 -08:00
colincross
d851df9ada
Merge pull request #283 from colincross/uppercase_properties
Support unpacking capitalized property names
2020-02-05 14:02:57 -08:00
Colin Cross
3bbbdf31e3 Support unpacking capitalized property names
Soong config variables may propagate an uppercase name from Make.
Blueprint properties have traditionally been all lowercase, and
using an uppercase property struct field name resulted in a strange
Blueprint property name with the first rune lowercase and the
remaining runes uppercase.

Update the rules for proptools.PropertyNameForField to not lowercase
the first rune if the field name has mulitple runes and is not all
uppercase.

Fixes: 148865218
Test: proptools_test.go
Change-Id: I8de2f65ffb00e5a8ce0aea0caf09f5859315f6b8
2020-02-05 13:50:28 -08:00
colincross
a5eeec2a89
Merge pull request #282 from colincross/ninjastring
Make ninjaString an interface
2020-01-29 16:45:27 -08:00
Colin Cross
2ce594e446 Make ninjaString an interface
There are 8935901 *ninjaString objects generated in an AOSP
aosp_blueline-userdebug build, and 7865180 of those are a literal
string with no ninja variables.
Each of those *ninjaString objects takes a minimum of 48 bytes for
2 slices, plus 8 bytes for the pointer to the ninjaString.  For
the literal string case, one of those slices has a single element,
(costing another 16 bytes for the backing array), and the other
slice is empty, for a total of 72 bytes.

Replace *ninjaString with a ninjaString interface.  This increases
the size of the reference from 8 bytes to 16 bytes, but using
a type alias of a string for the literal string implementation uses
only 16 bytes, saving 40 bytes per literal string or 314 MB.

Test: ninja_strings_test
Change-Id: Ic5fe16ed1f2a244fe6a8ccdf762919634d825cbe
2020-01-29 16:23:40 -08:00
Colin Cross
f46eed3e5d Merge remote-tracking branch 'aosp/upstream' into master
am: 5b1a877f0a

Change-Id: I10ba0c1a4aa766b65d39a384b900924cad6fa437
2020-01-28 17:20:33 -08:00
Colin Cross
5b1a877f0a Merge remote-tracking branch 'aosp/upstream' into master
* aosp/upstream:
  Make proptools functions consistently take *struct types
  Add isStruct and isStructPtr helpers
  Fix null pointer dereference printing an expression.

Test: m checkbuild
Change-Id: I547f3cd5f286edd32efee60a7300c7eabe293e7b
2020-01-28 11:20:41 -08:00
colincross
38e095ae8a
Merge pull request #281 from colincross/proptools_consistency
Proptools consistency
2020-01-28 10:02:20 -08:00
Colin Cross
5d57b2d347 Make proptools functions consistently take *struct types
The proptools functions took an inconsistent variety of
struct and *struct types.  Some methods even took a struct
but returned a *struct.  Make all the exported methods
take a *struct, with internal helpers for the ones that need
to take a struct.

Test: proptools tests
Change-Id: I60ce212606e96adcef66c531d57f69c39e1a1638
2020-01-28 09:51:19 -08:00
Colin Cross
6898d26054 Add isStruct and isStructPtr helpers
Test: proptools tests
Change-Id: I7814b2138cd19b538a3a33036a15119e118d7644
2020-01-28 09:51:19 -08:00
asmundak
30f225ba8a
Merge pull request #280 from asmundak/master
Fix null pointer dereference printing an expression.
2020-01-27 10:17:26 -08:00
Automerger Merge Worker
2e9dbb516b Merge remote-tracking branch 'aosp/upstream' into master am: a70b3381cb
Change-Id: Id96124baa24e4112b262fe2b9482c82450240a06
2020-01-23 21:46:24 +00:00
Sasha Smundak
77418b70b4 Fix null pointer dereference printing an expression.
Parser.parseVariable method should always set the value of the variable
it creates. Failure to do so may end up in the following:

```
$ androidmk  <(printf "FOO:=(X)\nFOO:=bar\n")
parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = %!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference) (%!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference)) false
```

The cause is that calling Parser.Parse to parse `FOO=abc` created
a Variable instance with nil value, causing panic on print attempt.

Test: m androidmk && androidmk  <(printf "FOO:=(X)\nFOO:=bar\n")
(should print:
ERROR:  parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = X = Not Evaluated (X = Not Evaluated) false)

Change-Id: I296d7984df6d8796e0075f9eb692b234f8c94f08
2020-01-23 13:32:43 -08:00
Colin Cross
a70b3381cb Merge remote-tracking branch 'aosp/upstream' into master
* aosp/upstream:
  Remove blueprint:"filter(*)" tag support
  Make FilterPropertyStructSharded smarter

Bug: 146234651
Test: m checkbuild
Change-Id: Ib3de8d8dd43e6354c17f1734705a9feb2ca7f701
2020-01-22 18:09:56 +00:00
colincross
0c4d1db0a0
Merge pull request #278 from colincross/filtershard
Make FilterPropertyStructSharded smarter
2020-01-21 12:12:08 -08:00
Colin Cross
571f77a60d Remove blueprint:"filter(*)" tag support
The filter tag is unused, replaced with FilterPropertyStruct to
generate a new type at runtime that only contains the filtered
fields.

Test: unpack_test.go
Change-Id: Id91cf99290832094d05426f3263279836f0fea73
2020-01-21 11:49:27 -08:00
Colin Cross
b89d91c67c Make FilterPropertyStructSharded smarter
FilterPropertyStructSharded was just sharding the top level
properties into groups of 10.  For nested property structs
this can be insufficient - there could be a single top level
property with many properties below it.

Take a maximum name size, and track the size used by parent
structs to determine when sharding a nested struct is necessary.

Bug: 146234651
Test: filter_test.go
Change-Id: I5b5ed11ea27a0325b2fd6c2c3fb427ea1e2af0c2
2020-01-21 11:49:27 -08:00
Automerger Merge Worker
6c9bd9ab34 Merge "Add mappings for Go extractor." am: 450c4f565a
Change-Id: I33befabb4d828c9b2a530c0b0e60fde53bc019aa
2020-01-17 08:08:26 +00:00
Treehugger Robot
450c4f565a Merge "Add mappings for Go extractor." 2020-01-17 07:58:43 +00:00
Colin Cross
8623fab804 Merge remote-tracking branch 'aosp/upstream' into master
am: 76a390c22e

Change-Id: Ibf73b384760e3f5fdb729be7e86032a9fabdca27
2020-01-16 13:32:43 -08:00
Colin Cross
76a390c22e Merge remote-tracking branch 'aosp/upstream' into master
* aosp/upstream:
  Fix missing glob filelists when sandboxed
  Fix primary builder reruns caused by globs
  Add EarlyModuleContext.BlueprintsFile

Test: m checkbuild
Change-Id: I239f16f6f9fe9ba0e83da28bdc4f32d8ee807351
2020-01-16 11:14:58 -08:00
colincross
6ea3f9ddd2
Merge pull request #277 from colincross/glob_incremental
Speed up incremental builds after globs
2020-01-16 11:01:00 -08:00
Colin Cross
67d0cbed90 Fix missing glob filelists when sandboxed
Sandboxing the primary builder caused the glob filelists to not
be written because they were using a relative path, causing
primary builder reruns on the second build.

Also report errors when writing the filelist files.

Test: m checkbuild
Change-Id: Id1706560d04c85f00f829cfb714967bb8600626f
2020-01-16 10:57:08 -08:00
Colin Cross
1cda3fd3e4 Fix primary builder reruns caused by globs
Globs produce a filelist using restat rules to avoid primary builder
reruns.  On the first execution of the primary builder the filelists
are written directly by the primary builder, so that the second
execution can restat them all but avoid rerunning the primary builder.
This wasn't working because the glob filelists were written after
the build.ninja file, but the build.ninja file depends on the
filelists.  Switching the order avoids the extra rerun.

Test: m nothing && m nothing
Change-Id: Ia8e0924598220d4ff34235907a8de2e3e03632df
2020-01-16 10:38:46 -08:00
Sasha Smundak
d7a5f952f2 Add mappings for Go extractor.
Bug: 137798757
Test: manual
Change-Id: I48905714d5fb4c4051c2ef3c77d8ff03d9935480
2020-01-14 17:02:29 -08:00
colincross
b92b466a46
Merge pull request #276 from colincross/blueprintsfile
Add EarlyModuleContext.BlueprintsFile
2020-01-14 14:14:41 -08:00
Colin Cross
c03a50c49c Add EarlyModuleContext.BlueprintsFile
EarlyModuleContext already has ModuleDir, add BlueprintsFile too.

Test: m checkbuild
Change-Id: Ia9e0c724562e1461dfcd5e01f56ff8abbea2ac06
2020-01-14 13:00:01 -08:00
Automerger Merge Worker
ca01e7e98f Merge remote-tracking branch 'aosp/upstream' into master am: de71320aa6 am: 1b5f9af317
Change-Id: I70c865fb198e30cd381b0b14c4fa2785d7908d7b
2020-01-11 07:17:14 +00:00
Colin Cross
1b5f9af317 Merge remote-tracking branch 'aosp/upstream' into master
am: de71320aa6

Change-Id: I3358f3d9f28e37922a1699d02ddd1f7b4722aed0
2020-01-10 23:00:52 -08:00
Colin Cross
de71320aa6 Merge remote-tracking branch 'aosp/upstream' into master
* aosp/upstream:
  Fix reading absolute paths through OsFs

Bug: 146437378
Test: m checkbuild
Change-Id: Iabee4c296902a227def128b6285118de50917885
2020-01-10 17:09:30 -08:00
colincross
b6ebce2c23
Merge pull request #275 from colincross/sandbox_primary_builder
Fix reading absolute paths through OsFs
2020-01-10 14:16:28 -08:00
Colin Cross
2f95ec7031 Fix reading absolute paths through OsFs
OsFs may be asked to read absolute paths if buildDir is absolute.
Check if the path is absolute before prepending srcDir to it.

Bug: 146437378
Test: fs_test.go
Change-Id: I2a67593e9d836ca3e11dc10b81f49a4fb49d2cdf
2020-01-10 13:52:22 -08:00
Automerger Merge Worker
cd0c3da180 Merge remote-tracking branch 'aosp/upstream' into master am: 40e4d10377 am: a722dcff9a
Change-Id: Ia893371a10874dc53fc1e624370746a8b030e7d7
2020-01-10 17:52:44 +00:00
Colin Cross
a722dcff9a Merge remote-tracking branch 'aosp/upstream' into master
am: 40e4d10377

Change-Id: I0c0849f3f7a9de20def3dccd401b17191f422807
2020-01-10 09:37:47 -08:00
Colin Cross
40e4d10377 Merge remote-tracking branch 'aosp/upstream' into master
* aosp/upstream:
  Allow primary builder to change working directory

Bug: 146437378
Test: m checkbuild
Change-Id: I95c476539cd2a74500b3f94baf048306ad5d9eca
2020-01-09 14:25:08 -08:00
colincross
5e8efd5f08
Merge pull request #274 from colincross/sandbox_primary_builder
Allow primary builder to change working directory
2020-01-09 12:56:10 -08:00