Commit graph

750 commits

Author SHA1 Message Date
Yo Chiang
e5a91f55e4 bpmodify: respect exitCode
main() method calls os.Exit(exitCode) upon return, respecting any
exitCode set by the report() method.

Test: bpmodify -w; echo "$?"
Test: bpmodify Android.bp; echo "$?"
Test: # echo command should output "2"
Change-Id: Iaf056301eaba3f249b256ecf0f0d87f8a4df1c58
2020-03-05 11:24:01 +08:00
asmundak
ca6940754e
Merge pull request #288 from asmundak/master
Fix the bug in buildPropertyMap in the previous commit.
2020-03-04 14:55:25 -08:00
Yo Chiang
3edfbc214a Set bpmodify usage function
golang `flag` package's default FlagSet `flag.CommandLine` calls
`flag.Usage` and `os.Exit(2)` on error to print the usage string.
Set `flag.Usage` to our custom usage function.

Test: m bpmodify; bpmodify -h; bpmodify --help
Change-Id: Ida107b0dbb07c291c3d7ea90eda9147d04a7cd51
2020-03-04 09:46:17 +08:00
Sasha Smundak
de4d9f94ac Fix bug in buildPropertyMap in previous commit.
Property value should be evaluated before proeprty map is populated with
it. Added the test for this case.
2020-03-03 17:42:38 -08:00
colincross
03d3ee51f6
Merge pull request #287 from colincross/mutator_ninja_file_deps
Fix missing dependencies from mutators
2020-03-03 14:31:17 -08:00
Colin Cross
ab0a83f09c Fix missing dependencies from mutators
Mutators were not propagating the results of ctx.AddNinjaFileDeps.

Test: examine out/soong/build.ninja.d
Fixes: 150689149
Change-Id: Ia1e69ebc9dfa94a05f4ecd9cc2a8691ee63c9dd5
2020-03-03 14:24:24 -08:00
asmundak
86497ca795
Merge pull request #286 from asmundak/master
Implement list of maps
2020-03-02 17:55:09 -08:00
Sasha Smundak
29fdcad56c Implement list of maps
Allow property value to be a list of maps, e.g.
my_module {
  my_list: [
    { name: "foo", value: 42, something: true, },
    { name: "bar", value: 34, something: false, },
  ],
}

Test: internal
Change-Id: I2fc37d692aac39f23c9aa7bda2859ab49f3bc672
2020-03-02 17:26:20 -08:00
colincross
c42ef965a0
Merge pull request #284 from colincross/uppercase_properties
Fix PropertyNameForField for X86.
2020-02-05 17:26:16 -08:00
Colin Cross
fc6efcb4a4 Fix PropertyNameForField for X86.
Field "X86" has no lowercase runes and was being left uppercase.
Change the new PropertyNameForField rules to lowercase the name unless
it has any uppercase rune after the first rune (which is always
uppercase) and no lowercase runes.

Bug: 148865218
Test: proptools_test.go
Change-Id: Ifd1c10fc03f5ae1765d25b3f73dba8fd61c5c956
2020-02-05 17:13:08 -08:00
colincross
d851df9ada
Merge pull request #283 from colincross/uppercase_properties
Support unpacking capitalized property names
2020-02-05 14:02:57 -08:00
Colin Cross
3bbbdf31e3 Support unpacking capitalized property names
Soong config variables may propagate an uppercase name from Make.
Blueprint properties have traditionally been all lowercase, and
using an uppercase property struct field name resulted in a strange
Blueprint property name with the first rune lowercase and the
remaining runes uppercase.

Update the rules for proptools.PropertyNameForField to not lowercase
the first rune if the field name has mulitple runes and is not all
uppercase.

Fixes: 148865218
Test: proptools_test.go
Change-Id: I8de2f65ffb00e5a8ce0aea0caf09f5859315f6b8
2020-02-05 13:50:28 -08:00
colincross
a5eeec2a89
Merge pull request #282 from colincross/ninjastring
Make ninjaString an interface
2020-01-29 16:45:27 -08:00
Colin Cross
2ce594e446 Make ninjaString an interface
There are 8935901 *ninjaString objects generated in an AOSP
aosp_blueline-userdebug build, and 7865180 of those are a literal
string with no ninja variables.
Each of those *ninjaString objects takes a minimum of 48 bytes for
2 slices, plus 8 bytes for the pointer to the ninjaString.  For
the literal string case, one of those slices has a single element,
(costing another 16 bytes for the backing array), and the other
slice is empty, for a total of 72 bytes.

Replace *ninjaString with a ninjaString interface.  This increases
the size of the reference from 8 bytes to 16 bytes, but using
a type alias of a string for the literal string implementation uses
only 16 bytes, saving 40 bytes per literal string or 314 MB.

Test: ninja_strings_test
Change-Id: Ic5fe16ed1f2a244fe6a8ccdf762919634d825cbe
2020-01-29 16:23:40 -08:00
colincross
38e095ae8a
Merge pull request #281 from colincross/proptools_consistency
Proptools consistency
2020-01-28 10:02:20 -08:00
Colin Cross
5d57b2d347 Make proptools functions consistently take *struct types
The proptools functions took an inconsistent variety of
struct and *struct types.  Some methods even took a struct
but returned a *struct.  Make all the exported methods
take a *struct, with internal helpers for the ones that need
to take a struct.

Test: proptools tests
Change-Id: I60ce212606e96adcef66c531d57f69c39e1a1638
2020-01-28 09:51:19 -08:00
Colin Cross
6898d26054 Add isStruct and isStructPtr helpers
Test: proptools tests
Change-Id: I7814b2138cd19b538a3a33036a15119e118d7644
2020-01-28 09:51:19 -08:00
asmundak
30f225ba8a
Merge pull request #280 from asmundak/master
Fix null pointer dereference printing an expression.
2020-01-27 10:17:26 -08:00
Sasha Smundak
77418b70b4 Fix null pointer dereference printing an expression.
Parser.parseVariable method should always set the value of the variable
it creates. Failure to do so may end up in the following:

```
$ androidmk  <(printf "FOO:=(X)\nFOO:=bar\n")
parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = %!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference) (%!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference)) false
```

The cause is that calling Parser.Parse to parse `FOO=abc` created
a Variable instance with nil value, causing panic on print attempt.

Test: m androidmk && androidmk  <(printf "FOO:=(X)\nFOO:=bar\n")
(should print:
ERROR:  parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = X = Not Evaluated (X = Not Evaluated) false)

Change-Id: I296d7984df6d8796e0075f9eb692b234f8c94f08
2020-01-23 13:32:43 -08:00
colincross
0c4d1db0a0
Merge pull request #278 from colincross/filtershard
Make FilterPropertyStructSharded smarter
2020-01-21 12:12:08 -08:00
Colin Cross
571f77a60d Remove blueprint:"filter(*)" tag support
The filter tag is unused, replaced with FilterPropertyStruct to
generate a new type at runtime that only contains the filtered
fields.

Test: unpack_test.go
Change-Id: Id91cf99290832094d05426f3263279836f0fea73
2020-01-21 11:49:27 -08:00
Colin Cross
b89d91c67c Make FilterPropertyStructSharded smarter
FilterPropertyStructSharded was just sharding the top level
properties into groups of 10.  For nested property structs
this can be insufficient - there could be a single top level
property with many properties below it.

Take a maximum name size, and track the size used by parent
structs to determine when sharding a nested struct is necessary.

Bug: 146234651
Test: filter_test.go
Change-Id: I5b5ed11ea27a0325b2fd6c2c3fb427ea1e2af0c2
2020-01-21 11:49:27 -08:00
colincross
6ea3f9ddd2
Merge pull request #277 from colincross/glob_incremental
Speed up incremental builds after globs
2020-01-16 11:01:00 -08:00
Colin Cross
67d0cbed90 Fix missing glob filelists when sandboxed
Sandboxing the primary builder caused the glob filelists to not
be written because they were using a relative path, causing
primary builder reruns on the second build.

Also report errors when writing the filelist files.

Test: m checkbuild
Change-Id: Id1706560d04c85f00f829cfb714967bb8600626f
2020-01-16 10:57:08 -08:00
Colin Cross
1cda3fd3e4 Fix primary builder reruns caused by globs
Globs produce a filelist using restat rules to avoid primary builder
reruns.  On the first execution of the primary builder the filelists
are written directly by the primary builder, so that the second
execution can restat them all but avoid rerunning the primary builder.
This wasn't working because the glob filelists were written after
the build.ninja file, but the build.ninja file depends on the
filelists.  Switching the order avoids the extra rerun.

Test: m nothing && m nothing
Change-Id: Ia8e0924598220d4ff34235907a8de2e3e03632df
2020-01-16 10:38:46 -08:00
colincross
b92b466a46
Merge pull request #276 from colincross/blueprintsfile
Add EarlyModuleContext.BlueprintsFile
2020-01-14 14:14:41 -08:00
Colin Cross
c03a50c49c Add EarlyModuleContext.BlueprintsFile
EarlyModuleContext already has ModuleDir, add BlueprintsFile too.

Test: m checkbuild
Change-Id: Ia9e0c724562e1461dfcd5e01f56ff8abbea2ac06
2020-01-14 13:00:01 -08:00
colincross
b6ebce2c23
Merge pull request #275 from colincross/sandbox_primary_builder
Fix reading absolute paths through OsFs
2020-01-10 14:16:28 -08:00
Colin Cross
2f95ec7031 Fix reading absolute paths through OsFs
OsFs may be asked to read absolute paths if buildDir is absolute.
Check if the path is absolute before prepending srcDir to it.

Bug: 146437378
Test: fs_test.go
Change-Id: I2a67593e9d836ca3e11dc10b81f49a4fb49d2cdf
2020-01-10 13:52:22 -08:00
colincross
5e8efd5f08
Merge pull request #274 from colincross/sandbox_primary_builder
Allow primary builder to change working directory
2020-01-09 12:56:10 -08:00
Colin Cross
c5fa50e057 Allow primary builder to change working directory
Bug: 146437378
Test: pathtools/fs_test.go
Change-Id: I513ceb9b8b0b4f18223bc34ecad9846fe220709b
2020-01-08 15:54:58 -08:00
colincross
55ca1c0860
Merge pull request #273 from colincross/scoped_module_factories
Scoped module factories
2020-01-06 12:53:54 -08:00
Colin Cross
f27c5e470b Move unpackProperties to proptools and export it
Test: unpack_test.go
Change-Id: I145369323bd7d5003a261c13dfb8ed31d0be51b5
2020-01-02 20:32:51 -08:00
Colin Cross
9672d86a87 Add scoped module factories
Add RegisterScopedModuleType to LoadHookContext that registers
a module type factory that will be visible for the remainder of
the file.  Also add ModuleFactories that returns the globally
register module factories.

Test: all blueprint tests
Change-Id: If646a73befe3b8e45c4b0984531c6a39ddc8d066
2020-01-02 20:32:51 -08:00
Colin Cross
da70fd0b84 Move LoadHooks from Soong to Blueprint
Move LoadHooks from Blueprint and run them during ParseBlueprintsFiles.
This will allow them to add scoped module types that are visible to the
rest of the Blueprints file.  Requires passing the config object to
ParseBlueprintsFiles.

Test: all blueprint tests
Change-Id: Ia2a2c9a0223d5458bfd48bd22ebed0fdbd0156c6
2020-01-02 20:32:51 -08:00
colincross
9999eddc46
Merge pull request #272 from colincross/sandbox_primary_builder
Add Context.SetFs
2019-12-17 14:42:15 -08:00
Colin Cross
8cde425b60 Add Context.SetFs
Allow the primary builder to set the filesystem directly.

Bug: 146437378
Test: m checkbuild
Change-Id: Ic8cc3e6a03943d5f47b041a66df6a11f4336c97b
2019-12-17 13:14:31 -08:00
Jaewoong Jung
cecffaf05e
Merge pull request #270 from jiyongp/replace_properties
Slice properties can be replaced
2019-11-18 18:16:30 -08:00
Jiyong Park
10f27f8139 Slice properties can be replaced
override_* in Soong requires a module to override certain properties of
other module. In that case, values of a slice property (e.g. []string)
should be replaced by the same property value in the overriding module.
However, since proptools only supports Append and Prepend orders where
the original values are kept for slice properties, the behavior
couldn't be implemented. To support the use case, Replace order is
introduced, in which case slice property values are completely replaced.
For other types of properties, the Replace order behaves exactly the
same as the Append order.

Bug: 144338929
Test: m

Change-Id: Iae9feda035177fe6a22e6e8319c0fdaa9e08e85e
2019-11-19 10:45:58 +09:00
colincross
eda08050f3
Merge pull request #269 from colincross/alias
Add support for module variant aliases
2019-11-15 07:20:19 -08:00
Colin Cross
f7beb89df5 Add support for module variant aliases
Adding a dependency on a module with variants can be problematic
if the code adding the dependency is not aware of every mutator
that has created variants.  Add an AliasVariations to
BottomUpMutatorContext, which allows a mutator to alias the
original variant of a module to one of the new variants of the
module, which will allow future dependencies to be added using
the original list of variations.  The aliases are transient,
and only exist until the next mutator that calls CreateVariations
when visiting the module without also calling AliasVariations.

Test: TestAlises
Change-Id: Ieaa04b5a6bdcb5a1ff5114b1e03460de795d4479
2019-11-14 12:58:52 -08:00
Colin Cross
9403b5a790 Delay allocating variationMaps until populating them
Modules are created with a nil variantionMap, which avoids allocating
an empty map if the module will never be split.
Context.addVariationDependencies constructs a variationMap to
compare against the variationMap of each variant, but constructed
an empty variationMap if no variations were specified.  A nil
map is not the same as an empty map, so consistently use a nil
map and create it when populating the first entry.

Change-Id: I48b604659f9cdb23326b504a093cdfe5a3eb4f68
2019-11-14 12:58:52 -08:00
Colin Cross
d03b59d03e Use module groups more widely
Use module groups instead of passing around the list of modules
extracted from a module group.

Test: blueprint tests
Change-Id: I02a79950c6377441c49129ebeb5f12be257df668
2019-11-14 12:58:52 -08:00
colincross
7e8c766ab8
Merge pull request #267 from colincross/maindir
Move outputs of bootstrap modules in main stage to separate directory
2019-10-17 15:51:43 -07:00
Colin Cross
16fec72dfc Move outputs of bootstrap modules in main stage to separate directory
Don't write to $buildDir/.bootstrap in the ninja file generated for
the main stage.

Test: examine main stage build.ninja file
Change-Id: I9c6183c9a434ff11d44cdd4f9844eae192ebfe03
2019-10-17 13:42:34 -07:00
colincross
2f55f24c46
Merge pull request #264 from colincross/create_module
Make CreateModule return the newly created module
2019-09-26 14:30:11 -07:00
colincross
93322c4d7b
Merge pull request #266 from colincross/filter
Add proptools.FilterPropertyStruct
2019-09-26 14:29:04 -07:00
Colin Cross
1907836ad9 Add proptools.FilterPropertyStruct
Move some code from Soong to support creating a property struct
at runtime by filtering fields out of another property struct.

Test: TestFilterPropertyStruct
Change-Id: Ic5ae390a885195bebad6f3ecb7c752c0582a60b1
2019-09-26 14:21:40 -07:00
colincross
5639ec157c
Merge pull request #265 from colincross/int_default
Add proptools.Int and proptools.IntDefault
2019-09-26 10:41:52 -07:00
Colin Cross
6af7284d08 Make CreateModule return the newly created module
Allow mutators to modify properties of the newly created module
by returning it.

Change-Id: Ia1acc70b6196195d1bad58c88ab3def70b99e42c
2019-09-25 14:53:25 -07:00