Commit graph

726 commits

Author SHA1 Message Date
c-parsons
fe2fa04c30
Export ModuleListFile in bootstrap (with fix) (#304)
* Export ModuleListFile in bootstrap (with fix)

* Run gofmt on command.go
2020-06-15 09:03:27 -07:00
colincross
743261c07e
Merge pull request #303 from colincross/blueprint_tests
Revert changes that broke the build or tests
2020-06-11 16:34:45 -07:00
Colin Cross
bae4de9f16 Revert "Merge pull request #301 from c-parsons/modulelistfile"
This reverts commit 1160e9826f, reversing
changes made to 3a153ecc7f.
2020-06-11 16:29:43 -07:00
Colin Cross
f89751462c Revert "Always emit rules for tests and add phony to run them"
This reverts commit 63085f9e7c.

It breaks the blueprint tests, and we aren't use it yet, so revert
it for now.

Change-Id: I1ac81371342285e8d57baf15fe3a223d4140c6cf
2020-06-11 16:23:10 -07:00
colincross
1160e9826f
Merge pull request #301 from c-parsons/modulelistfile 2020-06-04 14:42:05 -07:00
c-parsons
01e094f1b4
Revert changes to singleton_ctx.go 2020-05-27 21:34:26 -04:00
c-parsons
8690f8f196
Change bootstrap.moduleListFile to be exported 2020-05-27 21:31:07 -04:00
colincross
3a153ecc7f
Merge pull request #300 from marstj/master
Add ctx.OtherModule(Reverse)DependencyVariantExists.
2020-05-26 13:27:14 -07:00
Martin Stjernholm
2f212479e2 Add ctx.OtherModule(Reverse)DependencyVariantExists.
This allows guarding calls to AddVariationDependencies and
AddReverseDependency to register dependencies on optional modules.
2020-05-26 20:52:43 +01:00
Chris Parsons
56a06b3c06 Add a getter for ModuleListFile to SingletonContext 2020-05-20 13:50:58 -04:00
colincross
2bcc43c60f
Merge pull request #299 from paulduffin/master
Support checking syntax of generated Blueprint files
2020-05-14 16:10:00 -07:00
Paul Duffin
2a2c58ef46 Support checking syntax of generated Blueprint files
Adds a CheckBlueprintSyntax(...) method to check the syntax of a
Blueprint file.

Changes processModuleDef and newModule from being method on *Context to
being standalone functions. That ensures that CheckBlueprintSyntax(...)
does not need to take a context and so there is no chance that it can
change its state.
2020-05-13 09:06:17 +01:00
colincross
301749801e
Merge pull request #298 from colincross/blueprint_tests
Always emit rules for tests and add phony to run them
2020-05-12 20:45:35 -07:00
Colin Cross
63085f9e7c Always emit rules for tests and add phony to run them
Emit the rules to build blueprint tests even if runGoTests is not
set, and add a phony rule "blueprint_tests" to run them.  This will
allow Soong to stop running the tests at the beginning of every build
but still run them as part of checkbuild or with a manual
`m blueprint_tests`.

Bug: 156428456
Test: m
Test: m blueprint_tests
Change-Id: If293a0757766d3046e78bf230a1825f15adc68fd
2020-05-12 19:20:26 -07:00
colincross
b0fe51a0c2
Merge pull request #297 from paulduffin/master
Run LoadHooks before registering module
2020-05-04 08:27:39 -07:00
Paul Duffin
244033b20f Run LoadHooks before registering module
Previously a LoadHook could not modify the name of a module because the
module was registered before the LoadHooks were run. That made it very
complicated (requiring mutators and auto generated names) to create a
module type whose name was determined by say the directory in which it
is defined.

This change moves the LoadHook execution slightly earlier so it runs
before registration of the module.

That caused one slight side problem which was that the
moduleInfo.Name() would fail when called in a LoadHook. That was
because that gets the name from group.name but was group was nil
because it is only set when the module is registered.

Modifying the moduleInfo.Name() method to get the name from the module
logicModule.Name() if group is nil fixed that. The reason for getting
the name from the group.name rather than the logicModule.Name() is that
the former tracks renames but the latter does not. However that is not
an issue in this case as there has been no opportunity for the module
to be renamed until after the LoadHook has returned.
2020-05-04 14:16:03 +01:00
Dan Willemsen
2a062a2ef0
Merge pull request #296 from danw/emptyfile
Actually write an empty ninja file with --empty-ninja-file
2020-04-19 13:32:06 -07:00
Dan Willemsen
6c6c103612 Actually write an empty ninja file with --empty-ninja-file
Otherwise we'll re-run the primary builder every time this is used. This
triggered my new ninja checks that verify that all commands write their
outputs.

Change-Id: Ic92d1ea57203b953da8025a170b24e6420c5436c
2020-04-19 12:50:48 -07:00
colincross
6957a46d38
Merge pull request #293 from paulduffin/master
WalkDeps - only record module visited when it has been recursed into
2020-04-02 12:58:05 -07:00
Paul Duffin
72bab1707e WalkDeps - only record module visited when it has been recursed into
Previously, WalkDeps() would record that a module was visited after the
first time it encountered the module irrespective of whether it recursed
into or not. This change moves the recording so it happens only after it
has been recursed into.

Added TestWalkDepsDuplicates_IgnoreFirstPath to test the change. Without
the change the test fails because it does not visit E.

Test refactoring:
* A depsMutator was added instead of relying on blueprintDepsMutator to
  allow different tags to be used for different dependency types.
* Modified barModule and fooModule to support the new depsMutator and
  add support for another type of dependency that is ignored by the
  walking code.
* Extracted walkDependencyGraph() function to reuse common code.
2020-04-02 10:56:13 +01:00
colincross
ef33c90fe3
Merge pull request #292 from paulduffin/master
Allow missing variants when allowMissingDependencies=true
2020-04-01 15:13:27 -07:00
Paul Duffin
b77556bdca Allow missing variants when allowMissingDependencies=true 2020-03-24 19:17:27 +00:00
Dan Willemsen
215230a3e8
Merge pull request #291 from silverneko/bpmodify-exit
bpmodify: fix os.Exit() shouldn't shadow panic()
2020-03-05 10:29:27 -08:00
Yo Chiang
9342b43c95 bpmodify: fix os.Exit() shouldn't shadow panic()
The `defer func() { os.Exit() }()` in main() method shadows panic().
Make the exit handler recover() from panic(), log the panic(), and then
gracefully exit.

Test: m bpmodify
Change-Id: Icc89f8fce0b6096489baa0ba0f08c21d1ef623bc
2020-03-05 19:45:49 +08:00
Dan Willemsen
aa919ddcbb
Merge pull request #290 from silverneko/bpmodify-exitcode
bpmodify: respect exitCode
2020-03-04 20:59:31 -08:00
Dan Willemsen
571420830a
Merge pull request #289 from silverneko/bpmodify-usage
Set bpmodify usage function
2020-03-04 20:59:23 -08:00
Yo Chiang
e5a91f55e4 bpmodify: respect exitCode
main() method calls os.Exit(exitCode) upon return, respecting any
exitCode set by the report() method.

Test: bpmodify -w; echo "$?"
Test: bpmodify Android.bp; echo "$?"
Test: # echo command should output "2"
Change-Id: Iaf056301eaba3f249b256ecf0f0d87f8a4df1c58
2020-03-05 11:24:01 +08:00
asmundak
ca6940754e
Merge pull request #288 from asmundak/master
Fix the bug in buildPropertyMap in the previous commit.
2020-03-04 14:55:25 -08:00
Yo Chiang
3edfbc214a Set bpmodify usage function
golang `flag` package's default FlagSet `flag.CommandLine` calls
`flag.Usage` and `os.Exit(2)` on error to print the usage string.
Set `flag.Usage` to our custom usage function.

Test: m bpmodify; bpmodify -h; bpmodify --help
Change-Id: Ida107b0dbb07c291c3d7ea90eda9147d04a7cd51
2020-03-04 09:46:17 +08:00
Sasha Smundak
de4d9f94ac Fix bug in buildPropertyMap in previous commit.
Property value should be evaluated before proeprty map is populated with
it. Added the test for this case.
2020-03-03 17:42:38 -08:00
colincross
03d3ee51f6
Merge pull request #287 from colincross/mutator_ninja_file_deps
Fix missing dependencies from mutators
2020-03-03 14:31:17 -08:00
Colin Cross
ab0a83f09c Fix missing dependencies from mutators
Mutators were not propagating the results of ctx.AddNinjaFileDeps.

Test: examine out/soong/build.ninja.d
Fixes: 150689149
Change-Id: Ia1e69ebc9dfa94a05f4ecd9cc2a8691ee63c9dd5
2020-03-03 14:24:24 -08:00
asmundak
86497ca795
Merge pull request #286 from asmundak/master
Implement list of maps
2020-03-02 17:55:09 -08:00
Sasha Smundak
29fdcad56c Implement list of maps
Allow property value to be a list of maps, e.g.
my_module {
  my_list: [
    { name: "foo", value: 42, something: true, },
    { name: "bar", value: 34, something: false, },
  ],
}

Test: internal
Change-Id: I2fc37d692aac39f23c9aa7bda2859ab49f3bc672
2020-03-02 17:26:20 -08:00
colincross
c42ef965a0
Merge pull request #284 from colincross/uppercase_properties
Fix PropertyNameForField for X86.
2020-02-05 17:26:16 -08:00
Colin Cross
fc6efcb4a4 Fix PropertyNameForField for X86.
Field "X86" has no lowercase runes and was being left uppercase.
Change the new PropertyNameForField rules to lowercase the name unless
it has any uppercase rune after the first rune (which is always
uppercase) and no lowercase runes.

Bug: 148865218
Test: proptools_test.go
Change-Id: Ifd1c10fc03f5ae1765d25b3f73dba8fd61c5c956
2020-02-05 17:13:08 -08:00
colincross
d851df9ada
Merge pull request #283 from colincross/uppercase_properties
Support unpacking capitalized property names
2020-02-05 14:02:57 -08:00
Colin Cross
3bbbdf31e3 Support unpacking capitalized property names
Soong config variables may propagate an uppercase name from Make.
Blueprint properties have traditionally been all lowercase, and
using an uppercase property struct field name resulted in a strange
Blueprint property name with the first rune lowercase and the
remaining runes uppercase.

Update the rules for proptools.PropertyNameForField to not lowercase
the first rune if the field name has mulitple runes and is not all
uppercase.

Fixes: 148865218
Test: proptools_test.go
Change-Id: I8de2f65ffb00e5a8ce0aea0caf09f5859315f6b8
2020-02-05 13:50:28 -08:00
colincross
a5eeec2a89
Merge pull request #282 from colincross/ninjastring
Make ninjaString an interface
2020-01-29 16:45:27 -08:00
Colin Cross
2ce594e446 Make ninjaString an interface
There are 8935901 *ninjaString objects generated in an AOSP
aosp_blueline-userdebug build, and 7865180 of those are a literal
string with no ninja variables.
Each of those *ninjaString objects takes a minimum of 48 bytes for
2 slices, plus 8 bytes for the pointer to the ninjaString.  For
the literal string case, one of those slices has a single element,
(costing another 16 bytes for the backing array), and the other
slice is empty, for a total of 72 bytes.

Replace *ninjaString with a ninjaString interface.  This increases
the size of the reference from 8 bytes to 16 bytes, but using
a type alias of a string for the literal string implementation uses
only 16 bytes, saving 40 bytes per literal string or 314 MB.

Test: ninja_strings_test
Change-Id: Ic5fe16ed1f2a244fe6a8ccdf762919634d825cbe
2020-01-29 16:23:40 -08:00
colincross
38e095ae8a
Merge pull request #281 from colincross/proptools_consistency
Proptools consistency
2020-01-28 10:02:20 -08:00
Colin Cross
5d57b2d347 Make proptools functions consistently take *struct types
The proptools functions took an inconsistent variety of
struct and *struct types.  Some methods even took a struct
but returned a *struct.  Make all the exported methods
take a *struct, with internal helpers for the ones that need
to take a struct.

Test: proptools tests
Change-Id: I60ce212606e96adcef66c531d57f69c39e1a1638
2020-01-28 09:51:19 -08:00
Colin Cross
6898d26054 Add isStruct and isStructPtr helpers
Test: proptools tests
Change-Id: I7814b2138cd19b538a3a33036a15119e118d7644
2020-01-28 09:51:19 -08:00
asmundak
30f225ba8a
Merge pull request #280 from asmundak/master
Fix null pointer dereference printing an expression.
2020-01-27 10:17:26 -08:00
Sasha Smundak
77418b70b4 Fix null pointer dereference printing an expression.
Parser.parseVariable method should always set the value of the variable
it creates. Failure to do so may end up in the following:

```
$ androidmk  <(printf "FOO:=(X)\nFOO:=bar\n")
parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = %!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference) (%!s(PANIC=String method: runtime error: invalid memory address or nil pointer dereference)) false
```

The cause is that calling Parser.Parse to parse `FOO=abc` created
a Variable instance with nil value, causing panic on print attempt.

Test: m androidmk && androidmk  <(printf "FOO:=(X)\nFOO:=bar\n")
(should print:
ERROR:  parse error:
<input>:3:1: variable already set, previous assignment: FOO@<input>:1:5 = X = Not Evaluated (X = Not Evaluated) false)

Change-Id: I296d7984df6d8796e0075f9eb692b234f8c94f08
2020-01-23 13:32:43 -08:00
colincross
0c4d1db0a0
Merge pull request #278 from colincross/filtershard
Make FilterPropertyStructSharded smarter
2020-01-21 12:12:08 -08:00
Colin Cross
571f77a60d Remove blueprint:"filter(*)" tag support
The filter tag is unused, replaced with FilterPropertyStruct to
generate a new type at runtime that only contains the filtered
fields.

Test: unpack_test.go
Change-Id: Id91cf99290832094d05426f3263279836f0fea73
2020-01-21 11:49:27 -08:00
Colin Cross
b89d91c67c Make FilterPropertyStructSharded smarter
FilterPropertyStructSharded was just sharding the top level
properties into groups of 10.  For nested property structs
this can be insufficient - there could be a single top level
property with many properties below it.

Take a maximum name size, and track the size used by parent
structs to determine when sharding a nested struct is necessary.

Bug: 146234651
Test: filter_test.go
Change-Id: I5b5ed11ea27a0325b2fd6c2c3fb427ea1e2af0c2
2020-01-21 11:49:27 -08:00
colincross
6ea3f9ddd2
Merge pull request #277 from colincross/glob_incremental
Speed up incremental builds after globs
2020-01-16 11:01:00 -08:00
Colin Cross
67d0cbed90 Fix missing glob filelists when sandboxed
Sandboxing the primary builder caused the glob filelists to not
be written because they were using a relative path, causing
primary builder reruns on the second build.

Also report errors when writing the filelist files.

Test: m checkbuild
Change-Id: Id1706560d04c85f00f829cfb714967bb8600626f
2020-01-16 10:57:08 -08:00