Commit graph

727 commits

Author SHA1 Message Date
Colin Cross
67d0cbed90 Fix missing glob filelists when sandboxed
Sandboxing the primary builder caused the glob filelists to not
be written because they were using a relative path, causing
primary builder reruns on the second build.

Also report errors when writing the filelist files.

Test: m checkbuild
Change-Id: Id1706560d04c85f00f829cfb714967bb8600626f
2020-01-16 10:57:08 -08:00
Colin Cross
1cda3fd3e4 Fix primary builder reruns caused by globs
Globs produce a filelist using restat rules to avoid primary builder
reruns.  On the first execution of the primary builder the filelists
are written directly by the primary builder, so that the second
execution can restat them all but avoid rerunning the primary builder.
This wasn't working because the glob filelists were written after
the build.ninja file, but the build.ninja file depends on the
filelists.  Switching the order avoids the extra rerun.

Test: m nothing && m nothing
Change-Id: Ia8e0924598220d4ff34235907a8de2e3e03632df
2020-01-16 10:38:46 -08:00
colincross
b92b466a46
Merge pull request #276 from colincross/blueprintsfile
Add EarlyModuleContext.BlueprintsFile
2020-01-14 14:14:41 -08:00
Colin Cross
c03a50c49c Add EarlyModuleContext.BlueprintsFile
EarlyModuleContext already has ModuleDir, add BlueprintsFile too.

Test: m checkbuild
Change-Id: Ia9e0c724562e1461dfcd5e01f56ff8abbea2ac06
2020-01-14 13:00:01 -08:00
colincross
b6ebce2c23
Merge pull request #275 from colincross/sandbox_primary_builder
Fix reading absolute paths through OsFs
2020-01-10 14:16:28 -08:00
Colin Cross
2f95ec7031 Fix reading absolute paths through OsFs
OsFs may be asked to read absolute paths if buildDir is absolute.
Check if the path is absolute before prepending srcDir to it.

Bug: 146437378
Test: fs_test.go
Change-Id: I2a67593e9d836ca3e11dc10b81f49a4fb49d2cdf
2020-01-10 13:52:22 -08:00
colincross
5e8efd5f08
Merge pull request #274 from colincross/sandbox_primary_builder
Allow primary builder to change working directory
2020-01-09 12:56:10 -08:00
Colin Cross
c5fa50e057 Allow primary builder to change working directory
Bug: 146437378
Test: pathtools/fs_test.go
Change-Id: I513ceb9b8b0b4f18223bc34ecad9846fe220709b
2020-01-08 15:54:58 -08:00
colincross
55ca1c0860
Merge pull request #273 from colincross/scoped_module_factories
Scoped module factories
2020-01-06 12:53:54 -08:00
Colin Cross
f27c5e470b Move unpackProperties to proptools and export it
Test: unpack_test.go
Change-Id: I145369323bd7d5003a261c13dfb8ed31d0be51b5
2020-01-02 20:32:51 -08:00
Colin Cross
9672d86a87 Add scoped module factories
Add RegisterScopedModuleType to LoadHookContext that registers
a module type factory that will be visible for the remainder of
the file.  Also add ModuleFactories that returns the globally
register module factories.

Test: all blueprint tests
Change-Id: If646a73befe3b8e45c4b0984531c6a39ddc8d066
2020-01-02 20:32:51 -08:00
Colin Cross
da70fd0b84 Move LoadHooks from Soong to Blueprint
Move LoadHooks from Blueprint and run them during ParseBlueprintsFiles.
This will allow them to add scoped module types that are visible to the
rest of the Blueprints file.  Requires passing the config object to
ParseBlueprintsFiles.

Test: all blueprint tests
Change-Id: Ia2a2c9a0223d5458bfd48bd22ebed0fdbd0156c6
2020-01-02 20:32:51 -08:00
colincross
9999eddc46
Merge pull request #272 from colincross/sandbox_primary_builder
Add Context.SetFs
2019-12-17 14:42:15 -08:00
Colin Cross
8cde425b60 Add Context.SetFs
Allow the primary builder to set the filesystem directly.

Bug: 146437378
Test: m checkbuild
Change-Id: Ic8cc3e6a03943d5f47b041a66df6a11f4336c97b
2019-12-17 13:14:31 -08:00
Jaewoong Jung
cecffaf05e
Merge pull request #270 from jiyongp/replace_properties
Slice properties can be replaced
2019-11-18 18:16:30 -08:00
Jiyong Park
10f27f8139 Slice properties can be replaced
override_* in Soong requires a module to override certain properties of
other module. In that case, values of a slice property (e.g. []string)
should be replaced by the same property value in the overriding module.
However, since proptools only supports Append and Prepend orders where
the original values are kept for slice properties, the behavior
couldn't be implemented. To support the use case, Replace order is
introduced, in which case slice property values are completely replaced.
For other types of properties, the Replace order behaves exactly the
same as the Append order.

Bug: 144338929
Test: m

Change-Id: Iae9feda035177fe6a22e6e8319c0fdaa9e08e85e
2019-11-19 10:45:58 +09:00
colincross
eda08050f3
Merge pull request #269 from colincross/alias
Add support for module variant aliases
2019-11-15 07:20:19 -08:00
Colin Cross
f7beb89df5 Add support for module variant aliases
Adding a dependency on a module with variants can be problematic
if the code adding the dependency is not aware of every mutator
that has created variants.  Add an AliasVariations to
BottomUpMutatorContext, which allows a mutator to alias the
original variant of a module to one of the new variants of the
module, which will allow future dependencies to be added using
the original list of variations.  The aliases are transient,
and only exist until the next mutator that calls CreateVariations
when visiting the module without also calling AliasVariations.

Test: TestAlises
Change-Id: Ieaa04b5a6bdcb5a1ff5114b1e03460de795d4479
2019-11-14 12:58:52 -08:00
Colin Cross
9403b5a790 Delay allocating variationMaps until populating them
Modules are created with a nil variantionMap, which avoids allocating
an empty map if the module will never be split.
Context.addVariationDependencies constructs a variationMap to
compare against the variationMap of each variant, but constructed
an empty variationMap if no variations were specified.  A nil
map is not the same as an empty map, so consistently use a nil
map and create it when populating the first entry.

Change-Id: I48b604659f9cdb23326b504a093cdfe5a3eb4f68
2019-11-14 12:58:52 -08:00
Colin Cross
d03b59d03e Use module groups more widely
Use module groups instead of passing around the list of modules
extracted from a module group.

Test: blueprint tests
Change-Id: I02a79950c6377441c49129ebeb5f12be257df668
2019-11-14 12:58:52 -08:00
colincross
7e8c766ab8
Merge pull request #267 from colincross/maindir
Move outputs of bootstrap modules in main stage to separate directory
2019-10-17 15:51:43 -07:00
Colin Cross
16fec72dfc Move outputs of bootstrap modules in main stage to separate directory
Don't write to $buildDir/.bootstrap in the ninja file generated for
the main stage.

Test: examine main stage build.ninja file
Change-Id: I9c6183c9a434ff11d44cdd4f9844eae192ebfe03
2019-10-17 13:42:34 -07:00
colincross
2f55f24c46
Merge pull request #264 from colincross/create_module
Make CreateModule return the newly created module
2019-09-26 14:30:11 -07:00
colincross
93322c4d7b
Merge pull request #266 from colincross/filter
Add proptools.FilterPropertyStruct
2019-09-26 14:29:04 -07:00
Colin Cross
1907836ad9 Add proptools.FilterPropertyStruct
Move some code from Soong to support creating a property struct
at runtime by filtering fields out of another property struct.

Test: TestFilterPropertyStruct
Change-Id: Ic5ae390a885195bebad6f3ecb7c752c0582a60b1
2019-09-26 14:21:40 -07:00
colincross
5639ec157c
Merge pull request #265 from colincross/int_default
Add proptools.Int and proptools.IntDefault
2019-09-26 10:41:52 -07:00
Colin Cross
6af7284d08 Make CreateModule return the newly created module
Allow mutators to modify properties of the newly created module
by returning it.

Change-Id: Ia1acc70b6196195d1bad58c88ab3def70b99e42c
2019-09-25 14:53:25 -07:00
Colin Cross
66c0b13553 Add proptools.Int and proptools.IntDefault
Add proptools.Int and proptools.IntDefault that behave analogously
to proptools.String and proptools.StringDefault.

Change-Id: I41fd3417c973c9ff4a5aa6680546b4b893784745
2019-09-25 14:52:54 -07:00
colincross
884e7374fb
Merge pull request #263 from jiyongp/reverse_dep
Don't ignore local variations when creating reverse dep
2019-09-22 17:11:21 -07:00
Jiyong Park
df4ed95f43 Don't ignore local variations when creating reverse dep
This fixes a bug that reverse dependency can't be made for modules
having local variations. Previously, when module A having local variants
calls AddReverseDependency to module B having local variants, the match
is tested between the non-local variants of module A against all
variants of module B, which can never be successful.

This change fixes it by using all variants of module A when
findMatchingVariants is called for AddReverseDependency.

Test: m
Change-Id: Ib289188a5dd58c72bd6ba07e3c0f825f8b1c6b1b
2019-09-22 07:44:52 +09:00
colincross
658e01c1d7
Merge pull request #262 from skvadrik/master
Added VisitDirectDeps* methods to SingletonContext.
2019-09-21 08:17:02 -07:00
Ulya Trafimovich
811381a585 Added VisitDirectDeps* methods to SingletonContext.
A an example use case is generating the graph of module
dependencies, where modules are vertices and dependencies
are edges.

The patch was tested with a local implementation of such
a module in the Soong build system. The graph was generated
for all modules in the AOSP tree.
2019-09-20 23:58:23 +01:00
colincross
87050d9327
Merge pull request #261 from colincross/go1.13
Fix compiling binaries with go 1.13
2019-09-06 14:43:56 -07:00
Colin Cross
a2bc585afe Fix compiling binaries with go 1.13
The go compile command in go 1.13 needs -p main instead of
-p $pkgPath when compiling binaries.  This also works for older
go releases.

Also turn on travis for go 1.13.

Change-Id: I45ce185aa44cfa01a7933cc6e52290a85c851641
2019-09-06 14:25:28 -07:00
Dan Willemsen
3f22fcf3a8
Merge pull request #260 from jooyunghan/fix-caller
Fix checkCalledFromInit
2019-09-06 09:36:39 -07:00
Jooyung Han
5dd43cf9b4 Fix checkCalledFromInit
This function checks if it is called from init() by looking into
callers. By the way, it may fail when init() is inlined.

To fix this, CallersFrames() is used to translate PCs into symbolic
information accounting for inlined functions.

Test: go test ./...
Change-Id: I18b3c3ffdaf71f775e3522c87607aec5df7b09c5
2019-09-05 18:10:37 +09:00
colincross
a0b59d9d29
Merge pull request #259 from colincross/default_dependency_variation
Add SetDefaultDependencyVariation
2019-08-09 12:53:48 -07:00
Jiyong Park
1e2e56dc62 Add SetDefaultDependencyVariation
SetDefaultDependencyVariation sets the variation name that will be used
when a dangling dependency is found while a module is being split. A
dangling dependency can occur if a module is split to a variant that one
of its dependencies is not split into. When the default variation is not
set, such dangling dependency is a hard error. But with the new
function, the default variation can be set and subsequent calls to
CreateVariations and its variations on the same context uses the default
variation when necessary.

(If even the default variation does not exist for the dependent module,
it is an hard error)

Note that this is different from calling SetDependencyVariation("foo")
followed by CreateVariations("foo", "bar"). In that case, regardless of
whether a dependency of the current module has the variant 'bar' or not,
only the 'foo' variant is chosen.

With SetDefaultDependencyVariation("foo") followed by
CreateVariations("foo", "bar"), 'foo' variant is used only when the
'bar' variant of the current module depends on a module that does not
have 'bar' variant.

Bug: 138103882
Test: m
Change-Id: I4520ca87487994de024fdbacda3bef6636225f0d
2019-08-09 12:49:11 -07:00
colincross
876099e5e7
Merge pull request #258 from colincross/console
Revert "Put the primary builder into the console pool"
2019-07-18 15:18:22 -07:00
Colin Cross
0e112a8b97 Revert "Put the primary builder into the console pool"
This reverts commit 8ea996f61d.

Having the primary builder hinders automatic error reporting
by separating the error message from the failure status.
2019-07-18 13:53:43 -07:00
Jaewoong Jung
72ab2b4c95
Merge pull request #257 from gyias/master
Fix/improve comments on prepending/appending props
2019-07-08 16:38:34 -07:00
Jaewoong Jung
c067251da0 Fix/improve comments on prepending/appending props
Test: N/A
Change-Id: Id95afbea7fbccddafb9f2f0e068967d5547cc469
2019-07-08 14:07:45 -07:00
colincross
1589d6586b
Merge pull request #255 from colincross/mutator_name
Add MutatorName to mutator contexts
2019-07-02 11:05:28 -07:00
Colin Cross
d8367ff746 Add MutatorName to mutator contexts
Allow querying the mutator name for debugging.

Bug: 136473661
Change-Id: I6a9f1f8cc4fd456d32cc2f2ef85a497c193d7e09
2019-07-02 10:08:55 -07:00
colincross
e3fb08a2cd
Merge pull request #252 from colincross/console
Put the primary builder into the console pool
2019-07-01 23:09:02 -07:00
Jaewoong Jung
d744b671f8
Merge pull request #254 from gyias/master
Make off-the-shelf order funcs public.
2019-06-23 18:04:15 -07:00
Jaewoong Jung
4764fa76fe Make off-the-shelf order funcs public.
These are useful outside the package too when calling
proptools.ExtendMatchingProperties.

Change-Id: I054eb105e0dd5287aff99b8be137a8b09d52492d
2019-06-22 11:22:55 -07:00
colincross
94423de307
Merge pull request #253 from colincross/optimize
Optimize ninjaString.ValueWithEscaper
2019-06-20 11:25:59 -07:00
Colin Cross
19ff727ad5 Optimize ninjaString.ValueWithEscaper
ninjaString.ValueWithEscaper is a relatively hot function,
rewrite it with strings.Builder to avoid repeated string
concatenation, which requires an allocation each time.

Before:
BenchmarkNinjaString_Value/constant/1-72       	100000000	        11.9 ns/op
BenchmarkNinjaString_Value/constant/10-72      	100000000	        18.9 ns/op
BenchmarkNinjaString_Value/constant/100-72     	50000000	        22.1 ns/op
BenchmarkNinjaString_Value/constant/1000-72    	30000000	        39.3 ns/op
BenchmarkNinjaString_Value/variable/1-72       	20000000	        95.1 ns/op
BenchmarkNinjaString_Value/variable/10-72      	10000000	       223 ns/op
BenchmarkNinjaString_Value/variable/100-72     	 3000000	       437 ns/op
BenchmarkNinjaString_Value/variable/1000-72    	 2000000	       948 ns/op
BenchmarkNinjaString_Value/variables/1-72      	10000000	       161 ns/op
BenchmarkNinjaString_Value/variables/2-72      	 5000000	       368 ns/op
BenchmarkNinjaString_Value/variables/3-72      	 3000000	       560 ns/op
BenchmarkNinjaString_Value/variables/4-72      	 2000000	       795 ns/op
BenchmarkNinjaString_Value/variables/5-72      	 1000000	      1004 ns/op
BenchmarkNinjaString_Value/variables/10-72     	 1000000	      2275 ns/op
BenchmarkNinjaString_Value/variables/100-72    	   50000	     39667 ns/op
BenchmarkNinjaString_Value/variables/1000-72   	    1000	   2146592 ns/op

After:
BenchmarkNinjaString_Value/constant/1-72       	200000000	        11.3 ns/op
BenchmarkNinjaString_Value/constant/10-72      	100000000	        17.2 ns/op
BenchmarkNinjaString_Value/constant/100-72     	50000000	        21.7 ns/op
BenchmarkNinjaString_Value/constant/1000-72    	30000000	        38.3 ns/op
BenchmarkNinjaString_Value/variable/1-72       	20000000	        91.8 ns/op
BenchmarkNinjaString_Value/variable/10-72      	10000000	       199 ns/op
BenchmarkNinjaString_Value/variable/100-72     	 5000000	       377 ns/op
BenchmarkNinjaString_Value/variable/1000-72    	 2000000	       855 ns/op
BenchmarkNinjaString_Value/variables/1-72      	10000000	       141 ns/op
BenchmarkNinjaString_Value/variables/2-72      	 5000000	       312 ns/op
BenchmarkNinjaString_Value/variables/3-72      	 5000000	       362 ns/op
BenchmarkNinjaString_Value/variables/4-72      	 3000000	       417 ns/op
BenchmarkNinjaString_Value/variables/5-72      	 2000000	       621 ns/op
BenchmarkNinjaString_Value/variables/10-72     	 2000000	       837 ns/op
BenchmarkNinjaString_Value/variables/100-72    	  200000	      9141 ns/op
BenchmarkNinjaString_Value/variables/1000-72   	   20000	     95094 ns/op

Test: ninja_strings_test.go
Change-Id: I6c61e747d8e67f7f1e6cff0cc0c705745301a35f
2019-06-20 11:05:58 -07:00
Colin Cross
4f6631f74f Remove go 1.9 testing
strings.Builder is not in go 1.9.

Change-Id: If605026850af2819d2a31bb920837150e5031a71
2019-06-20 11:05:58 -07:00