Commit graph

727 commits

Author SHA1 Message Date
Colin Cross
8ea996f61d Put the primary builder into the console pool
The primary builder should always be running on its own with no
other processes running in parallel, so put it in the console
pool so that it can print to stdout without buffering.

Bug: 80165685
Test: build with print in Soong
Change-Id: If34ecdb5fa18de7e47c4cd6965d551c504850176
2019-06-19 14:39:41 -07:00
colincross
a466306423
Merge pull request #250 from gyias/master
Skip mutated struct properties in bpdoc.
2019-06-05 18:44:15 -07:00
Jaewoong Jung
bd0f6c3488 Skip mutated struct properties in bpdoc.
This change fixes a bug where the `blueprint:mutated` tag was ignored
when used for struct or interface properties. It also fixes a dormant
ExcludeByTag func bug.

Test: m soong_docs, properties_test.go, bpdoc_test.go
Change-Id: I926de2aa203ec552ced897174e9b78e817701a7d
2019-06-05 16:52:47 -07:00
colincross
e263995312
Merge pull request #251 from colincross/glob
Avoid exiting with errors in bpglob
2019-06-03 16:26:23 -07:00
Colin Cross
c708e1c9e3 Avoid exiting with errors in bpglob
bpglob is only used as a helper to check if the primary builder
needs to rerun due to the results of a glob changing.  A recent
change to glob support in pathtools made a glob format that was
accidentally previously accepted into an error.  If the bad
syntax was used in the most recent primary builder run, and
then an incremental build is performed after picking up the
change that made the syntax invalid, then bpglob will attempt
to rerun before the primary builder, see the now-invalid syntax,
and fail.  This will prevent the primary builder from rerunning,
which would have updated the bpglob rule with a corrected glob
syntax (or failed in the primary builder if the Blueprints file
still had the invalid glob syntax).

Avoid exiting with an error in bpglob.  Instead, write the
error to the output file along with a timestamp so that it is
always dirty, forcing the primary builder to rerun.

Bug: 129411151
Test: m nothing
Change-Id: Ib680570c33662f3c0f1f72425d60a963ed841ba6
2019-05-31 15:38:12 -07:00
colincross
abd66e6c82
Merge pull request #247 from colincross/mutator_ctx
Improve module, mutator and singleton contexts
2019-05-21 11:26:19 -07:00
colincross
2201ee55db
Merge pull request #248 from colincross/match
Simplify pathtools.Match
2019-05-21 11:26:10 -07:00
colincross
8ac6d8fcd9
Merge pull request #249 from colincross/bpfmt
Correctly report errors in bpfmt
2019-05-21 11:25:51 -07:00
Colin Cross
e52c33c61c Correctly report errors in bpfmt
bpfmt was always exiting with return code 0, make it use the
exitCode recorded when reporting errors.

Also use the usage function.

Test: bpfmt -o bpfmt.go
Test: bpfmt
Test: bpfmt -h
Fixes: 113069050
Change-Id: I717d524c5f43cd36f52e33964a37324cc04c5df0
2019-05-21 11:03:00 -07:00
Colin Cross
322cc01803 Report creating modules in errors on created modules
Bug: 133172285
Change-Id: I0782c722f5877508691868f96dbf7dc0bed3f9f4
2019-05-20 15:09:48 -07:00
Colin Cross
7bcc256dac Move module and singleton godoc from the implementation to the interface
The interface is visible in godoc, not the implementation, so
move the documentation onto the interface methods.

Test: view godoc
Change-Id: I3460003eed6bf57d8ae4dabe70ab306f049a0060
2019-05-20 15:08:39 -07:00
Colin Cross
99bdb2ab4f Consolidate mutator contexts
Move most of the mutator context methods into BaseModuleContext
so they are available to both top down and bottom up mutators.
Only CreateModule is unique to TopDownMutatorContext, and the
dependency and variation adding methods are unique to the
BottomUpMutatorContext.

The dependency visiting methods are now available on
BottomUpMutatorContext, which requires delaying making newly
added dependencies visible to maintain the invariant that
the mutator has been called on the dependency before the
dependency can be visited by its parents.

Test: m checkbuild
Change-Id: Ie14afc02ac76d0b5a66b0e52de2aa9e17fd1bec0
2019-05-20 14:58:14 -07:00
Colin Cross
818af3bfe0 Fix panic message in SingletonContext.VisitAllModules
The panic should be printing the module info string from the
moduleInfo, not trying to Stringify the Module interface.

Test: manual
Change-Id: I1a2b7c15c4216518c76b909b25d50ac2eb7d1baa
2019-05-20 14:58:06 -07:00
Colin Cross
a26ae89903 Simplify pathtools.Match
filepath.Match has always supported hierarchical patterns (a/*),
so pathtools.Match can be significantly simplified by reusing
filepath.Match directly instead of recursively stepping through
the path elements using filepath.Match on each one.

Test: glob_test.go
Change-Id: I8af59ee880f0402609b994922bafb1961fcabcf3
2019-05-20 14:57:36 -07:00
Colin Cross
a4cc6837db Add more pathtools.Match test cases
Add some more test cases for pathtools.Match before making
signficant changes to it.

Test: glob_test.go
Change-Id: I2eeb5ebf03fb645a2053852a1a9f4e368d22084a
2019-05-20 14:57:31 -07:00
Dan Willemsen
19ef680bc4
Merge pull request #246 from roman-mazur/patch-1
Fix Ninja build system site link
2019-03-23 07:40:25 -07:00
Roman Mazur
4d772202dc
Fix Ninja build system site link 2019-03-23 16:31:07 +02:00
Jaewoong Jung
08ab48903b
Merge pull request #244 from gyias/master
bpdoc preformatted text improvments
2019-03-11 14:25:29 -07:00
Jaewoong Jung
8bc6bf1090 bpdoc preformatted text improvments
1. Apply preformatted text (<pre>...</pre>) handling logic to module
type text too. It used to be applied to property texts only.
2. Improve <pre> handling logic itself for better readability.
- Insert a blank line before <pre>.
- Prevent from ending <pre> blocks prematurely by checking if an
unindented line isn't just a blank line between indented lines.

Change-Id: Id40fa668d4c6781caf7ed140b2f40784cdeb8c35
2019-03-11 11:16:31 -07:00
Jaewoong Jung
eaf410fe20
Merge pull request #243 from patricearruda/master
Moving the ParseBlueprintsFiles comment in context.go.
2019-03-08 14:28:58 -08:00
Patrice Arruda
b0a40a717a Moving the ParseBlueprintsFiles go comment in context.go.
The go comment for the blueprint.Context.ParseBlueprintsFiles was
actually place above the blueprint.Context.ParseFileList.

Bug: N/A
Test: N/A

Change-Id: I065d2f6ec034c614e40de745931f4c87cdb73422
2019-03-08 14:17:43 -08:00
Jaewoong Jung
5e9f9070d5
Merge pull request #242 from gyias/bpdoc
Panic when copying private properties.
2019-03-08 09:24:49 -08:00
Jaewoong Jung
7d699c37ec Panic when copying private properties.
When CopyProperties comes across private properties, it silently ignores
them. This is error-prone and can result in a bug very hard to debug if
the developer is unaware of the behavior.

Change-Id: Id6a0752e384ec7fed35728c1e87dbfa95fea84f2
2019-03-07 16:24:56 -08:00
colincross
a9df651359
Merge pull request #241 from colincross/tagindex
Add PropertyIndexesWithTag
2019-03-05 23:15:15 -08:00
Colin Cross
a4f6a3bebf Add PropertyIndexesWithTag
Add a function that returns all of the indexes to properties in
a property struct that are tagged with `name:"value"`.

Test: proptools/tag_test.go
Change-Id: I00294934c1a0383c8b64ecaabc0e138682efb2e5
2019-03-05 19:47:43 -08:00
colincross
17a0b96440
Merge pull request #240 from colincross/other_module
Add more OtherModule* methods
2019-03-03 17:06:47 -08:00
Colin Cross
d28b304d30 Add more OtherModule* methods
Add OtherModuleDir, OtherModuleSubDir, and OtherModuleType to
ModuleContext and TopDownMutatorContext, and add ModuleType to
BaseModuleContext.

Change-Id: If5c873a2620ef10333c0bdba5ab89d4256e5fdf2
2019-03-01 13:44:15 -08:00
colincross
bf8c1d31d9
Merge pull request #239 from colincross/go1.12
Support go 1.12
2019-02-28 19:52:33 -08:00
Colin Cross
16b01609ce Run go 1.11 and go 1.12 in travis
Change-Id: If8d53ca83fb97cc6b68e3e5f4025119bb3ea2502
2019-02-28 14:42:04 -08:00
Colin Cross
f7104c0ad6 Support go 1.12
go 1.12 changed the stack trace of a function call during global
variable initialization to contain a function called
"PKG.init.ializers".  Fix callerName to split the package path
and function name on the first "." after the last "/", and look
for functions called "init.ializers" or functions with the
prefix "init."

Change-Id: Ic2190837a8c4cde075f727bd69dd18d517a0ebc0
2019-02-28 14:38:42 -08:00
colincross
284c742a33
Merge pull request #219 from colincross/escape_list
Replace *Escape with *EscapeList
2019-02-28 11:14:02 -08:00
colincross
02c110735d
Merge pull request #238 from allight/rebase-add-java-services
GetDirectDepWithTag needs to check all tags before panicing
2019-02-27 16:13:57 -08:00
Alex Light
142de248fa GetDirectDepWithTag needs to check all tags before panicing
GetDirectDeWithTag would panic if the first possible directDep did not
have the correct tag. It should check to see if any version of that
dependency has the right tag.

Test: atest CtsJdwpTunnelHostTestCases
Bug: 124507633
Bug: 125933724
Change-Id: I00d269130e9f136a93fd30c58b8fd929372d5b37
2019-02-27 15:34:26 -08:00
colincross
8908a0a806
Merge pull request #237 from colincross/singleton
Allow Context to query Singletons
2019-02-26 10:54:08 -08:00
Colin Cross
9226d6c9f5 Allow Context to query Singletons
Add Context.Singletons and Context.SingletonName to allow primary
builders to query Singletons returned by all registered
SingletonFactories.

Change-Id: Iee85643dd47f7472e6bb5ae8004374bd6ea0419e
2019-02-26 10:15:40 -08:00
asmundak
d444506bf6
Merge pull request #235 from asmundak/collapseduplicate
Export CollapseDuplicateProperties, Soong writedocs uses it
2019-02-17 16:36:43 -08:00
colincross
58edc0fa5c
Merge pull request #236 from colincross/examples
Support examples as tests
2019-02-15 16:39:32 -08:00
Colin Cross
463eab7950 Support examples as tests
go test supports running examples as tests and verifying their
output, add support for running them to gotestmain.

Change-Id: If51abc64467d4701195cefc6db513f4d008794b5
2019-02-15 15:13:37 -08:00
Sasha Smundak
797563bec4 Export CollapseDuplicateProperties, Soong writedocs uses it 2019-02-14 10:58:48 -08:00
Jaewoong Jung
33cfa1c98a
Merge pull request #234 from gyias/bpdoc
bpdoc improvements
2019-02-12 11:32:37 -08:00
Jaewoong Jung
781f6b2896 bpdoc improvements
1. Extract module type documentation.
2. Support primary builder customization of factory function to use for
documentation for each module type.
3. Change the ModuleType list order so that they are grouped by package.

This is basically minor refactoring + readability improvement done on
top of https://github.com/google/blueprint/pull/232.

Change-Id: If7413e5ac23486b85f18d02fb3ba288a38730c32
2019-02-08 15:48:27 -08:00
colincross
3eeabc7991
Merge pull request #231 from colincross/pprof
Performance improvements
2019-01-24 10:32:25 -08:00
Colin Cross
937efab2e6 Use sync.Map for type field cache
The type field cache was using an atomic pointer to an immutable
map, which required copying the entire map each time a cache
entry was added.  Profiling showed that this was a significant
hot spot.  Use sync.Map instead.

Change-Id: Ie7c779c5e9e2be1cd530747d74025dcfd206763a
2019-01-23 13:26:42 -08:00
Colin Cross
de7afaaf74 Write ninja file directly to the output file
Writing the ninja file to a byte buffer causes a significant amount
of time to be spent in memmove when growing the byte slice.  Write
the file directly to disk instead.

This also fixes some unhandled error warnings, which become more
likely when doing disk IO instead of byte buffer writes.

Change-Id: I5094e4c45cab4012713037f60c5a4fb00718f92e
2019-01-23 13:26:42 -08:00
Colin Cross
3a8c025648 Add pprof labels
Add pprof labels to the larger stages of blueprint, including
labels for each mutator.

Change-Id: Ie91daff51811187c1b702a775c05d0b32f7170fc
2019-01-23 13:26:42 -08:00
Dan Willemsen
716137ec38
Merge pull request #230 from smore-lore/master
bpmodify: add argument if it is missing
2018-12-26 16:43:54 -08:00
Steven Moreland
b40aaadbca bpmodify: add argument if it is missing
before, adding to a list which had no entries would do nothing (and not
throw any errors). Now it will create the list and add a single element
as expected.

Change-Id: I8c48a42303f7d9b3741868ad86097e2071ec434a
2018-12-17 18:16:16 -08:00
Dan Willemsen
f8565113e1
Merge pull request #228 from danw/empty_ninja_file
Add --empty-ninja-file for test usecases
2018-10-26 10:00:37 -07:00
Dan Willemsen
dac90d33ca Add --empty-ninja-file for test usecases
In cases that we want to run blueprint-based builds in many
configurations to verify all of the logic works without errors, but
don't care about running the final ninja file, writing it out only
wastes time and disk space. So add a --empty-ninja-file option that
writes out an empty ninja file instead.

Our specific use case (Soong's build_test / multiproduct_kati) runs
Soong several hundred times for different configurations, and the ninja
files are around 1GB, which leads to several hundred gigabytes of disk
writes (and persistent use during incremental generation).

Change-Id: I0198dfb2f744ce22284c05d5214dac2ab5dc9700
2018-10-26 09:57:24 -07:00
colincross
19099e68bb
Merge pull request #227 from colincross/testdeps
Add link dependency on test package archive
2018-10-24 15:04:22 -07:00