Commit graph

1620 commits

Author SHA1 Message Date
Colin Cross
00890dd8f6 Optimize arg parsing in buildDef.WriteTo
Arguments to build definitions were copied from the input map to an
map with the name and value expanded, then to a list of names for
sorting, and then written, which required iterating over a map three
times.  Expand the name and value into a list of name value pairs,
and then do the rest of the operations on the list instead.

Test: ninja_writer_test.go
Change-Id: Id8ff644dafbaa3b4812747c60dc28cce22e21dbe
2021-01-21 22:02:30 -08:00
Colin Cross
c8b9e55289 Increase ninja file writer buffer
Increase the ninja file writer buffer from the default 4k to 16MB.

Test: m checkbuild
Change-Id: Ieb2c82218517b98469ef93f1ea4dd04b5651f7d1
2021-01-21 22:02:30 -08:00
Colin Cross
0335e0900d Use io.StringWriter in ninjaWriter
ninjaWriter repeatedly called io.WriteString() on its writer, which
does a type assertion every time.  Replace its io.Writer with an
io.StringWriter and call WriteString on it directly.

Test: ninja_writer_test.go
Change-Id: Ie073d996a319190242bf6a00af07a13a60d078b5
2021-01-21 22:02:30 -08:00
Colin Cross
eecfdd316b Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Return a copy of glob lists

Test: m checkbuild
Test: m nothing && m nothing
Change-Id: Ie015670828f3d6e0bf4fa22580cb7254966a2c82
2021-01-21 18:28:20 -08:00
colincross
bcd5686660
Merge pull request #341 from colincross/optimize_glob
Return a copy of glob lists
2021-01-21 18:27:49 -08:00
Colin Cross
a64ca94c0d Return a copy of glob lists
Callers to glob methods may do in-place modifications on the returned
list of globs, return a copy instead of the cached value.

Test: m nothing && m nothing
Change-Id: Ic9140d1e1900e8724ba0a484f27786e5c15dea90
2021-01-21 13:47:59 -08:00
Colin Cross
3d9a6b397e Merge remote-tracking branch 'aosp/upstream' am: e7bc78aa7b
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1553908

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I58c3604c7fd477418f30179663050767cfa4fb75
2021-01-21 00:58:23 +00:00
Colin Cross
ee9fe36a1c Merge remote-tracking branch 'aosp/upstream' am: 92ca1968cf
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1554574

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I4f5d18359f76614010b0f7c87ab92f644e388f92
2021-01-20 22:14:03 +00:00
Colin Cross
e7bc78aa7b Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Use info.IsDir() instead of info.Mode()&os.ModeDir != 0
  Remove redundant stat/lstat calls from glob

Test: glob_test.go
Change-Id: Iea8f093ca16a57de4c3ef0935f29dfe871ac4a72
2021-01-20 14:01:03 -08:00
colincross
2b2b9f28f2
Merge pull request #340 from colincross/optimize_glob
Use info.IsDir() instead of info.Mode()&os.ModeDir != 0
2021-01-20 13:52:33 -08:00
Colin Cross
4604a81721 Use info.IsDir() instead of info.Mode()&os.ModeDir != 0
Use the info.IsDir() shortcut.

Test: glob_test.go
Change-Id: Ibba8bbc6af295e3d86321b0c070a84c2b7008dd9
2021-01-20 13:47:37 -08:00
colincross
49a8b95d4e
Merge pull request #339 from colincross/optimize_glob
Remove redundant stat/lstat calls from glob
2021-01-20 13:34:51 -08:00
Colin Cross
e535c97cc9 Remove redundant stat/lstat calls from glob
Glob was calling IsSymlink and IsDir on each visited directory entry,
which resulted in an lstat and then a stat call on each.

Instead, use lstat when not following symlinks and use stat when
following symlinks, then use the result to check if the entry is a
directory.

Test: glob_test.go
Change-Id: I83d769e2de64ce8221e952e5204d365aeaf47687
2021-01-20 11:01:41 -08:00
Colin Cross
92ca1968cf Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Fix silently ignoring values assigned to map properties
  Use UnpackError for incorrect property type errors
  Improve unpack testing

Fixes: 177706602
Test: m nothing
Test: TestUnpackErrors
Change-Id: I2c6b0279dc6210d5c6fef12e19e54b98c473e78b
2021-01-19 11:09:10 -08:00
colincross
94a5c01693
Merge pull request #337 from colincross/map_assign
Fix silently ignoring values assigned to map properties
2021-01-19 10:35:33 -08:00
Colin Cross
3adb240964 Fix silently ignoring values assigned to map properties
Values assigned to map properties were silently ignored instead of
reported as an error.  Add a check when recursing into structs that
the value is a map.

Fixes: 177706602
Test: m nothing
Test: TestUnpackErrors
Change-Id: Ic56aeb1b9da6d5c86b6d98adae7bddb60c450404
2021-01-15 22:21:28 -08:00
Colin Cross
f6ef155884 Use UnpackError for incorrect property type errors
Returning a fmt.Errorf error causes Blueprint to identify the error
as an "internal error", return an UnpackError instead.

Bug: 177706602
Test: TestUnpackErrors
Change-Id: I19fba134ad778d08d5a4d90a0335bdf8cbea6a20
2021-01-15 22:21:28 -08:00
Colin Cross
461d95061e Improve unpack testing
Add more tests, give the tests names, run them as subtests, and add
benchmarks.

Test: unpack_test.go
Change-Id: Iff22538ce44ed503d5d088cfb55673448db998f1
2021-01-15 22:21:28 -08:00
Colin Cross
819320dabc Merge remote-tracking branch 'aosp/upstream' am: 52176e9851 am: 9492c29702
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1527662

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: If8f454d33ec6def0f7f13ab4866be05ed73cf0a7
2020-12-14 22:52:01 +00:00
Colin Cross
9492c29702 Merge remote-tracking branch 'aosp/upstream' am: 52176e9851
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1527662

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I836eb4971d005de6ca9144ea248790a0e5e86398
2020-12-14 22:27:39 +00:00
Colin Cross
52176e9851 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  bpmodify: support numerical sort (#332)
  Actually pause mutators when adding dependencies (#335)

Test: m checkbuild
Change-Id: I2ebccd43a2c5b7fd4796861fb519b6401a2c45b3
2020-12-14 11:37:37 -08:00
Jooyung Han
53e92a0de6
bpmodify: support numerical sort (#332)
bpmodify sorts touched list respecting numbers embedded in strings. For
example, "foo-2" comes before "foo-10".

Test: bpmodify_test.go
Change-Id: If2fe9bc871a463a47dd3c0b52982b34c9fde05f0

Co-authored-by: Jooyung Han <jooyung@google.com>
2020-12-14 11:30:48 -08:00
colincross
cfa2e91dc4
Actually pause mutators when adding dependencies (#335)
In the coordination between #316 and #318 the calls to mctx.pause()
were forgotten, causing dependencies returned by the Add*Dependency
calls to be in an undefined state.  Add the missing calls to
mctx.pause().

Change-Id: I648ad269449777363801785059b13b866424d4b5
2020-12-14 11:23:23 -08:00
Liz Kammer
1c4364276d Merge remote-tracking branch 'aosp/upstream' am: f41959b79a am: 14e63c229f
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1512683

Change-Id: Id4064a4b8ada8082c064b44a655a1eac6d2b09bb
2020-12-01 17:30:07 +00:00
Liz Kammer
14e63c229f Merge remote-tracking branch 'aosp/upstream' am: f41959b79a
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1512683

Change-Id: Ie8e7065b40c366b52100e346571547b2047f7181
2020-12-01 17:11:41 +00:00
Liz Kammer
f41959b79a Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Invalidate module group cache if deps are modified (#334)
  Fix github builds for deprecated set-env and add-path commands (#333)
  Improve formatting of some error messages. (#327)

Test: m nothing
Change-Id: I97d1f69cdb500a5083f5c6e228ce4cc35d937c5d
2020-11-30 16:08:41 -08:00
Liz Kammer
9ae14f12f9
Invalidate module group cache if deps are modified (#334)
* Invalidate module group cache if deps are modified

PreSingletons run after the blueprints have been parsed and can run
VisitAllModules; however, this seeds the cache of sorted modules which
may change after mutators have run. 

This causes recomputation of the cache if any deps have been
modified since the last time it was run.

Change-Id: I79fc822dd630f84790f309ba4e6024588a8fe28e
2020-11-30 16:30:45 -07:00
colincross
16754d14e2
Fix github builds for deprecated set-env and add-path commands (#333)
actions/setup-go@v2.0.3 seems to cause issues with
https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

Change the precise versions to major versions to get updates.

Also fix the "Install ninja" action to use $GITHUB_PATH.

Change-Id: Ia3ec693e405a2530350736e7856ad758af3fb027
2020-11-23 16:52:58 -08:00
Xin Li
28cb7d054c Mark ab/6881855 as merged am: 96e94e8469
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/blueprint/+/13111881

Change-Id: Ieffbfc154c2fa1843d20f210375a00277ef3fa61
2020-11-24 00:10:16 +00:00
Xin Li
96e94e8469 Mark ab/6881855 as merged
Bug: 172690556
Change-Id: Ib5ca116881a445241dc9d3ce6a178eaa05ed9c0d
2020-11-23 11:27:05 -08:00
Martin Stjernholm
0f1637b19b
Improve formatting of some error messages. (#327)
In particular the formatting of dependencies with variants which lacked
braces that caused them to float together with the dependency names.

Also add some context to the ReplaceDependenciesIf panic message.

Test: m
Change-Id: Ibd03690317ca53f3ecbdd95033404d89d849b4dd
2020-11-16 12:15:36 -08:00
Chris Parsons
aed8eb35de Merge remote-tracking branch 'upstream/master' into pull-upstream-3 am: e5dbb9140d am: ec7e2de0fb am: fed2602691
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1495096

Change-Id: Ia86903831cbe183ba55a13945ddc51b4e36fe34a
2020-11-11 02:02:18 +00:00
Chris Parsons
fed2602691 Merge remote-tracking branch 'upstream/master' into pull-upstream-3 am: e5dbb9140d am: ec7e2de0fb
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1495096

Change-Id: Ic5150736e1cea971cd9dcc74fe65ba59eb8c2ea2
2020-11-11 01:39:50 +00:00
Chris Parsons
ec7e2de0fb Merge remote-tracking branch 'upstream/master' into pull-upstream-3 am: e5dbb9140d
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1495096

Change-Id: Ib4a5b001573b85931be21db01874d3896beac3a3
2020-11-11 01:22:06 +00:00
Chris Parsons
e5dbb9140d Merge remote-tracking branch 'upstream/master' into pull-upstream-3
* upstream/master:
  Remove unnecessary depFile parameter from GlobFile
  Create early-exit hook StopBeforeWriteNinja

Test: m nothing
Test: Treehugger
Change-Id: I0f9ed211f99c2e4d3d208237a7b00f7cd9cd0af4
2020-11-10 16:14:21 -05:00
colincross
660bb7a3e5
Merge pull request #330 from colincross/glob
Remove unnecessary depFile parameter from GlobFile
2020-11-10 12:34:10 -08:00
Colin Cross
7ceeeaf364 Remove unnecessary depFile parameter from GlobFile
GlobRule uses $out.d for the dep file, it doesn't need an extra
parameter.

Test: glob_test.go
Change-Id: Ie753a119c3602c325450909f0240f6eeef9ef243
2020-11-10 12:24:17 -08:00
c-parsons
bcdfcceeb2
Create early-exit hook StopBeforeWriteNinja
This allows exiting bootstrap directly before writing ninja files.
This facilitates shorter runtime on integrations which do processing
which do not require ninja file output.

Test: Manually verified on Soong integration use case which involves
running bootstrap twice in a single program; stopping before ninja
output reduces runtime by ~20s, or ~11%.
2020-11-05 16:26:08 -05:00
Chris Parsons
5e834261a9 Create early-exit hook StopBeforeWriteNinja
This allows exiting bootstrap directly before writing ninja files.
This facilitates shorter runtime on integrations which do processing
which do not require ninja file output.

Test: Manually verified on Soong integration use case which involves
  running bootstrap twice in a single program; stopping before ninja
  output reduces runtime by ~20s, or ~11%.
2020-11-05 11:17:32 -05:00
Liz Kammer
22a4db7651 Merge remote-tracking branch 'aosp/upstream' am: 8e7ee04085 am: ea33b30d21 am: 1e67daa70c
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1487936

Change-Id: I2ed9840a8a362a47b83425d711bffe4053098f56
2020-11-05 05:30:37 +00:00
Liz Kammer
1e67daa70c Merge remote-tracking branch 'aosp/upstream' am: 8e7ee04085 am: ea33b30d21
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1487936

Change-Id: I4860ee7ef6747f6db577d55748b271e93dd39097
2020-11-05 05:03:50 +00:00
Liz Kammer
ea33b30d21 Merge remote-tracking branch 'aosp/upstream' am: 8e7ee04085
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1487936

Change-Id: Ia569f2cfb65e8eae41259656d70bcdb9764695e3
2020-11-05 04:29:22 +00:00
Liz Kammer
8e7ee04085 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Identify the type of the list for bpdocs (#326)

Test: m soong_docs
Change-Id: I2760596d9e6a1848f21600e92b259f482c4bc2c7
2020-11-04 17:33:13 -08:00
Liz Kammer
764a771917
Identify the type of the list for bpdocs (#326)
* Identify the type of the list for bpdocs

Test: go test bpdoc tests
Test: m soong_docs
Change-Id: I6a4a916e1f72b3fc702da90c32a2eddca70b3bac
2020-11-04 17:20:11 -08:00
Jingwen Chen
553283e969 Merge remote-tracking branch 'upstream/master' into update-upstream am: 1e649618d2 am: bc2659ab68 am: cb66253152
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1466244

Change-Id: I4ab7794d233fb6a51564800330e4f67a3dbe0518
2020-10-20 02:46:54 +00:00
Jingwen Chen
cb66253152 Merge remote-tracking branch 'upstream/master' into update-upstream am: 1e649618d2 am: bc2659ab68
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1466244

Change-Id: I3bed853fd7ce27bf133ca4742ba9a82a922df64e
2020-10-20 02:27:23 +00:00
Jingwen Chen
bc2659ab68 Merge remote-tracking branch 'upstream/master' into update-upstream am: 1e649618d2
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1466244

Change-Id: I7ad672d2b2be7d02b561925271805ab2950a4597
2020-10-20 01:57:15 +00:00
Jingwen Chen
1e649618d2 Merge remote-tracking branch 'upstream/master' into update-upstream
* upstream/master:
  Add support for symlink_outputs to Blueprint
  Add CODEOWNERS: @google/blueprint

Change-Id: I15ced72d913a6dc0b6d60b4772bcefb415a63537
2020-10-19 20:18:15 -04:00
Liz Kammer
ed35255eca
Merge pull request #322 from jin/symlink_outputs
Add support for symlink_outputs to Blueprint
2020-10-19 08:52:08 -07:00
Joe Onorato
e312377e2c Refactor OWNERS in platform/build/blueprint am: 7ae25b01ac am: 0e42907d7a am: 812c9665e7
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1455043

Change-Id: I72a451b2075a770828ec36f931d80436bea4421b
2020-10-09 05:13:11 +00:00