Commit graph

1861 commits

Author SHA1 Message Date
Lukacs T. Berki
e4b39574a3 cd / before generating build.ninja . am: f802ffcd12
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1592052

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I3b17e9de0999fc055d1adfe76b529ce70a649299
2021-03-04 07:58:46 +00:00
Lukacs T. Berki
122b3ee153 Make debugging easier:
- Add the -N -l command line arguments to Microfactory
- Pass the the Delve arguments to the primary builder on the command line

Test: Manual.

Change-Id: I4034f2d48b3e40d9863529053715183dc3dce1f5
2021-03-03 09:15:04 +01:00
Lukacs T. Berki
f802ffcd12 cd / before generating build.ninja .
This is so that soong_build doesn't know the current directory, thereby
avoiding one way to leak information.

Test: Manual.
Change-Id: I976a8663b11d99c5382726487102df10043c5a61
2021-03-03 09:14:22 +01:00
Treehugger Robot
d708933aba Merge "propertyIndexesWithTag can handle slice of struct" am: 2910183696 am: 8d9d43c46c am: 4d11943146
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1601875

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I5b7ad3a3b308708063af5ed10269730cd9d4a7af
2021-02-24 02:21:19 +00:00
Colin Cross
e36b5066e6 Add preupload hooks to build/blueprint am: 747b95206b am: 764d29e796 am: 5c9f69df0b
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1602333

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I7c1683dc1be5a101cba818c9669b79e6419f961f
2021-02-24 02:21:08 +00:00
Treehugger Robot
4d11943146 Merge "propertyIndexesWithTag can handle slice of struct" am: 2910183696 am: 8d9d43c46c
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1601875

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: Ibd6f37ba8edff1eb89469699a65f1790d1fc354d
2021-02-24 01:49:33 +00:00
Colin Cross
5c9f69df0b Add preupload hooks to build/blueprint am: 747b95206b am: 764d29e796
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1602333

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I3613c0603c8293117f67e6e008cee2000f8d7bdd
2021-02-24 01:45:31 +00:00
Treehugger Robot
8d9d43c46c Merge "propertyIndexesWithTag can handle slice of struct" am: 2910183696
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1601875

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I4908415cc4eb454030c0224fb990be6c2b89b443
2021-02-24 01:34:38 +00:00
Colin Cross
764d29e796 Add preupload hooks to build/blueprint am: 747b95206b
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1602333

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: Id329902522927d6fa678b17b10753a6f948523c4
2021-02-24 01:17:47 +00:00
Treehugger Robot
2910183696 Merge "propertyIndexesWithTag can handle slice of struct" 2021-02-24 00:54:56 +00:00
Colin Cross
747b95206b Add preupload hooks to build/blueprint
Run gofmt and bpfmt during preupload, and warn about uses of
DO NOT MERGE.

Test: repo upload
Change-Id: I5f8a318f9ec1dbca897d25f25107df36d160e1f6
2021-02-23 10:29:46 -08:00
Jiyong Park
b48d4aea85 propertyIndexesWithTag can handle slice of struct
The function now can traverse into a field whose type is slice of
struct. When reading field values from the returned indexes, Soong will
check if the next field is a slice of struct or not. If so, it will
recurse into all the values in the slice.

Bug: 181018147
Test: m nothing
Change-Id: Ib8a7b7911a0be37a6dc03079adeb906497e60875
2021-02-24 01:02:59 +09:00
Xin Li
f65a030188 [automerger skipped] Mark ab/7061308 as merged in stage. am: f4c3ff67ba -s ours am: 808f3f87a5 -s ours
am skip reason: Change-Id If8f454d33ec6def0f7f13ab4866be05ed73cf0a7 with SHA-1 819320dabc is in history

Original change: undetermined

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I84cf6e674c31b72c39827a827dcf32773a09c2c0
2021-02-20 09:54:33 +00:00
Xin Li
808f3f87a5 [automerger skipped] Mark ab/7061308 as merged in stage. am: f4c3ff67ba -s ours
am skip reason: Change-Id If8f454d33ec6def0f7f13ab4866be05ed73cf0a7 with SHA-1 819320dabc is in history

Original change: undetermined

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I7b58b381e2c12f537e588ad77398519ca0c06b63
2021-02-20 09:15:28 +00:00
Xin Li
f4c3ff67ba Mark ab/7061308 as merged in stage.
Bug: 180401296
Merged-In: If8f454d33ec6def0f7f13ab4866be05ed73cf0a7
Change-Id: I1e39637f944839c99cb6fa41fc8e914ad3455368
2021-02-20 00:24:00 -08:00
Treehugger Robot
30143e47f1 Merge "Make bpfmt use exit codes even when reading from stdin." am: 0335565677 am: e3abb5ce41 am: 1131633ceb
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1592922

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: Ieca90ac22d748f429ae51baef097b81135dfa0ea
2021-02-19 00:16:36 +00:00
Treehugger Robot
1131633ceb Merge "Make bpfmt use exit codes even when reading from stdin." am: 0335565677 am: e3abb5ce41
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1592922

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: Id2d2d069e7c9c0280d45afc3e6205a74006bcd02
2021-02-18 23:34:40 +00:00
Treehugger Robot
e3abb5ce41 Merge "Make bpfmt use exit codes even when reading from stdin." am: 0335565677
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1592922

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: Ifa0a2163ba8ffb6cc4cdd6610f24c00795079cd6
2021-02-18 23:11:12 +00:00
Treehugger Robot
0335565677 Merge "Make bpfmt use exit codes even when reading from stdin." 2021-02-18 22:10:22 +00:00
Hunter Knepshield
9033c957b9 Make bpfmt use exit codes even when reading from stdin.
Previously, an incantation like `bpfmt -o <$BAD_BP` would print to
stderr as expected, but then it would exit with 0. This is different
than the behavior of `bpfmt -o $BAD_BP`, which correctly exits with a
non-zero status code (in addition to printing the issues to stderr).

Test: m bpfmt; out/soong/host/linux-x86/bin/bpfmt -o <$BAD_BP; echo $?
Change-Id: I0a970cddf94fbd818525f9167d9113086b023c81
2021-02-17 20:24:50 -08:00
Thiébaud Weksteen
d6d8b94f0c Fix ReplaceExtension am: 8ec690764e am: 9d5b6043b5 am: c4f5f2ae7c
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1591572

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: Iefcf24ed5f3e462dd88a367ee6e51887dda7b5ff
2021-02-17 08:52:46 +00:00
Thiébaud Weksteen
c4f5f2ae7c Fix ReplaceExtension am: 8ec690764e am: 9d5b6043b5
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1591572

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: Ia8faa3c1c85777e9157cd36b61e39e272a462111
2021-02-17 08:18:22 +00:00
Thiébaud Weksteen
9d5b6043b5 Fix ReplaceExtension am: 8ec690764e
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1591572

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I915906293c6e245bd433460b0babfa0fe777b2ea
2021-02-17 08:00:50 +00:00
Thiébaud Weksteen
8ec690764e Fix ReplaceExtension
ReplaceExtension had an unexpected behaviour when the file did not have
an extension. In certain cases, the final path would be severely
trimmed: out/.intermediates/my_file would become out/.new_extension.
Explicitly handle the case by appending the new extension.

Test: Run checkbuild on Android Soong
Change-Id: Ie27a98845894cfaee5af5e2a02d44168c40ed821

This is an imported pull request from
https://github.com/google/blueprint/pull/345

GitOrigin-RevId: f9166c0e6151499b4b1a23b89b0bc133203a1116
Change-Id: I63f0798177545792440b8a84b04f1090590f1642
2021-02-16 21:45:01 -08:00
Colin Cross
a0bd64046e Merge remote-tracking branch 'aosp/upstream' am: e980b25b95 am: 8cb0bda1b1 am: 0381bdcc1d
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1580817

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I4aee599352d38243bbdbb20cfeda7f925df5a936
2021-02-09 23:44:27 +00:00
Colin Cross
0381bdcc1d Merge remote-tracking branch 'aosp/upstream' am: e980b25b95 am: 8cb0bda1b1
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1580817

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I6de36ceac8a7044b0cd360183c9864505d1d3f54
2021-02-09 22:49:18 +00:00
Colin Cross
8cb0bda1b1 Merge remote-tracking branch 'aosp/upstream' am: e980b25b95
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1580817

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I9f7803976201d41d2cd28aa4da5c93a166884023
2021-02-09 22:27:43 +00:00
Colin Cross
e980b25b95 Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Fix detecting cycles in parallelVisit
  Optimize HasTag function

Test: m checkbuild
Change-Id: Ia121f48bafa1461b8ea0ae6103d1b5bb503e6c16
2021-02-09 10:20:33 -08:00
colincross
aef5f3b7f2
Merge pull request #344 from colincross/parallel
Fix detecting cycles in parallelVisit
2021-02-09 09:17:17 -08:00
Colin Cross
7d4958d84c Fix detecting cycles in parallelVisit
Fix detecting cycles in parallelVisit when the first alphabetical
module is not part of the cycle, but depends on the cycle.  Instead of
starting from the first alphabetical module, check every module in a
determinsitic order and return the first time a cycle is found.

Test: Test_parallelVisit
Change-Id: I03726f838ec42975251088ba75158103940115c2
2021-02-08 15:56:43 -08:00
colincross
e09509592d
Merge pull request #338 from francois-berder/optimize-hastag
Optimize HasTag function
2021-01-28 10:28:04 -08:00
Francois Berder
1ac43bd65a Optimize HasTag function
The commits optimizes HasTag function by avoiding
the call to strings.Split that allocates a list of strings.

HasTag is called from the ExtendProperties family of functions.
So this new implementation would benefit any primary builder
that extensively uses property appending.

Add a small benchmark for HasTag function.
It shows that HasTag is speed up by a factor of 2-3 with the new
implementation.

Before:
goos: linux
goarch: amd64
BenchmarkHasTag/NoTag-56         	20000000	       118 ns/op
BenchmarkHasTag/EmptyTag-56      	20000000	       120 ns/op
BenchmarkHasTag/OtherTag-56      	10000000	       131 ns/op
BenchmarkHasTag/MatchingTag-56   	10000000	       177 ns/op
BenchmarkHasTag/ExtraValues-56   	 5000000	       392 ns/op
BenchmarkHasTag/ExtraTags-56     	10000000	       183 ns/op

After:
goos: linux
goarch: amd64
BenchmarkHasTag/NoTag-56         	200000000	        11.5 ns/op
BenchmarkHasTag/EmptyTag-56      	200000000	        11.5 ns/op
BenchmarkHasTag/OtherTag-56      	50000000	        25.2 ns/op
BenchmarkHasTag/MatchingTag-56   	20000000	        61.4 ns/op
BenchmarkHasTag/ExtraValues-56   	20000000	        94.3 ns/op
BenchmarkHasTag/ExtraTags-56     	20000000	        79.1 ns/op

Signed-off-by: Francois Berder <francois.berder@arm.com>
Change-Id: Ib45498e9ad6aebeca2beddea63543da40c0b1a21
2021-01-28 14:28:34 +00:00
Colin Cross
c8f823a16d Merge remote-tracking branch 'aosp/upstream' am: 3b6c68cd9e am: d7e36a2687 am: 9d6d659be3
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1557081

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: Ib8d9e9e40dbe0ba44c5dc2c05abd79ce3a6127aa
2021-01-25 17:14:54 +00:00
Colin Cross
9d6d659be3 Merge remote-tracking branch 'aosp/upstream' am: 3b6c68cd9e am: d7e36a2687
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1557081

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: If06c9aa406258607fdcac98a97afddd52bb1402a
2021-01-25 16:45:08 +00:00
Colin Cross
d7e36a2687 Merge remote-tracking branch 'aosp/upstream' am: 3b6c68cd9e
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1557081

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I468a1f890e0ff8122563edae6980f8b8180b5a3d
2021-01-25 16:32:12 +00:00
Colin Cross
3b6c68cd9e Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Optimize updateDependencies
  Write build definitions directly to output writer
  Memoize full names of variables, pools and rules
  Optimize arg parsing in buildDef.WriteTo
  Increase ninja file writer buffer
  Use io.StringWriter in ninjaWriter

Test: m checkbuild
Change-Id: I8ff3ca4a95c351809b62af841e9bb77b2de7d1ed
2021-01-22 17:05:34 -08:00
colincross
22343ff42c
Merge pull request #342 from colincross/optimize_updateDependencies
Optimize updateDependencies
2021-01-22 16:22:41 -08:00
colincross
2a29698186
Merge pull request #343 from colincross/optimize_ninja_writer
Optimize ninja writer
2021-01-22 16:22:22 -08:00
Colin Cross
513f6d6fcd Merge remote-tracking branch 'aosp/upstream' am: eecfdd316b am: 6715ce49d5 am: 502e70ef21
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1557261

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I8283fa51ceba4a5b0e506d01c01491913478f916
2021-01-22 19:29:23 +00:00
Colin Cross
502e70ef21 Merge remote-tracking branch 'aosp/upstream' am: eecfdd316b am: 6715ce49d5
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1557261

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: If29c50c2257e95bebba54bf49de9ad19690fae2b
2021-01-22 18:43:41 +00:00
Colin Cross
7ff2e8d2a4 Optimize updateDependencies
Avoid reallocating module.forwardDeps and module.reverseDeps every
time through updateDependencies by resetting the slices without
reducing their capacity.  Accumulate dependencies to visit directly
into module.forwardDeps.  Use a loop instead of a map to check
for duplicates, average number of dependencies is measured to be
9.5, although there are a few outliers with up to 2108.

Reduces mean soong_build execution time on internal master from 87s
to 82.7s (5%).

Test: context_test.go
Change-Id: I58fcd5514e494bafa965443461851b21b7bce382
2021-01-22 10:22:21 -08:00
Colin Cross
6715ce49d5 Merge remote-tracking branch 'aosp/upstream' am: eecfdd316b
Original change: https://android-review.googlesource.com/c/platform/build/blueprint/+/1557261

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I8442949eb048eda9112336f446c5eb683d1cf505
2021-01-22 17:54:22 +00:00
Colin Cross
8a40148408 Write build definitions directly to output writer
buildDef.WriteTo was calling valueList to convert all the build
parameter ninjaStrings into strings, which uses ValueWithEscaper
to build a strings.Builder.  This results in building a string
only to immediately copy it into the output writer's buffer.

Instead, pass an io.StringWriter to ValueWithEscaper so it can
build the string directly into the output writer's buffer.  This
requires converting ninjaWriterWithWrap into an io.StringWriter.

Test: ninja_writer_test.go
Change-Id: I02e1cf8259306267b9d2d0ebe8c81e13dd443725
2021-01-21 22:02:30 -08:00
Colin Cross
92054a49d2 Memoize full names of variables, pools and rules
Variables, pools and rules each computed their full names every time
they were referenced, which required string concatenations.  Since
every one is guaranteed to be accessed at least twice, once when the
definition is written into the ninja file and once for each reference,
precompute the full name.  For local variables that can be done
during initialization, but for global variables add a pass to
PrepareBuildActions to compute the name for each live variable using
the final package names.

Test: ninja_writer_test.go
Change-Id: I2264b05e0409e36651db2fb5d463c16c698d4d5e
2021-01-21 22:02:30 -08:00
Colin Cross
00890dd8f6 Optimize arg parsing in buildDef.WriteTo
Arguments to build definitions were copied from the input map to an
map with the name and value expanded, then to a list of names for
sorting, and then written, which required iterating over a map three
times.  Expand the name and value into a list of name value pairs,
and then do the rest of the operations on the list instead.

Test: ninja_writer_test.go
Change-Id: Id8ff644dafbaa3b4812747c60dc28cce22e21dbe
2021-01-21 22:02:30 -08:00
Colin Cross
c8b9e55289 Increase ninja file writer buffer
Increase the ninja file writer buffer from the default 4k to 16MB.

Test: m checkbuild
Change-Id: Ieb2c82218517b98469ef93f1ea4dd04b5651f7d1
2021-01-21 22:02:30 -08:00
Colin Cross
0335e0900d Use io.StringWriter in ninjaWriter
ninjaWriter repeatedly called io.WriteString() on its writer, which
does a type assertion every time.  Replace its io.Writer with an
io.StringWriter and call WriteString on it directly.

Test: ninja_writer_test.go
Change-Id: Ie073d996a319190242bf6a00af07a13a60d078b5
2021-01-21 22:02:30 -08:00
Colin Cross
eecfdd316b Merge remote-tracking branch 'aosp/upstream'
* aosp/upstream:
  Return a copy of glob lists

Test: m checkbuild
Test: m nothing && m nothing
Change-Id: Ie015670828f3d6e0bf4fa22580cb7254966a2c82
2021-01-21 18:28:20 -08:00
colincross
bcd5686660
Merge pull request #341 from colincross/optimize_glob
Return a copy of glob lists
2021-01-21 18:27:49 -08:00
Colin Cross
a64ca94c0d Return a copy of glob lists
Callers to glob methods may do in-place modifications on the returned
list of globs, return a copy instead of the cached value.

Test: m nothing && m nothing
Change-Id: Ic9140d1e1900e8724ba0a484f27786e5c15dea90
2021-01-21 13:47:59 -08:00