This is so that it doesn't need to abruptly call os.Exit(), denying
callers the opportunity to do cleanups.
Bug: 244730498
Test: Presubmits.
Change-Id: Ifd191d3bbbf2fdea2ca49e4fb552e5d1c557b80f
ninjaString is an interface, which uses 16 bytes of memory on top
of the size of the concrete type. A literalNinjaString is a string,
which is another 16 bytes for the string header for a total of 32
bytes. A varNinjaString is two slices, which are 24 bytes each
for the slice headers, for a total of 64 bytes. The slices contain
the first constant string, and then altenrating variable and string
parts of the ninjaString, resulting in 16 bytes plus 32 bytes per
variable.
This patch replaces the ninjaString interface with a *ninjaString
concrete struct type. The ninjaString struct is a string and a
pointer to a slice of variable references, for a total of 24 bytes.
ninjaStrings with no variable references (the equivalent of the old
literalNinjaString) have a nil slice, and now use 24 bytes instead
of 32 bytes.
ninjaStrings with variable references allocate a slice of variable
references that contain 32-bit start and end offsets and a Variable
interface, but reuse the original string and so avoid the extra
string headers, resulting in 24 bytes for the slice header, and
24 bytes per variable.
These savings reduce the peak memory usage averaged across 10 runs of
/bin/time -v build/soong/soong_ui.bash --make-mode nothing
on the internal master branch cf_x86_64_phone-userdebug build
from 50114842kB to 45577638kB, a savings of 4537204kB or 9%.
The new Benchmark_parseNinjaString shows savings in both time and
memory. Before:
Benchmark_parseNinjaString/constant/1-128 594251787 2.006 ns/op 0 B/op 0 allocs/op
Benchmark_parseNinjaString/constant/10-128 21191347 65.57 ns/op 16 B/op 1 allocs/op
Benchmark_parseNinjaString/constant/100-128 9983748 130.2 ns/op 112 B/op 1 allocs/op
Benchmark_parseNinjaString/constant/1000-128 2632527 445.1 ns/op 1024 B/op 1 allocs/op
Benchmark_parseNinjaString/variable/1-128 2964896 419.4 ns/op 176 B/op 4 allocs/op
Benchmark_parseNinjaString/variable/10-128 1807341 670.6 ns/op 192 B/op 7 allocs/op
Benchmark_parseNinjaString/variable/100-128 1000000 1092 ns/op 352 B/op 7 allocs/op
Benchmark_parseNinjaString/variable/1000-128 300649 3773 ns/op 1584 B/op 7 allocs/op
Benchmark_parseNinjaString/variables/1-128 2858432 441.6 ns/op 176 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/2-128 2360505 513.4 ns/op 208 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/3-128 1867136 635.6 ns/op 240 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/4-128 1584045 752.1 ns/op 272 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/5-128 1338189 885.8 ns/op 304 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/10-128 1000000 1468 ns/op 464 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/100-128 88768 12895 ns/op 3712 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/1000-128 8972 133627 ns/op 32896 B/op 4 allocs/op
After:
Benchmark_parseNinjaString/constant/1-128 584600864 2.004 ns/op 0 B/op 0 allocs/op
Benchmark_parseNinjaString/constant/10-128 19274581 64.84 ns/op 16 B/op 1 allocs/op
Benchmark_parseNinjaString/constant/100-128 9017640 127.6 ns/op 112 B/op 1 allocs/op
Benchmark_parseNinjaString/constant/1000-128 2630797 453.0 ns/op 1024 B/op 1 allocs/op
Benchmark_parseNinjaString/variable/1-128 3460422 347.0 ns/op 136 B/op 4 allocs/op
Benchmark_parseNinjaString/variable/10-128 2103404 519.9 ns/op 152 B/op 7 allocs/op
Benchmark_parseNinjaString/variable/100-128 1315778 906.5 ns/op 312 B/op 7 allocs/op
Benchmark_parseNinjaString/variable/1000-128 354812 3284 ns/op 1544 B/op 7 allocs/op
Benchmark_parseNinjaString/variables/1-128 3386868 361.5 ns/op 136 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/2-128 2675594 456.9 ns/op 160 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/3-128 2344670 520.0 ns/op 192 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/4-128 1919482 648.1 ns/op 208 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/5-128 1560556 723.9 ns/op 240 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/10-128 1000000 1169 ns/op 352 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/100-128 116738 10168 ns/op 2800 B/op 4 allocs/op
Benchmark_parseNinjaString/variables/1000-128 10000 105646 ns/op 24688 B/op 4 allocs/op
Bug: 286423944
Test: ninja_strings_test.go
Test: out/soong/build*.ninja is the same before and after this change
Change-Id: I1ecffbaccb0d0469a41fa31255c1b17311e01687
Many of the singletons are trivial and can be run in parallel, improving
the performance during analysis.
Bug: 281536768
Test: manual, presubmit
Change-Id: Ia63e4bc42a68e65dfa800e770982fa5826355fad
There is already a mutex for the structure, but several add functions do
not use the mutex to protect access.
Bug: 281536768
Test: manual, presubmits
Change-Id: I34e95d8722b8e5fb753c099d7aedee5c4734715d
When given as an input
```
array: [
"a",
// Unicorn
"b",
]
```
bpfmt with `-s` option was outputing
```
array: [
"a", // Unicorn
"b",
]
```
Which is not ideal because the comment was targetting
the second value and now it seems to be targetting the
first one
This patch preserve the difference in line number between
the value and the comment to give the same output when
```
array: [
"a",
// Unicorn
"b",
]
```
is given as input
Test: Manual tests + run bpfmt -w -s on packages/modules/Bluetooth
Change-Id: I2b58f20da463bea77c22a4e6978aa9beb4b4fcc8
Without this, we cannot correctly join action and module graph
information as we cannot distinguish variants on action graph.
Test: m json-module-graph
Change-Id: If7379845fc865d8a150f3995df6bf601456a71c3
Previously, warnings about missing modules were printed directly to
stderr. Instead we can pass these messages along as errors using the
existing pathways.
Bug: 269457150
Test: m nothing
Test: add -external to PRODUCT_SOURCE_ROOT_DIRS and observe missing
module errors
Change-Id: Ib6624b9edbd103247f7f6e6d4c3030f1959aa56c
Soong analyzes the entire source tree even though not every lunch target
needs to know about every module. For example, OEM sources can be
ignored for cuttlefish products. This functionality allows blueprint to
ignore a list of undesired directories.
Bug: 269457150
Change-Id: Icbbf8f3b66813ad639a7ebd27b1a3ec153cbf269
To build ninja hint including output path from module name
Test: m --ninja_weight_source=soong
Bug: 273282046
Change-Id: Ibb94c2c4efef4a6dedc973cbb90625231845d42e
This reduces the extract_phonys (now deduplicate_order_only_deps)
event's time from ~1.9s to ~1.5s on aosp-master, and from ~5.3s to ~4.6s
on internal master.
It does so by making keyForPhonyCandidate be based on a hash instead
of joining all the deps together. Having a hash allows us to also use
it as the name of the phony target, which simplifies the code a little.
Bug: None (original cl introducing extractPhonys also didn't have a bug)
Test: go tests
Change-Id: I2ff6e4614f19ccbfe99112ea7ae1ea33cd1df21b
1. scan if any set of order-only deps are repeated
2. if so extract them as a phony output to be shared
Test: m libc
Bug: NA
Change-Id: I0689111b97bbbd1f3b26650e8ae2e0a4ffb5085e
Collect additional metrics for individual mutators in order to
understand impact of individual mutators.
Test: m nothing
Change-Id: Ic3ecb1e79a79dd665c9f60d29f0dfd3732481c2d
With this close() missing, especially with large ninja files, soong paid
the runtime cost at arbitrary times during the rest of its execution (or
during process cleanup).
Test: Manual runs of `m nothing` and verifying timing metrics
Change-Id: Ibdfbf7371cfc4c58c485e76f96b8e9b9ad900972
