Writing the ninja file to a byte buffer causes a significant amount
of time to be spent in memmove when growing the byte slice. Write
the file directly to disk instead.
This also fixes some unhandled error warnings, which become more
likely when doing disk IO instead of byte buffer writes.
Change-Id: I5094e4c45cab4012713037f60c5a4fb00718f92e
before, adding to a list which had no entries would do nothing (and not
throw any errors). Now it will create the list and add a single element
as expected.
Change-Id: I8c48a42303f7d9b3741868ad86097e2071ec434a
In cases that we want to run blueprint-based builds in many
configurations to verify all of the logic works without errors, but
don't care about running the final ninja file, writing it out only
wastes time and disk space. So add a --empty-ninja-file option that
writes out an empty ninja file instead.
Our specific use case (Soong's build_test / multiproduct_kati) runs
Soong several hundred times for different configurations, and the ninja
files are around 1GB, which leads to several hundred gigabytes of disk
writes (and persistent use during incremental generation).
Change-Id: I0198dfb2f744ce22284c05d5214dac2ab5dc9700
The test package archive was being picked up through "-L" + testRoot,
but did not have a dependency on it. If the *_test.go file was
changed but not the implementation then the tests would not be
rerun.
Test: Add t.Fatal() to a *_test.go file and rebuild.
Change-Id: I32b8c7a0fa1d5b76b1242646887fe937e396e318
Throw an error if run into a module without a name when generating
contexts.
Test: context_test.go
Change-Id: I3976d86d1f15b8ac10a7a38aa42ae277740a8f3b
Linking tests needs the same dependencies that linking binaries
got in https://github.com/google/blueprint/pull/222.
Test: m checkbuild
Change-Id: I33330f3184b8c0fd2bc20b48736c20d6edeaea68
Property structs can now contain slices of structs and other
non-strings as long as they are tagged with `blueprint:"mutated"`.
Test: clone_test.go
Test: unpack_test.go
Change-Id: Ib77348dc6e7314a24f17caba10040f7d3ac54e54
Many Android build tools are written in go, and changes to Soong or
Blueprint can cause them to rebuild. Almost everything in an
Android build is downstream of at least one of these tools, so
they all rebuild too.
Go binaries are static, so their contents will always change if any
of their dependencies change. Only update output files of go compile
and go link if the contents change, and enable restat for the rules.
Test: m soong_zip && m soong_zip
Test: m soong_zip && touch build/soong/zip/cmd/main.go && m soong_zip
Change-Id: I9267580f644b42b44d43fb0a2674dc5234f127e5
Stat is used by soong_zip. Add it to the FileSystem interface
and add tests for it.
Test: fs_test.go
Change-Id: I1baa2b27398846a4e55bcf4fa291c62f507a4e9d
Readlink is used by soong_zip. Add it to the FileSystem interface
and add tests for it.
Test: fs_test.go
Change-Id: Ie8ca5cd7cae98a47980a50d2891501fe79fd47a5
Only check that functions that return an *os.SyscallError have
the correct errno, not that they have identical test text.
Test: fs_test.go
Change-Id: Iba050cb0474eaf2643858bcca4e52ba770702c2f
Both *os.File and *bytes.Buffer support all of io.Reader,
io.ReaderAt, io.Seeker and io.Closer. Return a combo interface
so that soong_zip can use the result in an io.SectionReader.
Test: m checkbuild
Change-Id: I31c3ce35e28c52bae20b536b5905de2f8a3d1478
Allow the caller to specify whether symlinks should be followed
(the old behavior) or not.
Test: glob_test.go
Test: fs_test.go
Change-Id: I550dc91b8e6370fb32a9a1cbdcb2edade48bda46
filepath.Walk does not walk symlinks to directories, which leads to
inconsitent behavior with following symlinks. Replace the use of
filepath.Walk in ListDirsRecursive with a helper function that lists
each directory and walks anything for which IsDir reports true, which
includes following symlinks, and then reconstructs the path through
the symlink.
Test: fs_test.go
Test: glob_test.go
Change-Id: Ie4dd0820e9c7c0a124caa65210ce20439a44da16
Add support for specifying symlinks in mock filesystems to prepare
for glob symlink tests.
This patch leaves incorrect behavior by not walking symlinks in
mockFs.ListDirsRecursive, but it matches what osFs does.
Test: fs_test.go
Change-Id: If87a83c00f21e14696faf890b7b09e88b18e95b9
IsDir on a dangling symlink produces ErrNotExist. Manually
check if the file exists but is a symlink to report a better
error.
Test: m checkbuild
Change-Id: I3181e74002436d74ec35a0923635835e561030dd
Globs that match a file that looks like a glob were causing duplicate
results because the prefix match would then re-match the
filename as a wildcard. Add escaping to prevent re-matching.
Also add tests for globs on files with wildcards, tests for
? and [a-z] glob matches supported by filepath.Match, and tests
for escaped wildcard characters.
Test: glob_test.go
Change-Id: Id11a754863979bb36cca0dbd18ea2e76dd1470e3
pathtools.Match was trimming the trailing slash from patterns. Make
it handle a trailing slash by first confirming that both the pattern
and name either both have or both do not have a trailing slash, and
then trimming it from both.
Bug: 111389216
Test: TestMatch in glob_test.go
Change-Id: I743e00c14d885de5b5a060aa9e2b22c81dc7e09d
Instead of sometimes re-running minibp/the primary builder during the
next phase, run bpglob earlier to check dependencies.
We've run into issues where the environment is slightly different
between bootstrapping phase and the main build phase. It's also a
problem because our primary builder (Soong) exports information used by
another tool (Kati) that runs in between the bootstrapping phases and
the main phase. When Soong would run in the main phase, it could get out
of sync, and would require the build to be run again.
To do this, add a "subninja" include a build-globs.ninja file to each
build.ninja file. The first time, this will be an empty file, but we'll
always run minibp / the primary builder anyway. When the builder runs,
in addition to writing a dependency file, write out the
build-globs.ninja file with the rules to run bpglob.
Since bpglob may need to be run very early, before it would normally be
built, build it with microfactory.
Change-Id: I89fcd849a8729e892f163d40060ab90b5d4dfa5d
This commit refines `compat` condition in `parseProperty()` so that a
module definition would either use the new syntax or the old syntax,
but not something in the between, such as
cc_library {
name: "bad_example",
srcs= ["bad.c"],
}
Test: lunch aosp_arm64-userdebug; make # runs unit test
Change-Id: If2d3e5d55edccc28d314d99b83b0f54e5c53ac35
WalkDeps was following every possible path through the dependency
tree, which can be enormous. Modify it to only call visit for
any particular (child, parent) pair once for each direct dependency
by not recursing into child if visitDown returns true but child
has already been visited.
Test: TestWalkDeps, TestWalkDepsDuplicates
Change-Id: Ieef28399bd10e744417cdeb661dfa04fbeb4ec60
Allow adding multiple dependencies on the same module. Adds a flag
to Context.walkDeps to determine whether dependencies should be
visited multiple times, and clarifies the behavior of
VisitDepsDepthFirst (continues to visit each module once, even if
there are multiple dependencies on it) and VisitDirectDeps (may
visit a module multiple times).
Test: TestWalkDepsDuplicates, TestVisit
Change-Id: I58afbe90490aca102d242d63e185386e1fe55d73
When someone used a multiline string:
cmd: "..." +
"...",
bpfmt used to print this out as:
cmd: "..." +
"...",
This change doesn't do the quote alignment like I see in some of our
user-created instances of this, but it does indent a single level:
cmd: "..." +
"...",
Test: unit tests
Test: Compared bpfmt results before and after across AOSP
Change-Id: I61bf790be9d08a187857b2725facf71e8b38e372
