Readlink is used by soong_zip. Add it to the FileSystem interface
and add tests for it.
Test: fs_test.go
Change-Id: Ie8ca5cd7cae98a47980a50d2891501fe79fd47a5
Only check that functions that return an *os.SyscallError have
the correct errno, not that they have identical test text.
Test: fs_test.go
Change-Id: Iba050cb0474eaf2643858bcca4e52ba770702c2f
Both *os.File and *bytes.Buffer support all of io.Reader,
io.ReaderAt, io.Seeker and io.Closer. Return a combo interface
so that soong_zip can use the result in an io.SectionReader.
Test: m checkbuild
Change-Id: I31c3ce35e28c52bae20b536b5905de2f8a3d1478
Allow the caller to specify whether symlinks should be followed
(the old behavior) or not.
Test: glob_test.go
Test: fs_test.go
Change-Id: I550dc91b8e6370fb32a9a1cbdcb2edade48bda46
filepath.Walk does not walk symlinks to directories, which leads to
inconsitent behavior with following symlinks. Replace the use of
filepath.Walk in ListDirsRecursive with a helper function that lists
each directory and walks anything for which IsDir reports true, which
includes following symlinks, and then reconstructs the path through
the symlink.
Test: fs_test.go
Test: glob_test.go
Change-Id: Ie4dd0820e9c7c0a124caa65210ce20439a44da16
Add support for specifying symlinks in mock filesystems to prepare
for glob symlink tests.
This patch leaves incorrect behavior by not walking symlinks in
mockFs.ListDirsRecursive, but it matches what osFs does.
Test: fs_test.go
Change-Id: If87a83c00f21e14696faf890b7b09e88b18e95b9
IsDir on a dangling symlink produces ErrNotExist. Manually
check if the file exists but is a symlink to report a better
error.
Test: m checkbuild
Change-Id: I3181e74002436d74ec35a0923635835e561030dd
NinjaEscape and ShellEscape operated on lists, which led to
awkward NinjaEscape([]string{s})[0]. Replace NinjaEscape
and ShellEscape with NinjaEscapeList and ShellEscapeList,
and add new NinjaEscape and ShellEscape functions that
operate on a string.
Test: m checkbuild
Change-Id: I283d92cdddc8e0066a300015863a3eab66f77c23
Globs that match a file that looks like a glob were causing duplicate
results because the prefix match would then re-match the
filename as a wildcard. Add escaping to prevent re-matching.
Also add tests for globs on files with wildcards, tests for
? and [a-z] glob matches supported by filepath.Match, and tests
for escaped wildcard characters.
Test: glob_test.go
Change-Id: Id11a754863979bb36cca0dbd18ea2e76dd1470e3
pathtools.Match was trimming the trailing slash from patterns. Make
it handle a trailing slash by first confirming that both the pattern
and name either both have or both do not have a trailing slash, and
then trimming it from both.
Bug: 111389216
Test: TestMatch in glob_test.go
Change-Id: I743e00c14d885de5b5a060aa9e2b22c81dc7e09d
Instead of sometimes re-running minibp/the primary builder during the
next phase, run bpglob earlier to check dependencies.
We've run into issues where the environment is slightly different
between bootstrapping phase and the main build phase. It's also a
problem because our primary builder (Soong) exports information used by
another tool (Kati) that runs in between the bootstrapping phases and
the main phase. When Soong would run in the main phase, it could get out
of sync, and would require the build to be run again.
To do this, add a "subninja" include a build-globs.ninja file to each
build.ninja file. The first time, this will be an empty file, but we'll
always run minibp / the primary builder anyway. When the builder runs,
in addition to writing a dependency file, write out the
build-globs.ninja file with the rules to run bpglob.
Since bpglob may need to be run very early, before it would normally be
built, build it with microfactory.
Change-Id: I89fcd849a8729e892f163d40060ab90b5d4dfa5d
This commit refines `compat` condition in `parseProperty()` so that a
module definition would either use the new syntax or the old syntax,
but not something in the between, such as
cc_library {
name: "bad_example",
srcs= ["bad.c"],
}
Test: lunch aosp_arm64-userdebug; make # runs unit test
Change-Id: If2d3e5d55edccc28d314d99b83b0f54e5c53ac35
WalkDeps was following every possible path through the dependency
tree, which can be enormous. Modify it to only call visit for
any particular (child, parent) pair once for each direct dependency
by not recursing into child if visitDown returns true but child
has already been visited.
Test: TestWalkDeps, TestWalkDepsDuplicates
Change-Id: Ieef28399bd10e744417cdeb661dfa04fbeb4ec60
Allow adding multiple dependencies on the same module. Adds a flag
to Context.walkDeps to determine whether dependencies should be
visited multiple times, and clarifies the behavior of
VisitDepsDepthFirst (continues to visit each module once, even if
there are multiple dependencies on it) and VisitDirectDeps (may
visit a module multiple times).
Test: TestWalkDepsDuplicates, TestVisit
Change-Id: I58afbe90490aca102d242d63e185386e1fe55d73
When someone used a multiline string:
cmd: "..." +
"...",
bpfmt used to print this out as:
cmd: "..." +
"...",
This change doesn't do the quote alignment like I see in some of our
user-created instances of this, but it does indent a single level:
cmd: "..." +
"...",
Test: unit tests
Test: Compared bpfmt results before and after across AOSP
Change-Id: I61bf790be9d08a187857b2725facf71e8b38e372
Misusing Rename was causing a nil pointer derefernece, return
an error if group was not found in s.modules.
Bug: 77922456
Change-Id: I7e7b20b1595d569ae07c755bd29d701e0e5dab78
Running with the data race detector enabled hits a 8192 active
goroutine limit. Reduce the maximum number of active goroutines
by limiting parallelVisit to 1000 goroutines. Also rewrite
cloneModule in terms of parallelVisit.
Change-Id: Icdd63648e8e010557b624e12592156490b40adb9
Patch and PatchList provide an API for making changes to substrings
of a Blueprint file by parsing the AST and using the token positions
to perform text replacements.
Test: modify_test.go
Change-Id: Ibb8993221982b54602ba5a05486198fab8d35a67
End() was previously only used to determine if a comment was within
a Node, so it used the expedient definition of the position of the
last token in the node. In the next patch it will be used for
capturing substrings of the Blueprint file, so make it point to
the character after the last token instead.
Also add tests for it.
Test: parser_test.go
Change-Id: Icaff3915b41e251ef9d0aad5615021bf37406aee
When passing "-c" to the Go compiler, any time this value changes, we'd
force all of the Go compiles to rebuild. This could trigger a
substantial portion of the tree to rebuild (anything that transitive
depends on a Go helper tool).
We're running into issues when moving output directories between
multiple GCE machines with different core counts, but are otherwise
identical. This could also hit users moving/mounting disks between
machines, though changes to other host tools can make an impact too.
On my 48-core machine, I get a ~15% benefit from going from -c 1 to -c
48, but also ~12% benefit from going from -c 1 to -c 8. So this will
still let us scale somewhat, but prevent rebuilds when transitioning
between machines that are more likely building Android.
In Go 1.10, runtime.GOROOT() will attempt to find the current location
of the go binaries, instead of using the GOROOT_FINAL that was baked
into the binaries. This means that GOROOT() will usually return an
absolute path.
We avoid putting absolute paths into the ninja file, since any change to
the paths would then cause all of the actions including it to rebuild.
Since we've got a decent number of build tools in Android using Go now,
this causes us to rebuild a decent portion of the tree.
Instead of passing the GOROOT around manually in a side channel, just
let the Go 1.10 detection do its thing, and always try to turn the
result into a relative path.
This makes it easy for users of Glob to detect whether the match is a
file or directory. Doing the check at this level means that the filelist
file used as a dependency will be updated if a directory is replaced
with a file of the same name, or vice versa.
Change-Id: I79ebba39327218bcdcf50b393498306119de9d6c
