osFs.acquire and osFs.release are surprisingly expensive, using a
combined 345.7s of runtime in an AOSP aosp_cf_x86_64_phone-userdebug
build. They are used to ensure we don't use too many simultaneous
open files, but many of the functions they are called from don't
actually open a file. Remove them from all the stat-based functions
(Exists, IsDir, IsSymlink, Lstat, Stat), and from ReadLink. After
this change the time spent in acquire and release is effectively
zero.
Test: SOONG_PROFILE_CPU=/tmp/cpu.pprof m nothing
Change-Id: Ie5e22e33c61794354821f05ab79ceb4efc3b276c
Mac builds keep running into too many files open. Restrict access of
filesystem to the current limit.
Test: m nothing
Change-Id: I2365da7c641f7c7f5d948396c6862eb3a0d1d8b9
Replace the individual matches and deps return values with a GlobResult
struct. Use the GlobResult to create the file list file in GlobWithDepFile
(used by bpglob) and in the glob singleton, as the generated files must
match exactly so that soong_build is not rerun.
Bug: 159845846
Test: glob_test.go
Change-Id: I2159cc9d85f388073198eac7456e5bf43e813096
Glob was calling IsSymlink and IsDir on each visited directory entry,
which resulted in an lstat and then a stat call on each.
Instead, use lstat when not following symlinks and use stat when
following symlinks, then use the result to check if the entry is a
directory.
Test: glob_test.go
Change-Id: I83d769e2de64ce8221e952e5204d365aeaf47687
OsFs may be asked to read absolute paths if buildDir is absolute.
Check if the path is absolute before prepending srcDir to it.
Bug: 146437378
Test: fs_test.go
Change-Id: I2a67593e9d836ca3e11dc10b81f49a4fb49d2cdf
Add some more test cases for pathtools.Match before making
signficant changes to it.
Test: glob_test.go
Change-Id: I2eeb5ebf03fb645a2053852a1a9f4e368d22084a
Stat is used by soong_zip. Add it to the FileSystem interface
and add tests for it.
Test: fs_test.go
Change-Id: I1baa2b27398846a4e55bcf4fa291c62f507a4e9d
Readlink is used by soong_zip. Add it to the FileSystem interface
and add tests for it.
Test: fs_test.go
Change-Id: Ie8ca5cd7cae98a47980a50d2891501fe79fd47a5
Both *os.File and *bytes.Buffer support all of io.Reader,
io.ReaderAt, io.Seeker and io.Closer. Return a combo interface
so that soong_zip can use the result in an io.SectionReader.
Test: m checkbuild
Change-Id: I31c3ce35e28c52bae20b536b5905de2f8a3d1478
Allow the caller to specify whether symlinks should be followed
(the old behavior) or not.
Test: glob_test.go
Test: fs_test.go
Change-Id: I550dc91b8e6370fb32a9a1cbdcb2edade48bda46
filepath.Walk does not walk symlinks to directories, which leads to
inconsitent behavior with following symlinks. Replace the use of
filepath.Walk in ListDirsRecursive with a helper function that lists
each directory and walks anything for which IsDir reports true, which
includes following symlinks, and then reconstructs the path through
the symlink.
Test: fs_test.go
Test: glob_test.go
Change-Id: Ie4dd0820e9c7c0a124caa65210ce20439a44da16
Add support for specifying symlinks in mock filesystems to prepare
for glob symlink tests.
This patch leaves incorrect behavior by not walking symlinks in
mockFs.ListDirsRecursive, but it matches what osFs does.
Test: fs_test.go
Change-Id: If87a83c00f21e14696faf890b7b09e88b18e95b9
IsDir on a dangling symlink produces ErrNotExist. Manually
check if the file exists but is a symlink to report a better
error.
Test: m checkbuild
Change-Id: I3181e74002436d74ec35a0923635835e561030dd
Recursive globs on MockFs were ending up in an unmocked os.Lstat
call in walkAllDirs. Move walkAllDirs into the FileSystem interface
as ListDirsRecursive.
Also duplicate the glob tests on both the real filesystem and on
MockFs.
Test: glob_test.go
Change-Id: Ia6b6b5eecdd17955a49d684a0fd5e55df05cfe62
Bug: 64600838
Test: mkdir errtest \
&& ln -s /tmp/dontexist errtest/Android.bp \
# and add errtest to ./Android.bp \
&& m nothing \
# and check that the error message mentions a symlink
Change-Id: I841ec12d613f61ccc3396538062bee48c8c1ca27
Add globbing to filesystem mocking so that more code can be tested
against the mock. Also moves the filesystem mock to pathtools,
and renames pathtools.GlobWithExcludes to pathtools.Glob, replacing
the existing pathtools.Glob.
Test: blueprint tests
Change-Id: I722df8121bc870c4a861d7c249245c57dcb607be
