Commit graph

19 commits

Author SHA1 Message Date
Liz Kammer
2df87f3cd9 Add a limit to blueprint filesystem accesses
Mac builds keep running into too many files open. Restrict access of
filesystem to the current limit.

Test: m nothing
Change-Id: I2365da7c641f7c7f5d948396c6862eb3a0d1d8b9
2022-02-08 15:16:59 -05:00
Liz Kammer
8b162be674 Rename saneSplit --> quickSplit
This better describes its goal and is more inclusive.

Test: go test blueprint pathtools tests
Change-Id: Ia382f1fe65d0b551dc5d92d40ddf3cb356f3204d
2022-02-07 09:41:01 -05:00
Colin Cross
67c99255cc Make Glob return a GlobResult
Replace the individual matches and deps return values with a GlobResult
struct.  Use the GlobResult to create the file list file in GlobWithDepFile
(used by bpglob) and in the glob singleton, as the generated files must
match exactly so that soong_build is not rerun.

Bug: 159845846
Test: glob_test.go
Change-Id: I2159cc9d85f388073198eac7456e5bf43e813096
2021-04-08 11:03:17 -07:00
Colin Cross
4604a81721 Use info.IsDir() instead of info.Mode()&os.ModeDir != 0
Use the info.IsDir() shortcut.

Test: glob_test.go
Change-Id: Ibba8bbc6af295e3d86321b0c070a84c2b7008dd9
2021-01-20 13:47:37 -08:00
Colin Cross
e535c97cc9 Remove redundant stat/lstat calls from glob
Glob was calling IsSymlink and IsDir on each visited directory entry,
which resulted in an lstat and then a stat call on each.

Instead, use lstat when not following symlinks and use stat when
following symlinks, then use the result to check if the entry is a
directory.

Test: glob_test.go
Change-Id: I83d769e2de64ce8221e952e5204d365aeaf47687
2021-01-20 11:01:41 -08:00
Colin Cross
2f95ec7031 Fix reading absolute paths through OsFs
OsFs may be asked to read absolute paths if buildDir is absolute.
Check if the path is absolute before prepending srcDir to it.

Bug: 146437378
Test: fs_test.go
Change-Id: I2a67593e9d836ca3e11dc10b81f49a4fb49d2cdf
2020-01-10 13:52:22 -08:00
Colin Cross
c5fa50e057 Allow primary builder to change working directory
Bug: 146437378
Test: pathtools/fs_test.go
Change-Id: I513ceb9b8b0b4f18223bc34ecad9846fe220709b
2020-01-08 15:54:58 -08:00
Colin Cross
a4cc6837db Add more pathtools.Match test cases
Add some more test cases for pathtools.Match before making
signficant changes to it.

Test: glob_test.go
Change-Id: I2eeb5ebf03fb645a2053852a1a9f4e368d22084a
2019-05-20 14:57:31 -07:00
Colin Cross
3316a5ee54 Add pathtools.Filesystem.Stat for soong_zip
Stat is used by soong_zip.  Add it to the FileSystem interface
and add tests for it.

Test: fs_test.go
Change-Id: I1baa2b27398846a4e55bcf4fa291c62f507a4e9d
2018-09-27 15:54:24 -07:00
Colin Cross
e81b432f09 Add pathtools.FileSystem.Readlink
Readlink is used by soong_zip.  Add it to the FileSystem interface
and add tests for it.

Test: fs_test.go
Change-Id: Ie8ca5cd7cae98a47980a50d2891501fe79fd47a5
2018-09-26 16:54:15 -07:00
Colin Cross
192dbc59c1 Make pathtools.FileSystem.Open return a ReaderAtSeekerCloser
Both *os.File and *bytes.Buffer support all of io.Reader,
io.ReaderAt, io.Seeker and io.Closer.  Return a combo interface
so that soong_zip can use the result in an io.SectionReader.

Test: m checkbuild
Change-Id: I31c3ce35e28c52bae20b536b5905de2f8a3d1478
2018-09-26 16:54:15 -07:00
Colin Cross
e98d0828c8 Add ShouldFollowSymlinks argument to pathtools.Glob
Allow the caller to specify whether symlinks should be followed
(the old behavior) or not.

Test: glob_test.go
Test: fs_test.go
Change-Id: I550dc91b8e6370fb32a9a1cbdcb2edade48bda46
2018-09-24 15:09:32 -07:00
Colin Cross
9e1ff7423b Fix recursive globs through symlinks
filepath.Walk does not walk symlinks to directories, which leads to
inconsitent behavior with following symlinks.  Replace the use of
filepath.Walk in ListDirsRecursive with a helper function that lists
each directory and walks anything for which IsDir reports true, which
includes following symlinks, and then reconstructs the path through
the symlink.

Test: fs_test.go
Test: glob_test.go
Change-Id: Ie4dd0820e9c7c0a124caa65210ce20439a44da16
2018-09-24 15:09:32 -07:00
Colin Cross
c64f26418e Add symlink support to mockFs
Add support for specifying symlinks in mock filesystems to prepare
for glob symlink tests.

This patch leaves incorrect behavior by not walking symlinks in
mockFs.ListDirsRecursive, but it matches what osFs does.

Test: fs_test.go
Change-Id: If87a83c00f21e14696faf890b7b09e88b18e95b9
2018-09-24 15:09:32 -07:00
Colin Cross
e3b7ec32c9 Improve error message for globbed dangling symlink
IsDir on a dangling symlink produces ErrNotExist.  Manually
check if the file exists but is a symlink to report a better
error.

Test: m checkbuild
Change-Id: I3181e74002436d74ec35a0923635835e561030dd
2018-09-21 15:55:03 -07:00
Colin Cross
a4fae90d1e Fix recursive glob on MockFs
Recursive globs on MockFs were ending up in an unmocked os.Lstat
call in walkAllDirs.  Move walkAllDirs into the FileSystem interface
as ListDirsRecursive.

Also duplicate the glob tests on both the real filesystem and on
MockFs.

Test: glob_test.go
Change-Id: Ia6b6b5eecdd17955a49d684a0fd5e55df05cfe62
2017-10-05 16:06:15 -07:00
Jeff Gaston
aca4220583 Clearer error in case of Android.bp being unreadable
Bug: 64600838
Test: mkdir errtest \
      && ln -s /tmp/dontexist errtest/Android.bp \
      # and add errtest to ./Android.bp \
      && m nothing \
      # and check that the error message mentions a symlink

Change-Id: I841ec12d613f61ccc3396538062bee48c8c1ca27
2017-09-01 17:29:30 -07:00
Colin Cross
616c278b70 Add . and / to MockFs
Add the final directory to MockFs so that Exists() on . and /
returns true.

Test: java_test.go
Change-Id: I1320f4f267ee6b6a7dbf1132630df306602e84d8
2017-07-14 08:16:00 -07:00
Colin Cross
b519a7e1b6 Add globbing to filesystem mocking
Add globbing to filesystem mocking so that more code can be tested
against the mock.  Also moves the filesystem mock to pathtools,
and renames pathtools.GlobWithExcludes to pathtools.Glob, replacing
the existing pathtools.Glob.

Test: blueprint tests
Change-Id: I722df8121bc870c4a861d7c249245c57dcb607be
2017-02-02 16:48:06 -08:00