Commit graph

13 commits

Author SHA1 Message Date
Chris Parsons
a798d96076 find bazel-related files and add them to bazel.list and ninja deps
This retriggers soong_build whenever a new bzl, WORKSPACE, or
BUILD.bazel file is changed or added.

Test: Manually verified on bionic/libc genrules with manual changes to
related BUILD/bzl/WORKSPACE files -- these all retrigger builds.
Test: Updated finder_test.go

Change-Id: I634384f88781a6b6db32f5d6bf9c07e179e14c39
2020-10-13 16:18:59 -04:00
Colin Cross
96e5e41b88 Dist module_paths/files.db
Help debugging finder issues by disting module_paths/files.db for
later analysis.

Bug: 157656545
Test: treehugger
Change-Id: Idfa4cffe7ef724b399a2667efd487f9c3c76bb6b
2020-07-06 17:15:05 -07:00
Colin Cross
25fd77310c Fix finder on symlinks pointing to directories
When finder encountered a symlink pointing to a directory it attempted
to ignore it, but becuase Lstat returns the a *os.FileInfo for the
symlink the child.IsDir() check always returned false.  Call Stat
on the symlink to get the *os.FileInfo for the target of the symlink.

Bug: 157656545
Test: finder_test.go
Change-Id: Ie33d2e05d5c435b48e83eaeadf7b3c9816398404
2020-06-29 23:30:38 -07:00
Colin Cross
a88c883e3e Add a DirEntryInfo interface that is a subset of os.FileInfo
ioutil.ReadDir returns []os.FileInfo, which contains information on
each entry in the directory that is only available by calling
os.Lstat on the entry.  Finder only the name and type (regular,
directory or symlink) of the files, which on Linux kernels >= 2.6.4
is available in the return values of syscall.Getdents.

In preparation for using syscall.Getdents, switch filesystem.ReadDir
to return an interface that only contains the information that will
be available from syscall.Getdents.

Bug: 70897635
Test: m checkbuild
Change-Id: Id2749d709a0f7b5a61abedde68549d4bf208a568
2017-12-22 13:56:17 -08:00
Colin Cross
8d6395c09d Move android/soong/fs to android/soong/finder/fs
The fs package is specific to finder, move it inside finder.

Bug: 70897635
Test: m checkbuild
Change-Id: Ie705f064a832141702a8e87fd59ed75c01018504
2017-12-22 13:56:17 -08:00
Jeff Gaston
821271b3b8 Fix the Finder's ability to ignore permission errors
It turns out the os was returning a different permissions error than
os.ErrPermission. Fortunately we can just use os.IsPermission().

Bug: 62455338
Test: mkdir not-permitted && chmod 000 not-permitted && m nothing

Change-Id: Ieb1653b9687ffdaae83184dcdcd4166ddb6a4613
2017-08-23 17:53:41 -07:00
Jeff Gaston
d311952e93 Add some tests for singlethreaded usage of the Finder.
Also change the existing tests to use exactly 2 threads.
Also update some tests to also same number of threads as will
be used by the finder.

Bug: 62455338
Test: m -j # which runs unit tests

Change-Id: I2b9d39680ecddd6d28c951df982ac51c077d2777
2017-08-22 16:55:21 -07:00
Jeff Gaston
996716a8e2 Fix hanging of finder_test
Previously there could be an infinite loop in the Finder's db export when
running tests on a system with few cores.

The Finder divides the cache-serialization work amongst cores. On systems with
enough cores, each path in TestFileSystemRoot would get its own core and the
infinite loop in the prefix calculation logic would be avoided. On systems with
fewer cores, the computation of the maximum common path could run forever.

This shouldn't have affected the general usage of the Finder in builds other
than in its unit tests, because the builds don't ask to include the
filesystem root as a path to scan. This should only have affected
finder_test.go which tested the filesystem root anyway.

Bug: 62455338
Test: m -j # which runs unit tests
Change-Id: I7a1467fb32e4538fb96833791180bfbfad5a3911
2017-08-22 14:14:40 -07:00
Jeff Gaston
b64fc1cab5 Run the Finder and make its results available to Kati
The Finder runs roughly 200ms faster than findleaves.py in aosp,
and runs roughly 400ms faster in internal master.

Bug: 64363847
Test: m -j

Change-Id: I62db8dacc90871e913576fe2443021fb1749a483
2017-08-16 14:56:00 -07:00
Jeff Gaston
b629e184dd Fail the Finder in case of unexpected fs error
Permissions errors are ignored
Errors on pruned dirs are also ignored

Bug: 62455338
Test: m -j blueprint_tools # which runs unit tests
Change-Id: I8ba85fdd0295deb7dc374a851212e7c850e76b75
2017-08-15 13:18:24 -07:00
Jeff Gaston
f1fd45e784 Revert "Revert "Cacheable, multithreaded finder.""
Bug: 62455338
Test: m -j

This reverts commit d1abeb9d98.

Change-Id: I9f73031636157511b5f1c6ce8a205e9bc91669ff
2017-08-10 12:20:30 -07:00
Jeff Gaston
d1abeb9d98 Revert "Cacheable, multithreaded finder."
This reverts commit b6d161bf16.

Reason for revert: New Build Breakage: aosp-master/sdk_mac @ 4260825

Change-Id: I8bda8c50c5e5c9f84621d11a4c15b168833bcd21
2017-08-10 01:16:34 +00:00
Jeff Gaston
b6d161bf16 Cacheable, multithreaded finder.
It can find every Android.bp in internal master in
about 2.5 sec the first time and 0.3 sec subsequent times

Bug: 62455338

Test: m -j blueprint_tools # which runs the unit tests

Test: m -j blueprint_tools && \
  out/soong/host/linux-x86/bin/finder \
    -v
    --db /tmp/mydb \
    --names Android.mk \
    --prune-files .android-out-dir \
    --exclude-dirs .git,.repo \
    . \
    >/tmp/finder-log 2>&1

Change-Id: I5ab2650459a1dae0d5d076faf411ec2d053c743d
2017-08-08 14:19:55 -07:00