Commit graph

15 commits

Author SHA1 Message Date
Colin Cross
b828213402 Fix data race in finder_test.go
Wait for newFinder to finish writing its dump before ticking the clock.

Test: go test -race ./...
Change-Id: Ib7527ebfb4e1c989deefdb0779a4f9c0928ae15d
2024-01-18 13:23:06 -08:00
Lukacs T. Berki
e3487c8848 Add a test for correctness of C++ compilation.
This required the following:

- Adding Platform_base_sdk_extension_version to default soong.variables
- Teaching the symlink tree creation code to understand symlinks
- Making finder.go follow symlinks when requested

Adding yet another knob is unfortunate, but I can't allow that
unconditionally because the Android code base contains a number of
symlinks giving rise to infinite directory trees because they point back
to their parent and this seemed preferable to adding complicated logic
like "follow symlink but if only its fully resolved version does not
point under the source tree".

I could be convinced about the latter, though.

Test: Presubmits.
Change-Id: I453f6b7e5334771f5832c700db00f9d24ed1d82f
2022-05-04 09:12:01 +02:00
Chris Parsons
a798d96076 find bazel-related files and add them to bazel.list and ninja deps
This retriggers soong_build whenever a new bzl, WORKSPACE, or
BUILD.bazel file is changed or added.

Test: Manually verified on bionic/libc genrules with manual changes to
related BUILD/bzl/WORKSPACE files -- these all retrigger builds.
Test: Updated finder_test.go

Change-Id: I634384f88781a6b6db32f5d6bf9c07e179e14c39
2020-10-13 16:18:59 -04:00
Colin Cross
405af07859 Revert "Make lots of tests run in parallel"
This reverts commit 323dc60712.

Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests

Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
2020-10-09 18:34:24 -07:00
Colin Cross
323dc60712 Make lots of tests run in parallel
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.

Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
2020-10-06 15:12:22 -07:00
Colin Cross
25fd77310c Fix finder on symlinks pointing to directories
When finder encountered a symlink pointing to a directory it attempted
to ignore it, but becuase Lstat returns the a *os.FileInfo for the
symlink the child.IsDir() check always returned false.  Call Stat
on the symlink to get the *os.FileInfo for the target of the symlink.

Bug: 157656545
Test: finder_test.go
Change-Id: Ie33d2e05d5c435b48e83eaeadf7b3c9816398404
2020-06-29 23:30:38 -07:00
Colin Cross
7f8aa39abe Move finder_test filesystem helper functions to fs/test.go
Move the filesystem helper functions used by finder_test.go into the
fs package so they can also be used by fs tests.

Bug: 157656545
Test: m checkbuild
Change-Id: I5ac3a1f502d8e2cb7c15dd70ca85a46d20f939a6
2020-06-29 23:30:38 -07:00
Colin Cross
2f5a484ee1 Fix data races in finder_test.go
Fix data races by incrementing clock before starting the finder.
Fixes:
==================
WARNING: DATA RACE
Read at 0x00c0003be1c0 by goroutine 99:
  android/soong/finder/fs.(*MockFs).WriteFile()
      build/soong/finder/fs/fs.go:159 +0x869
  android/soong/finder.(*Finder).dumpDb()
      build/soong/finder/finder.go:1163 +0x296
  android/soong/finder.(*Finder).goDumpDb.func1()
      build/soong/finder/finder.go:337 +0x3c

Previous write at 0x00c0003be1c0 by goroutine 43:
  android/soong/finder.TestDirectoryNotPermitted()
      build/soong/finder/fs/fs.go:155 +0x49a
  testing.tRunner()
      prebuilts/go/linux-x86/src/testing/testing.go:865 +0x163

Goroutine 99 (running) created at:
  android/soong/finder.(*Finder).goDumpDb()
      build/soong/finder/finder.go:336 +0x8d
  android/soong/finder.(*Finder).loadFromFilesystem()
      build/soong/finder/finder.go:380 +0x12b
  android/soong/finder.newImpl()
      build/soong/finder/finder.go:197 +0x38d
  android/soong/finder.newFinderAndErr()
      build/soong/finder/finder_test.go:58 +0x245
  android/soong/finder.newFinderWithNumThreads()
      build/soong/finder/finder_test.go:42 +0xb3
  android/soong/finder.TestDirectoryNotPermitted()
      build/soong/finder/finder_test.go:38 +0x386
  testing.tRunner()
      prebuilts/go/linux-x86/src/testing/testing.go:865 +0x163

Goroutine 43 (running) created at:
  testing.(*T).Run()
      prebuilts/go/linux-x86/src/testing/testing.go:916 +0x65a
  testing.runTests.func1()
      prebuilts/go/linux-x86/src/testing/testing.go:1157 +0xa8
  testing.tRunner()
      prebuilts/go/linux-x86/src/testing/testing.go:865 +0x163
  testing.runTests()
      prebuilts/go/linux-x86/src/testing/testing.go:1155 +0x523
  testing.(*M).Run()
      prebuilts/go/linux-x86/src/testing/testing.go:1072 +0x2eb
  main.main()
      out/soong/.bootstrap/soong-finder/test/test.go:157 +0x1dc
==================

Test: run finder_test.go with race detector enabled
Change-Id: Ia77090dabaf8b14a80ac9ce938fef75f1d382574
2019-06-20 15:24:05 -07:00
Colin Cross
8d6395c09d Move android/soong/fs to android/soong/finder/fs
The fs package is specific to finder, move it inside finder.

Bug: 70897635
Test: m checkbuild
Change-Id: Ie705f064a832141702a8e87fd59ed75c01018504
2017-12-22 13:56:17 -08:00
Jeff Gaston
d311952e93 Add some tests for singlethreaded usage of the Finder.
Also change the existing tests to use exactly 2 threads.
Also update some tests to also same number of threads as will
be used by the finder.

Bug: 62455338
Test: m -j # which runs unit tests

Change-Id: I2b9d39680ecddd6d28c951df982ac51c077d2777
2017-08-22 16:55:21 -07:00
Jeff Gaston
b64fc1cab5 Run the Finder and make its results available to Kati
The Finder runs roughly 200ms faster than findleaves.py in aosp,
and runs roughly 400ms faster in internal master.

Bug: 64363847
Test: m -j

Change-Id: I62db8dacc90871e913576fe2443021fb1749a483
2017-08-16 14:56:00 -07:00
Jeff Gaston
b629e184dd Fail the Finder in case of unexpected fs error
Permissions errors are ignored
Errors on pruned dirs are also ignored

Bug: 62455338
Test: m -j blueprint_tools # which runs unit tests
Change-Id: I8ba85fdd0295deb7dc374a851212e7c850e76b75
2017-08-15 13:18:24 -07:00
Jeff Gaston
f1fd45e784 Revert "Revert "Cacheable, multithreaded finder.""
Bug: 62455338
Test: m -j

This reverts commit d1abeb9d98.

Change-Id: I9f73031636157511b5f1c6ce8a205e9bc91669ff
2017-08-10 12:20:30 -07:00
Jeff Gaston
d1abeb9d98 Revert "Cacheable, multithreaded finder."
This reverts commit b6d161bf16.

Reason for revert: New Build Breakage: aosp-master/sdk_mac @ 4260825

Change-Id: I8bda8c50c5e5c9f84621d11a4c15b168833bcd21
2017-08-10 01:16:34 +00:00
Jeff Gaston
b6d161bf16 Cacheable, multithreaded finder.
It can find every Android.bp in internal master in
about 2.5 sec the first time and 0.3 sec subsequent times

Bug: 62455338

Test: m -j blueprint_tools # which runs the unit tests

Test: m -j blueprint_tools && \
  out/soong/host/linux-x86/bin/finder \
    -v
    --db /tmp/mydb \
    --names Android.mk \
    --prune-files .android-out-dir \
    --exclude-dirs .git,.repo \
    . \
    >/tmp/finder-log 2>&1

Change-Id: I5ab2650459a1dae0d5d076faf411ec2d053c743d
2017-08-08 14:19:55 -07:00