Commit graph

726 commits

Author SHA1 Message Date
Dan Willemsen
1e72321e58 Use microfactory to build the bootstrap minibp
This duplicates building common blueprint go packages between minibp and
the primary builder, but drastically simplifies the first stage,
removing the need to check in a generated build.ninja.in.

Change-Id: I639a9637f1ed36d4210823ef276c0f7a064a83bd
2017-07-24 14:02:51 -07:00
Dan Willemsen
7dafbbcf6a Add microfactory directly from Soong
From commit 38cef8ac3979da8e13037540981fe7aecc1e1fbe

Removes Go 1.7 from travis, since this depends on os.Executable

Change-Id: I58e2ae542621ffad66e9e145031091cb93ebb497
2017-07-24 14:02:15 -07:00
Dan Willemsen
7f2b2d229b Check for .bootstrap/build.ninja rebuilds too
Change-Id: Ic77b216e7153365cc6b897aba4e72ceee750c05e
2017-07-24 13:08:58 -07:00
colincross
a1501b96ce Merge pull request #161 from colincross/mockfs
Add . and / to MockFs
2017-07-16 09:35:50 -07:00
Colin Cross
616c278b70 Add . and / to MockFs
Add the final directory to MockFs so that Exists() on . and /
returns true.

Test: java_test.go
Change-Id: I1320f4f267ee6b6a7dbf1132630df306602e84d8
2017-07-14 08:16:00 -07:00
colincross
7a280738f3 Merge pull request #160 from colincross/testmain
Fix tests with TestMain
2017-07-13 16:42:32 -07:00
Colin Cross
5f303b9ee8 Fix tests with TestMain
If tests have a TestMain method the os import is unused and
breaks building the test.

Test: gotestmain/testmain_test.go
Change-Id: Ia46cdd0df71b0fc7a53d08644d220ecfd779b2ff
2017-07-13 11:10:39 -07:00
Colin Cross
26c141fe68 Add a script to regenerate build.ninja.in files
Add regen_build_ninja_in.sh to regenerate build.ninja.in and
tests/test_tree/build.ninja.in.

Change-Id: Ib196576fb39eb1adf4ed660b7aa2d24c84aed077
2017-07-13 11:10:39 -07:00
Colin Cross
d317b06164 Test against go 1.7.6 and 1.8.3
Go 1.5.1 is obsolete, test against the two currently supported versions
of go (1.7.6 and 1.8.3).

Change-Id: Ie463dd4d57e3d5c7795e780aedd8056072695342
2017-07-13 11:10:39 -07:00
colincross
fa54772e55 Merge pull request #159 from colincross/testmain
Support TestMain in tests
2017-07-12 16:45:10 -07:00
Colin Cross
83359c674e Support TestMain in tests
Use testing.MainStart to support test packages that implement
TestMain for initial setup and teardown.

Change-Id: I7330a8e922448063f291f7b5b2eed31d274248bd
2017-07-12 13:19:53 -07:00
Jeff Gaston
a1dae80bbb Utils to support removing duplicate shared_libs
Bug: 62957047
Test: m -j
Change-Id: I50d61d2c4a15af571d3bd5ca95f8ff7e3f18c382
2017-06-28 14:38:05 -07:00
Dan Willemsen
d6ffd7c59f Merge pull request #156 from danw/glob_deps
Fix glob deps for non-wild patterns
2017-06-23 11:14:41 -07:00
Dan Willemsen
75fec88fc6 Fix glob deps for non-wild patterns
Patterns that were not wild would return an empty "dirs" list if the
file was found. But then if they were removed, we wouldn't know to
update the glob file and re-run the primary builder.

In this case, instead of adding the final directory into the dirs list,
add the matched files themselves. Due to editors performing atomic
writes (the directory timestamp often gets updated at the same time as
file timestamp) this is probably more efficient. In either case, we're
only re-running the individual glob, which is rather cheap.

Rename startGlob/Glob return name from "dirs" to "deps" since it may
contain files now too.
2017-06-22 17:53:39 -07:00
colincross
b8a9f2e858 Merge pull request #155 from mathjeff/bpfix-utils
Some util functions to support bpfix
2017-06-19 11:12:33 -07:00
Jeff Gaston
5d4b9d8fde Some util functions to support bpfix
Change-Id: If53f696fbe937e007b902434f62d0d92435846dd
2017-05-23 17:51:44 -07:00
colincross
d1d60e7707 Merge pull request #154 from colincross/open_files
Cap concurrency when parsing blueprint files
2017-05-16 11:09:59 -07:00
Colin Cross
4a02a3019a Cap concurrency when parsing blueprint files
Darwin has a default limit of 256 open files per process.  Parsing
too many blueprint files in parallel can hit the limit.  Cap the
concurrency at 200.

Test: manual testing with limit set to 32
Change-Id: Ic64d21d2c0ffd7c86bf3f02fb51216ee5684a80c
2017-05-16 10:55:29 -07:00
colincross
ea10dedba9 Merge pull request #153 from colincross/build_description
Allow BuildParams to override ninja description
2017-05-09 16:47:57 -07:00
Colin Cross
8de48af6de Escape leading space in ninja strings
Spaces normally don't need to be escaped, but leading spaces are
trimmed.  Escape leading space to allow setting a variable to a
value with leading spaces.

Test: ninja_string_test.go
Change-Id: Ic0ffb076dbd603b7c0203720b9c1ea635c5ded75
2017-05-09 10:14:38 -07:00
Colin Cross
7aa318f83d Allow BuildParams to override ninja description
Allow the ninja description variable to be set on build statements
as well as rules.

Change-Id: I6abb2a8ec83a0c662348cc957fa1a307e6c9c6bb
2017-05-09 10:02:54 -07:00
Dan Willemsen
a30a741c22 Merge pull request #149 from danw/improve_variant_errors
Improve error messages when dependency variants are missing
2017-04-07 14:05:06 -07:00
Dan Willemsen
978c4aa92f Improve error messages when dependency variants are missing
Instead of just saying:

error: .../Android.bp:48:1: dependency "libc++" of "libtest" missing variant "arch:android_arm_armv7-a, link:shared, vndk:"

Include a list of currently existing variants:

error: .../Android.bp:48:1: dependency "libc++" of "libtest" missing variant:
  arch:android_arm_armv7-a, link:shared, vndk:
available variants:
  arch:android_arm_armv7-a, link:shared
  arch:linux_x86, link:shared
  arch:linux_x86_64, link:shared
  arch:windows_x86, link:shared
  arch:windows_x86_64, link:shared

This still isn't the best experience for users, but it at least provides
enough information for someone more familiar with the build to
understand the problem.
2017-03-20 14:23:43 -07:00
colincross
28c7837b3d Merge pull request #148 from colincross/visitdirectdeps
Add Context.VisitDirectDeps
2017-03-17 14:01:03 -07:00
Colin Cross
080c1336b5 Add Context.VisitDirectDeps
Context already has VisitDepsDepthFirst[If], add VisitDirectDeps[If].

Change-Id: Id550bc14275db230c19fd6ca14b67b305dd7d96b
2017-03-17 13:46:47 -07:00
Dan Willemsen
3f2d661087 Merge pull request #147 from danw/goos_tests_fix
Fix bad ninja file if only GOOS-specific tests exist
2017-03-16 09:45:51 -07:00
Dan Willemsen
5d05872cd5 Merge pull request #146 from Nan-Zhang/BaseDependencyTag
Added a check in Blueprint level for BaseDependencyTag
2017-03-16 09:45:39 -07:00
Dan Willemsen
aee3eac6ce Fix bad ninja file if only GOOS-specific tests exist
When only linux/darwin tests were specified, we weren't setting
testArchiveFile, so we had a corrupted ninja file (build : ...).
2017-03-15 23:25:20 -07:00
Nan Zhang
346b2d0d80 Added a check in BP level for BaseDependencyTag
It is not allowed to directly use BaseDependencyTag as customized
user dependency tag passed down to BP since it might cause issues
that different type of modules will be mixed when fetched based on
Tag.
2017-03-10 16:39:27 -08:00
Dan Willemsen
2831e690f0 Merge pull request #142 from danw/_GOOS.go
Support filtering OS-dependent go files using _darwin/_linux.go
2017-02-21 15:07:00 -08:00
Dan Willemsen
f14e096607 Support darwin/linux specific go sources
In order to use certain syscalls, some source must only be compiled on
one of darwin or linux. There are also cases where it's simplest to have
two implementations of a method (or struct) instead of choosing an
implementation at runtime.

The standard go tools use _<GOOS>.go for this, but the explicit structs
map better into the current blueprint format (and you can still use
_darwin.go / _linux.go to maintain compatiblity with those tools).

Test: Define _linux.go / _darwin.go files, ensure only one is compiled.
Change-Id: I58bffefbf839ba52dea3d8e7d3ec67eaa721c463
2017-02-18 15:12:13 -08:00
colincross
e8a09f95f6 Merge pull request #141 from colincross/fs
Add globbing to filesystem mocking
2017-02-02 16:53:03 -08:00
Colin Cross
b519a7e1b6 Add globbing to filesystem mocking
Add globbing to filesystem mocking so that more code can be tested
against the mock.  Also moves the filesystem mock to pathtools,
and renames pathtools.GlobWithExcludes to pathtools.Glob, replacing
the existing pathtools.Glob.

Test: blueprint tests
Change-Id: I722df8121bc870c4a861d7c249245c57dcb607be
2017-02-02 16:48:06 -08:00
Dan Willemsen
2eeb7ee14b Merge pull request #140 from danw/sort_tests
Sort testcases to make build (more) reproducible
2017-01-31 18:08:26 -08:00
Dan Willemsen
61caa240c8 Sort testcases to make build (more) reproducible
f.Scope.Objects is a map, so iteration order isn't defined. To fix this,
lets just sort the testcase list so that it's always consistent.

There are still some paths in the built go files, but this improves
repeatability using the same paths.

Test: Build twice, compare generated test.go files.
Change-Id: Ie3fc896f54314fee0e0e9ddc3a9e06ad2f72a78d
2017-01-31 10:45:08 -08:00
Dan Willemsen
188db1bdfc Merge pull request #139 from danw/tests
Improve testing tools
2017-01-18 16:18:22 -08:00
Dan Willemsen
b53f7beb72 gotestrunner: Make GOROOT absolute before chdir
If GOROOT is a relative path (like it is in Android), then changing
directories to run the test will cause GOROOT to become invalid. So make
it absolute if we're going to change to a different directory.

This allows tests to use the go tools without searching for a valid go
installation.

Change-Id: Ifab0a8533d236054ccf363dfb68b12e0bf66f6f8
2017-01-18 14:42:09 -08:00
Dan Willemsen
10e4357dbb Fix -test.run test selection with gotestmain
matchString was implemented separately on older go versions since they
didn't want a build dependency on the regex package from the testing
package.

Starting in Go 1.8, this is implemented using a internal package to
break the dependency, but until we require that everywhere, just copy
the regex implementation like the old `go test` did. I'm not sure we'll
be able to use the internal package in the future anyways.

Change-Id: Ief37542640026f9d73a75fd802f2691af50e2511
2017-01-18 14:37:13 -08:00
Dan Willemsen
e8a33ee623 Enable go tests for blueprint_tools
We haven't been building the tests for any Go tools that get built
during the primary stage, since we haven't been passing -t to the
primary builder. So enable them there so that we test those tools.

Change-Id: Ic7720c0679419f9e1ba61c7e151f37eb5c7bc8dc
2017-01-18 14:34:46 -08:00
colincross
8c7fd39c28 Merge pull request #138 from colincross/test_symlink
Make symlink during test
2017-01-11 14:02:33 -08:00
Colin Cross
da945bf33c Make symlink during test
The tests/test_tree/blueprint symlink causes find to print a warning
in the Android build:
find: File system loop detected; `./build/blueprint/tests/test_tree/blueprint' is part of the same file system loop as `./build/blueprint'.

Make the symlink when running the test instead.

Change-Id: I52788af7b262f8aa7a00a501f0b50695e469adb1
2017-01-11 13:50:58 -08:00
Dan Willemsen
b431941604 Merge pull request #108 from danw/globexclude
Prevent glob from accessing hidden files
2016-12-19 15:50:40 -08:00
Dan Willemsen
53f4950eea Prevent glob from accessing hidden files
Hidden files are often source control related (".git", ".repo", etc) or
editor related (vim's .*.swp), so are not guaranteed to exist, and may
be temporary. The build system shouldn't be using these files by
default.

If the glob pattern explicitly uses "." at the beginning of a path
component, allow returning hidden files for that component. Because of
this behavior, non-wildcard globs remain unchanged.

The one behavior that cannot be handled anymore is including hidden
files in recursive globs.

Change-Id: I583c506e9a18ed2ff7ca011a791165d9582de90f
2016-12-19 15:17:26 -08:00
colincross
f3218f61b6 Merge pull request #137 from colincross/name
Add ModuleContext method to get dependency by name
2016-12-14 21:45:11 -08:00
Colin Cross
dfb4c9f7af Add ModuleContext method to get dependency by name
Instead of forcing every mutator or module to use VisitDirectDeps
to iterate through its deps, allow it to request a dependency by
name to return the Module.

Change-Id: I18b23bffa324bb9d93a7743b6e2a19c07058c775
2016-12-14 10:42:00 -08:00
colincross
92844f0da8 Merge pull request #136 from colincross/race
Fix data race when applying replacements
2016-12-09 11:09:21 -08:00
Colin Cross
0ce142ca05 Fix data race when applying replacements
Mutator context goroutines appending directly to the global Context's
replacements list causes a data race.  Send them over a channel
instead.

The renames and replacements are local to the mutator, so move them
out of Context and into the runMutator method.

Change-Id: I797edb1e27ee29f8946c58101b40fcfb50a32eb9
2016-12-09 10:35:33 -08:00
colincross
0f1fa92e86 Merge pull request #135 from colincross/deps
Support deps and depfile properties on build statements
2016-11-22 13:51:07 -08:00
Colin Cross
aa873e1c65 Support deps and depfile properties on build statements
Ninja supports deps and depfile properties specified on build
statements, allow them to be specified in BuildParams.

Change-Id: I46eac5571350426f1419908def21f7d042f8739a
2016-11-22 13:40:45 -08:00
colincross
8c5b1d1f60 Merge pull request #134 from colincross/glob
Fix optional_subdirs globbing
2016-11-14 17:38:00 -08:00