Having separate .ninja_log files doesn't work very well after
04d886b110
since the last successful timestamp is not recorded in the ninja log.
This was triggering the primary builder to be run twice on every build
since it was always considered out of date in the primary, then main
stages.
The two bootstrap stages already were unified when minibootstrap was
simplified, but this combines the main stage as well. This means that
we'll save some time running globs that are shared between minibp and
the primary builder as well (subdirs globs).
The cleanup code needed to be refactored, since the ninja_log will have
entries from the main stage as well. So instead of looking at everything
in the ninja log, take a string prefix to limit what we may clean.
Change-Id: I8d43168643aa8a0c6c3e48c6101eaa45d174bbe5
Now that packages and binaries are only built when needed, they need to
be defined in both stages in case something building in the main stage
needs something that was defined, but didn't need to be built in the
earlier stage.
Change-Id: I67704fba1d670e82218eba5ce91f0c01896557a2
Instead of forcing things to be built alongside the primary builder,
mark all packages and binaries as optional, unless they're explicitly
marked as default.
Change-Id: Ie53c90a01b41ce886d651f9aa4c4900d1edb0157
This duplicates building common blueprint go packages between minibp and
the primary builder, but drastically simplifies the first stage,
removing the need to check in a generated build.ninja.in.
Change-Id: I639a9637f1ed36d4210823ef276c0f7a064a83bd
From commit 38cef8ac3979da8e13037540981fe7aecc1e1fbe
Removes Go 1.7 from travis, since this depends on os.Executable
Change-Id: I58e2ae542621ffad66e9e145031091cb93ebb497
If tests have a TestMain method the os import is unused and
breaks building the test.
Test: gotestmain/testmain_test.go
Change-Id: Ia46cdd0df71b0fc7a53d08644d220ecfd779b2ff
Use testing.MainStart to support test packages that implement
TestMain for initial setup and teardown.
Change-Id: I7330a8e922448063f291f7b5b2eed31d274248bd
Patterns that were not wild would return an empty "dirs" list if the
file was found. But then if they were removed, we wouldn't know to
update the glob file and re-run the primary builder.
In this case, instead of adding the final directory into the dirs list,
add the matched files themselves. Due to editors performing atomic
writes (the directory timestamp often gets updated at the same time as
file timestamp) this is probably more efficient. In either case, we're
only re-running the individual glob, which is rather cheap.
Rename startGlob/Glob return name from "dirs" to "deps" since it may
contain files now too.
Darwin has a default limit of 256 open files per process. Parsing
too many blueprint files in parallel can hit the limit. Cap the
concurrency at 200.
Test: manual testing with limit set to 32
Change-Id: Ic64d21d2c0ffd7c86bf3f02fb51216ee5684a80c
Spaces normally don't need to be escaped, but leading spaces are
trimmed. Escape leading space to allow setting a variable to a
value with leading spaces.
Test: ninja_string_test.go
Change-Id: Ic0ffb076dbd603b7c0203720b9c1ea635c5ded75
Instead of just saying:
error: .../Android.bp:48:1: dependency "libc++" of "libtest" missing variant "arch:android_arm_armv7-a, link:shared, vndk:"
Include a list of currently existing variants:
error: .../Android.bp:48:1: dependency "libc++" of "libtest" missing variant:
arch:android_arm_armv7-a, link:shared, vndk:
available variants:
arch:android_arm_armv7-a, link:shared
arch:linux_x86, link:shared
arch:linux_x86_64, link:shared
arch:windows_x86, link:shared
arch:windows_x86_64, link:shared
This still isn't the best experience for users, but it at least provides
enough information for someone more familiar with the build to
understand the problem.
It is not allowed to directly use BaseDependencyTag as customized
user dependency tag passed down to BP since it might cause issues
that different type of modules will be mixed when fetched based on
Tag.
In order to use certain syscalls, some source must only be compiled on
one of darwin or linux. There are also cases where it's simplest to have
two implementations of a method (or struct) instead of choosing an
implementation at runtime.
The standard go tools use _<GOOS>.go for this, but the explicit structs
map better into the current blueprint format (and you can still use
_darwin.go / _linux.go to maintain compatiblity with those tools).
Test: Define _linux.go / _darwin.go files, ensure only one is compiled.
Change-Id: I58bffefbf839ba52dea3d8e7d3ec67eaa721c463
Add globbing to filesystem mocking so that more code can be tested
against the mock. Also moves the filesystem mock to pathtools,
and renames pathtools.GlobWithExcludes to pathtools.Glob, replacing
the existing pathtools.Glob.
Test: blueprint tests
Change-Id: I722df8121bc870c4a861d7c249245c57dcb607be
f.Scope.Objects is a map, so iteration order isn't defined. To fix this,
lets just sort the testcase list so that it's always consistent.
There are still some paths in the built go files, but this improves
repeatability using the same paths.
Test: Build twice, compare generated test.go files.
Change-Id: Ie3fc896f54314fee0e0e9ddc3a9e06ad2f72a78d
If GOROOT is a relative path (like it is in Android), then changing
directories to run the test will cause GOROOT to become invalid. So make
it absolute if we're going to change to a different directory.
This allows tests to use the go tools without searching for a valid go
installation.
Change-Id: Ifab0a8533d236054ccf363dfb68b12e0bf66f6f8
matchString was implemented separately on older go versions since they
didn't want a build dependency on the regex package from the testing
package.
Starting in Go 1.8, this is implemented using a internal package to
break the dependency, but until we require that everywhere, just copy
the regex implementation like the old `go test` did. I'm not sure we'll
be able to use the internal package in the future anyways.
Change-Id: Ief37542640026f9d73a75fd802f2691af50e2511
We haven't been building the tests for any Go tools that get built
during the primary stage, since we haven't been passing -t to the
primary builder. So enable them there so that we test those tools.
Change-Id: Ic7720c0679419f9e1ba61c7e151f37eb5c7bc8dc
The tests/test_tree/blueprint symlink causes find to print a warning
in the Android build:
find: File system loop detected; `./build/blueprint/tests/test_tree/blueprint' is part of the same file system loop as `./build/blueprint'.
Make the symlink when running the test instead.
Change-Id: I52788af7b262f8aa7a00a501f0b50695e469adb1
Hidden files are often source control related (".git", ".repo", etc) or
editor related (vim's .*.swp), so are not guaranteed to exist, and may
be temporary. The build system shouldn't be using these files by
default.
If the glob pattern explicitly uses "." at the beginning of a path
component, allow returning hidden files for that component. Because of
this behavior, non-wildcard globs remain unchanged.
The one behavior that cannot be handled anymore is including hidden
files in recursive globs.
Change-Id: I583c506e9a18ed2ff7ca011a791165d9582de90f
