Commit graph

446 commits

Author SHA1 Message Date
Dan Willemsen
b8534bf280 Use go1.9 instead of go1.9rc1
Change-Id: Id698eff8efeb723f8665c0358cdb0cfca4d5f776
2017-09-13 16:50:27 -07:00
colincross
c40ef075f4 Merge pull request #158 from mathjeff/travis-link
Remove src.test after running Travis tests
2017-09-13 10:12:01 -07:00
Jeff Gaston
5f332c83ae Move generated Travis test dirs outside the tree
This should prevent the running of Travis tests from causing
a filesystem loop src.test/test_tree/blueprint -> ../..

Bug: 63100441
Test: NINJA=... ./tests/test_tree_tests.sh; stat src.test 2>/dev/null \
      && echo "src.test should not exist"

Change-Id: Ie01fe8999c738dca166e3fabde9d9391f6b70210
2017-09-12 17:58:10 -07:00
colincross
1ff2efa021 Merge pull request #170 from mathjeff/symlink-err
Clearer error in case of Android.bp being unreadable
2017-09-06 17:16:08 -07:00
Jeff Gaston
aca4220583 Clearer error in case of Android.bp being unreadable
Bug: 64600838
Test: mkdir errtest \
      && ln -s /tmp/dontexist errtest/Android.bp \
      # and add errtest to ./Android.bp \
      && m nothing \
      # and check that the error message mentions a symlink

Change-Id: I841ec12d613f61ccc3396538062bee48c8c1ca27
2017-09-01 17:29:30 -07:00
Dan Willemsen
18a0d2e9b5 Merge pull request #167 from danw/microfactory
Export microfactory as a package
2017-08-08 15:29:55 -07:00
Dan Willemsen
269248ff2d Support custom trace functions
Change-Id: I43207fd55f79d4adb783ad39eb5ee1b641ecce68
2017-08-08 13:50:44 -07:00
Dan Willemsen
5136c43117 Implement microfactory Config, Build
To make it easier to use as a package.

Change-Id: Idcb4856f34b91943a320e8281ef9e29af0faab98
2017-08-08 13:49:07 -07:00
Dan Willemsen
ff092863b3 Allow microfactory to be used as a package
In addition to running with `go run` and creating a microfactory binary,
allow microfactory to be used as a package from other go tools as well.

To allow other packages to use this, it needs to be in a non-main
package, but `go run` requires a main package. So microfactory.bash runs
a sed script before running microfactory with `go run`.

This could also be solved by using a relative import, but neither
blueprint nor microfactory currently support that.

Change-Id: I084163b14720102b3fb93a3c9d44b5d0225ff2c8
2017-08-08 13:19:26 -07:00
Dan Willemsen
f00c03da53 Stop passing path to microfactory source
It can be discovered via the existing pkg-path mappings.

Change-Id: I0b6ad7a9cccec195d602508104d6cf1e80936299
2017-08-08 13:05:02 -07:00
Dan Willemsen
3048433458 Merge pull request #165 from danw/go1.9
Go 1.9 changes
2017-07-25 16:53:23 -07:00
Dan Willemsen
a4363454c8 Workaround gimme not setting GOROOT
This appears to be fixed in gimme 1.2.0, but that's not available in
Travis yet (even in the latest trusty images). This is the gimme change
to always set GOROOT:

913dd17ff3

Change-Id: Ic9024b4bbbf5c75ed162a03fe119e4db9a1cdbba
2017-07-25 16:11:14 -07:00
Dan Willemsen
91ec9f4b7a Export GOROOT for minibp / primary builder
Otherwise they'll fall back to the GOROOT embedded in the distribution,
which may not be correct.

This used to be propagated through build.ninja.in, but exporting it in
the environment should catch any other Go users too.

Change-Id: I3d6de8494b266e9b127c0a528d31df03ddc5c3a0
2017-07-25 13:05:55 -07:00
Dan Willemsen
c54c072667 Support parallel Go compiles with Go 1.9
Change-Id: I80f3ad6e563fef1b4f22e936016675262f7ce8bf
2017-07-25 11:08:20 -07:00
Dan Willemsen
53d563facb Fix build on Go 1.9
Change-Id: Ie471f396b22cd88104b4f05dbcf3677c7ffd5f08
2017-07-25 10:45:51 -07:00
Dan Willemsen
f94ab1e486 Merge pull request #164 from danw/microfactory
Simplify bootstrapping with microfactory
2017-07-24 15:28:00 -07:00
Dan Willemsen
feb1a225f2 Update documentation
Change-Id: I59daf7802a1b04961d0372c2417869ad9bd6d0b2
2017-07-24 14:02:51 -07:00
Dan Willemsen
cd4e0cebc9 Unify .ninja_log files
Having separate .ninja_log files doesn't work very well after
04d886b110
since the last successful timestamp is not recorded in the ninja log.
This was triggering the primary builder to be run twice on every build
since it was always considered out of date in the primary, then main
stages.

The two bootstrap stages already were unified when minibootstrap was
simplified, but this combines the main stage as well. This means that
we'll save some time running globs that are shared between minibp and
the primary builder as well (subdirs globs).

The cleanup code needed to be refactored, since the ninja_log will have
entries from the main stage as well. So instead of looking at everything
in the ninja log, take a string prefix to limit what we may clean.

Change-Id: I8d43168643aa8a0c6c3e48c6101eaa45d174bbe5
2017-07-24 14:02:51 -07:00
Dan Willemsen
af456eaae4 Define everything in both stages
Now that packages and binaries are only built when needed, they need to
be defined in both stages in case something building in the main stage
needs something that was defined, but didn't need to be built in the
earlier stage.

Change-Id: I67704fba1d670e82218eba5ce91f0c01896557a2
2017-07-24 14:02:51 -07:00
Dan Willemsen
f60525a63d Mark intermediates as optional
Instead of forcing things to be built alongside the primary builder,
mark all packages and binaries as optional, unless they're explicitly
marked as default.

Change-Id: Ie53c90a01b41ce886d651f9aa4c4900d1edb0157
2017-07-24 14:02:51 -07:00
Dan Willemsen
91e2483024 Fix and test rebuilds of bootstrap ninja when using tests
Add testing of bootstrap.bash -t to travis, and fix spacing issues
causing rebuilds.

Change-Id: Ia30367e40f5cf32de9749b2742d68b57e9201278
2017-07-24 14:02:51 -07:00
Dan Willemsen
1e72321e58 Use microfactory to build the bootstrap minibp
This duplicates building common blueprint go packages between minibp and
the primary builder, but drastically simplifies the first stage,
removing the need to check in a generated build.ninja.in.

Change-Id: I639a9637f1ed36d4210823ef276c0f7a064a83bd
2017-07-24 14:02:51 -07:00
Dan Willemsen
7dafbbcf6a Add microfactory directly from Soong
From commit 38cef8ac3979da8e13037540981fe7aecc1e1fbe

Removes Go 1.7 from travis, since this depends on os.Executable

Change-Id: I58e2ae542621ffad66e9e145031091cb93ebb497
2017-07-24 14:02:15 -07:00
Dan Willemsen
7f2b2d229b Check for .bootstrap/build.ninja rebuilds too
Change-Id: Ic77b216e7153365cc6b897aba4e72ceee750c05e
2017-07-24 13:08:58 -07:00
colincross
a1501b96ce Merge pull request #161 from colincross/mockfs
Add . and / to MockFs
2017-07-16 09:35:50 -07:00
Colin Cross
616c278b70 Add . and / to MockFs
Add the final directory to MockFs so that Exists() on . and /
returns true.

Test: java_test.go
Change-Id: I1320f4f267ee6b6a7dbf1132630df306602e84d8
2017-07-14 08:16:00 -07:00
colincross
7a280738f3 Merge pull request #160 from colincross/testmain
Fix tests with TestMain
2017-07-13 16:42:32 -07:00
Colin Cross
5f303b9ee8 Fix tests with TestMain
If tests have a TestMain method the os import is unused and
breaks building the test.

Test: gotestmain/testmain_test.go
Change-Id: Ia46cdd0df71b0fc7a53d08644d220ecfd779b2ff
2017-07-13 11:10:39 -07:00
Colin Cross
26c141fe68 Add a script to regenerate build.ninja.in files
Add regen_build_ninja_in.sh to regenerate build.ninja.in and
tests/test_tree/build.ninja.in.

Change-Id: Ib196576fb39eb1adf4ed660b7aa2d24c84aed077
2017-07-13 11:10:39 -07:00
Colin Cross
d317b06164 Test against go 1.7.6 and 1.8.3
Go 1.5.1 is obsolete, test against the two currently supported versions
of go (1.7.6 and 1.8.3).

Change-Id: Ie463dd4d57e3d5c7795e780aedd8056072695342
2017-07-13 11:10:39 -07:00
colincross
fa54772e55 Merge pull request #159 from colincross/testmain
Support TestMain in tests
2017-07-12 16:45:10 -07:00
Colin Cross
83359c674e Support TestMain in tests
Use testing.MainStart to support test packages that implement
TestMain for initial setup and teardown.

Change-Id: I7330a8e922448063f291f7b5b2eed31d274248bd
2017-07-12 13:19:53 -07:00
Dan Willemsen
d6ffd7c59f Merge pull request #156 from danw/glob_deps
Fix glob deps for non-wild patterns
2017-06-23 11:14:41 -07:00
Dan Willemsen
75fec88fc6 Fix glob deps for non-wild patterns
Patterns that were not wild would return an empty "dirs" list if the
file was found. But then if they were removed, we wouldn't know to
update the glob file and re-run the primary builder.

In this case, instead of adding the final directory into the dirs list,
add the matched files themselves. Due to editors performing atomic
writes (the directory timestamp often gets updated at the same time as
file timestamp) this is probably more efficient. In either case, we're
only re-running the individual glob, which is rather cheap.

Rename startGlob/Glob return name from "dirs" to "deps" since it may
contain files now too.
2017-06-22 17:53:39 -07:00
colincross
b8a9f2e858 Merge pull request #155 from mathjeff/bpfix-utils
Some util functions to support bpfix
2017-06-19 11:12:33 -07:00
Jeff Gaston
5d4b9d8fde Some util functions to support bpfix
Change-Id: If53f696fbe937e007b902434f62d0d92435846dd
2017-05-23 17:51:44 -07:00
colincross
d1d60e7707 Merge pull request #154 from colincross/open_files
Cap concurrency when parsing blueprint files
2017-05-16 11:09:59 -07:00
Colin Cross
4a02a3019a Cap concurrency when parsing blueprint files
Darwin has a default limit of 256 open files per process.  Parsing
too many blueprint files in parallel can hit the limit.  Cap the
concurrency at 200.

Test: manual testing with limit set to 32
Change-Id: Ic64d21d2c0ffd7c86bf3f02fb51216ee5684a80c
2017-05-16 10:55:29 -07:00
colincross
ea10dedba9 Merge pull request #153 from colincross/build_description
Allow BuildParams to override ninja description
2017-05-09 16:47:57 -07:00
Colin Cross
8de48af6de Escape leading space in ninja strings
Spaces normally don't need to be escaped, but leading spaces are
trimmed.  Escape leading space to allow setting a variable to a
value with leading spaces.

Test: ninja_string_test.go
Change-Id: Ic0ffb076dbd603b7c0203720b9c1ea635c5ded75
2017-05-09 10:14:38 -07:00
Colin Cross
7aa318f83d Allow BuildParams to override ninja description
Allow the ninja description variable to be set on build statements
as well as rules.

Change-Id: I6abb2a8ec83a0c662348cc957fa1a307e6c9c6bb
2017-05-09 10:02:54 -07:00
Dan Willemsen
a30a741c22 Merge pull request #149 from danw/improve_variant_errors
Improve error messages when dependency variants are missing
2017-04-07 14:05:06 -07:00
Dan Willemsen
978c4aa92f Improve error messages when dependency variants are missing
Instead of just saying:

error: .../Android.bp:48:1: dependency "libc++" of "libtest" missing variant "arch:android_arm_armv7-a, link:shared, vndk:"

Include a list of currently existing variants:

error: .../Android.bp:48:1: dependency "libc++" of "libtest" missing variant:
  arch:android_arm_armv7-a, link:shared, vndk:
available variants:
  arch:android_arm_armv7-a, link:shared
  arch:linux_x86, link:shared
  arch:linux_x86_64, link:shared
  arch:windows_x86, link:shared
  arch:windows_x86_64, link:shared

This still isn't the best experience for users, but it at least provides
enough information for someone more familiar with the build to
understand the problem.
2017-03-20 14:23:43 -07:00
colincross
28c7837b3d Merge pull request #148 from colincross/visitdirectdeps
Add Context.VisitDirectDeps
2017-03-17 14:01:03 -07:00
Colin Cross
080c1336b5 Add Context.VisitDirectDeps
Context already has VisitDepsDepthFirst[If], add VisitDirectDeps[If].

Change-Id: Id550bc14275db230c19fd6ca14b67b305dd7d96b
2017-03-17 13:46:47 -07:00
Dan Willemsen
3f2d661087 Merge pull request #147 from danw/goos_tests_fix
Fix bad ninja file if only GOOS-specific tests exist
2017-03-16 09:45:51 -07:00
Dan Willemsen
5d05872cd5 Merge pull request #146 from Nan-Zhang/BaseDependencyTag
Added a check in Blueprint level for BaseDependencyTag
2017-03-16 09:45:39 -07:00
Dan Willemsen
aee3eac6ce Fix bad ninja file if only GOOS-specific tests exist
When only linux/darwin tests were specified, we weren't setting
testArchiveFile, so we had a corrupted ninja file (build : ...).
2017-03-15 23:25:20 -07:00
Nan Zhang
346b2d0d80 Added a check in BP level for BaseDependencyTag
It is not allowed to directly use BaseDependencyTag as customized
user dependency tag passed down to BP since it might cause issues
that different type of modules will be mixed when fetched based on
Tag.
2017-03-10 16:39:27 -08:00
Dan Willemsen
2831e690f0 Merge pull request #142 from danw/_GOOS.go
Support filtering OS-dependent go files using _darwin/_linux.go
2017-02-21 15:07:00 -08:00