Commit graph

2862 commits

Author SHA1 Message Date
Colin Cross
e4cfdf93a9 Optimize parseNinjaString
The closures in parseNinjaString seem to confuse go's escape
analysis, causing it to allocate a new stateFunc object on the
heap for every character in the string.  Convert the state
functions to normal functions, and pass a pointer to a state
object to provide access to the variables that were in the
closure.

The range iteration on the input string was iterating over
UTF-8 runes, when all of the rune comparisons for state
changes only look at ASCII bytes.  Switch to iterating
over bytes, stringState will pass through multi-byte
characters in strings untouched.

Together this reduces the time spent in parseNinjaString by
>50%, and the overall execution time by >20%.

Change-Id: I4c926b52b883d51e2f12a1673d03fcc2cfe83445
2015-01-23 13:41:49 -08:00
Christian Zander
6e2b232d79 Make Ninja string evaluation more robust
ninjaString's Eval() does not currently check if all of the variables
referenced by the Ninja string being evaluated are present in the
map provided by the caller.  This commit adds such a check.

Change-Id: I15cf3c44cc1eedaf6f93a9533b8cdcbffd74992c
2015-01-23 13:41:49 -08:00
Jamie Gennis
0bb5d8a8d9 Fix predicated visitors.
This change makes the Context.visitDepsDepthFirstIf() method descend into
modules for which the predicate is false rather than skipping their entire
sub-DAG.

Change-Id: I50564c69a714d5e199e1a51a8aa24162b0dc6f11
2015-01-23 13:41:49 -08:00
Colin Cross
b2e7b5d036 Add a PreGenerateBuildActions pass
Call PreGenerateBuildActions on each module that implments it after
resolving dependencies but before calling GenerateBuildActions on
any module.  This allows the build logic to propagate top-down
information, for example what variants of a library need to be
compiled for the binaries it is used in.

Change-Id: I2737504fa9d1a2b42ef747497de32c1c5129233d
2015-01-23 13:41:49 -08:00
Jamie Gennis
c4ed709a71 bootstrap: Fix abandoned file removal.
This change makes removeAbandonedFiles perform some of the string substitutions
that are done by the bootstrap script before considering whether a built target
still exists.  It fixes a bug where the bootstrap manifest would get deleted
and subsequently regenerated, resulting in more re-bootstrap iterations than
was necessary.

Change-Id: I946c883dcc31fb77496f64d2573b395ad2cf095d
2015-01-23 13:41:49 -08:00
Jamie Gennis
df935ac324 Make Context.AllTargets() evaluate variables.
Change-Id: Iaa7e134cb5cb9fd1491a2508a7953fb4cbc53d1e
2015-01-23 13:41:48 -08:00
Jamie Gennis
9e2a4c295b bootstrap: Don't export the script-substituted Ninja vars
Change-Id: I8999f9489415cc53661e079401df519f9d83258e
2015-01-23 13:41:48 -08:00
Colin Cross
42cad4b7dc Add pathtools.ReplaceExtension
Change-Id: I67d08700fce37b7949e6103132212766cdbe54ec
2015-01-23 13:41:48 -08:00
Colin Cross
46f8d184fb Pipe the Optional flag from BuildParams to the build definition
An Optional flag on buildDef is not very useful on its own,
BuildParams is what is exposed to modules.

Change-Id: I49b341654162adbc4eab884e88c21e68bbce8c75
2015-01-23 13:41:48 -08:00
Colin Cross
aa847f2893 Add support for Optional parameter to build definitions
Build definitions can now specify Optional.  If Optional is not
specified the build definition will be followed by a default definition
for all outputs.

Change-Id: Iada83b2ef32bfb599b3cc580263f12bc002e1cdb
2015-01-23 13:41:48 -08:00
David Allison
701fbad02b Add the ability to check if a property is present
This adds (to the ModuleContext) a facility to check if a property is
defined in a module in a Blueprints file.

Change-Id: I16a35fd236717d8797f15093c019f7cc1049fb3c
2015-01-23 13:41:48 -08:00
Nico Hailey
b6a5039e88 pathtools: Fixed absolute path bug in Glob.
Change-Id: I30ba550449787648f6dca052b89f4eb7722bcb55
2015-01-23 13:41:48 -08:00
Jamie Gennis
af43556ce3 Add support for removing abandoned files.
This change makes the bootstrapping process remove any files that were
previously created by invoking a Ninja rule (i.e. they appear in the .ninja_log
file) but are no longer a build output target.

Change-Id: I3c78e563393b97f8ca196ac85c7caa2b3866ffa6
2015-01-23 13:41:48 -08:00
Jamie Gennis
debef53c56 Add a Glob func that returns dirs searched
Change-Id: I35ab9a5f1c7425f42d0e667bac38450afa34c738
2015-01-23 13:41:48 -08:00
Colin Cross
daf6499a4a update golang build rules for go 1.3.3
Change-Id: I873d5f205bd0596ee3e0f992cfedb458be710e01
2015-01-23 13:41:47 -08:00
Colin Cross
5e3594f623 Add dependencies on golang tools
Add a dependency on the golang tool for each tool output file.  This
will allow updating the go version without cleaning the output
directory.

Change-Id: I55048e96c8847635256bd9491e2cd12ddf29771b
2015-01-23 13:41:47 -08:00
Colin Cross
535434a59d Move cgofilter to be built by bootstrap.ninja
Building cgofilter from main.ninja creates a circular dependency.
Move it to be built by the bootstrap stage instead, where go
files are built without cgofilter.

Change-Id: I853eb0b20a364adbfb93d44b039e1493214a6887
2015-01-23 13:41:47 -08:00
Colin Cross
add65dd4c1 Sort variables by name when writing ninja files
Regenerating build.ninja.in was non-deterministic, sort the maps
of name, value pairs before writing them.

Change-Id: I825734162b1adebc88ddb19d912751108322091e
2015-01-23 13:41:47 -08:00
Jamie Gennis
fbb27fe2d9 Add a utility function for writing depfiles.
Change-Id: Ic365e3e1632d518a5331646ed5deef1b5a36f2f6
2015-01-23 13:41:47 -08:00
Jamie Gennis
61e27fe331 Wrap a line in the docs
Change-Id: Ifea051b4ba89bcaefbaa0d8c8a8685595c377d40
2015-01-23 13:41:47 -08:00
Jamie Gennis
9914aeb703 Allow the multipass workaround to be disabled
Change-Id: I78f38ef4534a86cec9ce169e1bca91c758c66402
2015-01-23 13:41:47 -08:00
Jamie Gennis
30d8a3a2ed Fix a string parsing bug
Change-Id: I70faa1c06f19f70455a6d8f7db1da37203e21797
2015-01-23 13:41:47 -08:00
Jamie Gennis
68540dab06 Fix the context parse test
Change-Id: I411916b4d4d62cd53d870299f2a415371ed644a3
2015-01-23 13:41:47 -08:00
Jamie Gennis
8762292240 Add support for unpacking properties into nested structs.
Change-Id: Idb67dcb9cc7f857258b9b3409db68199d42a8001
2015-01-23 13:41:47 -08:00
Jamie Gennis
b9cbdae701 Add the blueprint/proptools package.
This change adds some utility functions for manipulating properties structs
using reflection.

Change-Id: I62017ef06bceb68d536421e50b853449a73daed3
2015-01-23 13:41:47 -08:00
Jamie Gennis
1174c695eb Revert "Add support for targets in Blueprints."
This reverts commit 63cdbb3c540349e877ac37a82b2b46565ee45fe7.

Change-Id: I5fd4db75ee04dfa706b26757cd0a247bd8748840
2015-01-23 13:41:47 -08:00
Jamie Gennis
6736ec30da Add the offending character to the bad-ninja-name error message.
Change-Id: I93260a064aece41c53afcecad73c037eb87e5aea
2015-01-23 13:41:47 -08:00
Jamie Gennis
2fb2095caa Stop determining package names from the call stack.
This change replaces the automatic caller package divination with a
PackageContext object that must be explicitly passed in by callers.

Change-Id: I139be29ecf75a7cf8488b3958dee5e44363acc22
2015-01-23 13:41:46 -08:00
Jamie Gennis
0c35b2db92 Ignore subdirs starting with a '.'
Change-Id: I67bb7c4fe02942c79973287cd3bf57627b6cdd1c
2015-01-23 13:41:46 -08:00
Jamie Gennis
b9e87f6af2 Add support for dynamic module dependencies.
This change adds the DynamicDependerModule interface.  Any Module type that
implements that interface will be called before Blueprint does its depenedency
analysis to allow the module to add additional dependencies.

Change-Id: I4d9ef8663d187262dcd9a1ec410c180c3ca4f57d
2015-01-23 13:41:46 -08:00
Jamie Gennis
c15544dbc7 Sort various things for deterministic output.
Change-Id: If6457713ad442ae3e4179e594820fa1e02e9b240
2015-01-23 13:41:46 -08:00
Jamie Gennis
b282d5c8d6 Fix a bug that allowed duplicate module names.
Change-Id: I9317e43ac435a26397557328036c11a8a2da7480
2015-01-23 13:41:46 -08:00
Jamie Gennis
7d5b2f82ce Simplify module and singleton registration.
This change eliminates blueprint.ModuleType and replaces it with simple factory
functions.  Rather than using the explicitly provided ModuleType names to
essentially identify the Go factory function in the generated Ninja files, we
now use the actual factory function name.

Change-Id: Ib7813e850322a82cc35cdc56bebff7d580a5c6ec
2015-01-23 13:41:46 -08:00
Romain Guy
285296519a Add support for targets in Blueprints.
The default target selector uses the name of the host OS.
Modules can implement their own target selector by implementing
the context.TargetSelector interface and its unique selectTarget()
method.

Targets are defined this way in Blueprint files:

cc_library {
    name:  "libmylib",
    deps:  ["libmath", "libutils"],
    zones: ["frontend"],
    srcs:  ["main.cpp"],
    targets: {
        darwin: {
            moduleCflags: "-framework OpenGL -framework GLUT",
        },
        linux:  {
            deps: ["libx11headers", "libglu"],
        },
    },
}

In this example, a set of C flags are defined on OS X only and on
Linux, two new dependencies are added.

When a target is selected, its properties are merged with the properties
of the modules:
- a list is appended to the original list (see deps above)
- a string is concatenated to the original string
- a bool replaces the original bool
- a map adds or replaces the key/value pairs of the original map

Change-Id: Ic627d47f795d6a4ff56ca5f6f099cad157621af1
2014-08-12 17:50:11 -07:00
Romain Guy
2476f81d84 Add support for map properties
Example:

foo {
  // string property
  name: "hello",
  // map property
  translations: {
    ge: "guten tag",
    fr: "bonjour",
  },
}

Change-Id: Ib9e6570cb6f355677b4ba0235e8fb4186e40f5f8
2014-07-25 17:01:20 -07:00
Romain Guy
5f7ef51f7a Add support for raw strings
Change-Id: Ib68f1a08c8707b8c955d25bf0a7fd6f491c0b470
2014-07-24 15:29:42 -07:00
Jamie Gennis
ae4430cd9e Stop processing modules when one fails.
This change causes Blueprint to stop calling GenerateBuildActions on modules
when one such call results in errors being generated.  If a module has failed
for some reason and another module depends on it, then processing the dependee
could end up trying to use the failed module, which may be in an invalid state.

Change-Id: I5cc165b7f4e169f90e9570ec7b2a5f9baac2895a
2014-07-23 14:37:21 -07:00
Jamie Gennis
6a40c19e1c Add some ModuleContext error functionality.
Change-Id: I0635de0440da27e1d7cf7e155f268e6f958047eb
2014-07-02 16:40:31 -07:00
Jamie Gennis
0ed63eff29 Fix scoping logic to respect the caller's package.
This change makes the ModuleContext and SingletonContext methods respect the
caller's Go package when performing name lookups.  It does this by re-parenting
the module or singleton's scope to the scope of the caller's package, thus
allowing a Module or Singleton to pass its context to a function defined in
another Go package and have the called function access variables and rules
defined in its own Go package.

Change-Id: Ifdec87ba3095a453b36fb00e38c0bb3a928a2b9b
2014-06-30 18:26:28 -07:00
Christopher Horvath
3159cb7673 Added extra ninja file dependencies (optional) to Main
This facilitates dependencies on extra files, such as global build configuration
files

Change-Id: I928688ad8b9867066d37dec2c00710755fae921e
2014-06-27 08:18:14 -07:00
Mathias Agopian
5b8477d423 add support for module generated ninja-file dependencies
Change-Id: I54b6d240adc89591fbe41ed1f9d63db9a791999a
2014-06-25 17:21:54 -07:00
Jamie Gennis
d4c53d8a50 A few minor changes.
- Make module 'new' functions return slices of properties structs rather than
  just one.
- Fix a couple places where errors were being reported incorrectly.
- Add ModuleContext methods for dealing with other modules.
- Make property value unpacking skip unexported struct fields rather than
  panicing.

Change-Id: I4775ef77ff0514fc2ce5abccbe2ce534c05272f4
2014-06-22 17:02:55 -07:00
Erik Gilling
310794f016 add ReplaceExtensions to pathtools
Change-Id: I6dc92323d4b0ef4b0f31076c868802dc568346c4
2014-06-19 21:22:52 +00:00
Jamie Gennis
e98b8a927d Add support for singletons to walk module deps.
Change-Id: I63405ee917a55ae92cf55070af94e01ddc266389
2014-06-18 12:57:56 -07:00
Jamie Gennis
48aed8cee0 Add more Build/Rule/Pool params validation.
This change adds two new kinds of checks to params validation.  First, all
BuildParams must have one or more outputs.  Second, any Pool or Rule referenced
must be visible within the Blueprint scope of the caller (e.g. if it's defined
in another Go package then an Import call must have been made).  If either of
these conditions are violated it will result in a panic.

Change-Id: Ibacb42513882d914c94eade23ef17269db5e8730
2014-06-18 12:57:56 -07:00
Jamie Gennis
71bd58a966 Improve Ninja string parse errors.
Change-Id: I8450befd19694e51bc3cb0658e8075b0d49ee614
2014-06-18 12:57:56 -07:00
Jamie Gennis
7330a23887 Clean up the bootstrap script a little.
This change makes the bootstrap script take two optional flag-based command
line arguments rather than using positional arguments.  The -i argument
replaces the old positional arguments (one of which was simply not needed), and
the -r flag now causes the script to regenerate the bootstrap Ninja file for
use when working on Blueprint itself.

Additionally this change allows all the configuration variables used in the
script to be overridden by environment variables.  This makes it possible to
create a very simple bootstrap script for a customized Blueprint-based build
system by invoking this script after setting up the proper environment
variables.

Change-Id: I60f890061ae237a68719e1e2d345d1a8e87c851c
2014-06-18 12:57:56 -07:00
Jamie Gennis
d4e1018e19 Document most of the blueprint package APIs.
This change adds docs to all the blueprint package APIs except for the Module
and Singleton types and their corresponding context types.

Change-Id: I74aa48c7743086ad79b3122d5813f5c4823c6519
2014-06-12 20:10:01 -07:00
Jamie Gennis
ec7012824a Fix a variable name
Change-Id: I0b7e14291df954de1023ab1396e6a0b8adbf5033
2014-06-12 20:10:01 -07:00
Jamie Gennis
04f106f23a Don't export the EOF constant.
Change-Id: I6f8578d0a6c62bdb1e384d5e512937d9489387b7
2014-06-12 19:41:57 -07:00