Refactor parsing Blueprints and walking the Blueprints tree to
allow reuse for tasks that don't involve creating moduleInfo
structures.
Change-Id: I677857a3462999426c8306432074ea97fdcb86c8
Add String() to print out the Comment, Text() to print out the
Comment while stripping /*, //, and */, and EndLine() to return
the line number of the last line of the comment.
Change-Id: I076bc0990143acfc03c42a8cc569894fced8ee24
The primary builder will now generate a rule to call itself with
--docs=.bootstrap/docs/<name>.html to produce an automatically
generated documentation file.
The documentation generation process is:
- Call each factory once to get empty property structs associated
with the module type
- Use reflection to determine the names of the type of each property
struct
- Use the bootstrap_go_package modules from reading the Blueprints files
to find the source files for each Go package used to build the primary
builder
- Use the go/parser module to find the type declaration for each
property struct
- Extract comments for the property struct and each property declaration
- Format all the comments into HTML
Change-Id: Icae9307cc10549a30bfc14d6922824099de5a9b0
Allow tagging a nested property structure with
`blueprint:"filter(key:\"value\")"`, which will only allow property
assignments to properites in the nested structure that are tagged
with `key:"value"`. Moving the filter into Blueprint instead of
the project build logic allows more reliable and consistent error
messages.
Change-Id: I06bc673dde647776fc5552673bdc0cdcd7216462
Users that want to enable this option can use the '-t' option to
bootstrap.bash when passing '-r'. Builders that want to enable this can
set the RUN_TESTS environment variable in their bootstrap.bash.
The gotestmain tools is needed to write the main functions for the test
binaries, since 'go test' doesn't work well in this environment.
Change-Id: Iec5c2b5c9c3f5e3ba0ac8677fb88f5e963f9bd3f
If the initial non-wild part of a glob path does not exist, return
the last existing part of the path as a dependency to detect if the
path is created later.
Change-Id: Ib5a39e6830cb386deed26e017279d0aac1bc9a20
Modules that want properties that vary by variant often need to create
zeroed copies of property structs. Add CloneEmptyProperties to proptools
that is the equivalent of calling CloneProperties and then ZeroProperties,
but is much faster because it directly creates zeroed objects.
Saves 200ms in Context.ParseBlueprintsFiles and Context.ResolveDependencies
in one case, which will be valuble when we start parsing Blueprints files
for cases where we are not regenerating the manifest, for example when
generating documentation or doing context-aware bpfmt.
Change-Id: I3d4a6af2f393886d95f27d15afc1a455d8dd5fc6
AddVariationDependencies is used to add dependency modules that
have the same variations as the depending module, but with additional
variataions. It cannot be used to add a dependency that is unrelated
to the depending module, for example a dependency on a code generator
that needs to run on the host to generate a target source file.
Add AddFarVariationDependencies, which adds a dependency on a module
that contains all the passed variations, but ignores the variations
of the depending module, as well as any unspecified variations on
the dependency module.
Change-Id: Ief696ec85cf33ad5fb187227d215c1c2e894f962
Add GlobWithExcludes, which takes a pattern and a list of exclude
patterns, and returns all files that match the pattern but do not
match any exclude patterns.
Change-Id: I8b94b3ba5a37409071b475b9a4035f52f47863f1
Recursive globs are supported by passing ** in any single non-final
path element. For example, path/**/*.java will find all files named
*.java under "path".
Change-Id: Ifebd76f8959289f7d0d378504053c1c6b88cdeed
Setting the "subname" variable will modify the name of Blueprints
files searched in sub directories. If the specified name is not
found it will fall back to trying "Blueprints". The subname
variable stays in scope for all subdirectories.
Change-Id: If3c8ac54735042076fb264d33719d9ea4b8934b3
The directory structure:
a/
a
b/
b
With the glob pattern */a would previously return []string{"a"} for
dirs, but it needs to return []string{"a", "b"} in order to re-run
the generator if a file called "a" is created inside b/.
Rewrite Glob to manually recurse through path elements, only calling
filepath.Glob for a pattern with wilds in the last element of the
path, and add the globbed directory to the dirs list each time.
Also add tests and test data for pathtools.Glob.
Change-Id: Ibbdb2f99809ea0826d4fa82066cf84103005ef57
build is a magic variable that acts similarly to subdirs, but specifies
files to build instead of subdirs. Using files provides a few advantages:
files can be named something more appropriate for the project than
"Blueprints", whatever you want instead of Blueprints, and multiple
Blueprints files can exist in the same directory.
A new variable is used instead of putting filenames into subdirs to avoid
unexpected behavior when a glob matches both files and directories.
subdirs= and build= entries that don't match any directories or files
are now reported as errors.
Change-Id: Id329504ace251eab4ccea1081a3c8665a4c52f5a
Apparently Go only allows a single thread to be executing by default,
so the increase in performance seen when concurrency was added to
Blueprint was entirely due to scheduling goroutines while the
executing goroutine was blocked in a syscall. Whoops.
Set GOMAXPROCS to the number of CPUs. Reduces wall thime by >2x on
one workload.
Change-Id: Ib5fc7761c09a076fc5a1b7f08f5ccf034304477c
Naively pre-allocate ninjaString slices by counting $ characters as
an estimate of how many variables will be needed. Saves 5% cpu time
on one workload.
Change-Id: Ib3a41df559d728b2db047f6dbbf9eb06d7045303
regexp is overkill for a simple extension replacement, just find
the last '.' and add the new extension after it. Saves 5% cpu time
on one workload.
Change-Id: Ic299c9ec5132d15bbea2c9c7778c000d6fdf509a
Prebuild a string of all spaces to avoid calling strings.Repeat for
every indented line. Replace appendWithWrap function with a class
that handles counting bytes on a line as they are written instead of
buffering lines, and avoids extra string concatenations by providing
a function that implicitly adds a space. Saves 10% wall time on one
workload.
Change-Id: I571d70366ee5db21c40c1c98d0fe92d640efefab
This fixes a bug where a split module could be reached by subsequent
mutator passes, but it is no longer in module group's list of modules.
Change-Id: I74c3d849ed2614dcdb6f54a6760122b933bb84c2
This commit makes deptools.WriteDepFile() escape spaces and the
following set of special characters in the dependencies it
writes to files used as Ninja depfiles: \, #, *, [, |. Without this
change, dependencies with paths including any of the above may not
function as intended.
Change-Id: I3560a34de930932422200985fb479515a21d9508
makeUniquePackageNames never assigned to pkgs, so all packages
were using their short names, even if another package had the
same short name.
Change-Id: I5420f7839efd25362d6c5d85265399ce7b29ddb2
If a $ sign occurs after a variable name, the ninja string parser
fails to check if it is a $$ or a ${. Go to the
parseDollarStartState instead of the parseDollarState.
Since it is not yet known if the $ is the beginning of a new
variable (${ or $<alphanumeric>) or a string ($$), an empty string
separator cannot be added to the ninjaString strings list. Instead,
add functions to push strings or variables onto the ninjaString,
and automatically add the blank separator if two variables are
pushed in a row.
Change-Id: Ia1cae6259b1d7e4f633f61b9eadb2a2028bbd5f0
The patch "Fix panic when dependency cycle includes the first
module visited" caused cycles to print incorrectly by initializing
the current module to be the last module in the cycle, when it
should be the first module in the cycle.
Change-Id: Iaf939283a48faa4cc6eeb9b19aed57993575a687
