This allows for a bazel-invocation hook in mixed builds, which allows
for mixed builds to take place in only a single pass, greatly improving
its performance and complexity.
Test: Conjunction with build/soong CL
Change-Id: If89fb56830b4eb06d3263d6ca6da7b285e7ba315
In conjunction with soong/build changes, this materialized runtime
metrics for various soong_build events.
Test: Manually verified materialized protos for bp2build, mixed builds,
and legacy build.
Change-Id: Ia92403605e3063028dbf6a1ded8449c190b9e63e
This is used to make GODEBUG=asyncpreemptoff=1 only be set in debug mode
so that production is not affected.
Test: Presubmits.
Change-Id: Idb15ad20bb74ba3f896699715e3208c1287a5e82
This makes debugging with Delve much faster, sometimes by an order of
magnitude because Delve does not have to deal with the many SIGURG
signals that are used to stop preempted goroutines this way.
This could in theory result in a performance loss, but running
`rm -f out/soong/build.ninja; time m nothing` five times demonstrated no
change in the execution time.
Test: Presubmits + performance testing as described above.
Change-Id: I438564bcaf70d2a50147cf54cdfac6dcfe925ce5
It's in reality built by Microfactory quite early in the build which
takes care of incrementality in bpglob itself.
Test: Presubmits.
Change-Id: I066a6e85f48e9cfec4efd46cb1f615c68b806080
Presently, entries in the Soong reference docs that
share the same type are condensed into a single entry.
This is very unintuitive, so this change removes that
functionality.
Fixes: 204441523
Test: bpdoc_test.go
Test: build and compare Soong docs
Change-Id: Ic03891a8a7a29b5f7ee58c01b2fa05a0c27e0a2b
ToolDir is going to become unstable when switching between KatiEnabled
and Soong-only builds while the duplication between out/soong/host and
out/host is resolved. bpglob gets executed very early during bootstrap,
before the primary builder has run to update the paths to match the
current configuration. Move it into SoongOutDir() so that its path
is more stable.
The copy of bpglob in ToolDir is still used when bpglob is used by the
primary builder through bootstrap.GlobFile.
Bug: 204136549
Test: m nothing
Change-Id: Ida51997b6408d7c265f3ba343278e5e2968467d3
struct types defined using reflection have empty PkgPaths(), due to
which they were not added to the generated bpdocs. This CL allows
us to generate bpdocs for such struct types.
For each field in the struct, the algorithm checks the following and
creates a PropertyStruct object accordingly
1. If field is a primitive type (Base condition)
2. If field is another exported struct (Base condition)
3. If field is another struct created using reflection (Recurse)
Test: m soong_docs
Test: java_sdk_library_import before
https://ci.android.com/builds/submitted/7710820/linux/latest/view/java.html#java_sdk_library_import
and after
https://spandandas.users.x20web.corp.google.com/docs/java.html#java_sdk_library_import
Bug: 172797653
Change-Id: I0349e405fd290d427fa0b38a109b4212aace50c6
PrimaryBuilderExtra flags was only used from soong_ui and has nothing to
do with Blueprint files or modules and Args contained a lot of members
that were likewise only used for plumbing withing soong_ui or for
command line processing for soong_build.
Test: Presubmits.
Change-Id: Ibf009e446b4044309a331592a88a4aa86b4801e3
This is because docgen will be called from the bootstrap Ninja file with
cwd == /.
Side cleanup: remove an unused template.
Test: Presubmits.
Change-Id: Ia5ff10247becbabc7a76c21f4ed05d654a7b20c3
Its value is a function of the call site, so it doesn't make a lot of
sense to plumb it through the configuration.
Test: Presubmits.
Change-Id: I5081650818664f283ebc5dc04daef90c26759353
It was confusing because there are two concepts with the same name:
thebootstrapping config special-cased in the bootstrap modules /
singletons and the "global" config.
This change folds the first into the second.
A further next step would be to split the Go parts and the Ninja parts
(subninjas + primary builder invocation) apart, but at some point, I
should stop gold-plating this.
Test: Presubmits.
Change-Id: Id341dee664a9d2527535f4f9980e5f0d10139eef
- Remove the bootstrap_go_binary module type in favor of
blueprint_go_binary
- Make it so that all the binaries used during bootstrapping are in the
host tool directory so that the bootstrap and main Ninja files are
consistent
Test: Presubmits.
Change-Id: I00744fec0474fbea89db6c0c0338856453138564
What's gone:
- TopFile (it's always Android.bp)
- GeneratingPrimaryBuilder (we don't make that distinction anymore)
Also rename BuildDir and NinjaBuildDir to SoongOutDir and OutDir,
respectively, to be consistent with the rest of the code.
Test: Presubmits.
Change-Id: I55ecb02a59fe68bf5674395a108181ea7557b21f
It is not used anymore; Soong assumes that the depfile is just the name
of the output file with a ".d" suffix.
Test: Presubmits.
Change-Id: Ifbf722dcdffc8420e12a032e23ab791a6be5a736
These are just out/ and out/soong/ and the old names were quite
confusing.
Test: Presubmits.
Merged-In: Idd9ce3c38a259faabcc56f0cd3fdac8b289123b1
Merged-In: I334ab40c668d2a94536f3e63d5f1fa0b401388ac
Change-Id: Ib7c568c6a97701f2240c5e3f0f2ce67397819ac0
This makes it possible to share work between the bootstrap and the main
Ninja file.
As a side fix, remove the reference to $srcDir from moduleSrcDir() since
that is hardwired to "." anyway.
Test: Presubmits.
Change-Id: Ia4a251dd34273ec3e0436ac7a2fbafe4ef46c73f
installPath was included by androidmk.go in the Soong-generated
Android.mk file, which means that the Ninja variable reference
`$buildDir` passe through Kati, with unpredictable results.
`$buildDir` is still used in a few places, but those are not problematic
because they are not copied to Android.mk (they could probably be
removed, but it's not trivial enough to be done as a drive-by fix)
Test: Presubmits.
Change-Id: I6a5ea8b1679001ff685d263310cb061d3a6a992e
From now on, the responsibility of writing build-globs.ninja and adding
the glob list files to build.ninja.d will be the responsibility of
whoever calls Blueprint.
It's nicer this way because the bootstrap Blueprint invocation does not
need globs and Blueprint can be invoked multiple times from the
soong_build binary depending on what it does.
Regrettably, the little Blueprint invocation that generates
build-globs.ninja still requires a reference to a BootstrapConfig
because GlobRule references ${buildDir}.
Side cleanup: remove Args.DocFile, which was unused.
Test: Presubmits.
Change-Id: I37586966e3d6b7557a490ed1ae81ec596ec66ee5
There used to be two:
- The absolute source directory is now threaded through the code instead
- The command line arguments are moved to soong_build
Test: Presubmits.
Change-Id: Ie09de1a2862ec4ced2c7385198300359f805127d
Now that Blueprint has been merged with Soong, a large amount of code is
unnecessary.
Test: Presubmits.
Change-Id: I8d30d8b80f37fdc6a9db630a1792c57c7d6edb38
This is not necessary anymore now that Blueprint doesn't have a separate
existence and Soong's documentation generation uses a completely
separate code path.
Test: Presubmits.
Change-Id: Ibc9e57379c5f8ee9a06d90fbe81fac314f01ee92
This makes the distinction between what the bootstrap .ninja file and
the main .ninja files do clearer: the bootstrap .ninja file includes the
build-globs.ninja file(s) which the main .ninja file writes.
This required removing the "default" statement from said bootstrap
.ninja file: now we can build more than one thing instead of just
build.ninja and thus soong_ui explicitly tells Ninja what to do instead.
On the side, stop catering for the use case where the bootstrap
Blueprint invocation uses globs: Soong's does not and that's our only
use case.
Test: Presubmits.
Change-Id: Icffce31928242cfe9cdab56b290a37598d32a58c
This enables bp2build to write its own build-globs.ninja file.
Test: soong integration tests
Fixes: 193650250
Change-Id: Icb35f3cf3c30929dab1b2c2e9b244321be30f967
An empty glob file would be created inside the source tree if
soong_build is invoked directly (instead of via soong_ui). This logic is
now moved to soong/ui/build to prevent this behavior
Bug: 187194795
Test: Ran the following command locally for the target
art-target-arm:git_master-art
```
. ./build/envsetup.sh && lunch armv8-eng &&
art/tools/buildbot-build.sh --target
```
Before change glob file is created in $TOP/.bootstrap
After change glob file is created in $TOP/out/soong/.bootstrap
Change-Id: I6745996c940d790d1f1f387ca08aafc6a8a669df
We have `blueprint_tools` that can build all Go binaries (and run their
associates tests). But if a package is not (yet) used, it won't be
included there. This makes testing the build of new Go packages
difficult.
Test: m blueprint_go_packages
Change-Id: I7a67cf2593fedcf499d40d82c266d3b0d7ebbef2
There are a few cases that force all globs to be rerun at the beginning
of the build (changes to bpglob or dependencies, second build after a
clean build). The number of globs has gotten high enough that rerunning
them all can have significant overhead to start bpglob for each one.
Replace the per-glob bpglob invocations with sharded invocations using
1024 hash buckets.
Bug: 159845846
Test: glob_test.go
Test: m nothing && m nothing
Test: build/soong/bootstrap_test.sh
Change-Id: Ife1f7a03c8f6b25d1be01531425d8dc2c76d1ea0
Args.BuildDir was accidentally hard-wired to "." in one of my previous
changes, and since Blueprint is not really independent of Soong anymore,
it's probably better to fix it in a way that doesn't involve putting
back the almost-but-not-completely redundant "-b" command line argument
to the primary builder invocation.
Test: Presubmits.
Change-Id: I76d064a99be16cd812cb47a6c448fe44b51893dc
The only thing that was done with it was to append it to the list of
dependencies returned, which can just as well be done by the caller.
Test: Presubmits.
Change-Id: Ibedf024de411a5881f19ed0c137a91b0898f1269
bpglob is executed through the rules in build-globs.ninja to determine
whether soong_build needs to rerun. That means when the arguments
accepted by bpglob change it will be called with the old arguments,
then soong_build will rerun and update build-globs.ninja with the new
arguments.
To avoid having to maintain backwards compatibility with old arguments
across the transition, a version argument is used to detect the
transition in order to stop parsing arguments, touch the output file
and exit immediately. The version number in
pathtools.BPGlobArgumentVersion should be manually incremented when
the bpglob argument format changes.
If the version argument is not passed then a version mismatch is assumed.
Bug: 159845846
Test: build/soong/tests/bootstrap_test.sh
Change-Id: Id6742c7acc51af8d0d224d51e47bddea78e6e363
- Make the parsed command line arguments accessible from other packages
- Separate writing the depfile from writing the Ninja file
- Extract a method to compute the name of the file list file for globs
Test: Presubmits.
Change-Id: I4d0daf0f3776a9336da305be0108ebf70a314776
This is accomplished by plumbing PrimaryBuilderInvocation in from
bootstrap.Args.
The confusing removal of "primaryBuilderExtraFlags" from documentation
generation is because it used to (erroneously) refer to the func defined
in command.go and its removal apparently doesn't break anything. I
suspect that this code path is dead within Android, but we'll need to
wait until May to remove it.
Test: Presubmits.
Change-Id: I8763579b82bd79d5ee0f233edfd48327282e401c
This makes it possible to invoke the primary builder more than once in a
single Ninja file.
This required adding arguments that were hard-wired in the build.ninja
rule to extraArgs and adding a "set of primary builder invocations"
member to bootstrap.Config . The only command line arguments that remain
hard-wired are those which are expected to be relevant to every
invocation of the primary builder.
As a welcome side effect, I was able to remove a number of random
environment variables that were so far special-cased ($SOONG_DELVE,
$SOONG_DELVE_PATH and $SOONG_OUTDIR). I was also able to move writing
the empty stub ninja glob file to command.go from the bowels of the
Blueprint machinery.
In theory, $TOP and $BUILDER could be removed, too, but this would
require hard-coding the value of $TOP into build.ninja and I don't know
what would break if I did that (it's okay to hard-wired $SOONG_DELVE and
$SOONG_DELVE_PATH because those are only used for debugging and
$SOONG_OUTDIR turned out to be superfluous)
Test: Presubmits.
Change-Id: Idbfd9976c4b270bc3e5a8926c8c760a8534596cf
