Add a VariableFuncContext argument to VariableFuncs that implements
GlobWithDeps. This will allow Soong to use optimized glob dependencies
in VariableFuncs.
Bug: 257079828
Test: no dependencies on directories in build.ninja.d
Change-Id: Iee5fc9c9ae3087662a5d1a3d7323a87462299205
This is to build bazel targets that are nearing readiness for prod mode.
Bug: 254265047
Test: m nothing
Test: m nothing --bazel-mode-dev
Test: m nothing --bazel-mode-staging
Change-Id: Ie8336ec4ad70c0c9bd6d74f3b0a278c85fd5df2d
also minor refactoring to limit var scopes
Test: m nothing and ensure ninja files are generated
Bug: N/A
Change-Id: If7bdede4e54687955896b03ab11ef771855d2225
This allows for a bazel-invocation hook in mixed builds, which allows
for mixed builds to take place in only a single pass, greatly improving
its performance and complexity.
Test: Conjunction with build/soong CL
Change-Id: If89fb56830b4eb06d3263d6ca6da7b285e7ba315
In conjunction with soong/build changes, this materialized runtime
metrics for various soong_build events.
Test: Manually verified materialized protos for bp2build, mixed builds,
and legacy build.
Change-Id: Ia92403605e3063028dbf6a1ded8449c190b9e63e
This is used to make GODEBUG=asyncpreemptoff=1 only be set in debug mode
so that production is not affected.
Test: Presubmits.
Change-Id: Idb15ad20bb74ba3f896699715e3208c1287a5e82
This makes debugging with Delve much faster, sometimes by an order of
magnitude because Delve does not have to deal with the many SIGURG
signals that are used to stop preempted goroutines this way.
This could in theory result in a performance loss, but running
`rm -f out/soong/build.ninja; time m nothing` five times demonstrated no
change in the execution time.
Test: Presubmits + performance testing as described above.
Change-Id: I438564bcaf70d2a50147cf54cdfac6dcfe925ce5
It's in reality built by Microfactory quite early in the build which
takes care of incrementality in bpglob itself.
Test: Presubmits.
Change-Id: I066a6e85f48e9cfec4efd46cb1f615c68b806080
Presently, entries in the Soong reference docs that
share the same type are condensed into a single entry.
This is very unintuitive, so this change removes that
functionality.
Fixes: 204441523
Test: bpdoc_test.go
Test: build and compare Soong docs
Change-Id: Ic03891a8a7a29b5f7ee58c01b2fa05a0c27e0a2b
ToolDir is going to become unstable when switching between KatiEnabled
and Soong-only builds while the duplication between out/soong/host and
out/host is resolved. bpglob gets executed very early during bootstrap,
before the primary builder has run to update the paths to match the
current configuration. Move it into SoongOutDir() so that its path
is more stable.
The copy of bpglob in ToolDir is still used when bpglob is used by the
primary builder through bootstrap.GlobFile.
Bug: 204136549
Test: m nothing
Change-Id: Ida51997b6408d7c265f3ba343278e5e2968467d3
struct types defined using reflection have empty PkgPaths(), due to
which they were not added to the generated bpdocs. This CL allows
us to generate bpdocs for such struct types.
For each field in the struct, the algorithm checks the following and
creates a PropertyStruct object accordingly
1. If field is a primitive type (Base condition)
2. If field is another exported struct (Base condition)
3. If field is another struct created using reflection (Recurse)
Test: m soong_docs
Test: java_sdk_library_import before
https://ci.android.com/builds/submitted/7710820/linux/latest/view/java.html#java_sdk_library_import
and after
https://spandandas.users.x20web.corp.google.com/docs/java.html#java_sdk_library_import
Bug: 172797653
Change-Id: I0349e405fd290d427fa0b38a109b4212aace50c6
PrimaryBuilderExtra flags was only used from soong_ui and has nothing to
do with Blueprint files or modules and Args contained a lot of members
that were likewise only used for plumbing withing soong_ui or for
command line processing for soong_build.
Test: Presubmits.
Change-Id: Ibf009e446b4044309a331592a88a4aa86b4801e3
This is because docgen will be called from the bootstrap Ninja file with
cwd == /.
Side cleanup: remove an unused template.
Test: Presubmits.
Change-Id: Ia5ff10247becbabc7a76c21f4ed05d654a7b20c3
Its value is a function of the call site, so it doesn't make a lot of
sense to plumb it through the configuration.
Test: Presubmits.
Change-Id: I5081650818664f283ebc5dc04daef90c26759353
It was confusing because there are two concepts with the same name:
thebootstrapping config special-cased in the bootstrap modules /
singletons and the "global" config.
This change folds the first into the second.
A further next step would be to split the Go parts and the Ninja parts
(subninjas + primary builder invocation) apart, but at some point, I
should stop gold-plating this.
Test: Presubmits.
Change-Id: Id341dee664a9d2527535f4f9980e5f0d10139eef
- Remove the bootstrap_go_binary module type in favor of
blueprint_go_binary
- Make it so that all the binaries used during bootstrapping are in the
host tool directory so that the bootstrap and main Ninja files are
consistent
Test: Presubmits.
Change-Id: I00744fec0474fbea89db6c0c0338856453138564
What's gone:
- TopFile (it's always Android.bp)
- GeneratingPrimaryBuilder (we don't make that distinction anymore)
Also rename BuildDir and NinjaBuildDir to SoongOutDir and OutDir,
respectively, to be consistent with the rest of the code.
Test: Presubmits.
Change-Id: I55ecb02a59fe68bf5674395a108181ea7557b21f
It is not used anymore; Soong assumes that the depfile is just the name
of the output file with a ".d" suffix.
Test: Presubmits.
Change-Id: Ifbf722dcdffc8420e12a032e23ab791a6be5a736
These are just out/ and out/soong/ and the old names were quite
confusing.
Test: Presubmits.
Merged-In: Idd9ce3c38a259faabcc56f0cd3fdac8b289123b1
Merged-In: I334ab40c668d2a94536f3e63d5f1fa0b401388ac
Change-Id: Ib7c568c6a97701f2240c5e3f0f2ce67397819ac0
This makes it possible to share work between the bootstrap and the main
Ninja file.
As a side fix, remove the reference to $srcDir from moduleSrcDir() since
that is hardwired to "." anyway.
Test: Presubmits.
Change-Id: Ia4a251dd34273ec3e0436ac7a2fbafe4ef46c73f
installPath was included by androidmk.go in the Soong-generated
Android.mk file, which means that the Ninja variable reference
`$buildDir` passe through Kati, with unpredictable results.
`$buildDir` is still used in a few places, but those are not problematic
because they are not copied to Android.mk (they could probably be
removed, but it's not trivial enough to be done as a drive-by fix)
Test: Presubmits.
Change-Id: I6a5ea8b1679001ff685d263310cb061d3a6a992e
From now on, the responsibility of writing build-globs.ninja and adding
the glob list files to build.ninja.d will be the responsibility of
whoever calls Blueprint.
It's nicer this way because the bootstrap Blueprint invocation does not
need globs and Blueprint can be invoked multiple times from the
soong_build binary depending on what it does.
Regrettably, the little Blueprint invocation that generates
build-globs.ninja still requires a reference to a BootstrapConfig
because GlobRule references ${buildDir}.
Side cleanup: remove Args.DocFile, which was unused.
Test: Presubmits.
Change-Id: I37586966e3d6b7557a490ed1ae81ec596ec66ee5
There used to be two:
- The absolute source directory is now threaded through the code instead
- The command line arguments are moved to soong_build
Test: Presubmits.
Change-Id: Ie09de1a2862ec4ced2c7385198300359f805127d
Now that Blueprint has been merged with Soong, a large amount of code is
unnecessary.
Test: Presubmits.
Change-Id: I8d30d8b80f37fdc6a9db630a1792c57c7d6edb38
This is not necessary anymore now that Blueprint doesn't have a separate
existence and Soong's documentation generation uses a completely
separate code path.
Test: Presubmits.
Change-Id: Ibc9e57379c5f8ee9a06d90fbe81fac314f01ee92
This makes the distinction between what the bootstrap .ninja file and
the main .ninja files do clearer: the bootstrap .ninja file includes the
build-globs.ninja file(s) which the main .ninja file writes.
This required removing the "default" statement from said bootstrap
.ninja file: now we can build more than one thing instead of just
build.ninja and thus soong_ui explicitly tells Ninja what to do instead.
On the side, stop catering for the use case where the bootstrap
Blueprint invocation uses globs: Soong's does not and that's our only
use case.
Test: Presubmits.
Change-Id: Icffce31928242cfe9cdab56b290a37598d32a58c
This enables bp2build to write its own build-globs.ninja file.
Test: soong integration tests
Fixes: 193650250
Change-Id: Icb35f3cf3c30929dab1b2c2e9b244321be30f967
An empty glob file would be created inside the source tree if
soong_build is invoked directly (instead of via soong_ui). This logic is
now moved to soong/ui/build to prevent this behavior
Bug: 187194795
Test: Ran the following command locally for the target
art-target-arm:git_master-art
```
. ./build/envsetup.sh && lunch armv8-eng &&
art/tools/buildbot-build.sh --target
```
Before change glob file is created in $TOP/.bootstrap
After change glob file is created in $TOP/out/soong/.bootstrap
Change-Id: I6745996c940d790d1f1f387ca08aafc6a8a669df
We have `blueprint_tools` that can build all Go binaries (and run their
associates tests). But if a package is not (yet) used, it won't be
included there. This makes testing the build of new Go packages
difficult.
Test: m blueprint_go_packages
Change-Id: I7a67cf2593fedcf499d40d82c266d3b0d7ebbef2
There are a few cases that force all globs to be rerun at the beginning
of the build (changes to bpglob or dependencies, second build after a
clean build). The number of globs has gotten high enough that rerunning
them all can have significant overhead to start bpglob for each one.
Replace the per-glob bpglob invocations with sharded invocations using
1024 hash buckets.
Bug: 159845846
Test: glob_test.go
Test: m nothing && m nothing
Test: build/soong/bootstrap_test.sh
Change-Id: Ife1f7a03c8f6b25d1be01531425d8dc2c76d1ea0
Args.BuildDir was accidentally hard-wired to "." in one of my previous
changes, and since Blueprint is not really independent of Soong anymore,
it's probably better to fix it in a way that doesn't involve putting
back the almost-but-not-completely redundant "-b" command line argument
to the primary builder invocation.
Test: Presubmits.
Change-Id: I76d064a99be16cd812cb47a6c448fe44b51893dc
