Soong writes out text protos for the sbox manifests, and if those change
every time soong changes, we trigger rebuilds of significant portions of
the tree. This is very annoying when attempting to iterate on Soong, so
disable this functionality at the soong_build level.
Bug: 200727888
Change-Id: Icf21e1775a4c4200e983c1b8ac66ddd780d7c90c
Note that this is an initial, rather crude attempt at this,
that unblocks generating BUILD files needed to build AOSP android apps.
In particular, note that
- Jetifier step currently ignored
- Visibility set to public for all output targets
- All dependencies are gathered in 'deps', though they are split under various categories in bp files
- Host, host-only notions are ignored for now
Test: Copy first 2 lines of, e.g. prebuilts/sdk/current/support/Android.bp to a BUILD.bazel file and run pom2bp -pom2build -regen BUILD.bazel
Bug: 198433754
Change-Id: I5efa93517e6c319fd828da5b16ba7efd2d90bca8
If SOONG_UI_ANSI_OUTPUT is set to a true value force the simple status
output to keep ANSI codes. This will allow buildbots to collect logs
with ANSI codes in them. Smart status output is not affected as it
always keeps ANSI codes.
Bug: 147310922
Test: manual
Change-Id: I0b78ceebb65125b8e8dafb4787816fb679d3eb3e
Under load sometimes the 1 millisecond timeouts are not run before
a 1 second sleep finishes, or `echo foo` is not run before a 1
second timeoutt. Increase the long sleeps and timeouts to 10 seconds.
This doesn't affect normal test runtime as the command should timeout
after 1 millisecond.
Bug: 199207577
Test: Test_runWithTimeout
Change-Id: I31166850a473b9cde37906a62d026580e7a7bd3f
Support three options for converting modules with unconverted
dependencies
1. (default) Warn when converting a module if it has unconverted deps.
2. Error when encountering a module with unconverted deps. (not hooked
up yet)
Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_libc.sh
Test: BP2BUILD_ERROR_UNCONVERTED=1 build/bazel/ci/bp2build.sh with
unconverted deps -- get appropriate error
Bug: 181155349
Change-Id: Ifaabf0cd2e43e963366dc137159c705294165c3d
This field was missing for Property objects that have non-empty inner
Property objects
Bug: 172797653
Test: m soong_docs
Change-Id: Iee9c66f8b85d68a6b5bf18fd9787143191c4f002
They were a bit repetitive.
In addition, make the environment checks more correct; they could
probably use an integration test or two to make sure that when the
environment changes, exactly those outputs are rebuilt that need to be,
but for now, this is an improvement already.
Test: Presubmits.
Change-Id: Idd79b81ca6975d57d00e5bf4699d266152505ff8
This makes soong_ui the only place where soong_build is invoked, thus
greatly simplifying the conceptual model of the build.
It comes with the slight limitation that now soong_docs (and queryview
and the JSON module graph) are not Make targets anymore, but I suppose
that's an acceptable loss.
The only place where someone depended on soong_docs from a Makefile is
removed in a separate change.
Test: Presubmits.
Change-Id: I3f9ac327725c15d84de725d05e3cdde1da3dcbe2
Its value is a function of the call site, so it doesn't make a lot of
sense to plumb it through the configuration.
Test: Presubmits.
Change-Id: If928b34de075969fd42932212ce9187808cbdf86
The breakage was because:
- aosp/1818236 removed the "rm -fr" invocation from the Ninja file that
removed the old queryview tree
- queryview files were created with permissions 440 to discourage manual
editing
These two together made incremental queryview invocations not be able to
overwrite its output files.
(Of course, it also made them incorrect since old files were lft in place)
Test: Presubmits.
Change-Id: Idaa3e1db82c9eb8325c48e37aca5146e69af7333
It used to be invoked from out/soong/build.ninja, which required two
soong_build invocations one after the other (ne to generate
out/soong/build.ninja, one to generate the queryview workspace). This
was slower and required some shell-quoted-in-ninja-quoted-in-Go .
Test: Presubmits.
Change-Id: Idda79c067606663b66e9f94626fa24f3b5af4114
They both used to return out/soong. Unfortunately, the tests seem to
assume the old behavior, so I had to keep tests inconsistent and I was
scared of dexpreopt, too.
Test: Presubmits.
Change-Id: Ib00c7956fd2cdb28cc11d481c17e5a3ff5ad563a
It was confusing because bootstrapping uses two configurations: the
"global" config and the special-cased bootstrap one.
This change merges them.
Test: Presubmits.
Change-Id: I82b482cbe28a343ab6991374b2a28667e1a06b48
I erroneously assumed that the continuous builds don't set DIST_DIR,
whereas they do.
The variable naming is quite questionable, but for now, the important
thing is that the code works and I wanted to do that with the least
amount of change possible.
Also add some more logging:
- Shell environment
- Command line arguments
- Location of logs.zip
- The logs directory
Test: Presubmits.
Change-Id: I651b1045eeef568aa860fd2457fa7799ad68ba38
These are: TopFile and GeneratingPrimaryBuilder.
Also re-shuffle the list of flags to make a bit more sense and finish
the rename of BuildDir and NinjaBuildDir to SoongOutDir and OutDir,
respectively.
Test: Presubmits.
Change-Id: I103ff5f09f1c0d16f695a7da5dea13b55028e33e
Before, we piggybacked on the implementation of regular soong_build and
wrote a fake build.ninja file to satisfy Ninja.
Now, instead, the JSON module graph is a a separate action in the Ninja
output file. This has the pleasant side effect that one can flip back
and forth between generating the JSON file and regular Soong without
loss of incrementality.
Side cleanup: write .d files in a slightly cleaner way.
Test: Presubmits.
Change-Id: Ia853383567b9dd31c53f3bdf56cfc8d517b498ec
These are just out/ and out/soong/ and the old names were quite
confusing.
Test: Presubmits.
Merged-In: I999ca62162062f27e449f83ffb00fa580d4065b8
Merged-In: I2fcd80a9e376a6842a06aa104a9f6f5c5e16f8c7
Change-Id: Ib481d2aac40df8da1b74174dc8ec576c1cb48be0
In particular, the writing of build-globs.ninja and the collection of
glob list files for build.ninja.d is now done outside of the Blueprint
invocation.
This is useful because RunBlueprint() is sometimes called multiple times
in a single soong_build invocation and because now glob processing is
not hidden by the machinery of Blueprint.
Test: Presubmits.
Change-Id: I6eb19b357901e452e6d1e7ce055f8a3bab4a74f4
They used to be defined in Blueprint but since it doesn't have a
separate existence anymore, we can consolidate all command line
arguments here.
Note that Blueprint is called directly from soong_ui, but that happens
by a simple Go function call and not by invoking a separate binary, so
command line arguments are not needed there.
Test: Presubmits.
Change-Id: Ifa2c101ddbe424b76fc5a508d3d41c329e7353ca
The following files are forked:
- build-globs.ninja
- The glob list files
- .d files for the glob list files
- The output file (build.ninja or the bp2build marker file)
This makes bp2build and soong_build not overwrite each other's files
they need for proper incrementality.
Test: Presubmits.
Change-Id: I69d192cbd36ecd9677f46f3fa095dfce6f872227
This fixes null bp2build builds.
When running bp2build, out/soong/.bootstrap/build-globs.ninja wasn't
generated, therefore none of the globs in out/soong/.primary/globs have
actions to generate them. This makes bp2build always re-run on a null
build, because it depends on those non-existent globs in its depfile.
Fix the above by generating a build-globs.ninja file with the globs that
bp2build encounters during conversion, and add tests to verify that null
builds and bp2build -> soong workflows correctly regenerates the
build-globs.ninja file.
Test: soong integration tests for null soong->soong, bp2build->bp2build and bp2build->soong builds.
Fixes: 193650250
Change-Id: Icfbad351dfa85541d1ea451f73e65cac87097926
This is done by moving SetupSignals() to its own little package.
There are a number of tiny little utility packages for soong_ui we might
be better of merging, but that's for another change (maybe)
Test: Presubmits.
Change-Id: I07b0ca98bfb8884ef4223d665e632183b9896a0d
This mostly cuts the dependency from multiproduct_kati to the build
package; the remaining logic is SetupSignals which will be dealt with in
a followup change.
This requires some code duplication (e.g. ensureFileExists() and
detectTotalRAM()).
This removes a Finder invocation added in aosp/452598; it looks like
it's not a measurable performance hit. The quoted half a second does not
matter much in build_test{1,2,3,4} which take an amount of time measured
in hours.
It also makes ANDROID_QUIET_BUILD unsupported for the UI of
multiproduct_kati; the function of that variable seems to be to suppress
the output from build actions, which don't exist in multiproduct_kati so
it should not matter. In addition, I checked whether anyone sets this
flag for multiproduct_kati and it doesn't appear to be the case.
Test: Presubmits.
Change-Id: I1c2901e1ea69c3a35429e6c967a7bba845676ef7
This serves to not link parts of soong_ui (and eventually soong_build)
into a separate, weird binary. This is in turn good because they contain
any number of global variables and no one really thought about what
happens when two instances are executing at the same time in the same
address space.
This comes with a slight performance hit: 5 aosp_* projects build 152
seconds instead of 146. I suppose this is a price worth paying for a
clean design?
Test: presubmits.
Change-Id: I5623dcab2290f0fc392dd2ede597b9794a3d2a4e
Print more messages in run_with_timeout to make it easier to
determine which log statements came before the timeout and the
on_timeout command.
Test: run_with_timeout_test.go
Change-Id: I92ad663529fa03d91185ba382654a73f0eba1ab7
Add __dlwrap_linker and __dlwrap_linker_end symbols for use by the
embedded linker trampoline to determine which load segments are
part of the embedded linker and should not be exposed to the
executable.
Use flags to name the sections to avoid colliding with __dlwrap_linker
when no section name can be determined.
Bug: 190084016
Test: build with musl
Change-Id: I51ccbf9c15a65c6194fa263b7504a598dbf4ad18
This is as part of an effort to move environment variables into config
files so that configs can be tied down to source.
Test:
Ran a build with "ANDROID_BUILD_ENVIRONMENT_CONFIG=googler m" which ran
a non-RBE build (since this is vanilla aosp).
Also ran a build with "ANDROID_BUILD_ENVIRONMENT_CONFIG= m" which ran a
non-RBE build.
https://paste.googleplex.com/5637282880028672
Bug: b/194679562
Change-Id: I416e8da75f84aa2b53995f525cf50501488dc972
Major operations:
* Fix the go_package entries to be consistent, as the new tool
complains when it doesn't have a '/'.
* Regenerate with the new protoc-gen-go tool.
* github.com/golang/protobuf -> google.golang.org/protobuf
* proto.[Un]MarshalText -> prototext.[Un]Marshal
Change-Id: Ie1147bd2457fafb66ba555461b3bf14f0561a25d
Use a note added to linker.s to place the linker in the linker script,
which will work musl, which doesn't have .note.android.ident, while
continuing to work for linux bionic.
Test: m checkbuild
Change-Id: I9cd4ad7c8e8fcb4d725746106d5beaa699a77807
I wrote these flags primarily for build-tools:external/kythe, where we
only use a subset of the Go packages defined in the project.
-limit allows defining the entry points to keep in the Android.bp, and
the utput will omit everything except those entry points and their
dependencies. This allows the kythe Android.gen.bp file to drop from 195
modules to 31 using two flags, without having to manually exclude the
other 160 packages.
-skip-tests does as you'd expect, though ideally it wouldn't be
necessary. We didn't import all of the test-only dependencies for kythe
into the source tree, so we're not able to build them.
Change-Id: I49e9f12dac53a74c6e05725ee41fb7c0ed8b0cbb
This CL adds support to bp2build/Soong to dump a BUILD file under
out/soong/soong_injection/targets containing alias targets to their real
targets for every converted Soong module, regardless of whether they are
handcrafted or generated.
Test: TH
Change-Id: Ic1816fda5d019c395301618134fac68b3057d752
Use an implicit linker script instead of flags in a file to specify
the locations of the host bionic embedded linker and to prevent it
from being stripped.
Test: build and run host bionic binary
Change-Id: I64e12118d33c67bab5e657cbc3ea8cde8f0fd7e6
Replace .linker.sect0, etc. with .linker, .linker.text, .linker.data
and .linker.data.rel.ro by extracting the name of the first section
in each program header.
Test: build and run host bionic binary
Change-Id: I25107547536a3a3963fdeb311c45a7ee53c0bc45
The host bionic bootstrapping no longer needs an injected symbol.
Replace host_bionic_inject with host_bionic_verify that validates
the resulting binary, and add it as a validation dependency of the
binary.
Test: build and run host bionic binary
Change-Id: I3e303d2a164b6eef851bdc8075e6ee456c05b0a8
