test_bp2build_generates_all_buildfiles uses the same directory for
--output_base and --sandbox_tmpfs_path. This is required on the build
servers, but causes issues locally. This commit adds a more helpful
error message when this test fails.
Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: Ib29a7edf456f8ae39bd429ebb367c8115e02c284
* We are interested mostly in the conversion the `default_applicable_licenses`
attribute, as `default_visibility` cannot be handled until every module's
`visibility` is handled.
* Several referenced license modules had to be manually enabled for
conversion, and likewise a few trivial Android.bp containing only
package and license modules.
* As Bazel allows only a single `package` rule, the package rules in a
manually converted BUILD.bazel files were removed (in
external/protobuf and prebuilts/clang/host/linux-x86 trees).
* The converted package rule is emitted before the `load` statements per
Bazel documentation.
Bug: 190817312
Test: treehugger
Change-Id: If8bf6fee1580177de3bb402923615bcd48923ed2
apex_comparison_test.sh defines a helper function to clean up state on
exit, but in error conditions the call_bazel function is not always
defined.
Test: added an exit 1 statement below trap
Change-Id: I73b0c56c0a4a2d9ed1d4c0bf38b6ac0ac985e716
RBE is already disabled for this test on CI. Locally, the mock checkout
created for the Soong integration tests does not set up RBE properly, so
we should just disable it explicitly so that the integration tests can
be run locally.
Test: build/soong/tests/run_integration_tests.sh
Change-Id: I38875a02b549a9d8c8368d6287c4516518d20e34
apex_diff_test targets were the last remaining dependents of make_injection. Rewriting them allows us to delete make_injection.
Test: CI
Change-Id: I928d8fe136d709b9f819c00c3127af66e720a7db
SC2236: use [[ -n "$var" ]] instead of [[ ! -z "$var" ]]
SC2035: use -- end of options, to accomodate filename w\ hyphens
SC2155: `local -r var=..` for readonly variable shellcheck ignores return value masking
SC2045: prefer glob to iteration over `ls`
Bug: N/A
Test: ran ./run_intergration_tests.sh
Change-Id: Iaf313857ecc417e855afd70c7455841b54b1c363
In the isDir check, there's a possibility that the Stat check fails
because the path is an unresolved symlink. Verify it with Lstat (which
would succeed, since it doesn't follow links), and treat it like a file
if Lstat succeeds.
Test: new integration test
Fixes: 232370097
Change-Id: I9807ca363a5dbdc20639b489b54627bd2cc1ca60
This required the following:
- Adding Platform_base_sdk_extension_version to default soong.variables
- Teaching the symlink tree creation code to understand symlinks
- Making finder.go follow symlinks when requested
Adding yet another knob is unfortunate, but I can't allow that
unconditionally because the Android code base contains a number of
symlinks giving rise to infinite directory trees because they point back
to their parent and this seemed preferable to adding complicated logic
like "follow symlink but if only its fully resolved version does not
point under the source tree".
I could be convinced about the latter, though.
Test: Presubmits.
Change-Id: I453f6b7e5334771f5832c700db00f9d24ed1d82f
Currently, mk2rbc/rbcrun aren't built in certain phases
of the build, which causes the Starlark product configuration
to fail when using get_build_var if a regular build hasn't
been run before.
Bug: 226677850
Test: Manually
Change-Id: If85608eef63be25fddc4d916c82247c13eb0195f
1. Added unit tests for license conversions in the androidmk tool, by
using a mocked filesystem;
2. Added integration tests in a shell script with a mocked filesystem.
Bug: 205615944
Test: m androidmk
Test: bash androidmk_test.sh
Test: TreeHugger
Change-Id: Id3d1f914190b52b82aadda7ea019192787b839d6
We already have a rbc product config ci postsubmit target,
but we want to add a small presubmit check as well.
run_integration_tests.sh is already run on presubmit, so
add it there.
Test: Presubmits
Change-Id: Id6fc282261507fb49a57b578ce2fdf20855dd6fd
It's better not to have state hidden in dotfiles (or rather,
dotdirectories) if we can help.
It's questionable whether the "linux-x86" path segment makes sense since
soong_build only ever runs on one operating system, but I didn't want to
rock the boat now.
Drive-by fixed some quoting fixes in rbc-run. Notably, I didn't wrap
`$@` into double quotes because I don't know whether the lack of double
quotes was intended or not.
Also drive-by fixed the fact that "out/soong" was added twice to the
directory name of bpglob. This turned out not to be a problem because
bpglob doesn't need to be explicitly built: if it's needed, it's
declared as an input of the glob files so it'll be built automatically
(at the cost of the first "null build" not actually being a null build)
Test: Presubmits.
Change-Id: I710d8d16cd8212059a0ca1ee95378505303eed83
--skip-make is deprecated because of its odd behavior around the
out/.soong.kati_enabled file. Replace it with --skip-config --soong-only
instead, which will unambigiously skip running kati.
Bug: 204136549
Test: build/soong/tests/run_integration_tests.sh
Test: build/soong/scripts/build-mainline-modules.sh
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I629ee7542687aaafd68eea50346b0cf53eafe1ef
Revert submission 1854375-update-bazel-daecf427ec0bf0e963c324783062c4f5b61ff679
Reason for revert: introduces 503d56eda2, which conflicts with our current cc_binary->cc_shared_library work
Reverted Changes:
I40a511195:Fix test to use --experimental_platform_in_output_...
I05c2c11f3:Update Bazel to https://github.com/bazelbuild/baze...
Ie772d1665:Set --experimental_platform_in_output_dir.
Id66674d72:Update Bazel to https://github.com/bazelbuild/baze...
Change-Id: I0e5a7548feb13c2f3b4deae1921542b15d23a283
This makes soong_ui the only place where soong_build is invoked, thus
greatly simplifying the conceptual model of the build.
It comes with the slight limitation that now soong_docs (and queryview
and the JSON module graph) are not Make targets anymore, but I suppose
that's an acceptable loss.
The only place where someone depended on soong_docs from a Makefile is
removed in a separate change.
Test: Presubmits.
Change-Id: I3f9ac327725c15d84de725d05e3cdde1da3dcbe2
It used to be invoked from out/soong/build.ninja, which required two
soong_build invocations one after the other (ne to generate
out/soong/build.ninja, one to generate the queryview workspace). This
was slower and required some shell-quoted-in-ninja-quoted-in-Go .
Test: Presubmits.
Change-Id: Idda79c067606663b66e9f94626fa24f3b5af4114
This includes the JSON graph generator and bp2build.
Before:
GENERATE_BAZEL_FILES=1 m nothing
GENERATE_JSON_MODULE_GRAPH=1 m nothing
Now:
m json-module-graph
m bp2build
They can now also be combined with other targets or each other.
The longer-term goal is to run "m queryview" and "m soong_docs" using
the same infrastructure. There are two alternate approaches:
1. Call soong_build from within the main Ninja invocation. This requires
two sequential soong_build invocations and is thus slower.
2. Do everything requested in the same soong_build invocation. This
would be faster, but one AFAIU can't tell Ninja that multiple possible
actions can build the same output so that doesn't work.
(1) is somewhat more desirable because soong_docs seems to be built
from build/make/core/main.mk ; I assume that that can be worked around
although I haven't checked where the output of "m soong_docs" goes.
Test: Presubmits.
Change-Id: If5ba36490d9f3f60733e6d6be9286eb2b67c3ff5
Before, we piggybacked on the implementation of regular soong_build and
wrote a fake build.ninja file to satisfy Ninja.
Now, instead, the JSON module graph is a a separate action in the Ninja
output file. This has the pleasant side effect that one can flip back
and forth between generating the JSON file and regular Soong without
loss of incrementality.
Side cleanup: write .d files in a slightly cleaner way.
Test: Presubmits.
Change-Id: Ia853383567b9dd31c53f3bdf56cfc8d517b498ec
This requires some explanation: turns out, Ninja uses the $builddir
variable declared in the top-level Ninja file to figure out where the
command log should go. This resulted in a loss of incrementality between
the manual Ninja invocation in test_null_build_after_docs because they
used different Ninja files and therefore different values of $builddir.
The reason why this was not a problem before is that the output files of
the main Ninja file and the bootstrap one were separate, but with that
changing, this will start to matter.
Test: Presubmits.
Change-Id: I28959cc7d659e698c7694a58326c8731eac060eb
The following files are forked:
- build-globs.ninja
- The glob list files
- .d files for the glob list files
- The output file (build.ninja or the bp2build marker file)
This makes bp2build and soong_build not overwrite each other's files
they need for proper incrementality.
Test: Presubmits.
Change-Id: I69d192cbd36ecd9677f46f3fa095dfce6f872227
This fixes null bp2build builds.
When running bp2build, out/soong/.bootstrap/build-globs.ninja wasn't
generated, therefore none of the globs in out/soong/.primary/globs have
actions to generate them. This makes bp2build always re-run on a null
build, because it depends on those non-existent globs in its depfile.
Fix the above by generating a build-globs.ninja file with the globs that
bp2build encounters during conversion, and add tests to verify that null
builds and bp2build -> soong workflows correctly regenerates the
build-globs.ninja file.
Test: soong integration tests for null soong->soong, bp2build->bp2build and bp2build->soong builds.
Fixes: 193650250
Change-Id: Icfbad351dfa85541d1ea451f73e65cac87097926
In particular, Bazel sometimes creates directories whose files cannot be deleted.
Test: m clean used to fail after b build //system/timezone/apex:com.android.tzdata, but now it works
Test: Added integration test which fails without this change
Change-Id: I08c8feed21c31ec187fe40be513f7eb4865c8ac3
Major operations:
* Fix the go_package entries to be consistent, as the new tool
complains when it doesn't have a '/'.
* Regenerate with the new protoc-gen-go tool.
* github.com/golang/protobuf -> google.golang.org/protobuf
* proto.[Un]MarshalText -> prototext.[Un]Marshal
Change-Id: Ie1147bd2457fafb66ba555461b3bf14f0561a25d
The source tree will eventually be made ReadOnly, and recipes that write
directly to the source tree will fail. Use a pattern-match approach on
the results of stdout/stderr to provide hints to the user in such a
scenario.
If multiple patterns are found in raw output, print error hint
corresponding to first pattern match. first pattern match is chosen
since the failing function will be at the top of the stack, and hence
will be logged first
Test: Wrote a unit test to assert errorhint is added to output.
Wrote an integration test that writes to a file in the source tree
1. When source_tree is RO, the recipe fails and an error hint is printed
to stdout
2. When source tree is RW, the recipe succeeds and no error hint is
printed
Bug: 174726238
Change-Id: Id67b48f8094cdf8a571c239ae469d60464a1e89c
In mixed builds, Bazel operates in <root>/out/soong/workspace, but the real
OUT_DIR is just <root>out/, not <root>/out/soong/workspace/out. This CL
ensures that the correct OUT_DIR is passed to mixed builds's bazel by
making OUT_DIR absolute.
Also add STANDALONE_BAZEL to build_test's Bazel smoke test.
Test: TH
Change-Id: If8b4058d43bf821f5196016f9b0ad38edfe18716
This CL fixes a typo in writeFakeNinjaFile to correctly write
a fake out/soong/build.ninja and its depfile. It also modifies bootstrap
phase to *not* run the main soong build phase (which takes more than a
minute) if GENERATE_BAZEL_FILES=1.
This change has the side effect that `GENERATE_BAZEL_FILES=1 m nothing`
no longer generates the real build.ninja, which is fine because one
shouldn't be using GENERATE_BAZEL_FILES=1 for that anyway (or, use
USE_BAZEL_ANALYSIS=1). This change has no effect on mixed builds.
Time on a change to Soong or any Android.bp files:
Before: bp2build_workspace_marker (~20 seconds) + build.ninja (1 min)
After: bp2build_workspace_marker (~20 seconds)
Time on the second of two consecutive `b build //bionic/...`: 2.070s
Test: TH
Test: Soong integration tests
Change-Id: I43720641815994caba97b8d165d7c3fc254cbd06
