Commit graph

225 commits

Author SHA1 Message Date
Rupert Shuttleworth
4949557d50 Merge "Generate BUILD files for every directory that has an Android.bp file." 2021-04-21 14:51:28 +00:00
Lukács T. Berki
6b3fb3a12a Merge "Reenable test_glob_noop_incremental." 2021-04-21 14:28:52 +00:00
Rupert Shuttleworth
2a4fc3ecdc Generate BUILD files for every directory that has an Android.bp file.
Test: Added an integration test
Test: bazel build --package_path=out/soong/workspace //bionic/...

Change-Id: Ie34bd23ab3c5428e6c9c9919e5fb6fcb4e709adc
2021-04-21 07:15:50 -04:00
Lukacs T. Berki
b21166e236 Make symlink_forest.go prefer generated files.
Now, if the same file exists in the generated tree and the source tree,
it symlinks in the generated file instead of failing outright.

Drive-by fix: print errors for all conflicts instead of bailing out on
the first one.

Test: Presubmits (including the two new tests)
Change-Id: Ifb5b3fc89b5454d231966bfa4e61c22cd69834f3
2021-04-21 12:24:27 +02:00
Lukacs T. Berki
69ed2a2b9e Reenable test_glob_noop_incremental.
It accidentally got disabled in aosp/1676621 due to a merging mishap.

Test: Presubmits.
Change-Id: I10a090480008c72c2a8390fe30c64f666a177ea3
2021-04-21 12:05:08 +02:00
Christopher Parsons
4e78814200 Merge "Subsume INTEGRATED_BP2BUILD into other env modes" 2021-04-20 23:33:40 +00:00
Chris Parsons
ec1a3dc6f8 Subsume INTEGRATED_BP2BUILD into other env modes
This refactors bazel-build mode determination logic in soong_ui so it's
clearer which of three possible modes are being used in a given
invocation (NO_BAZEL, GENERATE_BUILD_FILES, or MIXED_BUILDS).

Test: bootstrap tests
Change-Id: I41d2baebf8d560c2cc42db8daa8b936101d453e3
2021-04-20 19:25:48 -04:00
Colin Cross
14d09018eb Merge "Update for sharded globs" 2021-04-20 16:16:40 +00:00
Lukács T. Berki
3784448c9e Merge "Create a synthetic Bazel workspace." 2021-04-20 10:44:08 +00:00
Lukacs T. Berki
b353cca496 Create a synthetic Bazel workspace.
It's under out/soong/workspace and is a symlink forest that merges BUILD
files generated by bp2build and the source tree.

bazel-* symlinks and $OUTDIR are currently ignored.

Test: Presubmits.
Change-Id: If14df23bd0f6a6f2d7d1140f95213f2aee66d652
2021-04-20 10:00:02 +02:00
Lukacs T. Berki
731bb91b8e Propagate errors from integration tests scripts.
Test: Presubmits.
Change-Id: Ie1f83c562a357e472bcaa95a4c1567a3e974d496
2021-04-19 09:28:21 +02:00
Colin Cross
7d9d2405f3 Merge "Fix test_glob_noop_incremental" 2021-04-17 02:22:06 +00:00
Colin Cross
b72877f749 Fix test_glob_noop_incremental
I6a98660cfe6a40bcaa978e8d7544cdffad86a7fa was submitted concurrently
with test_glob_noop_incremental and broke the test by making the
first build incremental.  Add an rm -rf out to make the test start
from a clean build.

Bug: 185591972
Test: tests/bootstrap_test.sh
Change-Id: I529aaf707b2790e5e1022ba9b2637ed7d2435bd5
2021-04-16 10:09:54 -07:00
Lukács T. Berki
86b38024c5 Merge "Colorize the name of the test to be run." 2021-04-16 05:55:05 +00:00
Colin Cross
cb33a00992 Update for sharded globs
Update the path bootstrap_test.sh uses to check for bpglob reruns
to follow the changes made for sharding globs.

Use bootstrap.GlobFileListFiles to get dependencies during bpbuild.

Test: tests/bootstrap_test.sh
Bug: 159845846
Change-Id: Ibaa7c4360881ed6b666a811bf34b19ea0cdcafe9
2021-04-15 11:03:00 -07:00
Colin Cross
9e7cea2c0b Merge "Add test for globs on noop incremental builds" 2021-04-15 16:42:08 +00:00
Lukacs T. Berki
7a51907346 Colorize the name of the test to be run.
Test: Presubmits.
Change-Id: Ie1335f9fb91c199874e37d74d0853e8f441cc294
2021-04-15 18:18:45 +02:00
Lukacs T. Berki
d518e1a407 Make bp2build be more correct.
It now handles adding .bp files and changing globs.

In order to do this, depfiles are now written separately from RunBlueprint.

This is necessary due to the confluence of a number of seemingly
unrelated factors:

1. The glob filelist dependencies are discovered in globSingleton
2. Singletons need to be registered because otherwise singleton module
   types panic
3. Singletons don't work because they require mutators bp2build does not
   run

Due to (1), we would need to run the glob singleton. However, due to (2)
and (3), we can't run singletons and have to run Blueprint with
StopBeforeGeneratingBuildActions, which is when the build actions
writing glob files would be generated. So what happens is:

1. When bp2build is run, the glob singleton is disabled
2. At the end of bp2build, the list of glob files is artifically added
   to the depfile of the workspace marker file
3. When build.ninja is generated, the Ninja file containing the glob
   list file is written by the now-active glob singleton

Test: Presubmits.
Change-Id: I3c5898d8c57c554a93520276c64a952afc912dbe
2021-04-15 13:06:16 +02:00
Lukács T. Berki
3d76d5d6b8 Merge "Allow running bp2build as part of a regular build." 2021-04-15 11:03:51 +00:00
Lukacs T. Berki
686965baeb Speed up Soong integration tests.
This is done by running a build before the first test case, tarring up
$TOP (including out/) then untarring it for every test case instead of
rebuilding soong_build each time.

The speedup is approximately 2x.

Also add some cute ANSI escape sequences and delineation of individual
test cases.

Test: The aforementioned integration tests.
Change-Id: I6a98660cfe6a40bcaa978e8d7544cdffad86a7fa
2021-04-15 08:47:39 +02:00
Lukacs T. Berki
f8e2428c5d Allow running bp2build as part of a regular build.
This is done by setting the INTEGRATED_BP2BUILD environment variable
when invoking the build.

Even though the name of the marker file insinuates that a Bazel
workspace is already created, this is not the case yet.

An issue that remains is that a .d file is not written for the marker
file so it won't be rebuilt if a .bp file changes. Fixing this requires
delicate surgery because writing the .d file is the result of delicate
interplay between Soong and Blueprint.

There are also a number of semi-related fixes:

- The name of soong.environment.{used,available} is now on the command
  line of soong_build (soong_docs is still special cased because its
  command line in the Ninja file is taken from the os.Args of
  soong_build so it's not trivial to remove the --{available,used}_env
  from it
- bp2build writes a separate soong.environment.used file
- I had to call SetAllowMissingDependencies() separately when creating
  the android.Context for bp2build so that bp2build runs in the
  integration tests (it was not obvious how not to do this)
- Fixed a number of integration tests where a command with an expected
  exit code of 1 was used as the last one in a test case, thereby
  breaking the test suite

Test: Presubmits.
Change-Id: Ibeb61c26022cf801dcb98505b4039151b3409873
2021-04-15 08:46:07 +02:00
Lukacs T. Berki
b448c34bfd Make integration tests clean up after themselves.
Well, this was embarrassing.

Test: These tests.
Change-Id: Id25f9f1f4bd2d2d01d49d11a5af8eb50dd51fe6e
2021-04-14 16:45:22 +02:00
Colin Cross
1042595508 Add test for globs on noop incremental builds
Test that an incremental build with a glob doesn't rerun soong_build, and
only regenerates the globs on the first but not the second incremental build.

Bug: 159845846
Test: tests/bootstrap_test.sh
Change-Id: Ib9e3f2f22422d567253a366d28255e3f39d97ca8
2021-04-12 22:35:46 -07:00
Colin Cross
c02504edd6 Add test for globs used during bootstrapping
bootstrapping can read extra Android.bp files that could affect
soong_build.

Bug: 159845846
Test: bootstrap_test.sh
Change-Id: Ib053ff9c2ee70638419f122a36438a515640e0b4
2021-04-09 09:46:09 -07:00
Lukacs T. Berki
3b730c4bc8 Organize Soong's integration tests:
- Move them into a new directory
- Split tests by theme and add a library file
- Add a shell script that runs them all

Test: Manually ran build/soong/run_integration_tests.sh .
Change-Id: I758b91d679f41aee47d15472cc02547ce89f6386
2021-04-08 13:21:13 +02:00