Commit graph

59 commits

Author SHA1 Message Date
Romain Jobredeaux
41fd5e402f Set --noautodetect_server_javabase in mixed build bazel.
If a valid JAVA_HOME is set in the environment when running
bazel with --experimental_repository_disable_download, bazel
will unsuccessfully attempt to download rules_java. The above
flag has the practical effect of preventing the download attempt
(some clues as to why in the doc at go/aosp-bazel-jdk).

Note that this also prevents building java targets in mixed mode.
Eventually, we will pull in the java_rules into AOSP so they don't
need to be downloaded.

Test: build/bazel/ci/mixed_libc.sh
Change-Id: Icdb186c18a3334f4caa17243a8620a1cc8c1db0e
2021-08-27 17:34:16 +00:00
Lukacs T. Berki
9f6c24a887 Rename BuildDir and NinjaBuildDir.
These are just out/ and out/soong/ and the old names were quite
confusing.

Test: Presubmits.
Merged-In: I999ca62162062f27e449f83ffb00fa580d4065b8
Merged-In: I2fcd80a9e376a6842a06aa104a9f6f5c5e16f8c7
Change-Id: Ib481d2aac40df8da1b74174dc8ec576c1cb48be0
2021-08-27 10:08:49 +02:00
Lukacs T. Berki
f9008075d9 Define command line args in soong_build .
They used to be defined in Blueprint but since it doesn't have a
separate existence anymore, we can consolidate all command line
arguments here.

Note that Blueprint is called directly from soong_ui, but that happens
by a simple Go function call and not by invoking a separate binary, so
command line arguments are not needed there.

Test: Presubmits.
Change-Id: Ifa2c101ddbe424b76fc5a508d3d41c329e7353ca
2021-08-16 15:29:02 +02:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
a05a255eac Add infrastructure to support PythonBinary
Add a new request type with its own StarLark function
Hook it up via GetPythonBinary
Add to MockBazelContext a LabelToPythonBinary
Add a test for the new request type

Test: request_type_test.go:TestGetPythonBinaryParseResults
Change-Id: I05f6506adfbbdac8b3f40475509ed02ab8e844e5
2021-08-12 18:48:15 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
0d99045b69 Lift BazelHandler interface into android/
Because it's commonly useful and not cc/ specific
Also export GenerateBazelBuildActions and update uses

Test: Existing tests pass
Change-Id: Ibc6858bb1129afba181a7686dda432defe33b00d
2021-08-11 18:19:40 +00:00
Liz Kammer
c0c6609ef6 Set mixed builds use product-config based target
Rather than specifying a specific arch/os, use the android_target
platform that incorporates arch/os based on product config.

Test: build/bazel/ci/mixed_libc.sh
Change-Id: I2b30cfb9778444acbfd7648af2d331303cab416a
2021-07-26 17:41:49 -04:00
Jingwen Chen
442b1a4070 Use IsEnvTrue for USE_BAZEL_ANALYSIS check.
Make this consistent with the check in soong_ui: https://cs.android.com/android/platform/superproject/+/master:build/soong/ui/build/config.go;l=930;drc=0b55bdb7af7ab2a039fefc88086f0ee41c00ac69

Change-Id: I8d57fafa954ce3dd041cc38250819531125d81ac
Test: TH
2021-06-17 07:02:23 +00:00
Chris Parsons
94a0bba5a9 Support cc_library_shared for mixed builds
Authors: cparsons, eakammer, jingwen

This CL also contains .toc file integration between Bazel and Make.

Fixes: b/190524879

Test: build/bazel/ci/mixed_droid.sh

Co-authored-by: Christopher Parsons <cparsons@google.com>
Co-authored-by: Liz Kammer <eakammer@google.com>
Co-authored-by: Jingwen Chen <jingwen@google.com>

Change-Id: If484042a58cb9f0db6d30a460f415f5684b4cbf6
2021-06-15 12:40:28 +00:00
Liz Kammer
c49e682f37 Handle simple symlinks in mixed builds
Bug: 180945121
Test: build/bazel/ci/mixed_libc.sh
Change-Id: I49fba569a41dcb8cd4c2e58560817443697f58f1
2021-06-09 10:40:32 -04:00
Jingwen Chen
8c52358ad7 Fix OUT_DIR lookup in mixed builds.
In mixed builds, Bazel operates in <root>/out/soong/workspace, but the real
OUT_DIR is just <root>out/, not <root>/out/soong/workspace/out. This CL
ensures that the correct OUT_DIR is passed to mixed builds's bazel by
making OUT_DIR absolute.

Also add STANDALONE_BAZEL to build_test's Bazel smoke test.

Test: TH
Change-Id: If8b4058d43bf821f5196016f9b0ad38edfe18716
2021-06-01 12:07:40 +00:00
Liz Kammer
50d601731b Merge "Dump bazel product config in Soong" 2021-05-20 14:09:18 +00:00
Jingwen Chen
6333b0e460 Set the default --platforms value to android_arm.
This is because the default lunch target is aosp_arm.

Change-Id: I3d99f6ea99862dc2eb579be10db123fe578f9f75
Test: TH
2021-05-20 01:38:53 +00:00
Liz Kammer
09f947d67e Dump bazel product config in Soong
Bug: 187862880
Test: build/bazel/ci/bp2build.sh
Test: build/soong/tests/bp2build_bazel_test.sh
Change-Id: I24b09baad973e25bec4476e1ea4a7692b72b7d20
2021-05-19 17:38:56 -04:00
Lukacs T. Berki
3069dd9d71 Make @soong_injection usable in standalone builds.
Test: Presubmits.
Change-Id: I942e627f9f7b7428834258ef3ccefb7f1f5c7606
2021-05-11 16:54:29 +02:00
Lukacs T. Berki
d6cd813abc Call mixed mode Bazel in the symlink forest.
Test: Presubmits + manual execution.
Change-Id: Idef2d821222ccbf9385d0ea3fc92178b3206df0a
2021-04-20 17:43:02 +02:00
Lukacs T. Berki
d518e1a407 Make bp2build be more correct.
It now handles adding .bp files and changing globs.

In order to do this, depfiles are now written separately from RunBlueprint.

This is necessary due to the confluence of a number of seemingly
unrelated factors:

1. The glob filelist dependencies are discovered in globSingleton
2. Singletons need to be registered because otherwise singleton module
   types panic
3. Singletons don't work because they require mutators bp2build does not
   run

Due to (1), we would need to run the glob singleton. However, due to (2)
and (3), we can't run singletons and have to run Blueprint with
StopBeforeGeneratingBuildActions, which is when the build actions
writing glob files would be generated. So what happens is:

1. When bp2build is run, the glob singleton is disabled
2. At the end of bp2build, the list of glob files is artifically added
   to the depfile of the workspace marker file
3. When build.ninja is generated, the Ninja file containing the glob
   list file is written by the now-active glob singleton

Test: Presubmits.
Change-Id: I3c5898d8c57c554a93520276c64a952afc912dbe
2021-04-15 13:06:16 +02:00
Liz Kammer
fe23bf3eb0 Add tests for request_type ParseResult functions.
Test: go test request_type_test
Test: bp2build generate & sync; mixed build libc
Change-Id: Id0b813e9de4d02d8625e42549999659ccb005c6d
2021-04-12 16:23:53 -04:00
Liz Kammer
b71794d8e6 Consolidate cc-specific cqueries.
There is little overlap at this point, but we expect these to converge
more over time, to handle exported includes, libs, etc., this will allow
those changes to be handled in one place and allow more consolidation of
code.

Test: bp2build generate & sync; mixed build libc
Change-Id: I51685dad9f4fc11a634965a3c9e84f4a0e279ecb
2021-04-12 15:59:08 -04:00
Liz Kammer
8d62a4f725 Add tests for bazel_handler
Test: go test
Test: m nothing
Test: bp2build generate && sync; mixed build libc
Change-Id: Id4307a9e735b50221a8effb963325aaa57c3c7d0
2021-04-12 15:59:06 -04:00
Treehugger Robot
c1c2352d95 Merge "Make MockBazelContext more specific to cquerys" 2021-04-12 13:25:38 +00:00
Jingwen Chen
a42c707572 Merge "bp2build: cc_library converter for //bionic/libdl:libdl_android" 2021-04-12 04:30:27 +00:00
Liz Kammer
a92e844d16 Make MockBazelContext more specific to cquerys
Rather than having a single storage container, make multiple so that it
is easier to unit test building modules with bazel with MockBazelContext

Test: go test genrule_test
Change-Id: I1da85d28f096d5102ad889b9518fdda6914342b1
2021-04-09 11:49:48 -04:00
Jingwen Chen
6393098ecf bp2build: cc_library converter for //bionic/libdl:libdl_android
This CL contains the converter for libdl_android, a cc_library that
expands into a cc_shared_library and a regular cc_library.

Test: TH
Test: bp2build; bazel test //build/bazel/tests/...

Change-Id: If70641a538211b0d6b2aac0e4d0d06912318304d
2021-04-09 09:42:53 +00:00
Liz Kammer
3f9e155f83 Handle simple prebuilt static libraries from bazel
Test: generate & sync BUILD files via bp2build && mixed build droid
Bug: 184192619
Change-Id: I27f0d76c88cbff25f3c7a805f3dfbb1eeaf8e771
2021-04-07 11:07:36 -04:00
Liz Kammer
f29df7cc52 Move cquery RequestType interface to bazel_handler
go idiom is to define the interface where it is used rather than where
it is defined. This makes it obvious that ParseResult is not a used part
of the interface, removing the need to return an interface{} and cast
results.

Test: go test soong tests
Test: generate & sync bp2build; mixed build libc
Change-Id: I0d8d99c1d8d0125588522cc86502286b83c91bf7
2021-04-07 10:56:22 -04:00
Rupert Shuttleworth
b809bcf331 Merge "Fix syntax error in bazel_handler.go." 2021-04-06 20:19:11 +00:00
Rupert Shuttleworth
a29903fb22 Fix syntax error in bazel_handler.go.
Test: go fmt no longer complains.
Change-Id: I725fb1d13f0f88a94e44b1fdbe9080590af2347a
2021-04-06 16:17:33 +00:00
Liz Kammer
66ffdb74ee Only include used requests in cquery starlark
Test: go test soong tests
Test: generate & sync bp2build, mixed build libc
Change-Id: I1fbf854cf31f40b7721788956d0f08da3bda2cba
2021-04-05 10:41:05 -04:00
Liz Kammer
0202caa0fd Remove unused constants for cquery requests
Test: go test soong tests
Change-Id: I63bfff2b1fb2efecf0811b842ffcad874c83fc50
2021-04-05 08:47:43 -04:00
Jingwen Chen
1a6bbbd990 Merge "Add os/target configurable selects for label list attributes." 2021-04-05 07:09:48 +00:00
Liz Kammer
5ad66aaf41 Merge "Add depfile handling for bazel_handler." 2021-04-02 17:06:07 +00:00
Jingwen Chen
91220d7334 Add os/target configurable selects for label list attributes.
This CL is pretty large, so I recommend starting with reading the newly
added tests for the expected behavior.

This change works in conjunction with the linked CLs in the Gerrit topic.
Those CLs add support for new platform() definitions for OS targets
specified in Soong's arch.go, which are configurable through
Android.bp's `target {}` property. It works similary to previous CLs
adding support for the `arch {}` property.

These configurable props are keyed by the OS: android, linux_bionic,
windows, and so on. They map to `select` statements in label list
attributes, which this CL enables for cc_library_headers' header_libs
and export_header_lib_headers props.

This enables //bionic/libc:libc_headers to be generated correctly, from:

    cc_library_headers {
        name: "libc_headers",
        target: {
            android: {
                header_libs: ["libc_headers_arch"],
                export_header_lib_headers: ["libc_headers_arch"],
            },
            linux_bionic: {
                header_libs: ["libc_headers_arch"],
                export_header_lib_headers: ["libc_headers_arch"],
            },
        },
        // omitted props
    }

to:

    cc_library_headers(
        name = "libc_headers",
        deps = [] + select({
            "//build/bazel/platforms/os:android": [
                ":libc_headers_arch",
            ],
            "//build/bazel/platforms/os:linux_bionic": [
                ":libc_headers_arch",
            ],
            "//conditions:default": [],
        }),
    )

Test: TH
Test: Verify generated //bionic/libc:libc_headers
Fixes: 183597786

Change-Id: I01016cc2cc9a71449f02300d747f01decebf3f6e
2021-04-02 08:17:34 +00:00
Liz Kammer
7dfaa3a732 Merge "Don't use custom mixed build logic for cc_object" 2021-03-31 18:43:55 +00:00
Liz Kammer
de116856fb Add depfile handling for bazel_handler.
Test: go test soong tests
Test: TODO mixed build change header, mixed build
Change-Id: I7c51faf2d5b1a8717cbab6bb0b3eb75c307fcd85
2021-03-30 15:55:02 -04:00
Chris Parsons
ad0b5ba42c Refactor cquery processing to generalize transitions
This is a step in the right direction to allow generalized specification
of variant information in the form of configuration transitions, such
that not all transitions must be specified manually. This allows for
much more reasonable addition of new variations in mixed builds.

Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I95898da06af6d0e3e1f7c073aaf8ec0ebe26c951
2021-03-29 21:14:57 -04:00
Liz Kammer
8206d4ff98 Don't use custom mixed build logic for cc_object
Starlark implementation of cc_object now performs partial linking, which
better matches Soong's implementation of cc_object.

Bug: 181794963
Test: generate build files with bp2build, mixed build droid
Change-Id: I5a325aa1c608981deaee3671ac28c6014e6cbf08
2021-03-25 12:00:30 -04:00
Lukacs T. Berki
d1e3f1ff08 Embed minibp into soong_ui.
This requires linking Blueprint into soong_ui. It lets us avoid the
complicated dance of Ninja files and shell scripts: now the information
as to how soong_build is built is passed directly to Blueprint using a
struct that contains all the information the command line arguments used
to contain.

The ability to run Blueprint from the command line is kept (for now).

Some variables in bootstrap/command.go needed public accessor functions
because soong_build reads them. This will be disentangled by moving the
flag parsing to soong_build.

The presence of the flag definitions in Blueprint means that soong_ui
now also accepts them. This is not a problem in practice because they
are ignored and because soong_ui itself is hidden behind a few layers of
shell scripts.

Test: Presubmits + the new bootstrap_test.sh .
Change-Id: I6dca478f356f56a8aee1e457d71439272351390b
2021-03-17 08:35:52 +01:00
Chris Parsons
944e7d01aa Refactor and cleanup of cquery processing
Test: USE_BAZEL_ANALYSIS=1 m libc
Change-Id: Iaf9a92e84d39c132e2444a8aaafd79505a12b8ec
2021-03-16 14:39:16 -04:00
Chris Parsons
808d84c45d mixed builds for cc_static_library without deps
Test: Manual mixed builds testing of `libc` target with manually
migrated "libc_nopthread" and "libc_init_dynamic".

Change-Id: If7d67e95eca9899271b1eeb662c7c2e571f64afa
2021-03-10 13:24:59 -05:00
Liz Kammer
15b04e2cbd Correct fmt printf error in bazel_handler
When running tests locally, it fails to build due to using the incorrect
formatting.

Test: go test soong test
Change-Id: Iccdc3fbc93e409d74f1b71d929dd9650492b401d
2021-03-04 09:45:21 -05:00
Chris Parsons
8d6e433c1e Support cc_object modules in mixed builds
Test: With a handwritten conversion of crtbegin_so1, USE_BAZEL_ANALYSIS=1 m crtbegin_so1
Change-Id: I7c777d7f46b37aa1827cc04205e2014f9293bf35
2021-03-01 18:22:34 -05:00
Chris Parsons
0703d514e3 Remove lingering debug statement from previous submission
While this may be helpful in the immediate-term, it will cause
incredibly bloated logs as more of the build is managed by Bazel.

Test: m nothing
Change-Id: I84f9ea5457c8180bc323a3835f70f5ec23e5c00e
2021-02-09 19:05:59 -05:00
Chris Parsons
ee423b04b4 Set platform and toolchain flags to canonicalized for cquery/aquery
Test: USE_BAZEL=1 USE_BAZEL_ANALYSIS=1 m libc
Change-Id: If9194d5e00d4d53a3c012ea2d602f0b8e73213c7
2021-02-08 23:06:05 -05:00
Chris Parsons
07c1e4aba1 avoid error if bazel intermediates dir already exists
Previously, the build failed if the directory already existed,
so mixed builds had to be null builds.

Test: lunch aosp_flame && USE_BAZEL=1 USE_BAZEL_ANAYSIS=1 m libc &&
  USE_BAZEL=1 USE_ANALYSIS=1 m libc
Change-Id: I167cb82b3e47e9388a1ebca7daffe45f91474125
2021-01-19 17:19:16 -05:00
Chris Parsons
4f069891ae Improve aquery-related validation and error handling
Also introduce test suite for aquery handling

Test: m nothing
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I2493d42782099ea0b575968fca38bce6f0d59015
2021-01-15 13:53:57 -05:00
Treehugger Robot
f33786d3ec Merge "Correct mismatched string format in bazel_handler" 2021-01-05 23:57:53 +00:00
Jaewoong Jung
18aefc1977 Remove unnecessary snake case variables.
Test: m nothing + TreeHugger
Change-Id: I99f7162944daa6c57c6ae4763261e108bb5cb6b1
2020-12-22 12:38:35 -08:00
Liz Kammer
13548d7fa8 Correct mismatched string format in bazel_handler
Test: go test soong tests
Change-Id: Ic48d2e75e8d246f490f46f4619e739b0fa565902
2020-12-16 11:13:30 -08:00
Chris Parsons
dbcb1ff469 Use aquery to declare bazel actions in the ninja file.
This effectively moves execution of Bazel actions outside of soong_build
and alongside ninja execution of the actual ninja files, whether that be
by ninja or by Bazel itself.

This almost allows for mixed builds and Bazel-as-Ninja-executor to
coexist, but requires hacks explained in b/175307058.

Test: Treehugger
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Test: lunch aosp_flame && USE_BAZEL=1 USE_BAZEL_ANALYSIS=1 m libc,
though this requires a hack of the main BUILD file. See b/175307058

Change-Id: Ia2f6b0f1057e8cea3809de66d8287f13d84b510c
2020-12-15 17:39:23 -05:00
Patrice Arruda
05ab2d0731 Enable bazel profiling in soong_build.
Bazel is executed several times during the execution of soong_build.
For each bazel execution, generate a profile and save under the
BAZEL_METRICS_DIR which is defined in soong_ui.

Bug: b/174479924
Test: * USE_BAZEL_ANALYSIS=1 USE_BAZEL=1 m nothing and checked
        if the cquery and graph build bazel profiles were generated.
      * Verified that the generated bazel profiles were uploaded
        to the local dev metrics pipeline.

Change-Id: I3d20204484dc6c5a1525a5d3eec1d62cfb33535b
2020-12-14 18:48:11 +00:00