platform_build_soong/cmd/soong_build/queryview.go

67 lines
1.8 KiB
Go
Raw Normal View History

Create a AOSP Bazel overlay workspace with Soong The Bazel overlay is a directory at out/soong/bazel_overlay that replicates the layout of the AOSP Soong module tree, but as a Bazel workspace. Each Soong module variant is represented as a BUILD target created with the `soong_module` rule. To create this overlay, run `m bazel_overlay`. A `soong_module` target can depend on other `soong_module` targets. These dependencies replicate each module's `directDeps` in the Blueprint graph, just before `PrepareBuildActions`. This enables users to use bazel query as a way to introspect the Soong module graph. For example, - Direct reverse dependencies of //bionic/libc:generated_android_ids in //bionic/libc/...: $ bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' //bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static //bionic/libc:generated_android_ids - Why does com.android.runtime depend on lzma? $ bazel query 'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image, //external/lzma/...)' //bionic/apex:com.android.runtime--android_common_com.android.runtime_image //bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime //system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime //external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime - What does the dep graph of //bionic/libc:crtbegin_so look like? $ bazel query 'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)' --output=graph > graph.in && dot -Tpng < graph.in > graph.png https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8 Test: croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel query //... && bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' Signed-off-by: Jingwen Chen <jingwen@google.com> Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-07-15 12:06:41 +02:00
// Copyright 2020 Google Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"io/ioutil"
"os"
"path/filepath"
"android/soong/android"
"android/soong/bp2build"
Create a AOSP Bazel overlay workspace with Soong The Bazel overlay is a directory at out/soong/bazel_overlay that replicates the layout of the AOSP Soong module tree, but as a Bazel workspace. Each Soong module variant is represented as a BUILD target created with the `soong_module` rule. To create this overlay, run `m bazel_overlay`. A `soong_module` target can depend on other `soong_module` targets. These dependencies replicate each module's `directDeps` in the Blueprint graph, just before `PrepareBuildActions`. This enables users to use bazel query as a way to introspect the Soong module graph. For example, - Direct reverse dependencies of //bionic/libc:generated_android_ids in //bionic/libc/...: $ bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' //bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static //bionic/libc:generated_android_ids - Why does com.android.runtime depend on lzma? $ bazel query 'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image, //external/lzma/...)' //bionic/apex:com.android.runtime--android_common_com.android.runtime_image //bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime //system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime //external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime - What does the dep graph of //bionic/libc:crtbegin_so look like? $ bazel query 'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)' --output=graph > graph.in && dot -Tpng < graph.in > graph.png https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8 Test: croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel query //... && bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' Signed-off-by: Jingwen Chen <jingwen@google.com> Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-07-15 12:06:41 +02:00
)
func createBazelQueryView(ctx *bp2build.CodegenContext, bazelQueryViewDir string) error {
os.RemoveAll(bazelQueryViewDir)
ruleShims := bp2build.CreateRuleShims(android.ModuleTypeFactories())
res, err := bp2build.GenerateBazelTargets(ctx, true)
if err != nil {
panic(err)
}
filesToWrite := bp2build.CreateBazelFiles(ruleShims, res.BuildDirToTargets(), bp2build.QueryView)
for _, f := range filesToWrite {
if err := writeReadOnlyFile(bazelQueryViewDir, f); err != nil {
return err
}
}
return nil
Create a AOSP Bazel overlay workspace with Soong The Bazel overlay is a directory at out/soong/bazel_overlay that replicates the layout of the AOSP Soong module tree, but as a Bazel workspace. Each Soong module variant is represented as a BUILD target created with the `soong_module` rule. To create this overlay, run `m bazel_overlay`. A `soong_module` target can depend on other `soong_module` targets. These dependencies replicate each module's `directDeps` in the Blueprint graph, just before `PrepareBuildActions`. This enables users to use bazel query as a way to introspect the Soong module graph. For example, - Direct reverse dependencies of //bionic/libc:generated_android_ids in //bionic/libc/...: $ bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' //bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static //bionic/libc:generated_android_ids - Why does com.android.runtime depend on lzma? $ bazel query 'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image, //external/lzma/...)' //bionic/apex:com.android.runtime--android_common_com.android.runtime_image //bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime //system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime //external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime - What does the dep graph of //bionic/libc:crtbegin_so look like? $ bazel query 'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)' --output=graph > graph.in && dot -Tpng < graph.in > graph.png https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8 Test: croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel query //... && bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' Signed-off-by: Jingwen Chen <jingwen@google.com> Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-07-15 12:06:41 +02:00
}
// The auto-conversion directory should be read-only, sufficient for bazel query. The files
// are not intended to be edited by end users.
func writeReadOnlyFile(dir string, f bp2build.BazelFile) error {
dir = filepath.Join(dir, f.Dir)
if err := createDirectoryIfNonexistent(dir); err != nil {
return err
Create a AOSP Bazel overlay workspace with Soong The Bazel overlay is a directory at out/soong/bazel_overlay that replicates the layout of the AOSP Soong module tree, but as a Bazel workspace. Each Soong module variant is represented as a BUILD target created with the `soong_module` rule. To create this overlay, run `m bazel_overlay`. A `soong_module` target can depend on other `soong_module` targets. These dependencies replicate each module's `directDeps` in the Blueprint graph, just before `PrepareBuildActions`. This enables users to use bazel query as a way to introspect the Soong module graph. For example, - Direct reverse dependencies of //bionic/libc:generated_android_ids in //bionic/libc/...: $ bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' //bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static //bionic/libc:generated_android_ids - Why does com.android.runtime depend on lzma? $ bazel query 'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image, //external/lzma/...)' //bionic/apex:com.android.runtime--android_common_com.android.runtime_image //bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime //system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime //external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime - What does the dep graph of //bionic/libc:crtbegin_so look like? $ bazel query 'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)' --output=graph > graph.in && dot -Tpng < graph.in > graph.png https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8 Test: croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel query //... && bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' Signed-off-by: Jingwen Chen <jingwen@google.com> Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-07-15 12:06:41 +02:00
}
pathToFile := filepath.Join(dir, f.Basename)
Create a AOSP Bazel overlay workspace with Soong The Bazel overlay is a directory at out/soong/bazel_overlay that replicates the layout of the AOSP Soong module tree, but as a Bazel workspace. Each Soong module variant is represented as a BUILD target created with the `soong_module` rule. To create this overlay, run `m bazel_overlay`. A `soong_module` target can depend on other `soong_module` targets. These dependencies replicate each module's `directDeps` in the Blueprint graph, just before `PrepareBuildActions`. This enables users to use bazel query as a way to introspect the Soong module graph. For example, - Direct reverse dependencies of //bionic/libc:generated_android_ids in //bionic/libc/...: $ bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' //bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static //bionic/libc:generated_android_ids - Why does com.android.runtime depend on lzma? $ bazel query 'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image, //external/lzma/...)' //bionic/apex:com.android.runtime--android_common_com.android.runtime_image //bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime //system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime //external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime - What does the dep graph of //bionic/libc:crtbegin_so look like? $ bazel query 'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)' --output=graph > graph.in && dot -Tpng < graph.in > graph.png https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8 Test: croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel query //... && bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' Signed-off-by: Jingwen Chen <jingwen@google.com> Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-07-15 12:06:41 +02:00
// 0444 is read-only
err := ioutil.WriteFile(pathToFile, []byte(f.Contents), 0444)
Create a AOSP Bazel overlay workspace with Soong The Bazel overlay is a directory at out/soong/bazel_overlay that replicates the layout of the AOSP Soong module tree, but as a Bazel workspace. Each Soong module variant is represented as a BUILD target created with the `soong_module` rule. To create this overlay, run `m bazel_overlay`. A `soong_module` target can depend on other `soong_module` targets. These dependencies replicate each module's `directDeps` in the Blueprint graph, just before `PrepareBuildActions`. This enables users to use bazel query as a way to introspect the Soong module graph. For example, - Direct reverse dependencies of //bionic/libc:generated_android_ids in //bionic/libc/...: $ bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' //bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static //bionic/libc:generated_android_ids - Why does com.android.runtime depend on lzma? $ bazel query 'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image, //external/lzma/...)' //bionic/apex:com.android.runtime--android_common_com.android.runtime_image //bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime //system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime //external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime - What does the dep graph of //bionic/libc:crtbegin_so look like? $ bazel query 'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)' --output=graph > graph.in && dot -Tpng < graph.in > graph.png https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8 Test: croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel query //... && bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' Signed-off-by: Jingwen Chen <jingwen@google.com> Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-07-15 12:06:41 +02:00
return err
Create a AOSP Bazel overlay workspace with Soong The Bazel overlay is a directory at out/soong/bazel_overlay that replicates the layout of the AOSP Soong module tree, but as a Bazel workspace. Each Soong module variant is represented as a BUILD target created with the `soong_module` rule. To create this overlay, run `m bazel_overlay`. A `soong_module` target can depend on other `soong_module` targets. These dependencies replicate each module's `directDeps` in the Blueprint graph, just before `PrepareBuildActions`. This enables users to use bazel query as a way to introspect the Soong module graph. For example, - Direct reverse dependencies of //bionic/libc:generated_android_ids in //bionic/libc/...: $ bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' //bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime //bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static //bionic/libc:generated_android_ids - Why does com.android.runtime depend on lzma? $ bazel query 'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image, //external/lzma/...)' //bionic/apex:com.android.runtime--android_common_com.android.runtime_image //bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime //system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime //external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime - What does the dep graph of //bionic/libc:crtbegin_so look like? $ bazel query 'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)' --output=graph > graph.in && dot -Tpng < graph.in > graph.png https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8 Test: croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel query //... && bazel query 'rdeps(//bionic/libc/..., //bionic/libc:generated_android_ids, 1)' Signed-off-by: Jingwen Chen <jingwen@google.com> Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-07-15 12:06:41 +02:00
}
func createDirectoryIfNonexistent(dir string) error {
if _, err := os.Stat(dir); os.IsNotExist(err) {
return os.MkdirAll(dir, os.ModePerm)
} else {
return err
Surface module properties as Bazel BUILD target attributes in the Bazel overlay This patchset changes bazel_overlay to generate soong_module as a macro, instead of a rule, and generate module properties in the BUILD files as kwargs to the soong_module macro. Here's a sample of the new BUILD files with module properties: bionic/libdl/BUILD.bazel: https://paste.googleplex.com/6484466996346880?raw art/build/apex/BUILD.bazel: https://paste.googleplex.com/5461276001042432?raw bionic/apex/BUILD.bazel: https://paste.googleplex.com/4932795173437440?raw soong_module is now a macro that conditionally expands to underlying soong_<module type> rules with statically defined attributes. In this CL, we are starting with a hardcoded filegroup rule definition to demonstrate the conditional rule loading within the soong_module macro. If the module_type matches an existing Bazel rule, soong_module forwards the entire **kwargs into the rule, which Bazel typechecks. Non-filegroup module types will be expanded into generic_soong_module, but with the kwargs dropped. This approach allows us to: 1) Programmtically generate soong_<module type> rules for all module types available in Soong, together with the statically defined attribute types in `attrs`. 2) Incrementally migrate and test individual module types from generic_soong_module to their module rule shims. 3) Swap out the module rule shims to the actual Bazel rules (e.g cc_library, java_library) and perform attribute manipulation in Starlark itself. Example of querying against the 'srcs' attribute in soong_filegroup: ``` $ bazel cquery 'kind(soong_filegroup, //...)' | wc -l 590 $ bazel cquery --output=build 'attr(srcs, "linker.cpp", kind(soong_filegroup, //bionic/...))' INFO: Analyzed 3907 targets (0 packages loaded, 0 targets configured). INFO: Found 3907 targets... /usr/local/google/home/jingwen/aosp/out/soong/bazel_overlay/bionic/linker/BUILD.bazel:4144:13 soong_filegroup( name = "linker_sources", generator_name = "linker_sources", generator_function = "soong_module", generator_location = "bionic/linker/BUILD.bazel:4144:13", srcs = ["dlfcn.cpp", "linker.cpp", "linker_block_allocator.cpp", "linker_dlwarning.cpp", "linker_cfi.cpp", "linker_config.cpp", "linker_debug.cpp", "linker_gdb_support.cpp", "linker_globals.cpp", "linker_libc_support.c", "linker_libcxx_support.cpp", "linker_namespaces.cpp", "linker_logger.cpp", "linker_mapped_file_fragment.cpp", "linker_phdr.cpp", "linker_relocate.cpp", "linker_sdk_versions.cpp", "linker_soinfo.cpp", "linker_tls.cpp", "linker_utils.cpp", "rt.cpp"], deps = [], ) /usr/local/google/home/jingwen/aosp/out/soong/bazel_overlay/soong_module.bzl:32:23 in <toplevel> ``` This CL is known to be lacking the following features, and will be looked at in follow up CLs: 1) Pretty printing reflect.Interface properties, like arch, multilib and dists. 2) Generating module Bazel rule shims for all module types, instead of hardcoding them like `soong_filegroup`. Bug: 162720644 Test: bazel_overlay_test.go (soong build test) Test: m bazel_overlay && cd out/soong/bazel_overlay && bazel cquery //... Signed-off-by: Jingwen Chen <jingwen@google.com> Change-Id: Ic1e448887eb540ed15a55bc4090cf75a4d832d41
2020-08-07 16:16:34 +02:00
}
}