Commit graph

3815 commits

Author SHA1 Message Date
Chris Parsons
494eef3dc8 Support host platform cc builds
This involves handling of linux-* platforms (identifying them
appropriately as host), disabling unsupported Windows platform, and
denylisting of still-problematic modules.

Test: mixed_droid, with verification that shared library artifacts
originate from bazel-out

Change-Id: Ib52db49a2d2a3c1ff9b76af23fd4f22cfc9182d0
2021-11-10 21:57:15 +00:00
Colin Cross
02ce1e7833 Merge changes Ib3f4ee14,Iac22c9fd,Ibd78758c,I40d8696c,I2a2b10e4, ...
* changes:
  Fix InstallBypassMake symlink dependencies
  Add tests for ctx.InstallFile
  Fix OutDir vs SoongOutDir in tests
  Fix go vet error
  Add missing os.MkdirAll to WriteFileToOutputDir
  Use pathtools.WriteFileIfChanged in translateAndroidMk
2021-11-10 18:49:08 +00:00
Treehugger Robot
8a709b149b Merge changes from topic "soong-java-install"
* changes:
  Revert "Temporarily add method to get java binary tool"
  Move java module installation into Soong
  Fix ctx.InstallFile calls for java modules
  Make HostJavaToolPath use pathForInstall
2021-11-10 17:05:51 +00:00
Colin Cross
64002afc56 Fix InstallBypassMake symlink dependencies
Host symlinks cannot use order-only dependencies because they may be
used as part of the dependency chain on a tool, and an order-only
dependency would cause the target of the symlink not to be updated.
Use regular dependencies instead.

Bug: 204136549
Fixes: 205674000
Test: TestInstallBypassMake
Change-Id: Ib3f4ee143e94d1995ec6c60d314e7c91e57cc775
2021-11-09 17:25:57 -08:00
Colin Cross
6ac9576eef Add tests for ctx.InstallFile
Add tests that cover Soong-only installation as well as installation with
InstallBypassMake.

Bug: 204136549
Test: TestInstall
Test: TestInstallBypassMake
Change-Id: Iac22c9fdf99994e06b419623ee5fa399ef6957fb
2021-11-09 17:25:57 -08:00
Colin Cross
7b6a55f5a2 Fix OutDir vs SoongOutDir in tests
Tests were using the same value for OutDir and SoongOutDir, separate
them to allow a test that needs to distinguish between them.

Test: all soong tests
Change-Id: Ibd78758c133a7f07bead3f51b699052676f92fbd
2021-11-09 17:25:57 -08:00
Colin Cross
d642113643 Add missing os.MkdirAll to WriteFileToOutputDir
Fix tests when out/soong doesn't already exist by adding os.MkdirAll
to WriteFileToOutputDir.

Test: soong tests
Change-Id: I2a2b10e43b967d0c61d0dbe6a3f8bf381babe73c
2021-11-09 17:25:57 -08:00
Colin Cross
836e387323 Use pathtools.WriteFileIfChanged in translateAndroidMk
pathtools.WriteFileIfChanged was reimplemented in translateAndroidMk,
but without a call to os.MkDirsAll.  Reuse pathtools.WriteFileIfChanged.

Test: soong tests
Change-Id: If3f6a7bfc172b4a1a8945109801146c2b7e72b96
2021-11-09 17:25:57 -08:00
Jingwen Chen
8c10020ffd Merge "bp2build: add support for soong_config_module_type." 2021-11-09 06:59:34 +00:00
Jingwen Chen
a47f28d28e bp2build: add support for soong_config_module_type.
Test: CI, go unit test
Bug: 198556411
Change-Id: Idf862904d51d822f92af0c072341c31b7a02fc64
2021-11-08 13:38:28 +00:00
Colin Cross
c899d7808e Revert "Temporarily add method to get java binary tool"
Java binaries and cc binaries are in the same directory again, the
separate method can be removed.

Bug: 204136549
Test: m checkbuild
Change-Id: I2e7ca82448aba5c7e54d3c3c376fe14a953ef4af
2021-11-05 14:26:53 -07:00
Colin Cross
24cc4be60b Move java module installation into Soong
Move java module installation rules into Soong by overriding
InstallBypassMake.  Update the locations that find host java tools
to look in the Make installation directory instead of the Soong
installation directory, which will no longer be used.

Bug: 204136549
Test: m checkbuild
Change-Id: I5af6d764c97e7ddb5ee121fc9830166c25d831b1
2021-11-05 14:24:51 -07:00
Colin Cross
1d0eb7a9d0 Fix ctx.InstallFile calls for java modules
Call ctx.InstallFile on the primary install file last so that the
primary install file can depend on the extra install files, and so that
the primary install file can be inferred from the last installed file.

Add missing ctx.InstallFile calls for the dexpreopt and hostdex outputs.

Fix the install subdirectory for modules installing to the testcases
directory.

Bug: 204136549
Test: m checkbuild
Change-Id: I7edd7647be27439d3ca0ecc589ca5e89d4ba8474
2021-11-05 14:20:28 -07:00
Colin Cross
ae5330a2d9 Make HostJavaToolPath use pathForInstall
Use pathForInstall instead of PathForOutput for HostJavaToolPath
so that it internally produces an InstallPath that can later support
being converted to Make installpath.

Bug: 204136549
Test: m checkbuild
Change-Id: If4f5d3552b27ffe6b9bc709c4a08d9513c49ef7d
2021-11-05 14:20:28 -07:00
Colin Cross
acfcc1f682 Move cc module installation into Soong
Move cc module installation rules into Soong by overriding
InstallBypassMake.  Update the locations that find host tools
to look in the Make installation directory instead of the Soong
installation directory, which will no longer be used.

The methods that find host tools are also used on go binaries,
so update the config methods that tell Blueprint where to install
go binaries to the Make installation directory too.

Bug: 204136549
Test: m checkbuild
Change-Id: Id172592c195e506102982a4af0084f6d9c68a896
2021-11-04 10:28:58 -07:00
Colin Cross
3e3eda6eb6 Temporarily add method to get java binary tool
cc binaries are moving from out/soong/host/linux-x86/bin to
out/host/linux-86/bin first, to be followed by java binaries.
Temporarily add a method that gets the java binary path while
it is different from the cc binary path.

Bug: 204136549
Test: m checkbuild
Change-Id: I04a7e741436da6be68078f23d074870413a5bb5e
2021-11-04 10:28:58 -07:00
Treehugger Robot
c37663aad9 Merge "Convert .meta_lic files to textproto" 2021-11-03 18:31:02 +00:00
Treehugger Robot
a3bd963988 Merge changes Id11f4fb1,Id573d970,Ia94a0b5c
* changes:
  Fix //conditions:default excludes computation for LabelListAttribute.
  bp2build: split Bazel conversion context into smaller ones, and change TopDownMutatorContext signatures to use Bazel conversion context.
  Add an error check in `bazelPackage` for malformed labels.
2021-11-03 15:36:28 +00:00
Paul Duffin
8380ee66a5 Merge "Use module-lib system modules when building from prebuilts" 2021-11-03 15:32:42 +00:00
Liz Kammer
5506c1658c Merge "Update bp2build denylist cause" 2021-11-03 14:19:16 +00:00
Wei Li
c989eaf4c3 Keep the following targets disabled in mixed build since there are errors in mixed_droid CI after go/oag/1861755 is submitted.
func_to_syscall_nrs
libseccomp_policy_app_zygote_sources
libseccomp_policy_app_sources
libseccomp_policy_system_sources

Test: CI
Bug: 204949078

Change-Id: Iaf9d7cd8831da984bcee82399be138ebe3aab9e8
2021-11-03 04:15:36 +00:00
Liz Kammer
b84b8c9c9a Update bp2build denylist cause
The binary is converted; however, there are still issues with the host
toolchain.

Test: build/bazel/ci/bp2build.sh
Change-Id: Idf864ea6c64c0c7bbbaa0c9d43d6eac5120e0268
2021-11-02 15:03:12 -04:00
Wei Li
664a4fd0af Fix python_binary_host module in mixed build.
Test: USE_BAZEL_ANALYSIS=1 m genfunctosyscallnrs
Test: USE_BAZEL_ANALYSIS=1 m func_to_syscall_nrs
Test: USE_BAZEL_ANALYSIS=1 m libseccomp_policy
Bug: 201094425, 197135289

Change-Id: Ibc8b99a92149410c8a879b7a4facf6c8961a7b9f
2021-11-02 17:04:38 +00:00
Jingwen Chen
55bc820d66 bp2build: split Bazel conversion context into smaller ones,
and change TopDownMutatorContext signatures to use Bazel conversion context.

This minimizes the context interfaces/functions actually needed to
convert a module, and makes such interfaces easier to mock/test.

Test: CI
Change-Id: Id573d97023d59e06ef70e1f54437024d3f7aadbd
2021-11-02 06:51:27 +00:00
Jingwen Chen
80b6b64db5 Add an error check in bazelPackage for malformed labels.
Bug: 204281595
Test: CI
Change-Id: Ia94a0b5c8dd3a0294475e9bc816a19ae03f7342d
2021-11-02 06:24:31 +00:00
Colin Cross
34c7832aaa Convert .meta_lic files to textproto
Make it easier to write tools against .meta_lic files and store complex
data by writing them in textproto.

Test: builds

Change-Id: I54bb82cc5581d17078fd0f56eed43a7364dc70db
2021-11-01 16:32:31 -07:00
Dan Willemsen
8528f4ec5e Add Darwin+Arm64 toolchain support
This just sets up the toolchain and allows Darwin+Arm64 to be specified
as a HostCross target. These variants will not be exported to Make, or
be installed on a Soong-only build. A future CL will add support for
universal binaries using these variants.

This config is a bit stranger than the regular 64/32 multilib, as it's
two primary 64-bit configs. And on a Darwin/X86 machine, the Arm64
versions are HostCross (doesn't work on the current machines), while a
Darwin/Arm64 machine, either version works (if Rosetta is installed).

Bug: 203607969
Change-Id: Iacaed77d267773672da027cd74917e33fb1c1e94
2021-11-01 15:07:37 -07:00
Colin Cross
1caea35278 Merge "Export Soong install rules to Make" 2021-11-01 21:31:56 +00:00
Paul Duffin
004547facd Use module-lib system modules when building from prebuilts
When building from source the build uses the java system modules for
the public or module APIs as needed. However, previously when building
from prebuilts it would always use the public API. That difference lead
to build failures when building from prebuilts.

This change makes the selection of java system modules when building
from prebuilts consistent with the selection when building from
sources.

As API levels 30 and 31 (which are the only previous releases to
provide system modules) did not provide separate java system modules
for the module-lib API those levels always use the public APIs.

Bug: 204189791
Test: - before applying these change
      m TARGET_BUILD_APPS=framework-connectivity
      - build fails with compilation error due to missing module APIs
      m sdk dist
      cp out/dist/system-modules/module-lib/core-for-system-modules.jar prebuilts/sdk/current/module-lib/core-for-system-modules.jar
      - apply these changes
      m TARGET_BUILD_APPS=framework-connectivity
      - build passes as expected
Change-Id: Id113ff014e7892b1009fbcaad89b1ae23a7c3b79
2021-11-01 16:40:28 +00:00
Colin Cross
91eaa6316a Merge changes I04f2f558,Ie16a6264,I83a3be9f,Icb3eeef3
* changes:
  Rename amod variable in AndroidMkEntries.fillInEntries to base
  Make HostToolPath, HostJNIToolPath and sboxPathForToolRel use pathForInstall
  Add PathForGoBinary
  Remove PathForOutput from InstallPathToOnDevicePath
2021-10-30 19:19:53 +00:00
Chih-hung Hsieh
7b2239f648 Merge "Add -{checkbuild,tidy} targets only for modules in mm" 2021-10-30 03:57:06 +00:00
Chih-Hung Hsieh
a3d135b1af Add -{checkbuild,tidy} targets only for modules in mm
Test: lunch aosp_coral-userdebug; WITH_TIDY=1
      make 7-libgptutils.coral-checkbuild 7-libgptutils.coral-tidy
Test: only coral libgptutils in out/build-aosp_coral.ninja
Test: lunch aosp_arm64-eng; make has no *libgptutils* tagets
Test: no libgptutils in out/build-aosp_arm64.ninja
Test: lunch aosp_cf_arm64_phone-userdebug; make libskia-checkbuild
Test: lunch aosp_arm64-eng; WITH_TIDY=1 make tidy-soong

Bug: 202763874
Change-Id: I2d5a3fdc627d62e43c3f10e61b338a03c10440be
2021-10-29 18:02:27 -07:00
Colin Cross
6301c3cffa Export Soong install rules to Make
Previously Soong's install rules have been disabled when embedded
in Make (ctx.Config().KatiEnabled() == true).  The primary blocker
for moving installation into Soong has been the `required` proeprty,
which is too vague to be easily handled in Soong.  Keeping
installation in Make has resulted in two host bin directories,
the Make-owned directory (e.g. out/host/linux-x86/bin), and the
Soong-owned directory (e.g. out/soong/host/linux-x86/bin).  The
lack of knowledge in Soong about the final, Make-owned installation
location makes it hard to support NOTICE files entirely in Soong.

This patch begins to solve this problem by supporting the creation of
the installation rules into Soong, but rather than writing the rules
to the ninja file it writes them to a Makefile and lets Kati convert
them to ninja.  This allows Kati to inject extra dependencies to
handle the `required` property.

Converting all modules to create their installation rules in Soong
would be too complex, so only modules that return true from
InstallBypassMake will use the Soong installation rules.  This
is currently only set for robolectric tests.

Bug: 204136549
Test: m checkbuild
Change-Id: I28af9fa7fadece8ea1f98f5efd140c823751cae7
2021-10-29 16:34:55 -07:00
Colin Cross
f1f763a981 Rename amod variable in AndroidMkEntries.fillInEntries to base
AndroidMkEntries.fillInEntries calls its android.ModuleBase amod,
despite also handling an android.Module.  Rename amod to base to
match other locations, and add a new amod for the android.Module.
This will simplify the next patch that needs to access the
android.Module.

Bug: 204136549
Test: m checkbuild
Change-Id: I04f2f558959def22e8b3f5b8c534b8d655b06a4e
2021-10-29 16:34:55 -07:00
Colin Cross
790ef35d1e Make HostToolPath, HostJNIToolPath and sboxPathForToolRel use pathForInstall
Use pathForInstall instead of PathForOutput for HostToolPath,
HostJNIToolPath and sboxPathForToolRel so that they internally produce
an InstallPath that can later support being converted to Make install
path.

Bug: 204136549
Test: m checkbuild
Change-Id: Ie16a62641d113873daeec4d1dd4261251bc0d0eb
2021-10-29 16:34:55 -07:00
Colin Cross
a44551fec6 Add PathForGoBinary
Add PathForGoBinary that uses pathForInstall to return the install
path of a GoBinaryTool.  This will replace various places that used
PathForOutput to reconstruct a path to a Go tool, and will support
moving Go tools to the Make install directory outside of the
PathForOutput directory in a future patch.

Bug: 204136549
Test: m checkbuild
Change-Id: I83a3be9f5c621975540f5ed601a0b9e2611c98b9
2021-10-29 16:34:55 -07:00
Colin Cross
b1692a3468 Remove PathForOutput from InstallPathToOnDevicePath
The next patches will make more InstallPaths use the Make output
directory, which is not inside PathForOutput.  Fix the assumption
that the InstallPath is relative to PathForOutput by storing
enough information in the InstallPath to find the on-device path
without PathForOutput.

Bug: 204136549
Test: soong tests
Change-Id: Icb3eeef3f1c72f773f333267f8a7dfc503feacb5
2021-10-29 16:34:55 -07:00
Paul Duffin
12e311d958 Add tests for prebuilts of all API surfaces
Previously, the tests only covered checking the sdk dependencies added
when using a prebuilt current (public) and system SDKs, i.e. with
sdk_version set to "current_30" or "system_30". This change adds tests
to cover other APIs, e.g. "test_30", "module_30" and
"system_server_30". It adds tests for "module" and "system-server" up
to API level 32 as following changes will give them different behavior
as API levels less than or equal to 31 do not have a separate
core-for-system-modules.jar file for the module-lib API.

Bug: 204189791
Test: m nothing
Change-Id: I585a88615439a24abf75250114a86113b5f5da57
2021-10-29 17:57:12 +01:00
Treehugger Robot
c9e85ca825 Merge "Make a pass over our codebase cleaning up non-precise / non-inclusive language." 2021-10-28 20:51:17 +00:00
Joe Onorato
b4638c1ac3 Make a pass over our codebase cleaning up non-precise / non-inclusive language.
Test: treehugger
Bug: 204369779
Change-Id: I6e63cae824d323535cdb14b493edd868df47738d
2021-10-28 10:32:07 -07:00
Chih-hung Hsieh
4dceb3705f Merge "Do not add .tidy files as RBE link inputs" 2021-10-27 20:29:04 +00:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
34d60d6672 Remove vestigial GetPrebuiltCcStaticLibraryFiles
Test: mixed_{libc,droid}.sh
Change-Id: Ie6e22e1994f8abf1b66f4abfd834bf38a1f097b2
2021-10-26 15:03:18 +00:00
Chih-Hung Hsieh
3bb934f0ad Do not add .tidy files as RBE link inputs
* RBE adds implicit dependent files as inputs
  and creates too long commands.
* Adds .tidy files as validation paths.
* Change type of tidyFiles to WritablePaths.

Bug: 203577539
Test: lunch aosp_cf_arm64_phone-userdebug; USE_RBE=true WITH_TIDY=1 make
Test: USE_RBE=true WITH_TIDY=1 mm in external/skia
Change-Id: I5fc9d6f8c9c58abcd3721ca9fbae1bc785e4e609
2021-10-25 12:24:30 -07:00
Jiyong Park
d163d4d7f5 Add the device config GenerateAidlNdkPlatformBackend
The config is used to force the generation of the AIDL ndk_platform
backend which will eventually be removed in favor of the ndk backend.
The switch is needed as an escape hatch for some devices whose BSP
(outsourced and thus hardly modifiable) depends on the ndk_backend
libraries.

Bug: 161456198
Test: m
Merged-In: Ie11ca62be05754bb2593da16e1380103c670b393
Change-Id: Ie11ca62be05754bb2593da16e1380103c670b393
(cherry picked from commit 65d9718328)
2021-10-25 08:55:57 +09:00
Liz Kammer
2dafba7da9 Merge "bp2build: Add support for cc_binary." 2021-10-20 14:02:39 +00:00
Chris Parsons
787fb36189 Add OS to configuration key in mixed builds
This also removes the special-case filegroup from mixed builds
buildroot; no special handling is required. Since we're currently
hardcoding linux_x86_64 as our host platform, it should be fine
to continue assumping that hardcoded host for now.

Test: USE_BAZEL_ANALYSIS=1 m adbd
Change-Id: I35509f4eb33ba7a243fab4c34b35958f6f2fceab
2021-10-19 16:55:52 -04:00
Liz Kammer
2b8004b1ef bp2build: Add support for cc_binary.
Bug: 197920036
Test: build/bazel/ci/bp2build.sh
Change-Id: I2c4200967653af15a330ab8cbaf796b70d43f32d
2021-10-19 14:12:39 -04:00
Dan Willemsen
2a109db10b Merge "Reduce modules exposed to Make in Mac builds" 2021-10-19 00:32:54 +00:00
Liz Kammer
503f65da36 Merge "Add tests for defaults used in conditions_default" 2021-10-18 22:03:33 +00:00
Dan Willemsen
def7b5d198 Reduce modules exposed to Make in Mac builds
Device builds are no longer supported on Mac, but we do support building
various host tools, including the SDK build-tools and platform-tools
packages. These have dependencies on [java] device modules, so we don't
completely disable device modules, only hide them from Make (which makes
them more difficult to trigger from the command line).

Also fix the mac build of multiproduct_kati, so that `m blueprint_tools`
works on Mac.

Bug: 187222815
Test: `m`, `m dist`, etc on Mac
Change-Id: I92f16605d5cd173d431cbcb79081234d45cc6e2e
2021-10-18 12:44:45 -07:00