Commit graph

154 commits

Author SHA1 Message Date
Sasha Smundak
c180dbd4c7 Handle SymlinkTree action, ignore PythonZipper action.
Introduce bazelBuildRunfiles to build runfiles symlink tree, allowing to
ignore a bogus PythonZipper action.

Bug: 232085015
Test: treehugger
Change-Id: I81267f523d8237fddbc7d65955cdd08ea6369046
2022-07-06 11:30:40 -07:00
Sasha Smundak
1da064c1e6 Handle WriteFile and SourceSymlinkManifest actions.
Plus minor editorial changes.

Bug: 232085015
Test: treehugger
Change-Id: I966e9d6d306382dbb8eac6f8a495a2f152c7a22e
2022-07-06 11:30:40 -07:00
Sasha Smundak
e3cf1abefc Reformat the test to make them more readable.
Test: treehugger
Change-Id: I28ccd531a5dadcee3cbdebd5f5ba6db1605662d9
2022-06-30 14:20:55 -07:00
Usta Shrestha
ef92225a67 null build upon repeated mixed build
no implicit deps on bazel-tools

Test: USE_BAZEL_ANALYSIS=1 ../bazel/ci/incremental_mixed_build.sh
Bug: b/216194240
Change-Id: Ibbd87c6a6cc2fddf21fba37a6bb4e72adc209576
2022-06-28 15:37:12 -04:00
Colin Cross
2bce04f76f Merge "Add linux_musl arm+arm64" 2022-06-27 18:37:07 +00:00
Usta Shrestha
1c4a3ea497 More helpful error message in test
Test: run aquery_test.go
Bug: N/A
Change-Id: I47920d536cb81d148bf6aa6f9c340e21d1f5be86
2022-06-24 23:03:05 -04:00
Colin Cross
a9b2aacf07 Add linux_musl arm+arm64
Add toolchains to support cross compiling to aarch64-linux-musl and
arm-linux-musleabihf.

Bug: 236052820
Test: build arm and arm64 musl sysroots
Change-Id: I47a9322929baff2492c6e8db989ece01fcbeb133
2022-06-24 13:46:36 -07:00
Sasha Smundak
f10c3aca7e Fix minor printf format problems.
Test: treehugger
Change-Id: I806ea8d249e40ecb1c6eb7903177ad7213a92753
2022-06-23 18:08:30 -07:00
Usta Shrestha
16ac13506e cosmetic changes
Test: m nothing
Bug: N/A
Change-Id: Ia74a09a6ee42560562e2e1a15e972860e77a5724
2022-06-22 11:02:43 -04:00
Christopher Parsons
0e36bc11e4 Merge "Use builtin CcSharedLibraryInfo for cquery" 2022-06-14 16:27:18 +00:00
Yu Liu
57c1edc4d9 Merge "Support cc code coverage for mixed build" 2022-06-13 18:57:50 +00:00
Yu Liu
8d82ac58b4 Support cc code coverage for mixed build
Bug: 231322627
Test: Manual tests and unit tests
Change-Id: I786042af0d612192c54c3572f63a86a47174a242
2022-06-06 12:29:25 -07:00
Usta Shrestha
93a9b9f471 Merge "minor dead code: unused format string" 2022-06-06 17:00:34 +00:00
Liz Kammer
62ac8a0707 Merge "Handle restricting to device only builds" 2022-06-02 20:29:27 +00:00
Usta Shrestha
2ccdb42098 minor dead code: unused format string
Using URL base64 encoding
(without padding, i.e. with trailing '=' or '==')
hash string width is 256/6 = 43 characters.

file size virtually unchanged,
  out/soong/build.ninja:  7,188,669,041 => 7,186,999,370 ie 1.6M

Bug: N/A
Test: manually verified (e.g. ran `m nothing` successfully)
Change-Id: I166d613e1fd857555da9611d420d6691806571c7
2022-06-02 11:37:20 -04:00
Treehugger Robot
354b49e9f1 Merge "cosmetics: readability" 2022-05-31 23:27:45 +00:00
Liz Kammer
dfeb120324 Handle restricting to device only builds
By default, most module types are only enabled for device. Converting
this behavior and host_supported/device_supported properties allows us
to skip building incompatible targets.

Test: soong tests
Test: bp2build.sh
Change-Id: If1da523b4cc8c4cbf2bb26da063d9923b662cc32
2022-05-31 17:46:48 -04:00
Usta Shrestha
6298cc5e67 cosmetics: readability
Note go.mod file changes go version to 1.18 (for generics)
You might need to refresh/reload your IDE (e.g. IntelliJ)

Bug: N/A
Test: manually verified (e.g. ran `m nothing` successfully)
Change-Id: I2a6dadd673fd170961d1fd1e1756f7581e9a1091
2022-05-31 21:39:06 +00:00
Chris Parsons
cf553a6575 Use builtin CcSharedLibraryInfo for cquery
Test: mixed_droid.sh CI script
Change-Id: I1571ede5f3320344f69c136843eb662044ba9c5f
2022-05-31 16:47:12 -04:00
Cole Faust
5317be265a Merge "Add bp2build converter for python protobuf files" 2022-05-31 19:22:32 +00:00
Usta Shrestha
c23724905b cosmetic: name receiver consistently
see go/go-style/decisions#receiver-names

Bug: N/A
Test: N/A
Change-Id: Ifee42ff66cdb4db024a82ebd03cdee1c6d4fb5f7
2022-05-27 10:59:02 -04:00
Cole Faust
53b62098d0 Add bp2build converter for python protobuf files
Bug: 196084681
Test: b run //build/bazel/examples/python/protobuf:build_bazel_examples_python_protobuf_main --config=linux_x86_64
Change-Id: I4d806902d262351231f64686a5d24513a25d9749
2022-05-26 15:54:23 -07:00
Trevor Radcliffe
ef9c900ec3 Generate genlex rules from bp2build for cc targets
This change will cause bp2build to generate genlex targets any
time a .l or .ll file is present in the srcs for a cc target and
add those genlex targets to the srcs attribute of the original
target.

Bug: 207408632
Test: unit tests
Change-Id: I1bce82c9d3c3d458eae1cef547ffae3d6e975134
2022-05-24 14:42:51 +00:00
MarkDacek
7564127cf2 Grammar nit.
Test: N/A

Change-Id: Ied10037b662cd49573680d392cddd11b628618ff
2022-05-13 20:44:07 +00:00
Chris Parsons
0bfb1c0556 Deterministic aquery details in mixed builds
This change constitutes a number of fixes which cause mixed builds to
have deterministic ninja file output:

1. Depsets are identified based on a hash of their contents instead of
   an arbitrary ID integer from Bazel
2. Depset definitions in the ninja file are sorted by the above hashes
3. BuildStatements (action information from Bazel's aquery) are sorted
   by their contents

Test: Ran `USE_BAZEL_ANALYSIS=1 m nothing` three times and verified the
md5sum of out/soong/build.ninja was identical all three runs.
Test: mixed_droid

Change-Id: Iffdf6cc62c31d76fbbfa78726827497516171f4f
2022-05-13 13:45:56 -04:00
Chris Parsons
1a7aca075b Preserve depset structure from bazel aquery
Each depset now corresponds to a phony rule which depends on other
depsets or on full paths; thus, bazel's depset structure is preserved in
the form of phony rules of name bazel_depset_{id}.

Previously, flattening and recopying large lists of file path strings
was quite inefficient. This was particularly evident as we enumerated
hundreds of clang headers for each cc compile action.

This reduces soong_build analysis time by about 30% for mixed builds.
It also reduces ninja file size by ~750MB.

Fixes: 229405615
Test: Unit tests, manually verified metrics, mixed_droid CI

Change-Id: I78df152ac1488ae0c6807afdde4b4ad5e6d26287
2022-04-28 12:44:28 -04:00
Sam Delmerico
cc51843e52 add unit test for bp2build allowlist
Test: go test ./android
Change-Id: I0ea1d97444cbedfd285f1fc4bd7ff246ce699dc9
2022-04-14 13:31:09 +00:00
Sam Delmerico
24c5603815 refactor bp2build module allowlists into struct
To make testing easier, refactor existing module-global variables into a
struct that can be mocked.

Test: build/bazel/bp2build.go
Change-Id: I9d177677644ea743641a745b1839a3a8b29f902a
2022-04-14 13:31:05 +00:00
Sam Delmerico
7f88956c16 refactor Bazel variable export
Most of the variable export code for cc modules can be re-used for
exporting variables for java modules. Refactor this code into a more
composable structure for reuse.

Test: build/bazel/bp2build.sh
Test: manual comparison of
  out/soong/soong_injection/cc_toolchain/constants.bzl
  with previous output
Change-Id: Ie5a6fee08cc888b7dc69c3e324e5c3f8aa269a8f
2022-04-06 14:48:32 +00:00
Sam Delmerico
c016143602 emulate java_library static_deps with Bazel exports
In Soong, java_library can specify static_deps which are dependencies
that get aggregated into the final jar (akin to static linking). This is
useful because it allows dependencies higher up in the chain to compile
against the APIs exported by transitive dependencies. Bazel does not
support this functionality directly, but it can be emulated via the
exports attribute which makes any targets listed in the attribute public
to targets further up the chain.

Bug: 217236083
Bug: 219908977
Test: b build //external/error_prone:error_prone_core
Test: b build //external/bouncycastle:bouncycastle-host
Test: b build --platforms=//build/bazel/platforms:linux_x86
  //prebuilts/sdk/tools/jetifier/jetifier-standalone:jetifier
Change-Id: I2867e3f816de720a6f4bd9ff7a847d1b0c2da2d6
2022-03-30 18:58:38 +00:00
Liz Kammer
8c46bf96f5 Update cqueries for bazel rules dir rename
Test: USE_BAZEL_ANALYSIS=1 m updater
Change-Id: I1356828a00037c8ec42108438a2975d2ab44d1d7
2022-02-10 10:26:21 -05:00
Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
c3b97c3568 libclang_rt_prebuilt_library_shared mixed builds
Also fix nil deref in setting $LOCAL_SOONG_UNSTRIPPED_BINARY

Bug: 201802518
Test: request_type_test.go
Test: prebuilt_test.go:TestPrebuiltLibrarySharedWithBazel
Test: mixed_{libc,droid}.sh
Change-Id: I22afb56c4b42d3412c2b2e1f079f1bcf8f3129a7
2022-01-20 12:03:18 -05:00
Liz Kammer
07e106ffa1 Change format string %s -> %#v
Make go stop complaining about format string:
    "Errorf format %s has arg ba of wrong type"

Test: go test soong/...
Change-Id: I4615f5ff77e1c2c41b84fd227462cdb3564d4369
2022-01-13 22:29:29 +00:00
Liz Kammer
5430953c82 bp2build: Remove duplicate system shared libs
If a system shared lib is specified in shared_libs, this results in
duplicate values appearing, causing a failure in Bazel. This change
removes any system shared libraries that appear in shared libraries from
bionic OS axes where system_shared_libraries takes the default value.

Test: go soong tests
Test: temporarily allowlist directory with this issue, no longer hits
      duplicate library failure.
Change-Id: I9dce570b73c24973f695b815bce8d50f7259798d
2021-12-22 15:53:13 -05:00
Chris Parsons
58852a05f3 Handle the 'enabled' property in bp2build
Also fix some bugs pertaining to configurable attribute handling of bool
attributes and label sttributes, so that they may support values across
multiple different axes at the same time.

Test: unit tests for bp2build
Test: mixed_droid

Change-Id: I411efcfddf02d55dbc0775962068a11348a8bb2c
2021-12-21 16:37:32 -05:00
Liz Kammer
eb2d6d1ffd Propagate headers from bazel to mixed builds
Bug: 208503274
Test: mixed_libc.sh
Change-Id: I0be57f2a22f48be3a919208db4034d2bd03c18c0
2021-12-07 10:25:11 -05:00
Chris Parsons
77acf2e5c1 bp2build: split full_cc_library into shared/static
Test: mixed_droid.sh in conjunction with rule changes
Test: bp2build.sh
Change-Id: If0577065fd39a0446eab16b62c15204d43207e19
2021-12-06 14:11:40 -05:00
Liz Kammer
12615dbbca bp2build: support full/lite protos in cc libs
Test: bp2build.sh
Bug: 200601772
Change-Id: I3a7e00546726bc63b5eb8d5604557c5988a5320b
2021-12-02 11:00:46 -05:00
Jingwen Chen
01812020c1 Add support for writing all Soong config variables into @soong_injection.
Also remove the need to use bp2build_available on
soong_config_module_types as we want to convert every single of them
into the tree for a complete soong_injection soong_config_variables.bzl
file.

The variables are split into their bool, value and string types
respectively, as they all need to be handled differently on the Bazel
product_platform side, as well as for generating constraint values and
settings. For example, value variables need to integrate with
TemplateVariableInfo, and string variables need to include the string
value itself into the select key/constraint value.

Sample soong_config_variables.bzl file: https://gist.github.com/jin/cef700bfb20c8656a931306dd71d47e1

Test: CI
Bug: 198556411
Change-Id: I8665dd1269a507edb37de62407ed3641564bea5c
2021-11-23 08:37:30 +00:00
Jingwen Chen
58ff6801f4 Fix bp2build select generation for inter-attribute soong config
variable usage.

There's bug a in the current soong_config_variable handling
implementation where a soong_config_variable sets conditions_default
value for an attr, and a non-conditions_default value for another attr.
This results in the former attr not properly setting the zero value for
the non-conditions_default select key, resulting in the pretty printer
omitting the attribute totally.

The current implementation in this CL ensures that the zero value is set
whenever this happens at the module level. This is seen in
library_linking_strategy_cc_defaults (see comments in code, and the new
tests)

Test: CI
Bug: 198556411

Change-Id: Ibaeb94508c51a7429fb7a08df610cbb5470f76d2
2021-11-23 08:37:30 +00:00
Liz Kammer
c31929962b Quote aquery commands.
Bazel run shell actions begin `bin/bash -c <command>`, without escaping,
the command can be treated as an argument to `bin/bash`.

Test: build/bazel/mixed_droid.sh
Change-Id: I423cb393da2e6ac97448ec77b2596f12670dfd31
2021-11-16 17:31:05 -05:00
Wei Li
455ba83a94 Fix the output handling of Bazel aquery for py_binary targets so soong's python_binary_host modules can be converted to py_binary and built with Bazel dependent targets in mixed build.
Test: USE_BAZEL_ANALYSIS=1 m genfunctosyscallnrs
Test: USE_BAZEL_ANALYSIS=1 m func_to_syscall_nrs
Test: USE_BAZEL_ANALYSIS=1 m libseccomp_policy
Bug: 201094425
Bug: 197135289
Bug: 204949078
Change-Id: If98630830d687ca814d6ae09c1e610f2c3a371b1
2021-11-16 20:49:28 +00:00
Jingwen Chen
a47f28d28e bp2build: add support for soong_config_module_type.
Test: CI, go unit test
Bug: 198556411
Change-Id: Idf862904d51d822f92af0c072341c31b7a02fc64
2021-11-08 13:38:28 +00:00
Liz Kammer
c773778bfe Make bazel action symlinks absolute paths
hardlinks are incompatible with sandboxing
relative links are incopatible with Soong's use of `cp -d`

Test: build/bazel/ci/mixed_libc.sh
Change-Id: I8c776cda6a27c680c51466d9a7af1b499f2f566d
2021-11-04 12:42:52 -04:00
Jingwen Chen
9af49a49f8 Fix //conditions:default excludes computation for LabelListAttribute.
Previously, excludes computation for the default condition in label list
selects was clobbering pre-existing values. This CL fixes it by
performing a union of the new values with existing ones instead.

Test: properties_test.go
Bug: 198556411
Change-Id: Id11f4fb14e359201304afde0d8ba856851d41395
2021-11-03 01:56:42 +00:00
Dan Willemsen
8528f4ec5e Add Darwin+Arm64 toolchain support
This just sets up the toolchain and allows Darwin+Arm64 to be specified
as a HostCross target. These variants will not be exported to Make, or
be installed on a Soong-only build. A future CL will add support for
universal binaries using these variants.

This config is a bit stranger than the regular 64/32 multilib, as it's
two primary 64-bit configs. And on a Darwin/X86 machine, the Arm64
versions are HostCross (doesn't work on the current machines), while a
Darwin/Arm64 machine, either version works (if Rosetta is installed).

Bug: 203607969
Change-Id: Iacaed77d267773672da027cd74917e33fb1c1e94
2021-11-01 15:07:37 -07:00
Chris Parsons
d80b3c8218 Use hardlinks for Bazel's symlink actions
This prevents failures resulting from using `cp -d` on symlinks.

Test: USE_BAZEL_ANAYSIS=1 m adbd
Change-Id: Ic1756ffe407df81b0e99f99c50bee6af4ecf7796
2021-10-14 15:24:05 -04:00
Liz Kammer
9bad9d645e Do not sort after subtraction.
This allows labels/strings to remain in their original order.

Test: go test bazel tests
Change-Id: I69f575df9e4a358fee4392ae48edf4550e463efb
2021-10-12 13:36:27 -04:00
Chris Parsons
2dde0cb3de bp2build: Handle target.linux
Also refactor target.bionic to be handled not as its own configuration
axis, but instead to be grouped into os selects handling.

This allows us to remove libbase and its dependencies from the bp2build
denylist.

Test: mixed_droid.sh CI
Change-Id: I92f30074d286306207653fe37589835ae3db16c4
2021-10-01 14:56:39 -04:00
Liz Kammer
46f8dbe06a Merge "Refactor code for partitions c srcs" 2021-09-24 17:35:16 +00:00