A single proto providing directory can be used by multiple soong modules. Some of these
can be
1. Host specific
2. Device specific
3. Both host and device
Since the generated proto_library can have 1:many mapping, it should
have an empty target_compatible_with. Compatiblity will be enforced at
the top-level {cc|java|python}_proto_library.
(This is a followup to aosp/2727054 which did not handle this correctly)
Test: Added a unit test
Change-Id: I09b3def70e3d043fd8ba0d1eb4ffff1910f097d1
createProtoLibraryTargetsForIncludeDirs uses a lock-protected plain map
to see if a directory has been handled previously during concurrent visits
of modules in bp2build.
From golang's docs https://pkg.go.dev/sync#Map, replacing sync.Mutex
with sync.Map will be faster in this use case since the keys (directores
here) are written just once but read many times.
Test: m bp2build
Change-Id: Ia2a471b4db5d2890fa6048bc05a17cebc5f686af
This should hopefully fix an intermittent flake in bp2build, which is
now run by default.
Test: go build ./android
Change-Id: Ic257a34448ab323df1680cf1990b087ed415a592
The fixes are
- Dedupe the dir list. Since we partition the proto srcs per pacakge and
then iterate the map, a single include directory was being listed
multiple times
- Drop target_compatible_with from these proto_library targets. The
compatibility will be enforced at the top-level <lang>_proto_library.
Test: Added a unit test
Change-Id: Ia18c0f8c495585010fd4e372092afe80c5d8290c
If a python_library uses a pkg_path foo/bar, then the proto srcs in
that libray need to import the dep .proto as foo/bar/proto.proto.
This behavior is restricted to python modules. To implement this is in
bp2build, this CL creates a new interface with a single method
`PkgPath`. Only python module structs implement this interface, and
this method is only available during bp2build
Test: Added a bp2build unit test
Test: TH
Change-Id: If8d207c0b321f75337a053795826b283a5eaaf46
This is a followup to aosp/2711093 which added support for
proto.include_dirs. local_include_dirs is simlar, except that is
relative to the module directory.
Test: go test ./bp2build
Bug: 285140726
Change-Id: I32ddc7371048672d6935f827c8aee9d767305e2c
This is a fix similar to aosp/2707793. This adds manual tags to the
top-level proto_library created in the root package, plus any
dynamically created proto_library targets for proto.include_dirs. These
proto_library targets might not specify their deps correctly and might
be unbuildable.
(We need to keep these targets because they provide a ProtoInfo for
cc/java/py source gen)
Test: go test ./bp2build
Change-Id: Ic00f05186327fcfcc8d33a0a2c0891ed619b7acb
Soong's proto.include_dirs becomes the -I path for aprotoc cpp code
generation. This does not translate well to Bazel because it runs this
action in a sandbox. Even if we construct an -I path, the .proto files
will not be there. This CL attempts to handle this kind of dependency
automatically via bp2build.
For this hypothetical example
```
foo.proto # contains `import bar.proto"
Android.bp # cc_library {srcs:"foo.proto", p.include_dirs:["subdir"]},
subdir/bar.proto
```
Implementation details for CcProtoGen of foo
- Glob the labels of .proto files for each includeDir, and create a
proto_library that encapsulates them.
- Since Bazel poses a contraint that proto_library target needs to be
in the same pacakge as the .proto file, the proto_library might be created
in a subdirectory with an import_prefix
- Add bar's proto_library as transitive deps of foo's cc_proto_library.
This will be added to -I during aprotoc. We cannot add them to `deps`,
otherwise bar's symbols will be statically linked into foo's .a
file.
Implementation details for clang compile of foo
At the end of CcProtoGen, we have converted foo.proto
to .cpp/.h files. To compile them to .a files, we need the .h files
generated from bar.proto. Soong specifies this at the
top-level cc_library soong module, so add those deps as the
implementation deps for clang compile.
(Will add support for java in a follow-up CL)
Test: go test ./bp2build
Test: built some internal modules that were previously blocked by this
Bug: 285140726
Change-Id: I7e1f9f0d1b1ba916a7ba8278f6cfb342b381d695
This is a fix for aosp/2693190 that handled .proto files that end up in
different bazel packages. It did it by creating proto_library targets in
the correct bazel package.
Changing the granularity causes issues if the the new proto_library in
the subpackage imports a .proto file from a parent package or a
different package. e.g.
```
tmp
├── foo.proto
└── subdir/import_foo.proto # contains an `import "foo.proto"`
└── subdir/Android.bp # package boundary
├── Android.bp # contains a cc_library with foo.proto and
# subdir/import_foo.proto
```
At ToT, the ProtoInfo we provide to CcProtoGen is correct, but the
proto_library in subdir/BUILD will not compile because it does not have
a dep on the proto_library in ./BUILD
This CL creates a workaround by adding `manual` to the proto_library
targets. This CL is based on the assumption that the buildable unit in
bp2build is cc_library_*, and not proto_library necessarily (atleast
till we do a manual/automated cleanup)
Test: Created an integration test in build/bazel
Test: go test ./bp2build
Bug: 292583584
Bug: 246997908
Change-Id: I73120be2411967cb144f37ed4417f76ecf1a6ffa
Protos in another directory were using import prefix, which was
prepending the repository-relative path with the value, instead, we want
to strip the prefix of the directory of the module the protos were used
in such that they can be referenced at the appropriate relative path.
Reference on import_prefix:
https://bazel.build/reference/be/protocol-buffer#proto_library.import_prefix
Test: b build //packages/modules/adb:libfastdeploy_host
Change-Id: If050b0f5fc5103bd9cc5a99703bd604325aa4204
Bazel poses a strict requirement that .proto files and proto_library
must be in the same package. This CL handles this automatically by
creating the proto_library in a separate dir/package if necessary
Implementation details
- Partition the `srcs` by package. `srcs` has been computed using
`transformSubpackagePath`, so the information about packages is
available at this point
- Create a proto_library in each package by using
`CommonAttributes.Dir`. Collect all these additional libraries
and put them in `info.Proto_libraries` so that they get added as deps
of (cc|python|...)_proto_library
- Add an import_prefix to the proto_library in subpackages relative to
the current directory. This relies on the assumption that every src is
beneath the current directory (Soong will complain if a path in
Android.bp contains ../)
filegroup module type uses a separate code-path to create proto_library.
This will be handled in the next CL in stack.
Test: bp2build unit tests
Test: TH
Test: Built the failing internal module mentioned in
b/292583584#comment1
Bug: 292583584
Change-Id: I437fc89092321b26c5f0511387cde9e84084d6f9
Soong does not enforce apex_available on the contents of test apex. To
prevent special-casing test apexes in the apex validation aspect in
Bazel, drop the test apexes from the tags altogether.
( The core problem I am trying to solve is making sure that stub
libraries in Bazel have a single apex available. apex validation happens
to be a nice side benefit)
Bug: 277651159
Test: go test ./bp2build
Change-Id: Ibb3cfedb5c0f2cda0464bf3758c70b67cb5885d1
The Tags attribute is now available in CommonAttributes, and if we use
that one, we won't get errors with conflicting definitions of the
attribute if a global Tags value is added via CommonAttributes.
Test: m bp2build
Change-Id: I422ed817b84e6808ef60fe3599fe84332bc51d8e
This is just a partial fix.
Bug: 246997908
Test: Manual build //frameworks/proto_logging/stats:libstats_proto_host_proto
Change-Id: I9e4e62bce22cb68bcd7f917c57f3d1438fcec716
For each package in the include_dirs property a proto_library target
should be added to the BUILD file in that package and a mapping
should be added to the bp2build code, by this way a proper dependency
relationship can be established and used by bazel.
Bug: 239944064
Test: Added unit tests and manually verified include_dirs can be
properly converted to bazel and used by bazel to build the targets.
Change-Id: I50d8ee21fabcfec0a44487f6e5f3d8a3845e79c3
Test: go test ./bp2build
Test: b build //frameworks/base/services/tests/servicestests/aidl:servicestests-aidl
Test: enable //packages/modules/NetworkStack/common/networkstackclient
&& disable restriction on Android SDK in javaLibraryBp2Build
&& b build //packages/modules/NetworkStack/common/networkstackclient:ipmemorystore-aidl-interfaces-V10-java
Change-Id: Ifb817daf09a3983ea1c84948ed9f02a79f95784b
Allow java_libraries that depend on protobufs to be converted with
bp2build.
Bug: 215230097
Test: build/bazel/ci/bp2build.sh
Change-Id: I3ce52389e7e4e82755605ee277c1e527a6aebc6b
The logic was inverted from what it should have been,
canonical_from_root --> no strip_import_prefix attribute.
Test: build/bazel/ci/bp2build.sh
Test: build/bazel/ci/mixed_droid.sh
Change-Id: Ic6685d8f0b88279d4444bab3b5e03a544d225f77
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.
Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
The arch variants are hardcoded in every module type. Refactor
them out into a Target.Variations() method in preparation for
splitting the arch mutator into two, which will require using
different variations.
Test: m checkbuild
Change-Id: I28ef7cd5168095ac888fe77f04e27f9ad81978c0
Replace the common pattern of:
cmd.Tool(ctx.Config().HostToolPath(ctx, "tool"))
with:
cmd.BuiltTool("tool")
And similarly for PrebuiltBuildTool.
Test: m checkbuild
Change-Id: I7d63188505362c7df6a3b3e7330b4a2cca5a2409
Add a proto.plugin property to allow specifying a custom protoc
plugin to generate the code.
Fixes: 70706119
Test: m am StreamingProtoTest
Change-Id: I1ecdd346284b42bbcc8297019d98d2cd564eb94c
Using blueprint.Rule for protoc commands was causing code duplication
because there was no good way to run the same protoc for cc, java and
python but then run custom source packaging steps for java and python.
Move most of the code into a common function that returns a
RuleBuilder, and then let java and python add their own commands at
the end of the rule.
Bug: 70706119
Test: All Soong tests
Test: m checkbuild
Change-Id: Ic692136775d273bcc4f4de99620ab4878667c83a
Historically, we've always passed '-I .' as the first argument to
protoc, essentially treating all proto file package names as their full
path in the android source tree. This would make sense in a monorepo
world, but it makes less sense when we're pulling in external projects
with established package names.
So keep the same default (for now), but allow individual builds to opt
into using local paths as the default names with
'canonical_path_from_root: false'. A cleanup effort and/or large scale
change in the future could change the default to false.
As part of this, run protoc once per input proto file, since the flags
may need to change per-file. We'll also need this in order to specify
--dependency_out in the future.
Bug: 70704330
Test: aosp/master build-aosp_arm.ninja is identical
Test: aosp/master soong/build.ninja has expected changes
Test: m
Test: Build protobuf test
Change-Id: I9d6de9fd630326bbcced1c62a4a7e9546429b0ce
The first include directory affects where protoc places the
output files. Ensure -I . is always the first protoc argument.
Bug: 70704330
Test: m checkbuild
Change-Id: I4992e4074f612409865e6e18dc8993c6f68385fd
