Previously, it stored an android.Context (no pointer) which while it
worked (because the current contents are themselves pointers) it is
inconsistent with how the rest of the code references it and could
cause weird behavior if additional fields were added to the Context.
Test: m nothing
m bp2build
Change-Id: I9c70f925dde85e5b15431cc232b0c2eb6371ec6d
This is a second attempt at aosp/2273288 which got rolled back because
it turned out that:
1. We make ~120K symlinks in AOSP (!), all of which need to be deleted
2. System calls are sometimes slow
3. Golang spawns a new OS-level thread for each blocking system calls to
keep cores busy
All this together means that we sometimes had 10K system calls in
flight, which meant 10K OS-level threads, which is when Go gives up and
says "I created too many threads, please help".
The fix is to move the system calls into a pool of goroutines, which
soon end up on a pool of threads (since they mostly do blocking system
calls)
Test: Presubmits.
Change-Id: Ia9aefff3b0ed373f09bb6c8b2ec1d8b0f00b213b
Bug: b/256212479
Test: Prior to thi CL bp2build.symlink_forest event was missing in bp2build_metrics.pb after a clean mixed build
Change-Id: I53bfc4114a383c0d1f9c4c7945e7d4c69bc50b0c
Since Bazel's java_import requires a jars attribute to be specified,
the generated neverlink-duplicated module is of type java_library
Change-Id: I14a866dfc583507a9462add50d95060cbfe540c5
Bug: 244210934
Test: m bp2build, go test ./bp2build, manual inspection of generated Build and jar files
This makes symlink forest creation ca. 2x faster again, taking 2-3
seconds instead of 5.
Who would have thought that os.RemoveAll() is slow.
Test: Presubmits.
Change-Id: I91e41319c972dbf1113cf723e383c785433c18b9
This makes it take ~5 seconds on AOSP instead of ~10. Frankly, the
speedup is somewhat disappointing but at least the code is not
complicated.
Test: Presubmits.
Change-Id: Icf94d7ca8bd80c458d014f4cf4cc1be7138deaa6
This helps with incrementality a lot: the symlink forest must depend on
almost every directory in the source tree so that if a new file is added
or removed from *anywhere*, it is regenerated.
Previously, we couldn't do this without invoking bp2build, which is
quite wasteful because bp2build takes way more time than the symlink
forest creation, even though we do the latter in a very suboptimal way
at the moment.
This means that if a source file is added or removed (which does not
affect globs), we don't pay the cost of bp2build anymore.
Also refactored symlink_forest.go on the side. Too much state was being
passed around in arguments.
This change reimplements aosp/2263423 ; the semantics of not touching an
output file is the exact same as order-only inputs and the latter is a
bit fewer lines of code.
Test: Presubmits.
Change-Id: I565c580df8a01bacf175d56747c3f50743d4a4d4
bp2build (i.e. bootstrap.ninja#bp2build_workspace_marker) generate $OUT/soong/bp2build/**/BUILD.bazel files
Having them as a dependency would thus make bp2build_workspace_marker stale upon incremental builds (because we don't re-touch the marker file if it already exists)
See Also: https://android-review.googlesource.com/c/platform/build/soong/+/2263423
Bug: b/253450880
Test: NINJA_ARGS='-d explain' m --bazel-mode adbd
repeat and see if bp2build is rerun
prior to this CL, one would see the input /usr/local/google/home/usta/aosp/out/soong/bp2build/build/make/tools/BUILD.bazel trigger bp2build over and over again
Change-Id: I904cd333a5d6ef506fc4019eda7623ef96a1daa3
Soong supports string properties, but they are overloaded, and can mean
one of three things:
* path reference
* module reference
* string literal
Bazel has different types: label and string attributes. Thus there needs
to be a way to categorize them correctly in bp2build.
This CL introduces a new function to be used on properties like
apex_key.private_key / apex_key.public_key, as well as
android_app.certificate / apex.certificate.
It is important to disambiguate the prop betenn a string literal
attribute or file/rule target label attribute, so this functions does
just that. The new attributes are then further handled by their
respective macros (apex_key, android_binary, apex).
Bug: 253557437
Fixes: 253557437
Test: presubmits, new tests
Change-Id: Id8111cdd60d3aabcae7d17fe9da84d0ee3966023
Also disallow arch variant of genrule.out.
This is to be consistent with bazel where we are migrating to.
Bug: 254114674
Test: Manual
Change-Id: I685a2e64102b7bb68128b39931f0bc85878bc6de
In Soong, decodeMultilib, used to get multilib to determine the dep variations, return "first" if defaultMultilib is set to "common". apex sets defaultMultilib to "common" which means equivalent compileMultilib in bp2build for apex should be "first" (See new Soong unit tests for more context).
This CL fixes bp2build for apex to be more correct.
Bug: 251559512
Test: go tests
Change-Id: Id1cb4407980fc1fab91822c81326f37fb4adfa0a
1) Bp2build convert python_binary_host main attribute as LabelAttribute. Currently "main" attribute in python_binary_host is handled as string but for some modules (e.g certify_bootimg) the "main" attribute points to a file in its subpackage like "subpackage/file.py" and should be converted to "//.../subpackage:file.py".
2) Filter out duplicated labels in the merged list of "required" attributes of python_binary_host and its defaults.
Test: b build //system/tools/mkbootimg:certify_bootimg
Test: b build //build/make/tools/releasetools:check_target_files_signatures
Bug: 253081249
Bug: 253101186
Change-Id: Ic2cb4cadec2c1348da70af9f0730da9914d3a8ca
The module types in scope of this conversion are
1. cc_library and cc_library_shared (non-null llndk or stubs prop)
2. cc_library_headers (all)
For (2), we need some postprocessing on the results of the parser
bp2BuildParseBaseProps. This is necessary because arch and os specific
API exports need to be flattened and added to the generateed API headers
target along NoConfigAxis
e.g.
```
The api equivalent of
cc_library_headers (
name = "lifoo",
deps = select({
"//build/bazel/platforms/arch:arm": ["arm_deps"],
"//build/bazel/platforms/arch:arm64": ["arm64_deps"],
}),
)
should be
cc_api_library_headers (
name = "lifoo",
deps = ["arm_deps", "arm64_deps"],
)
```
For (1), we also need to generate 1:many header api targets so that
arch-specific deps can propagate arch metadata to the top-level
api_domain rule
Test: go test ./bp2build
Test: go test ./cc
Change-Id: Ie40cba1ac8e89f290b3d926c190d5e93abd52859
The merged build files should be added as ninja dependencies,
so we rerun when they're changed.
Fixes: 246552590
Test: m bp2build, m bp2build again and observe it didn't rerun. Then add a comment in external/protobuf/BUILD.bazel, run m bp2build again, and observe it reruns
Change-Id: I26ed035cc0a894500a192f9aa3371fb46519689b
(and remove unused keyName field.)
The private_key and public_key props of an apex_key can point to either
a module or a string. If it's a module, then respect it. If it's a
string, there's additional product variable lookup to find the apex_key
files in product_vars's DefaultAppCertificate parent dir.
This is similar to android_app_certificate.
Test: presubmits
Change-Id: Ib258da14cb0c2df8b5f817fcbc46afebcf225db8
The module types in scope are
1. ndk_library
2. ndk_headers, versioned_ndk_headers
3. api_domain
These modules will no longer generate Bazel targets in bp2build
workspace. A new soong mode `api_bp2build` will be used to generate API
specific targets in a separate Bazel workspace at out/soong/api_bp2build
Test: go test ./bp2build
Change-Id: I01d06b6d8b8364c0d56d6d3f07f8f8bf21db31d7
The generated Bazel workspace will only contain api specific targets.
This is feasible since these targets do not have any cross dependencies
with the targets in the bp2build workspace
The advantages of a new mode are
1. Does not pollute bp2build workspace with api targets
2. Does not block api targets with the current allowlist conversion
mechansims in bp2build
(In the future we might want to combine these two workspaces)
A Soong module type will generate a Bazel target if it implements
ApiProvider interface
Test: m apigen
Test: m nothing
Change-Id: I69c57ca6539f932e0ad554ce84a87fb7936fdba0
But do the splitting before adding $(location...) flags.
Fixes: 247830974
Test: Unit tests
Test: m nothing
Change-Id: I4a7e8e20f720a1074a5ffa17da4fe9b96ca6ed58
Add barebones riscv64-linux-android support. This should be enough
to add riscv64-specific entries to Android.bp files, but can't
actually compile anything until there are riscv64 toolchains.
Test: arch_test.go
Change-Id: I0dcc7e797d9352dd38243be908a7f19004ff3db1
The certificate module is handled the same in Soong between android apps
and apexes, so share the bp2build code as well.
There are a few changes in this CL:
- If override_apex.certificate is unset, the generated apex also unsets
it. This prevents the generated apex from using the base apex's
certificate, which is most likely incorrect (e.g. google variant using
the cert for the aosp variant). Instead, rely on the default
certificate handling in the macro.
- If the certificate prop is a string, then it gets generated into
certificate_name in order to disambiguate. This behavior is identical
to android_app.
Test: added various unit tests.
Bug: 249089160
Fixes: 249089160
Change-Id: I99e18964ff546429a985d0f64dc21e2c69d35d9d
Add it to the `CommonAttributes` struct and handle it in fillCommonBp2BuildModuleAttrs
Bug: 190817312
Test: treehugger
Change-Id: I7b21056e680384d4046372ee844512029721026e
bazel_path conversion would treat "./" as a package boundary
which would be incorrect
Bug: b/243072036
Test: unit test build_conversion_test.go and `m libc`
Change-Id: I0fc44d2fd7a4270442c8755219a11fd585244c5c
