Use the same logic for both go and non-go actions. This also has a
slight performance improvement for go actions by allocating the slice
in advance instead of using append()
Test: Presubmits
Change-Id: I92b60c547d8470816a67ac1ef6e334fd5f547701
See aosp/I5962b27fe31f7103d2af6cef159f11747cff3ed9
where proto classes were used directly making
these structs unused.
Test: go test soong tests
Test: m nothing
Change-Id: Ib452aaa4ce069d882888fc2556ebf3bf7a94c2ca
Since these bazel labels are removed from depsets anyways,
there is little reason to convert they to ninja build statements
Test: m nothing (with bazel af426041)
Bug: 301638491
Change-Id: Ie920477231d147d0b5b7dbcd1c59ed9985a80abb
When an Action fails to convert to a Ninja build statement,
include the bazel label in the error message.
Test: NA
Bug: 301638491
Change-Id: I592978c5e530043769b0232fdd200ad13b72c39b
Tree artifacts aren't fully correct in incremental builds with ninja,
so add a check that we don't write rules using them.
Bug: 293609369
Test: m nothing
Change-Id: I2e49cb6ec24124baf00adb0860f3c1f1f80178bb
Now that aquery includes this information.
Also added rm -f $out to some rules, because since they write files
by shell redirection, if the file existed before and was executable,
they wouldn't make the file non-executable and visa versa.
Fixes: 297366783
Test: m bazel_sandwich
Change-Id: Ie5b6c4275b162601f51deaec9912eea4be16988d
Revert submission 2651299-bazel_sandwich
Reason for revert: b/293883239, checking if this breaks the build.
Reverted changes: /q/submissionid:2651299-bazel_sandwich
Change-Id: I5f1d4bb13d21e6599f5c353dcaba2375f5ec234d
The "bazel sandwich" is a mechanism for bazel to depend on make/soong
outputs. The name comes from the fact that bazel is now at the top
and bottom of the build graph. This is intended to allow us to work
on converting the partition builds to bazel while not all of the
dependencies of the partition have been converted.
It works by adding the bazel_sandwich_import_file rule, which emits a
dangling symlink that starts with bazel_sandwich:, and includes
information that the aquery handler in soong reads. The aquery handler
rewrites the symlink so that it points to a file generated by
make/soong, and adds a ninja dependency from the symlink to the file
it's targeting.
This allows us to depend on make-built files from bazel, but notably
it doesn't allow us to depend on analysis-time information from make.
This shouldn't be a problem for the partitions, but limits the use of
the bazel sandwich to similar, less complicated types of builds.
go/roboleaf-bazel-sandwich
Bug: 265127181
Test: m bazel_sandwich
Change-Id: Ic41bae7be0b55f251d04a6a95f846c50ce897adc
This mnemonic has '' args, which would become literal empty when
combined by strings.Join. To prevent this, add it to the
special-handling function.
Bug: 284483729
Test: N/A
Change-Id: Ibd3c3ebc55284882e7336487319070b9710076d3
The use case for this is for building rules_go's root builder which runs
into issues when built in a directory that contains a symlink to
prebuilts/go
The implementation will involve two changes of working dir
- `sbox` to change the working directory to
__SBOX_SANDBOX_DIR__
- the generated manifest will change the working
directory to mixed build execution root relative to that
Implemenation details
1. Create a unique intermediate path by hashing the outputs of a buildAction.
"out/bazel/output/execroot/__main__/" was deliberately not chosen as
the outpuDir for the sandbox because ruleBuilder would wipe it.
`sbox` will generate the files in __SBOX_SANDBOX_DIR__ and then place
the files in this intermediate directory.
2. After the files have been generated in (1), copy them to
out/bazel/output/execroot/__main__/...
3. For bazel depsets that are inputs of an action, copy the direct
artifacts into the sandbox instead of the phony target
4. Make sandboxing an opt-in. Currently we will only use it for
`GoToolchainBinaryBuild`
In the current implementation, (3) will increase the size of the ninja
file. With sboxing turned on for only GoToolchainBinaryBuild, this will
increase the size of the ninja file by around 1.3% on aosp's cf
Test: m com.android.neuralnetworks (will build soong_zip from source
using rules_go)
Test: OUT_DIR=out.other m com.android.neuralnetworks
Bug: 289102849
Change-Id: I7addda9af583ba0ff306e50c1dfa16ed16c29799
These are essentially FileWrite actions, like SourceSymlinkManifest.
However, currently aquery appears to give non-deterministic results
for the file content. Emit an empty file for now to avoid the
non-determinism.
Fixes: 288292933
Bug: 287539324
Test: Presubmits
Change-Id: Ieaf8aff0d58d2d69087e4e34dc99074a015e700c
GoCompilePkg action's aquery results contains ''. e.g (the interesting
ones)
```
/bin/bulder ... -embedroot '' -gcflags '' -asmflags ''
```
strings.Join would cause it to appear as a whitespace and cause issues.
(in ninja without this CL)
```
/bin/builder ... -embedroot -gcflags -asmflags
```
Convert the empty literal to '' before strings.Join
To limit inadvertent side-effects, this limits it to GoCompilePkg mnemonic
Test: TH
Change-Id: I67161c194dabac6f857ff49b85d4b2471970a9b2
Creation of build statements is largely parallelizable because each
action is independent apart from updates/reads to
depsetHashToArtifactPathsCache. Locally resulted in build statements
taking ~.45 seconds on staging mode to ~.02 seconds
Test: CI
Change-Id: Iab00c8394a9eab17353f71230885ff0870e17f24
This prevents unnecessary copies of data from the query by using a
pointer to the proto or its contents
Test: go test soong tests
Test: m nothing
Change-Id: I5962b27fe31f7103d2af6cef159f11747cff3ed9
Instead of using a sentinel file that was created under bazel_workspace
prune them out. The sentinel file was being created under bazel_workspace,
which would be deleted and thus rendered useless from the perspective of
not retriggering a build.
Test: `touch Android.bp` then `NINJA_ARGS="-d explain" m --bazel-mode-staging adbd_test`
Bug: 265155778
Change-Id: If5d8f0ea7f3b8828fda0646faafd1a621f0cb27c
* Add ApexCqueryInfo to obtain apex artifacts used by the makefile
generator and downstream modules
* Refactor code common to GenerateAndroidBuildActions and ProcessBazelQueryResponse
* Implement android.MixedBuildBuildable for modules
* Enable mixed build for apex modules with payload_type:"image"
The first take 6a2b7c40b was setting compressed APEX suffix incorrectly, and
was reverted in 8a3c91494.
Fixes: 239925080 239695521 232085015
Test: treehugger
Change-Id: I1720f8db3c7cc773183d25a815d9b7eeaf7c73ad
* Add ApexCqueryInfo to obtain apex artifacts used by the makefile
generator and downstream modules
* Refactor code common to GenerateAndroidBuildActions and ProcessBazelQueryResponse
* Implement android.MixedBuildBuildable for modules
* Enable mixed build for apex modules with payload_type:"image"
Bug: 232085015
Test: treehugger
Change-Id: I9f1e03e5e7a5b5dde35a5db10c253069543ac973
Note go.mod file changes go version to 1.18 (for generics)
You might need to refresh/reload your IDE (e.g. IntelliJ)
Bug: N/A
Test: manually verified (e.g. ran `m nothing` successfully)
Change-Id: I2a6dadd673fd170961d1fd1e1756f7581e9a1091
This change will cause bp2build to generate genlex targets any
time a .l or .ll file is present in the srcs for a cc target and
add those genlex targets to the srcs attribute of the original
target.
Bug: 207408632
Test: unit tests
Change-Id: I1bce82c9d3c3d458eae1cef547ffae3d6e975134
This change constitutes a number of fixes which cause mixed builds to
have deterministic ninja file output:
1. Depsets are identified based on a hash of their contents instead of
an arbitrary ID integer from Bazel
2. Depset definitions in the ninja file are sorted by the above hashes
3. BuildStatements (action information from Bazel's aquery) are sorted
by their contents
Test: Ran `USE_BAZEL_ANALYSIS=1 m nothing` three times and verified the
md5sum of out/soong/build.ninja was identical all three runs.
Test: mixed_droid
Change-Id: Iffdf6cc62c31d76fbbfa78726827497516171f4f
Each depset now corresponds to a phony rule which depends on other
depsets or on full paths; thus, bazel's depset structure is preserved in
the form of phony rules of name bazel_depset_{id}.
Previously, flattening and recopying large lists of file path strings
was quite inefficient. This was particularly evident as we enumerated
hundreds of clang headers for each cc compile action.
This reduces soong_build analysis time by about 30% for mixed builds.
It also reduces ninja file size by ~750MB.
Fixes: 229405615
Test: Unit tests, manually verified metrics, mixed_droid CI
Change-Id: I78df152ac1488ae0c6807afdde4b4ad5e6d26287
Bazel run shell actions begin `bin/bash -c <command>`, without escaping,
the command can be treated as an argument to `bin/bash`.
Test: build/bazel/mixed_droid.sh
Change-Id: I423cb393da2e6ac97448ec77b2596f12670dfd31
hardlinks are incompatible with sandboxing
relative links are incopatible with Soong's use of `cp -d`
Test: build/bazel/ci/mixed_libc.sh
Change-Id: I8c776cda6a27c680c51466d9a7af1b499f2f566d
