Commit graph

17 commits

Author SHA1 Message Date
Usta Shrestha
1c4a3ea497 More helpful error message in test
Test: run aquery_test.go
Bug: N/A
Change-Id: I47920d536cb81d148bf6aa6f9c340e21d1f5be86
2022-06-24 23:03:05 -04:00
Sasha Smundak
f10c3aca7e Fix minor printf format problems.
Test: treehugger
Change-Id: I806ea8d249e40ecb1c6eb7903177ad7213a92753
2022-06-23 18:08:30 -07:00
Usta Shrestha
16ac13506e cosmetic changes
Test: m nothing
Bug: N/A
Change-Id: Ia74a09a6ee42560562e2e1a15e972860e77a5724
2022-06-22 11:02:43 -04:00
Usta Shrestha
2ccdb42098 minor dead code: unused format string
Using URL base64 encoding
(without padding, i.e. with trailing '=' or '==')
hash string width is 256/6 = 43 characters.

file size virtually unchanged,
  out/soong/build.ninja:  7,188,669,041 => 7,186,999,370 ie 1.6M

Bug: N/A
Test: manually verified (e.g. ran `m nothing` successfully)
Change-Id: I166d613e1fd857555da9611d420d6691806571c7
2022-06-02 11:37:20 -04:00
Chris Parsons
0bfb1c0556 Deterministic aquery details in mixed builds
This change constitutes a number of fixes which cause mixed builds to
have deterministic ninja file output:

1. Depsets are identified based on a hash of their contents instead of
   an arbitrary ID integer from Bazel
2. Depset definitions in the ninja file are sorted by the above hashes
3. BuildStatements (action information from Bazel's aquery) are sorted
   by their contents

Test: Ran `USE_BAZEL_ANALYSIS=1 m nothing` three times and verified the
md5sum of out/soong/build.ninja was identical all three runs.
Test: mixed_droid

Change-Id: Iffdf6cc62c31d76fbbfa78726827497516171f4f
2022-05-13 13:45:56 -04:00
Chris Parsons
1a7aca075b Preserve depset structure from bazel aquery
Each depset now corresponds to a phony rule which depends on other
depsets or on full paths; thus, bazel's depset structure is preserved in
the form of phony rules of name bazel_depset_{id}.

Previously, flattening and recopying large lists of file path strings
was quite inefficient. This was particularly evident as we enumerated
hundreds of clang headers for each cc compile action.

This reduces soong_build analysis time by about 30% for mixed builds.
It also reduces ninja file size by ~750MB.

Fixes: 229405615
Test: Unit tests, manually verified metrics, mixed_droid CI

Change-Id: I78df152ac1488ae0c6807afdde4b4ad5e6d26287
2022-04-28 12:44:28 -04:00
Liz Kammer
c31929962b Quote aquery commands.
Bazel run shell actions begin `bin/bash -c <command>`, without escaping,
the command can be treated as an argument to `bin/bash`.

Test: build/bazel/mixed_droid.sh
Change-Id: I423cb393da2e6ac97448ec77b2596f12670dfd31
2021-11-16 17:31:05 -05:00
Wei Li
455ba83a94 Fix the output handling of Bazel aquery for py_binary targets so soong's python_binary_host modules can be converted to py_binary and built with Bazel dependent targets in mixed build.
Test: USE_BAZEL_ANALYSIS=1 m genfunctosyscallnrs
Test: USE_BAZEL_ANALYSIS=1 m func_to_syscall_nrs
Test: USE_BAZEL_ANALYSIS=1 m libseccomp_policy
Bug: 201094425
Bug: 197135289
Bug: 204949078
Change-Id: If98630830d687ca814d6ae09c1e610f2c3a371b1
2021-11-16 20:49:28 +00:00
Liz Kammer
c773778bfe Make bazel action symlinks absolute paths
hardlinks are incompatible with sandboxing
relative links are incopatible with Soong's use of `cp -d`

Test: build/bazel/ci/mixed_libc.sh
Change-Id: I8c776cda6a27c680c51466d9a7af1b499f2f566d
2021-11-04 12:42:52 -04:00
Rupert Shuttleworth
ffc4cc41a3 Revert "Fix python_binary_host module in mixed build."
Revert submission 1861755-fix_python_binary_host_in_mixed_build

Reason for revert: Broke Bazel CI, please see https://android-build.googleplex.com/builds/submitted/7879108/mixed_droid-clean/latest/view/logs/build_error.log and https://android-build.googleplex.com/builds/submitted/7879108/mixed_droid-incremental/latest/view/logs/build_error.log
Reverted Changes:
Ica457ee71:Add platform needed by python_binary_host.
Ibc8b99a92:Fix python_binary_host module in mixed build.

Change-Id: I3d6c4f6edee2d00bdc78efd5cee64f4c915baeaa
2021-11-03 09:07:26 +00:00
Wei Li
664a4fd0af Fix python_binary_host module in mixed build.
Test: USE_BAZEL_ANALYSIS=1 m genfunctosyscallnrs
Test: USE_BAZEL_ANALYSIS=1 m func_to_syscall_nrs
Test: USE_BAZEL_ANALYSIS=1 m libseccomp_policy
Bug: 201094425, 197135289

Change-Id: Ibc8b99a92149410c8a879b7a4facf6c8961a7b9f
2021-11-02 17:04:38 +00:00
Chris Parsons
d80b3c8218 Use hardlinks for Bazel's symlink actions
This prevents failures resulting from using `cp -d` on symlinks.

Test: USE_BAZEL_ANAYSIS=1 m adbd
Change-Id: Ic1756ffe407df81b0e99f99c50bee6af4ecf7796
2021-10-14 15:24:05 -04:00
Liz Kammer
c49e682f37 Handle simple symlinks in mixed builds
Bug: 180945121
Test: build/bazel/ci/mixed_libc.sh
Change-Id: I49fba569a41dcb8cd4c2e58560817443697f58f1
2021-06-09 10:40:32 -04:00
Chris Parsons
c4fb133380 Support middleman actions in mixed builds
This allows support of generated hdrs / srcs in mixed builds.

Test: Manually verified that libc_bionic_ndk passes compilation (failed
previously due to missing generated heaer)
Test: bp2build and mixed_libc CI scripts
Test: New aquery test

Change-Id: I88e359a4bd9eba383c207d5cf812272725ff0a3d
2021-05-18 14:07:44 -04:00
Liz Kammer
de116856fb Add depfile handling for bazel_handler.
Test: go test soong tests
Test: TODO mixed build change header, mixed build
Change-Id: I7c51faf2d5b1a8717cbab6bb0b3eb75c307fcd85
2021-03-30 15:55:02 -04:00
Chris Parsons
943f243bc2 Support aquery depsets with depth
In Bazel aquery responses, Bazel represents action inputs by preserving
the form of the depset data structure which contains them.
(https://docs.bazel.build/versions/master/skylark/lib/depset.html)
Thus, Soong's aquery handler must appropriately "flatten" this
data structure in cases where the depset consists of multiple levels.

Test: m nothing
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I2ebacac1deea7538eb34ad1975594caae71091a2
2021-01-19 21:50:32 +00:00
Chris Parsons
4f069891ae Improve aquery-related validation and error handling
Also introduce test suite for aquery handling

Test: m nothing
Test: lunch aosp_flame && USE_BAZEL_ANALYSIS=1 m libc
Change-Id: I2493d42782099ea0b575968fca38bce6f0d59015
2021-01-15 13:53:57 -05:00