Commit graph

248 commits

Author SHA1 Message Date
David Srbecky
789fe0fdeb Revert^2 "[strip.sh] Move remaining GNU binutils usage to llvm binutils"
This reverts commit 33dfba1d9c.

Bug: 141010852
Test: art/test.py -t 137-cfi
Test: llvm-objcopy --dump-section=.gnu_debugdata=mdi.xz libhwui.so && \
      unxz -fk mdi.xz && readelf -a mdi | less
Change-Id: I751b51274a961194647ec11fd3fd49886df1e1b3
2021-04-08 18:31:46 +01:00
Artur Satayev
ca6fe3453a Add a converter for classpaths.proto between proto and json/text formats.
The proto is used at build time to define classpath fragments to be read
and parsed at runtime by `derive_classpath` service; in order to define
and set BOOTCLASSPATH, DEX2OATCLASSPATH, and SYSTEMSERVERCLASSPATH
environ variables.

The tool is used for debugging and generation of binary protos.

Bug: 180105615
Test: m && launch_cvd with a follow up changes
Change-Id: I7e8b084e5f63e9aaad11da3221b909818e69e235
2021-04-06 18:37:13 +01:00
Yi Kong
6f43f54264 Fix toc.sh, llvm-nm does not recognise '-f P' option
llvm-nm only accepts full name for format option, or just '-P' alias.

Test: build
Bug: 184360305
Change-Id: Ie25f5410403f4dbca564430149a5753e0d48d60d
2021-04-03 03:21:24 +08:00
Yi Kong
948205699b Merge "Move toc.sh to use LLVM binutils" 2021-04-02 02:38:16 +00:00
Ulyana Trafimovich
400a1b1309 Merge "Emit a better error message in manifest_check.py." 2021-04-01 09:58:15 +00:00
Yi Kong
4ad44e713b Move toc.sh to use LLVM binutils
Test: build
Change-Id: Ib965b6ea3fba6ae007135ac7e167cc50f0f6e3de
2021-04-01 17:45:42 +08:00
Yi Kong
9d9e053964 Merge "Move gen_sorted_bss_symbols.sh to use LLVM binutils" 2021-04-01 03:10:08 +00:00
Treehugger Robot
fdfb9540f6 Merge "Do not build stats-log-api-gen-exports in mainline builds" 2021-03-31 18:48:18 +00:00
Ulya Trafimovich
bb7513d80a Emit a better error message in manifest_check.py.
Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Test: temporarily patch Gallery2 to fail the check, observe the error
Change-Id: Id67e359188396f68dcecd8b3f4d1bc26271af56d
2021-03-31 14:31:49 +01:00
Yi Kong
bf1db7b149 Move gen_sorted_bss_symbols.sh to use LLVM binutils
Test: build
Change-Id: I496b92d963d127087daef405dafa24377d6bac97
2021-03-31 17:13:43 +08:00
Sophie Zheng
dd3797b44e Merge "NDK API coverage" 2021-03-30 20:44:44 +00:00
Sophie Zheng
a4c09e1894 Merge "NDK API coverage" 2021-03-30 20:40:41 +00:00
Eric Holk
406765b15a Do not build stats-log-api-gen-exports in mainline builds
This no longer needs to be a prebuilt since stats-log-api-gen can be
built from proto_logging without extra dependencies.

Test: Treehugger
Bug: 170149255
Change-Id: I6925c2526cee1cbff3ba78bedd7f6c2f58dfff0f
2021-03-29 15:53:32 -07:00
sophiez
d3e2742ad2 NDK API coverage
Update OWNERS file under build/soong/scripts to include coverage team as owner of related scripts.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Change-Id: I6150d5bda0d59828a985cd2ab5dd0aa38a7987c3
2021-03-29 20:29:35 +00:00
Ulya Trafimovich
9f12df9abb Don't fail the build if manifest_check cannot extract targetSdkVersion.
Instead, return "any" SDK version 10000, which will cause dexpreopt to
not add any compatibility libraries to class loader context. This is as
good guess as any, and the build system already uses "any" version if
there is not manifest or APK from which to extract targetSdkVersion.

This fixes broken build on cf_x86_auto-userdebug.

Bug: 132357300
Bug: 183943146
Test: lunch cf_x86_auto-userdebug \
  && m out/target/product/vsoc_x86/obj/APPS/playback_intermediates/dexpreopt.zip
Change-Id: I5ee75639ece716c3a9800ddea346fe7c73da7c3a
2021-03-29 15:43:38 +00:00
Ulyana Trafimovich
a8f6db972f Merge changes I1f0ab4af,Iea3be0fc
* changes:
  Fix AAPT path in dexpreopt.
  Allow modules that don't run verify_uses_libraries to have nonempty CLC.
2021-03-29 09:24:00 +00:00
sophiez
ae5da5ba78 NDK API coverage
Update the gen_ndk_usedby_apex.sh script to generate NDK API list with
library name.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Change-Id: I799e03ad7d81a4184229f658fd6c8824b9ba6e1f
2021-03-25 22:12:04 +00:00
Treehugger Robot
c29088b727 Merge "Remove .rustc ELF section from mini-debug-info." 2021-03-23 09:13:53 +00:00
Ulya Trafimovich
bd6b076f80 Allow modules that don't run verify_uses_libraries to have nonempty CLC.
There are modules that would have passed verify_uses_library check, but
don't run it for some reason (the check gets enabled either with an
explicit setting, or if the module has nonempty <ues-library> lists in
the build properties). Previously all such modules were assumed to have
empty CLC, which is not always true. In particular, compatibility
libraries are ignored, which affected e.g. Calendar and messaging apps.

This CL gives such apps a chance to have correct CLC. The goal for the
future is to enforce verify_library_check by default.

Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: Iea3be0fc9d7775c52950848b5a3fd3b7fcd36c53
2021-03-22 10:49:31 +00:00
satayev
2b077baa5e Merge changes from topic "move_allowed_deps_txt"
* changes:
  Treat allowed_deps.txt source file as optional.
  Move allowed_deps.txt to packages/modules/common.
2021-03-22 10:48:45 +00:00
David Srbecky
b474fc0ad6 Remove .rustc ELF section from mini-debug-info.
The section is stripped from binaries (but preserved in /symbols).

However, is it also unintentionally copied into the minidebuginfo
compressed section, which is included in the final shipped binary.
(This does not affect symtab symbols, which are handled as usual)

Removing it from mini-debug-info shrinks some rust binaries by
up to 10x, saving >10MB on the system partition overall (AOSP).

Test: extract minidebuginfo and check it manually with readelf
Change-Id: I462d81455626bb2b5244e1bf3d5f8b71646f3401
2021-03-20 20:31:29 +00:00
Treehugger Robot
f2be52c4dc Merge "Revert "[strip.sh] Move remaining GNU binutils usage to llvm binutils"" 2021-03-20 19:25:23 +00:00
David Srbecky
33dfba1d9c Revert "[strip.sh] Move remaining GNU binutils usage to llvm binutils"
This reverts commit ad50ce8349.

Bug: 183237575
Reason for revert: Breaks stack unwinding

Change-Id: I256dc77525687fb3e670355587b1eac72158b274
2021-03-20 14:46:15 +00:00
Treehugger Robot
8cc04e59e1 Merge "manifest_check.py: translate library names using dexpreopt configs." 2021-03-20 12:12:52 +00:00
Julien Desprez
2dc86b2956 Migrate manifest_*_test out of test mapping
Unit tests do not require an explicit TEST_MAPPING config

Change-Id: Ieccebc78a26b9555803a2278e2962d0b388a6957
Test: presubmit
Bug: 183209529
2021-03-19 17:16:16 +00:00
Yi Kong
ad50ce8349 [strip.sh] Move remaining GNU binutils usage to llvm binutils
With clang r377782, llvm binutils implements all the necessary
funtionaility for strip.sh. We can finally get rid of all the fallback
GNU binutils usage.

Test: m
Bug: 141010852
Bug: 135627985
Change-Id: I110f6028dab7f599decf59a5cb1b927b35e11857
2021-03-19 12:28:44 +08:00
Artur Satayev
fdb61edf43 Move allowed_deps.txt to packages/modules/common.
Bug: 179234385
Test: run update-apex-allowed-deps.sh locally
Change-Id: I8e8864468b87342c688d001bc5f6e6f8416863ed
Merged-In: I8e8864468b87342c688d001bc5f6e6f8416863ed
2021-03-17 13:44:16 +00:00
Ulya Trafimovich
3c902e7acc manifest_check.py: translate library names using dexpreopt configs.
Java modules that are defined in makefiles are not processed in
topological order, so it is necessary to communicate information from
dependencies via dexpreopt.config files. This has already been done in
make/core/dex_preopt_config_merger.py, and now manifest_check.py also
needs to get library names from their dexpreopt.config files.

This is to accommodate Java libraries which name differs from their
modules name. Soong properties `uses_libs`/`optional_uses_libs` and
makefile vars `LOCAL_USES_LIBRARIES`/`LOCAL_OPTIONAL_USES_LIBRARIES`
contain module names, not library names, so it is necessary to translate
them when comparing against library names in the manifest.

Bug: 132357300

Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
      adb wait-for-device && adb root && adb logcat \
      | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
      # empty grep output, no errors

Change-Id: I7cfb17d6d74e37fc9f37d977f15f0d711bab5c8e
2021-03-16 15:43:05 +00:00
Ulya Trafimovich
0aba252e15 Reimplement verify_uses_libraries.sh in manifest_check.py.
Previously there were two different scripts that did similar things:
1) build/soong/scripts/manifest_check.py
2) build/make/core/verify_uses_libraries.sh

Both scripts extracted <uses-library> tags and `targetSdkVersion` from
the manifests of Java modules, but 1) worked for XML manifests, and 2)
worked for APKs. This CL reimplements the functionality from 2) in 1),
so that one script can handle both XML manifests and APKs.

Bug: 132357300

Test: lunch cf_x86_64_phone-userdebug && m && launch_cvd \
      adb wait-for-device && adb root && adb logcat \
      | grep -E 'ClassLoaderContext [a-z ]+ mismatch'
      # empty grep output, no errors

Change-Id: Id1b66e4f3f30f307dba70cb111c7571762cb546a
2021-03-15 13:50:49 +00:00
Martin Stjernholm
4f0f3f01d9 Build heapprofd_client_api for use in the ART chroot.
Test: build/soong/scripts/build-mainline-modules.sh
Bug: 179915934
Change-Id: I4653bf705563ad2ddb077ecae7938c2dd1b818e9
2021-03-09 17:13:51 +00:00
Treehugger Robot
7ebe464748 Merge "Build statsd-module-sdk-for-art" 2021-02-24 01:08:28 +00:00
David Srbecky
bb96b56b0d Merge "Set block-size for XZ mini-debug-info compression." 2021-02-24 00:43:12 +00:00
Eric Holk
ff0529d209 Build statsd-module-sdk-for-art
Bug: 178236337
Test: build-mainline-modules.sh
Change-Id: I51dab13416be8ea1cd65234d560d84ca89b53e13
2021-02-23 22:49:57 +00:00
Treehugger Robot
7eacdbf8a6 Merge "Add non-fatal mode for verify_uses_libraries check." 2021-02-23 20:45:42 +00:00
Paul Duffin
ece454400d Merge changes from topic "hiddenapi_additional_annotations"
* changes:
  Sort hiddenapi monolithic files by signature
  Remove duplicates in monolithic hidden API files
  Remove implicit dependency from <x> -> <x>-hiddenapi
2021-02-23 12:48:24 +00:00
Ulya Trafimovich
8c35fcf4d5 Add non-fatal mode for verify_uses_libraries check.
The new mode is enabled with environment variable
RELAX_USES_LIBRARY_CHECK. If the variable is set to true, then a
verify_uses_libraries check failure does not fail the build, instead it
sets a special compiler filter "extract" for dexpreopt, which means that
the DEX file will be extracted, but it won't be compiled to native code.
Class loader context will be set to empty in this case (not &, as it is
going to be deprecated soon).

If the variable RELAX_USES_LIBRARY_CHECK is unset or set to something
other than "true", then the old behaviour of the verify_uses_libraries
check is preserved.

The intended use case for this flag is to have a smoother migration path
for the Java modules that need to add <uses-library> information in
the build files. The flag allows to quickly silence build errors. This
flag should be used with caution and only as a temporary measure, as it
masks real errors and affects performance.

verify_uses_libraries check is reworked so that it writes the error
message to a status file (which is used instead of the former timestamp
file). Currently the stored error message is not used, but it may be
used later to produce a warning. Dexpreopt command checks if the status
file exists and is nonempty; if that is the case, then compiler filter
is set to "extract".
Bug: 132357300

Test: Manually add some mismatch between the libraries in the Android.bp
      and Android.mk files for dexpreopted apps, build with
      RELAX_USES_LIBRARY_CHECK=true and obsserve that the build doesn't
      fail and they are compiled with compiler-filter "extract".
      Unset RELAX_USES_LIBRARY_CHECK and observe that the build fails.

Change-Id: Ibb5d993a25b1df1d2e70b7d5aafc6997f9d64e67
2021-02-22 15:41:03 +00:00
Anton Hansson
a6bf4a189c Merge "Disallow using error-prone merge strategies" 2021-02-19 15:27:29 +00:00
Anton Hansson
771fd524f2 Disallow using error-prone merge strategies
DO NOT MERGE / RESTRICT AUTOMERGE and its cousin DO NOT MERGE
ANYWHERE are almost always misused. Make it an error to upload
commits with those merge strategies.

Bug: 156896646
Test: repo upload -n (with DNM and RM commits)
Change-Id: Iae3540fa35dbd7b87a14b53a20be0cd23a206f3a
2021-02-19 13:05:54 +00:00
Treehugger Robot
999314fd98 Merge "Add bloaty_merger" 2021-02-19 09:45:39 +00:00
Eric Holk
277b0e7f41 Make statsd apex available as a prebuilt
Bug: 178236337
Test: build-mainline-modules.sh
Change-Id: I807eca7c5296227f4b593fc9f52d29bd20e78645
2021-02-18 17:39:51 +00:00
Thiébaud Weksteen
713db480cd Add bloaty_merger
This script will be used to merge the csv outputs from bloaty into a
single proto file.

Bug: 172339795
Test: atest --host bloaty_merger_test
Change-Id: I9a3c12d7b0c4b5e347fec04e6758bcc1fbc8ab40
2021-02-18 10:25:25 +01:00
Bob Badour
1bf6765def Add license to hiddenapi
Test: m all
Change-Id: Ic61a74d94bea0e4de2a47be7e15fcf874fb397c3
2021-02-17 09:46:26 -08:00
Paul Duffin
2c36f24082 Sort hiddenapi monolithic files by signature
Adds a new --key_field option to merge_csv.py which specifies the name
of the field that should be used to sort the input. If specified it
causes that field to be the first in each row and performs the merge
operation of a merge sort on the input files. That assumes that each
input file is already sorted into the same order.

Modifies the rules that use merge_csv.py to pass in:
    --key_field signature
to sort the rows by signature.

Bug: 180387396
Test: Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change other than
      changing the order.
Change-Id: Idcd5f0fea373b520b604889e1c280f21ed495660
2021-02-16 17:38:08 +00:00
Paul Duffin
031d8693b3 Allow explicitly specified additional annotations for hiddenapi
Adds the hiddenapi_additional_annotations to allow a library to list
the libraries that provided additional hiddenapi related annotations
for a library.

Modifies merge_csv.py so it can process multiple zip files at the same
time and uses that to merge the embedded .uau files from a module and
those it depends upon.

Bug: 180102243
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: I796520021c7357398a9e2a09f1029e4a578b05b3
2021-02-16 13:28:26 +00:00
Eric Holk
7529f06707 Add statsd exports to be consumed by unbundled modules
This is needed first for ART to enable statsd metrics reporting, but
this will be needed by more modules as they become unbundled.

Test: build-mainline-modules.sh
Bug: 178500083
Change-Id: I5af0b8db7a24c86ffd3f34732ac747a4fc6a7a55
2021-02-11 19:02:40 +00:00
Treehugger Robot
b68036f44b Merge "Add LOCAL_LICENSE_KINDS to build/soong" 2021-02-10 07:18:29 +00:00
Paul Duffin
fdada68320 Move hiddenapi tools used by build/soong from frameworks/base
Also, creates a python_binary_host module for generate_hiddenapi_lists
and uses that when constructing the build rule rather than using the
file directly.

Bug: 177317659
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: Ia11bb203ce5a74740d35f1b7e86716e15aad336e
2021-02-09 10:46:36 +00:00
Colin Cross
69f0a24762 Pass list of module libraries to gen_ndk_backed_by_apex.sh
Running find on the entire module directory can lead to flaky failures
when a directory is modified by another rule while find is traversing
it.  Pass a list of libraries in the module into the script instead
of finding them on disk.

Fixes: 178509901
Test: TARGET_BUILD_APPS=com.android.runtime dist
Change-Id: Ie4bf288f0e3d99cbbed5ca8770687a9bbe5377e4
2021-02-08 16:52:07 -08:00
Bob Badour
02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
David Srbecky
412322f2a2 Set block-size for XZ mini-debug-info compression.
This allows the compression to be done on multiple threads
and more importantly, it allows libunwindstack to decompress
only the needed blocks, which reduces unwind time and memory.

Bug: 110133331
Test: verify lazy decompression in libunwindstack
Change-Id: Ia3f597928edcbc85acdf05cb5c83ac7fced39148
2021-02-03 20:48:50 +00:00