Commit graph

253 commits

Author SHA1 Message Date
Treehugger Robot
add0b0880e Merge "Deprecate group_static_libs property" 2021-08-31 16:35:18 +00:00
LaMont Jones
5569d6eb47 Deprecate group_static_libs property
We now default to lld for platform builds, removing all need for this
property.  For more details, see "Effecient archive file handling" in
https://lld.llvm.org/NewLLD.html#key-concepts

Bug: 189475744
Test: Manual build succeeds
Change-Id: If1104d68b13de8c7afab35c1741a68f64394b448
2021-08-30 23:13:03 +00:00
Chih-Hung Hsieh
062e934a9e No clang-analyzer-* checks by default for generated files
Test: make with WITH_TIDY=1 CLANG_ANALYZER_CHECKS=1

Bug: 198098397
Change-Id: I386be0b4ee0fcc1785b2eeb8d1eb26ced68246d7
2021-08-30 15:08:24 -07:00
Chih-Hung Hsieh
fde8e6d387 RBE should not copy source file back to local
Test: make with default RBE variables plus:
WITH_TIDY=1 CLANG_ANALYZER_CHECKS=1
RBE_CLANG_TIDY=true
RBE_CLANG_TIDY_EXEC_STRATEGY=remote_local_fallback

Bug: b/157147559
Change-Id: I1c0106312910919cfda99b816cc8d0f9731b8bd1
2021-08-30 12:49:39 -07:00
Hsin-Yi Chen
79781a94f0 Specify working directory and OUT_DIR to ABI tools
This commit adds `--root-dir . --root-dir $OUT_DIR:out` to the commands
of ABI tools. If the user sets non-default $OUT_DIR, the tools write
"out" instead of the actual $OUT_DIR to the ABI dumps.

Test: export OUT_DIR=~/aosp_out ; \
      make libstagefright_bufferqueue_helper.vendor
Bug: 192157880
Change-Id: I4e2d1b252273106fb6a3ecfa1727d5c91ba03444
2021-07-19 03:46:31 +00:00
Colin Cross
0523ba2675 Rename ninja variables with Clang in the name
Rename the ninja variables that have an extraneous Clang in the name,
since there are no longer any non-Clang variables.

Bug: 68947919
Test: no change to command lines used to build mmma bionic
Change-Id: I081d5e1d423a66d55faece6cb1d265cd5e504d5f
2021-07-14 20:55:23 -07:00
Colin Cross
c465efd5d4 Support multiple CrtBegin and CrtEnd files
Host bionic will use multiple files for CrtBegin, make CrtBegin and
CrtEnd Paths instead of OptionalPath.

Test: go test ./build/soong/cc/...
Change-Id: Ie2e954cd87808a903617696da443009f6173e312
2021-06-11 18:00:04 -07:00
Colin Cross
f04eb99acc Stop injecting symbols into host bionic binaries
The host bionic bootstrapping no longer needs an injected symbol.
Replace host_bionic_inject with host_bionic_verify that validates
the resulting binary, and add it as a validation dependency of the
binary.

Test: build and run host bionic binary
Change-Id: I3e303d2a164b6eef851bdc8075e6ee456c05b0a8
2021-06-11 15:22:41 -07:00
Elliott Hughes
5f314b1613 Rename gccCmd mingwCmd now that's all it's used for.
Bug: http://b/185257607
Test: treehugger
Change-Id: Ida079183828444f122ba6ea5662f63796677de7d
2021-05-18 11:56:02 -07:00
Colin Cross
58dd6a92e5 Add order-only dependency on shared libraries
Improve the dependency accuracy by adding an order-only dependency
on the shared library alongside the existing depenency on the
table of contents file.  The dependency won't cause the module
to be rebuilt when the the shared library changes, but will allow
anything using the ninja graph for sandboxing to know the dependency
exists.

Test: manual
Change-Id: I3accbcabee62fa0ad3eb3d1aaedc5a4bffe27308
2021-05-13 18:01:24 -07:00
Elliott Hughes
afcd339d96 Remove unused parts of the strip implementation.
The --use-gnu-strip option was removed from the shell script already,
and it no longer uses the $CROSS_COMPILE environment variable.

Bug: http://b/185257607
Bug: http://b/147452927
Test: treehugger
Change-Id: If9f7b75d52c1e9cd167d73eeba8f77054ffd63a7
2021-05-06 12:12:43 -07:00
David Srbecky
69315e4ea7 Use create_minidebuginfo tool instead of bash script.
The behaviour is semantically identical, however,
the tool additionally sorts the symbols by address,
compresses frame unwind information more efficiently,
and improves random-accessibility for lazy decompression.

Overall, the changes balance and the output size is same,
however, libunwindstack can access the data much faster
while using less memory (due to the lazy decompression).
It will also enable further improvements in the future.

Bug: 110133331
Test: ART unwinding tests, run prefetto on the device.
Change-Id: Id48f9fe67fb67fcf2b90cc3b217b71bb8f5147ca
2021-05-01 00:06:07 +01:00
Elliott Hughes
2e70f79273 Move builder.go over to llvm-objcopy.
Bug: http://b/147452927
Test: treehugger
Change-Id: I60440e44669d226ca2ef2bcd8d0dbba59f01d43f
2021-04-19 12:58:43 -07:00
David Srbecky
789fe0fdeb Revert^2 "[strip.sh] Move remaining GNU binutils usage to llvm binutils"
This reverts commit 33dfba1d9c.

Bug: 141010852
Test: art/test.py -t 137-cfi
Test: llvm-objcopy --dump-section=.gnu_debugdata=mdi.xz libhwui.so && \
      unxz -fk mdi.xz && readelf -a mdi | less
Change-Id: I751b51274a961194647ec11fd3fd49886df1e1b3
2021-04-08 18:31:46 +01:00
Yi Kong
4ad44e713b Move toc.sh to use LLVM binutils
Test: build
Change-Id: Ib965b6ea3fba6ae007135ac7e167cc50f0f6e3de
2021-04-01 17:45:42 +08:00
Colin Cross
a4eafddc41 Support multiple rsp files in REParams
rewrapper supports a comma separate list of rsp files, replace
REParams.RSPFile with REParmas.RSPFiles.

Test: remoteexec_test.go
Change-Id: I7850c071c23d368d6fad4480dd527d146c13c6d3
2021-03-25 11:06:45 -07:00
Treehugger Robot
f2be52c4dc Merge "Revert "[strip.sh] Move remaining GNU binutils usage to llvm binutils"" 2021-03-20 19:25:23 +00:00
David Srbecky
33dfba1d9c Revert "[strip.sh] Move remaining GNU binutils usage to llvm binutils"
This reverts commit ad50ce8349.

Bug: 183237575
Reason for revert: Breaks stack unwinding

Change-Id: I256dc77525687fb3e670355587b1eac72158b274
2021-03-20 14:46:15 +00:00
Colin Cross
d2db7953a0 Merge changes Iab4e09d9,Icf2f24dd,I15be5ef1,Ic0db9619
* changes:
  Run lint actions in sbox
  Support sbox-in-RBE
  Move android package on top of remotexec
  Support sandboxing inputs in RuleBuilder
2021-03-19 17:25:14 +00:00
Yi Kong
ad50ce8349 [strip.sh] Move remaining GNU binutils usage to llvm binutils
With clang r377782, llvm binutils implements all the necessary
funtionaility for strip.sh. We can finally get rid of all the fallback
GNU binutils usage.

Test: m
Bug: 141010852
Bug: 135627985
Change-Id: I110f6028dab7f599decf59a5cb1b927b35e11857
2021-03-19 12:28:44 +08:00
Colin Cross
77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Treehugger Robot
61cb7ed6ff Merge "Soong: add missing header-abi-dumper inputs" 2020-12-09 07:26:36 +00:00
Liz Kammer
e2861131e6 Add context for invalid extensions for cc compile
Change-Id: I342e8ece56bbe632232d83670d14f9368ed9f0ff
Test: treehugger
2020-12-04 15:19:22 +00:00
Chris Parsons
bf4f55f180 Improve commenting for cc/builer.go, and kill dead code
Test: m nothing
Change-Id: I836c717d3243e901257120be71246e419de4d28e
2020-12-01 18:28:38 -05:00
Ulf Adams
1893176532 Soong: add missing header-abi-dumper inputs
When soong runs the header-abi-dumper remotely, it must provide a few
dynamic libraries as toolchain inputs, or the action will fail
unconditionally on the remote execution system.

This patch allows disabling fallback for the header abi dumper
actions, i.e., this now works with remote execution:
export RBE_ABI_DUMPER_EXEC_STRATEGY=remote

Test: ran against a remote execution server; actions no longer fail remotely
Change-Id: I0c48484c03d0923dae63004fea6632704b317e95
2020-11-25 23:01:15 +01:00
Kousik Kumar
d207cbed65 Revert "Add -fdebug-compilation-dir option"
Revert submission 1461902-debug-compilation-dir

Reason for revert: "-Xclang" isn't being uniformly respected everywhere. For example, in ".S" compilations, when I pass `-Xclang -fdebug-compilation-dir=.", the assembler seems to be ignoring it and then inserting the `pwd` into the command, however when I pass "-fdebug-compilation-dir=.", it strips out the path to the current working directory.
This indicates that we need to update re-client's input processor so that we can pass -fdebug-compilation-dir=. without "-Xclang" and then remove `PWD` setting.
I'll followup with a patch to add -fdebug-compilation-dir.

Reverted Changes:
Ib0f271e55:Add -fdebug-compilation-dir option
Ifa0592af5:Remove env-var-allowlist

Change-Id: I12fa0bf2fb4975ac3d1bffaf1358331548abc339
2020-10-20 09:29:34 -04:00
Kousik Kumar
578ba0021f Add -fdebug-compilation-dir option
The main use of this flag is to be to make both the debug info
and the build command line be independent of the compilation dir.
See: https://reviews.llvm.org/D63387 & https://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html

Removed -fdebug-prefix-map and PWD variable being set in the compile
command line - these were workarounds we originally did to ensure that
the compile output is independent of the working directory and are no
longer needed now that we have -fdebug-compilation-dir.

Test: Ran "m
out/target/product/crosshatch/obj/SHARED_LIBRARIES/libexif.vendor_intermediates/same_vndk_variants.timestamp"
which fails on an RBE hardened stack without this change and passes with
this change.

Bug: b/169266636
Change-Id: Ib0f271e55f6cf0c89a30e65d3a0b25fe3f3a8f48
2020-10-15 15:25:08 -04:00
Treehugger Robot
a6b38f2d3e Merge "Remove InputRootAbsolutePath spec to header-abi-dumper" 2020-09-28 17:45:41 +00:00
Kousik Kumar
3e0b9c031c Merge "Check UseRBE is set before replacing any template with the RE version." 2020-09-10 09:24:17 +00:00
Ramy Medhat
16f23a4038 Check UseRBE is set before replacing any template with the RE version.
Test: presubmit
Change-Id: I6df58b4e700f0d231367af2710672d731d20a736
2020-09-03 01:29:49 -04:00
Ramy Medhat
3618f0a02e Merge "Pass cpp linking implicit deps to RBE." 2020-09-01 11:19:28 +00:00
Matthias Maennich
bea94d3525 Merge "soong: add support for module specific lexer flags" 2020-08-28 19:24:01 +00:00
Ramy Medhat
6797edc70e Pass cpp linking implicit deps to RBE.
Test: build with RBE_CXX=1 RBE_CXX_EXEC_STRATEGY=remote
Change-Id: Ie769b9a99f080b75c1b82e458b5220261e9c1b47
2020-08-28 14:53:44 -04:00
Thiébaud Weksteen
d458745f15 cc: export Stripper struct
The cc stripping logic can be reused for Rust. Export the Stripper
structure for that purpose. Extract the strip-related flags from
builderFlags into StripFlags. Add the method flagsToStripFlags
(similarly to flagsToBuilderFlags).

Add the helper method disableStripping on libraryDecorator.

Test: m
Bug: 153430439
Change-Id: I11aef1abb8d498a4c1672500a7398279edf7f548
2020-08-27 10:13:42 +02:00
Matthias Maennich
22fd4d1b82 soong: add support for module specific lexer flags
To support module specific lexer flags, this follows the same strategy
as the yacc flags:
 - add LexProperties to the BaseCompilerProperties
 - propagate those flags to the generator generation (i.e. genLex)
 - add a placeholder for custom flags
 - replace the placeholder with the concatenated flags

This might not support escaping very well, but I figured that this is a
very edge case. Support for escaping etc. could be added later on.

Bug: 159682555
Signed-off-by: Matthias Maennich <maennich@google.com>
Change-Id: I31a3b783bb05213fe1621191031952b41b318103
2020-08-26 21:41:33 +01:00
Kousik Kumar
118686b4ef Remove InputRootAbsolutePath spec to header-abi-dumper
We used this when doing cache-only for header-abi-dumper to prevent
cross-branch cache hits since abi-dumper had the absolute path of the files
in its output. Since header-abi-dumper no longer outputs absolute paths,
we no longer need to prevent cross-branch cache-hits.

Bug: b/162045672
Change-Id: Iecc3ad9614cda6202cc5f8a9d538f806a79bdf3b
2020-08-24 10:06:35 +00:00
Colin Cross
053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Kousik Kumar
4e30bbaca1 Add remote execution support for clang-tidy actions
Test: Ran with the following command and it succeeded
RBE_CLANG_TIDY="true" RBE_CLANG_TIDY_EXEC_STRATEGY="remote" RBE_CXX_EXEC_STRATEGY="remote_local_fallback" RBE_METALAVA="true" RBE_METALAVA_EXEC_STRATEGY="local" RBE_ABI_LINKER="true" RBE_ABI_LINKER_EXEC_STRATEGY="remote_local_fallback" RBE_CXX_LINKS="true" RBE_CXX_LINKS_EXEC_STRATEGY="remote" use_rbe m out/soong/.intermediates/external/android-clat/clatd/android_arm_armv7-a-neon/obj/external/android-clat/icmp.tidy

Bug: b/157147559
Change-Id: I110b6157fc090abd14ac32330fc59a3d76cdfa82
2020-06-18 09:34:19 -07:00
Ivan Lozano
f3717eec6c Match Rust gcda output to cc via -Z profile-emit.
Use the -Z profile-emit flag to ensure that rust gcda files have
/proc/self/cwd/ appended to them similar to cc modules. This makes sure
our gcda output is consistent no matter what language the binary is
written in.

Bug: 156482307
Test: gcda outputs to the new path.

Change-Id: Ife4d55f885c7e33dffa66f7436bc2bf8b5916586
2020-06-11 17:16:27 +00:00
Martin Stjernholm
391d94c283 Append whole_static_libs deps from .a files instead of the list of
objects.

Necessary to make whole_static_libs work with
cc_prebuilt_library_static since it doesn't propagate the list of
object files.

Test: Build & boot
Test: m libsigchain && \
  ar t out/soong/.intermediates/art/sigchainlib/libsigchain/android_arm64_armv8-a_cortex-a73_static/libsigchain.a
  (Check that the list is sigchain.o followed by async_safe_log.o, both
  in a normal build and in one where async_safe is a prebuilt static
  lib.)
Bug: 154248570
Change-Id: Iaada8490ce713c13804b5771ad606f4a27e72a2f
2020-05-14 17:54:20 +01:00
Ramy Medhat
808594c668 Add support for the remote execution of Abi links.
Test: built crosshatch userdebug with RBE_ABI_LINKS=1
Change-Id: Idb4e15b59878af4a98d9a594127c21deff669e16
2020-05-07 23:55:03 -04:00
Sasha Smundak
a4ef83b252 Canonicalize saved paths, Kythe cannot handle symlinks
Bug: 141385766
Test: build kzip
Change-Id: I87c4af719135ee89298d160bb8d6395ad7d6046c
2020-05-03 18:30:13 -07:00
Kousik Kumar
3fb6126309 Add implicit outputs to link actions
Change-Id: I9b6ae97f5bbfe8f9c4b9c089544a768982794f32
Bug: b/154733231
Test: Built the failing action with / without this change
2020-04-27 21:53:43 -07:00
Treehugger Robot
867dfd52f5 Merge "Add support for remotely executing header-abi-dumper actions." 2020-04-27 19:16:41 +00:00
Ramy Medhat
31ec942ff7 Add support for remotely executing header-abi-dumper actions.
Test: built with and without RBE_ABI_DUMPER set.
Change-Id: Ie752c07325453076d191813eaa50da03e3a0c2d8
2020-04-27 08:13:42 -07:00
Hsin-Yi Chen
9105fa385a Merge "Add check_all_apis option for header ABI checker" 2020-04-27 03:34:08 +00:00
Oliver Nguyen
0452678a40 Only package gcno files for gcov coverage builds.
Bug: 154550223
Test: m -j NATIVE_COVERAGE=true droid dist tests
Test: m -j CLANG_COVERAGE=true droid dist tests
Merged-In: I81598bcab8db105de6692156c001fc961409ce63
Change-Id: I81598bcab8db105de6692156c001fc961409ce63
2020-04-22 13:30:07 -07:00
Ramy Medhat
9a90fe5e23 Allow remote execution of link actions.
This CL adds a remoteexec package that allows adding a configurable RBE
prefix to the template.

Test: built aosp crosshatch userdebug with and without RBE_CXX_LINKS.
Change-Id: Ica920c3d7f79f2996210b9cbd448126451c1707c
2020-04-16 21:18:10 -04:00
Logan Chien
2a65dda27d Add check_all_apis option for header ABI checker
This commit adds a header ABI checker option to check all APIs that can
be found.  Without this option, the checker by default only checks the
class/struct/enum that are directly or indirectly referred by one of the
exported symbols.  With `check_all_apis: true,`, the checker will check
all class/struct/enum.

Bug: 141709599
Test: Add `header_abi_checker: { check_all_apis: true, }` to a library
      and see breakage if I change some enum.
Change-Id: I61f90e07b60a6752fc6be4398420c1ad1291102f
2020-02-26 09:25:40 +00:00
Kousik Kumar
2976bfd568 Do not add ccWrapper to ccNoDeps rule
The "ccNoDeps" rule was introduced to separate out ".s" compilations
from other ".S" and ".c*" compilations. The ".s" compilation does not
produce a dependency file and does not support running a preprocessor
through it.
However, it does have ".include" directives, that do NOT take a macro,
but still does the equivalent of a "#include". The compilation of ".s"
assembly files also do NOT produce a dependency file.

Because they don't produce a dependency file AND because RBE's input
processor does not yet support finding dependencies for these files, I'm
making this change to not prepend rewrapper for these commands.

Test: Tested by running a build with this change and ensuring the ".s"
compilations aren't being sent to rewrapper.

Change-Id: I60bb14ff92596e4992e9f675bdc199f1440d4327
2020-02-18 08:00:17 -08:00
Treehugger Robot
1562752fc6 Merge "build: soong: windres: use clang to preprocess" 2020-01-30 02:21:02 +00:00
Nick Desaulniers
18eeffa4ce build: soong: windres: use clang to preprocess
Otherwise binaries under:
prebuilts/gcc/linux-x86/host/x86_64-w64-mingw32-4.8/bin/
are being invoked.

https://sourceware.org/binutils/docs-2.33.1/binutils/windres.html
states:
When windres reads an rc file, it runs it through the C preprocessor first.

Use the `--preprocessor` flag to specify the exact command to run. The
args to the preprocessor get wiped out when setting that flag, so
`-E -xc-header -DRC_INVOKED` must be re-passed, else clang will error as
it tries to invoke ld.

Once AOSP packages llvm-rc, aosp/1206346 can be rebased on top, so this
patch should be a short lived change.

Bug: 147295872
Test: <remove mingw gcc> && lunch aosp_x86-eng && \
  cd external/mdnsresponder && mm
Change-Id: Ic0e6b75e7e964ee9f9757e534cddd16438604c34
Suggested-by: Pirama Arumuga Nainar <pirama@google.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2020-01-29 16:28:06 -08:00
Ramy Medhat
8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Sasha Smundak
6c2d4f91f8 Save compilation units in protobuf format.
Bug: 146224091
Test: manual
Change-Id: I7a856bab13f54c78efa0061421c3fcb9341bc6e7
2020-01-15 12:40:03 -08:00
Colin Cross
1f38237c12 Merge changes from topic "reorder-cflags"
* changes:
  Reorder local and global flags
  Split local and global cflags
2019-11-08 05:07:34 +00:00
Colin Cross
6d88dbad0e Reorder local and global flags
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags

This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags.

The previous CL split the global and local flags into separate
variables.  Rearrange the order that the variables are applied
to be:
global cflags
global conlyflags
global cppflags
local cflags
local include dirs
local conlyflags
local cppflags
global include dirs

Bug: 143713277
Test: m native

Change-Id: I171524ab40096a636a8e549e1e4bc3347ef9f97a
2019-11-07 15:27:58 -08:00
Colin Cross
4af21ed26f Split local and global cflags
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags

This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags.  A better order
would be:
global cflags
global conlyflags
global cppflags
local cflags
local conlyflags
local cppflags
local include dirs
global include dirs

We are mixing both the global and local cflags into a single
variable, and similar for conlyflags and cppflags, which
prevents reordering them.  This CL prepares to reorder them
by splitting the global and local cflags into separate variables.

Bug: 143713277
Test: m native
Change-Id: Ic55a8c3516c331dc5f2af9d00e59ceca9d3e6c15
2019-11-07 15:27:58 -08:00
Ramy Medhat
dd0418a4d4 Run non-RBE supported actions in the local pool when USE_RBE is set.
Bug: 143938974
Test: ran CTS build at -j500 successfully.
Change-Id: I55074bd67308cd716972e24fb56a20bc393d5d9d
2019-11-05 22:57:35 +00:00
Yi Kong
c49c393f73 Repack libgcc.a to only include required objects
Previous solution by using objcopy uses a quirky behaviour of the GNU
objcopy and there is no equivalent option in llvm-objcopy.

Instead of removing symbols, extract and repack libgcc to only include
required objects.

Bug: 142585047
Test: presubmit
Change-Id: I58af74c18838f797e481da38c3265f0624fddf99
2019-10-24 16:34:54 -07:00
Colin Cross
2e2dbc250a Use localPool consistently for UseGoma() == true
Remove the distinction between pctx.StaticRule and
pctx.AndroidStaticRule so that all of the local rules correctly
get assigned to the localPool.  Also put Module and Singleton
rules into the localPool.

Test: compare out/soong/build.ninja
Change-Id: Id2bb38eff3c7209340fe55bc9006f00bd3661d81
2019-10-01 14:11:20 -07:00
Sasha Smundak
651436460a Pass filename mappings to C++ and Java extractors.
Android builds by default put artifacts into out/ subdirectory of
the source tree, causing the extractor to record their names as
relative. The indexer considers such files as sources, which is wrong.
Fortunately, the extractor can be fed a set of filename rewriting
rules (see build/tools/vnames.json).
Also, undo previous unsuccessful attempt use to absolute path for the
output directory to distinguish between source code and artifacts.

Bug: 141385476
Test: run the build, inspect compilation units of the kzip file
Change-Id: I89ec3aed8fd14f43ea6e0b226d54f643346f6125
2019-09-27 10:28:11 -07:00
Dan Willemsen
724ab5d5f7 Fix missing deps found through RBE
Bug: 130111713
Test: run with RBE, no longer see error about file not found
Change-Id: Ib6192f2a537f49efdb69b3f3bf28aef1660dec01
2019-09-19 10:51:03 -07:00
Yi Kong
ee96a7956a Add back the llvm-ar P flag
The P flag is supported as of llvm r354044, add back the flag to reduce
the size of intermediate archieve files. This does not affect the final
binaries.

Test: built
Bug: 71618641
Change-Id: I017780e4dcaa31c7fbe10b5e7482db1bba83e716
2019-09-06 15:11:57 -07:00
Dan Willemsen
98ab311767 Save deps when asflags contains -xassembler-with-cpp
Usually, ".S" files are processes with the c preprocessor, and ".s"
files are not, so they don't have any dependency information, since it
is generated by the preprocessor.

But with the -xassembler-with-cpp flag, ".s" files are processed with
the preprocessor, so we should ask for dependency information from them.

Test: NINJA_ARGS="-t deps out/soong/.intermediates/external/sonivox/arm-wt-22k/libsonivox/android_arm_armv7-a-neon_core_static/obj/external/sonivox/arm-wt-22k/lib_src/ARM-E_filter_gnu.o" m
Test: treehugger
Change-Id: Iee7baeebc2b205b5a2f33e7c1705ea4a5b4fc95a
2019-08-27 21:20:40 -07:00
Treehugger Robot
792942de47 Merge changes I75b4a761,I779f28c6,If1422372,I26307dd1
* changes:
  Introduce inject_bssl_hash library property.
  BoringSSL FIPS build - introduce extraLibFlags and use for STL libs.
  Allow linker scripts when building objects.
  Allow .o files as srcs.
2019-08-23 00:15:08 +00:00
Pete Bentley
99f2fc27e6 BoringSSL FIPS build - introduce extraLibFlags and use for STL libs.
Rationale: On non-bionic, stl.go currently adds system libraries to
ldFlags, this causes problems for partialLd rules.  However adding the
same libraries to libFlags breaks some existing modules due to symbol
conflicts as the system libraries are linked before some module code.

Introduced a general mechanism for adding libraries to be linked
last rather than making this STL-specific.

Bug: 134581881
Bug: 137267623
Test: TH
Change-Id: I779f28c6586b3fea85cc6299b686e4fde95262d3
2019-08-22 13:21:37 -07:00
Pete Bentley
fcf55bf656 Allow .o files as srcs.
Test: m nothing
Test: TreeHugger
Bug: 134581881
Bug: 137267623

Change-Id: I26307dd1129e58878f0468da3b61c53f074bd674
2019-08-22 13:21:37 -07:00
Hsin-Yi Chen
eef3366ab3 Merge "Add tags to the list of lsdump paths" 2019-08-15 02:20:30 +00:00
Dan Willemsen
6b4419ccc7 Include dependencies for clang-tidy and header-abi-dumper
We've been getting these dependencies transitively through the
dependency on the object itself (which is a workaround for the lack of
dep file support in these tools). But for remote builds to work, we need
to know about these dependencies like any other object compilation.

For regular builds, this increases the size of the ninja file by a few
tens of megabytes (~1-2%).

WITH_TIDY builds were already larger (~40-50%), but are now about 90%
larger than a normal build.

Test: treehugger
Change-Id: Icdb4ca3d4d08d5706593d96d5c627149fa14fed8
2019-08-09 13:03:07 -07:00
Hsin-Yi Chen
5348964723 Add tags to the list of lsdump paths
This commit adds tags, such as NDK, VNDK-core, and PLATFORM, to
LSDUMP_PATHS. The script updating the reference ABI dumps uses the tags
to determine the directories where the dumps should be created.

Test: make findlsdumps
Bug: 133176785
Change-Id: I8540286238cf0ec55c65e1c4f60cb9c12e5e57a1
2019-08-07 13:40:11 +08:00
Sasha Smundak
2a4549ec98 Support source code cross-referencing for C++ and Java
Use Kythe (https://kythe.io) to build cross reference for the Android
source code. ~generate the input for it during the build. This is done
on demand: if XREF_CORPUS environment variable is set, build emits a
Ninja rule to generate Kythe input for each compilation rule. It
also emits two consolidation rules (`xref_cxx` and `xref_java`),
that depend on all Kythe input generation rules for C++ and Java.

The value of the XREF_CORPUS environment variable is recorded in the
generated files and thus passed to Kythe. For the AOSP master branch it is
`android.googlesource.com/platform/superproject`, so the command to build
all input for Kythe on that branch is:
```
XREF_CORPUS=android.googlesource.com/platform/superproject m xref_cxx xref_java
```

Each Kythe input generation rule generates a single file with .kzip
extension. Individual .kzip files have a lot of common information, so
there will be a post-build consolidation step run to combine them.
The consolidated .kzip file is then passed to Kythe backend.

The tools to generate .kzip files are provided by Kythe (it calls them
'extractors'). We are going to build them in toolbuilding branches
(clang-tools and build-tools) and check them in as binaries into master
and other PDK branches:
For C++,  `prebuilts/clang-tools/linux-x86/bin/cxx_extractor`
for Java, `prebuilts/build-tools/common/framework/javac_extractor.jar`

Bug: 121267023
Test: 1) When XREF_CORPUS is set, build generates Ninja rules to create
.kzip files; 2) When XREF_CORPUS is set, building
`xref_cxx`/`xref_java` creates .kzip files; 3) Unless XREF_CORPUS is
set, build generates the same Ninja rules as before

Change-Id: If957b35d7abc82dbfbb3665980e7c34afe7c789e
2019-07-26 09:16:47 -07:00
Peter Collingbourne
84d8385ae1 Use LLD for partialLd.
Clang will soon start using relocations that are currently unsupported by
the version of ld.bfd that Android uses when HWASAN is enabled. This will
cause partialLd to fail since the clang driver defaults to ld.bfd.

Switch to using LLD, since it will support those relocations.

Change-Id: I0964f53dc63773f3b52bb377f863df9c39961a07
2019-07-16 14:14:36 -07:00
Colin Cross
d34ab7c6b2 Treat .cxx files as c++
Fixes: 135749365
Fixes: 135667566
Test: m checkbuild
Change-Id: If47274812dfe781ebbb9f8a60a395650573727d5
2019-06-27 14:46:10 -07:00
Yi Kong
823532df31 Move darwin to llvm-ar
Test: build
Bug: 71618641
Change-Id: Ib45963bc9e295afe39752a270f7b5a1197335c69
2019-06-12 19:16:25 -07:00
Treehugger Robot
7268fd494d Merge "Package coverage files as a zip." 2019-06-12 02:00:01 +00:00
Oliver Nguyen
c743414d42 Package coverage files as a zip.
Test: make NATIVE_COVERAGE=true COVERAGE_PATHS="*"
Change-Id: I955212a20ea19c854ebc88635252a1d174ea8f7f
2019-06-11 16:04:11 -07:00
Josh Gao
75a50a2c15 Revert "Revert "Revert "Revert "Enable lld for windows""""
This reverts commit b20200b3f5.

Bug: http://b/110800681
Test: manual
Change-Id: Id3eb45ed29f3977fef3da241ec2c43755116d60c
2019-06-10 12:59:11 -07:00
Josh Gao
b20200b3f5 Revert "Revert "Revert "Enable lld for windows"""
This reverts commit 6d8c0a50bc.

The switchover to lld appears to be making adb crash on exit for unclear
reasons. Revert it as a quick fix pending further investigation.

Bug: http://b/134613180
Test: adb.exe under windbg
Change-Id: Ibdf5981c3c6828b3684974b7c30a7c96449c64c5
2019-06-07 12:34:18 -07:00
Colin Cross
badf8d691f Cut darwinStripPool from 10 to 5
Failures in strip.sh are still occurring with darwinStripPool set to
10, try 5.

Bug: 132822437
Test: none
Change-Id: I2df247c89b6d8f6e741d4e17057b06ff9ec72b40
2019-05-23 02:09:07 +00:00
Christopher Ferris
8a7469a0d7 Merge "Add an option to preserve symbols and debug_frame." 2019-05-21 18:09:08 +00:00
Christopher Ferris
b43fe7a838 Add an option to preserve symbols and debug_frame.
New strip option named keep_symbols_and_debug_frame, that will keep the
symbols and the .debug_frame. This is meant for use by libc.so only on
arm32. Other libraries might want to use it to keep better unwinding
information on device.

Bug: 132992102

Test: Built libc.so with this option and verified that it contains
Test: the .debug_frame section.
Change-Id: I823a28199dec8316e8b26fe31ff9f17e6b11d406
2019-05-17 17:52:18 -07:00
Colin Cross
ee3ea31a24 Limit calls to strip.sh on darwin
strip.sh can use a file descriptor per .o file when run on .a files,
which can hit the system file descriptor limit on darwin.  This
causes failures when manay variants of libgcc_stripped are built
simultaneously.  Put all strip rules on darwin into a pool that
limits them to 10 concurrent processes, which will limit the file
descriptor usage to ~7500.

Fixes: 132822437
Test: no mention of darwinStripPool in out/soong/build.ninja on linux
Test: m libgcc_stripped on darwin
Change-Id: I3d4fbbd8d44d2e9059a79df113ab95336ec2c658
2019-05-17 22:51:00 +00:00
Yi Kong
acee27cd72 Strip libgcc to only keep fallback symbols
We use libgcc as fallback for symbols not present in libclang_rt
builtins, however we didn't know what exact symbols were being used,
some may not be intended to fallback.

Create libgcc_stripped, which only contains unwind symbols from libgcc.

Bug: 29275768
Test: bionic-unit-tests
Change-Id: I5b349fa6138e51663bf3b67109b880b4356da8e8
2019-04-30 10:07:00 -07:00
Pirama Arumuga Nainar
6d8c0a50bc Revert "Revert "Enable lld for windows""
This reverts commit 61166dc047.

One difference from the earlier change is that import libraries are now
using the '.lib' extension instead of '.a' to prevent clash with
AdbWinApi.a.

Bug: http://b/110800681

The following flags that the binutils linkers support are not
available in lld for Windows:
  -soname
  --no-undefined
  -rpath

Windows also uses "import libraries", which are stub libraries used only
for linking.  The binutils linkers accepted a DLL and treated them as an
import library.  But lld issues the following error:

  lld-link: error: ...DLL: bad file type. Did you specify a DLL instead
  of an import library?

To resolve this, pass '-out-implib=libFoo.lib' to lld when linking
libFoo.dll to get lld to generate an import library.  Add libFoo.lib as
an implicit output to the 'ld' build rule.

Rewrite the shared libraries when building a library/binary to use the
import library instead of the DLL.  As a side-effect, this also uses the
newly-created AdbWinApi.lib that's alongside
development/host/windows/prebuilt/usb/AdbWinApi.dll

Test: Run Windows tests (go/android-llvm-windows-testing) and check
absence of regressions.  Also check that the following commands pass:
      $ adb.exe devices
      $ fastboot.exe devices

Change-Id: I34e07d345e0207086ac8e8ea12525d8c322b20fd
2019-04-16 16:22:03 -07:00
Dan Willemsen
4e0aa23dd3 Convert yacc to a single RuleBuilder rule
So that <module>/gen/yacc/... is (re)created by a single rule, previous
files are removed, and location.hh is in the build graph when it is
produced.

Test: treehugger
Change-Id: I2f6e47ea07f315e10ae1cb8ad50697e7123d0285
2019-04-15 14:52:05 -07:00
Logan Chien
62f1f9488e Add --llndk for NDK libs as well
This commit add `--llndk` to the command line option for
`create_reference_dumps.py` as well.

Bug: 124620774
Bug: 130324828
Test: lunch aosp_arm64-userdebug && make
Change-Id: I117864209b8cf8c934fd3aa1086b7691c1c449e3
2019-04-11 05:40:57 +00:00
George Burgess IV
c4624c0189 Remove our clang-tidy wrapper
Looks like the reason for it existing has been fixed. It should probably
just be removed.

Bug: None
Test: WITH_TIDY=1 m

Change-Id: I770b2fec4ac44f265ff31731c9c0bd4da14d5b0f
2019-04-04 17:58:00 -07:00
Josh Gao
61166dc047 Revert "Enable lld for windows"
This reverts commit 3c21c0b1d9.

Reason for revert: appears to break AdbWinUsb.dll usage by fastboot (and adb?)

Bug: http://b/129420419
Bug: http://b/110800681
Change-Id: I36d6d87f2e830e08e257ab42d470e49cdbb95819
2019-04-03 00:59:14 +00:00
Colin Cross
fe17f6f0e8 Add support for protoc plugins
Add a proto.plugin property to allow specifying a custom protoc
plugin to generate the code.

Fixes: 70706119
Test: m am StreamingProtoTest
Change-Id: I1ecdd346284b42bbcc8297019d98d2cd564eb94c
2019-04-02 16:38:55 +00:00
Colin Cross
19878da6a0 Move proto compilation to RuleBuilder
Using blueprint.Rule for protoc commands was causing code duplication
because there was no good way to run the same protoc for cc, java and
python but then run custom source packaging steps for java and python.
Move most of the code into a common function that returns a
RuleBuilder, and then let java and python add their own commands at
the end of the rule.

Bug: 70706119
Test: All Soong tests
Test: m checkbuild
Change-Id: Ic692136775d273bcc4f4de99620ab4878667c83a
2019-04-02 16:38:47 +00:00
Colin Cross
36ae135434 Restat clang linker outputs
clang -Wl,--out-implib doesn't update its output file if it hasn't
changed, always restat the outputs.

Fixes: 129553377
Test: m checkbuild && m checkbuild
Change-Id: I2cb19e1b8280fd69c8a9b2143f55760ec05630f8
2019-03-30 00:03:24 +00:00
Pirama Arumuga Nainar
3c21c0b1d9 Enable lld for windows
Bug: http://b/110800681

The following flags that the binutils linkers support are not available
in lld for Windows:
  -soname
  --no-undefined
  -rpath

Windows also uses "import libraries", which are stub libraries used only
for linking.  The binutils linkers accepted a DLL and treated them as an
import library.  But lld issues the following error:

  lld-link: error: ...DLL: bad file type. Did you specify a DLL instead
  of an import library?

To resolve this, pass '-out-implib=libFoo.a' to lld when linking
libFoo.dll.  Add libFoo.a as an implicit output to the 'ld' build rule.

Rewrite the shared libraries for a library/binary to use the import
library instead of the DLL.  As a side-effect, this also (correctly)
uses the AdbWinApi.a that's alongside
development/host/windows/prebuilt/usb/AdbWinApi.dll

Test: Run Windows tests (go/android-llvm-windows-testing) and check
absence of regressions.

Change-Id: I15a178589aa6882caa6e7e38650cc6ef48109764
2019-03-20 21:07:47 +00:00
Logan Chien
6227fed029 Print --llndk to create_reference_dumps command
This commands adds `--llndk` to `create_reference_dump.py` command if
the module with incompatible ABI changes is an LLNDK or a NDK lib. This
prevents people from uploading a CL that simply removes ABI dumps (the
result even when the uploader runs the command).

Bug: 124620774
Test: Make an incompatible ABI changes in libaaudio and see the flag.
Change-Id: I9f07f879193bd4d73f5924baff298d28d6fe1506
2019-02-18 13:35:36 +08:00
Yi Kong
b5c34d7f40 Use llvm-{strip,objcopy} by default
... except for Darwin Mach-O, as it is not supported by llvm-strip.

Test: m checkbuild
Bug: 119221035
Change-Id: I021637b6dd3530bff1f563f2ec7c2168e1083b7e
2019-02-04 18:20:28 +08:00
Logan Chien
e3d7a0d062 Add a header_abi_checker section
This commit adds a header_abi_checker section so that the library owner
can have a fine-grained control over the ABIs that must be checked.

For example, a library "libexample" may have following configurations:

    cc_library {
        name: "libexample",
        header_abi_checker: {
            symbol_file: "libexample.map.txt",
            exclude_symbol_versions: ["LIBEXAMPLE_PRIVATE"],
            exclude_symbol_tags: ["platform", "apex"],
        },
    }

Bug: 122845490
Test: Add header_abi_checker to libc to filter out LIBC_PRIVATE
Change-Id: I60cfea868f815afe6213c242ed0ca818161d55c6
2019-01-30 22:18:59 +08:00
Logan Chien
8f74fe6e14 Fix a typo related to abidiffs
This commit fixes a typo related to abidiffs.  The generated abidiffs
should be copied to `$$DIST_DIR/abidiffs` instead of
`$$DIST_DIR/abidiff`.

(See also. https://android-review.googlesource.com/796458/)

Bug: 123491909
Test: lunch aosp_x86-userdebug && make dist
Change-Id: I89c13580344a27cac5ea0d00497f5bba3227cf61
2019-01-28 14:30:04 +08:00
Dan Willemsen
fcabb1c518 Don't expect depfile from .s files
.s files (unlike .S files) aren't run through the preprocessor, so -M*
doesn't actually write out a depfile.

Since our ninja is now going to be verifying that the depfile is created
(https://android-review.googlesource.com/861510), don't specify a
depfile for .s files.

Bug: 121058584
Test: apply https://android-review.googlesource.com/861510
Test: cd external/libavc; mma
Change-Id: I1697aa020c63639317c8f4771147026601ae72fc
2019-01-03 23:25:11 -08:00
Dan Willemsen
60e62f0c44 Add nanopb-c support
Test: cd hardware/ril; mma
Change-Id: Id1481940d15a2a3f6eb29af54ee30080ff2286cb
2018-11-17 15:28:25 -08:00
Peter Collingbourne
abde584fd1 Stop passing the gold plugin path to llvm-ar.
llvm-ar takes a --plugin argument but it is ignored for compatibility,
so passing it has no effect.

Change-Id: I4fc51d226d66cf2a43462d3d4ccc12e6e5ebb226
2018-11-08 20:13:16 -08:00
Dan Willemsen
2d31a44b8a Stop using DIST_DIR in Soong
We're only using it to distribute files in case of failure, which isn't
well supported currently, but can be handled for now by using the
DIST_DIR environment variable during the command execution.

This was at least one cause that we'd be re-running Soong during every
build server build, as the DIST_DIR values are unique.

Test: m dist
Change-Id: Ibd5e6b6c46695350de80b745bfb6a6aa685033a0
2018-10-20 21:33:41 -07:00