On second thought, this is actually counter-productive rather than an
optimisation. LTOing test code is usually quite cheap, however since
tests can pull in additional static libraries (e.g. to set up test
environment), we will have to create non-LTO variants for these
dependent libraries, which can be much more expensive.
This reverts commit 85d7297318.
Bug: 195134194
Bug: 203737712
Change-Id: I1a105136075133a5b5895d00847d8796c419964c
Test: m GLOBAL_THINLTO=true dist device-tests platform_tests
There's not a great return on investment for spending additional
compilation/link time on tests that may only be run once or very few
times (as compared to actual library code that ships and is run orders
of magnitude more times).
Bug: 195134194
Bug: 203737712
Test: m GLOBAL_THINLTO=true dist device-tests platform_tests
Change-Id: I7149c4140f7c52b22bedbe322f1bf2c81f7ad66b
Instead of making everything into ThinLTO variant by default (it works
but many Soong tests don't like this, and got bit rot due to lack of
active builder for this configuration), let the default option be
ThinLTO and no LTO be a special variant.
Test: m GLOBAL_THINLTO=true
Test: m
Bug: 195134194
Change-Id: I2fd98061ba55eba1fdfdd056fb2f8c2051fd2553
With global ThinLTO enabled, vndk version has different symbol ordering
from the platform version. Opt out any binaries that has vndk enabled as
a temporary workaround.
Test: GLOBAL_THINLTO=true m
Bug: 169217596
Change-Id: I75b060cbe6c74421d283c6dfbd669af20f466d1f
For global ThinLTO, don't implicitly turn on LTO for static libraries,
but instead rely on mutator to generate correct variants.
Bug: 169004486
Test: GLBOAL_THINLTO=true m
Change-Id: I9cdeea706ec6dd4ad31f55b9e12a96b42176aa89
Experimental global ThinLTO mode for compilation time evaluation.
Bug: 169004486
Test: GLBOAL_THINLTO=true m
Change-Id: I2ff06397be8b65c7972f1e5f09ab1496949567ee
We previously disabled inlining and unrolling completely during ThinLTO
in absense of PGO profile. For global ThinLTO, we want to better balance
binary size and performance.
We evaluated a number of combination of heuristics with global ThinLTO
configuration:
binary size change
no LTO baseline
no inline, no unroll -0.54%
no inline, unroll -0.50%
import-instr-limit=5, unroll +0.02%
import-instr-limit=10, unroll +0.13%
Loop unrolling does not contribute much to the binary size, therefore
it is re-enabled.
import-instr-limit=5 balances the binary size savings from ThinLTO and
size incrase due to additional optimisation.
Bug: 78485207
Bug: 169004486
Test: TreeHugger
Change-Id: I1c21153605e2ae42daa8857b06e27c081ee8ad85
-fwhole-program-vtables cflag requires LTO to be enabled. Some projects
add this flag unconditionally, breaking builds that turn off LTO.
Make this an LTO property so that build system will handle adding the
flag when LTO is on.
Test: build
Bug: 169004486
Change-Id: Ifce12c09e1d6b8c2807314c3bb982b076efcaaa4
The implementation before libraryDependencyTag was added failed to
treat the libgcc_unwind dependency as a static library when traversing
dependencies for LTO because it didn't handle staticUnwinderDepTag.
Bug: 162437057
Test: m checkbuild
Change-Id: Iccc96c0daa0379f51eb975c51fd5e8c507a0daba
dependencyTag uses a set of predefined tags to identify different types
of dependencies. There are already multiple bits of metadata stored
in the dependency tag (Library, Shared, ReexportFlags), and supporting
them all requires a combinatorial explosion of predefined tags and
causes issues when using equality comparisons if a new bit of metadata
is added.
Add a new libraryDependencyTag type that will contain the metadata
bits, and replace the quality comparisons with checks on the metadata
bits.
There are 5 TODOs where modifying the checks identified problems with
the existing checks. These were left in place to produce identical
build output and will be fixed separately.
Bug: 162437057
Test: no change to build.ninja or {Android,make_vars,late}-${TARGET_PRODUCT}.mk
Change-Id: I72d4207dcf381c07c92e00e5a03968ebb5ed8d30
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags
This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags. A better order
would be:
global cflags
global conlyflags
global cppflags
local cflags
local conlyflags
local cppflags
local include dirs
global include dirs
We are mixing both the global and local cflags into a single
variable, and similar for conlyflags and cppflags, which
prevents reordering them. This CL prepares to reorder them
by splitting the global and local cflags into separate variables.
Bug: 143713277
Test: m native
Change-Id: Ic55a8c3516c331dc5f2af9d00e59ceca9d3e6c15
Adds an interface, CcLinkableInterface, for cc linkable dependencies
which come from other toolchains such as Rust.
Bug: 140726209
Test: Soong tests pass, rust modules still compile.
Change-Id: I7378a46fad94fd0b735746aaf4e265fd2c2c04d8
Bug: 131771163
LTO is currently broken when building with SANITIZE_TARGET=fuzzer. The
compiler bug is currently being addressed upstream (see linked bug), but
we have applied a local workaround in the build system to disable LTO
when building using the fuzzer config.
There is a bug here however. In the sanitizer mutator we explicitly
remove -flto and add -fno-lto. The sanitizer mutator runs after the LTO
mutator, so (in general) this works just fine. The problem exists when a
target specifies an explicit 'lto: { ... }' flag in their Android.bp. In
this case, the sanitizer mutator disables LTO, then the flags are parsed
from the Android.bp, re-enabling LTO.
This patch fixes this issue. If the sanitizer mutator has added the
-fsanitize=fuzzer-no-link flags, then the LTO mutator won't add the LTO
flags after this fact.
Test: Build a target with SANITIZE_TARGET=fuzzer (or a cc_fuzz target),
where there is an explitiy 'lto: { ... }' and watch it now succeed in
building.
Change-Id: I6643909417f666539c23469816926b806e204b06
LLD has different option syntax for specifying ThinLTO cache. Change it
to LLD syntax and drop the support for GOLD, since there aren't many
projects that are linked with it.
Test: USE_THINLTO_CACHE=true m
Change-Id: I5c6c5281a05793414a6387d52ae48d9b43822ee9
llvm-ar takes a --plugin argument but it is ignored for compatibility,
so passing it has no effect.
Change-Id: I4fc51d226d66cf2a43462d3d4ccc12e6e5ebb226
This reverts commit 98d8580c3a.
Change no longer needed with the new LLVM toolchain update.
Bug: 112907825
Bug: 111759196
Change-Id: Ife3ce47a602dd850ac84bb51abc70c0ca157776e
This is achieved via the compiler flag -ffixed-x18 for non-LTO builds
and the linker flag -plugin-opt -mattr=+reserve-x18 for LTO builds.
This change should be reverted once we upgrade past LLVM r340889
which does this by default on Android.
Bug: 112907825
Bug: 111759196
Change-Id: I05473ddbb98319d87d442425b4d715647eae3a38
This should always be true now. It will be a change for side branches
(build_tools, etc) that weren't setting UseClangLld in the
soong.variables file.
Test: treehugger
Change-Id: I9fd6157fda630bf8bb939677dbcb026f02685f19
* USE_CLANG_LLD is unedefined in current builds.
* When USE_CLANG_LLD is defined to 'true' or '1',
use clang's lld instead of ld or ld.gold.
* When lld is enabled:
* ld-only flags are not passed to 'lld'.
* location_packer is disabled.
* Use new lld's --pack-dyn-relocs=android.
* When lld does not work:
* In Android.mk files use LOCAL_USE_CLANG_LLD := false.
* In Android.bp files use use_clang_lld: false.
* Only arm, arm64, x86, and x86_64_devices have LLD flags;
all other hosts and targets do not call lld yet.
Bug: 73768157
Test: make checkbuild and boot
Change-Id: I06b8a1e868a600997a7e70fe05c299d751d23d5f
* Suppress more noisy new warnings at global level.
* Add -no-pie to partial link .o files, with -r.
* Revert workaround of b/72706604, no need of
-Wl,-plugin-opt,-emulated-tls
* Filter out clang 7.0 unknown flag "-Wno-extended-offsetof"
Bug: 72706604
Bug: 72412006
Test: make checkbuild
Change-Id: I7ff45465c4bd771991f42b40f68dc35586045656
Allow developers to enable ThinLTO caching on their local build by
setting USE_THINLTO_CACHE environment variable.
This significantly speeds up incremental ThinLTO builds by caching
intermediate results. An incremental ART build (by `touch
compiler/compiler.cc`) reduced from 01:32 to 00:45.
We put the cache under out/soong/thinlto-cache as the cache can be
valid across different targets, and allows us to set a global cache
pruning policy.
The current cache policy is sufficient for current list of
ThinLTO-enabled projects, we can tune it up/down in the future.
Test: USE_THINLTO_CACHE=true m
Bug: 62839002
Change-Id: I4838fe833a1d8fbc73d743da6d19e3698dc89600
Such optimisations may significantly increase the binary size when
compiler heuristics are off. Disabling these helps cut down the
binary sizes with negligible decrease in performance, but allows us to
be more comfortable enabling LTO across various projects.
Test: m
Test: dex2oat, hwui, skia benchmark
Bug: 62839002
Change-Id: Id63e8dd295df2972f76ae4e29ee367080fff8429
LLVM-AR does not allow passing --plugin options more than once. The
--plugin ARFLAGS that lto want to add, may already exist if sanitizer is
also turned on.
Fixed this by adding a new bool Flags.ArGoldPlugin. Set this variable to
true whenever LLVM gold plugin is needed for ArFlags. In function
TransformObjToStaticLib(), add this option to arFlags using global value
${config.LLVMGoldPlugin} if the bool value is true.
Bug: http://b/73160350
Test: build the image with make and succeeded.
Change-Id: I62785829b0a4b663225926e4aed98defc1b6da2c
(cherry picked from commit 4917049f6e)
-fsanitize=cfi requires LTO. Normally, the Clang driver automatically
enables emutls, but it's broken with -flto, so work around it by passing
-Wl,-plugin-opt=-emulated-tls.
See https://buganizer.corp.google.com/issues/72706604#comment15
Enable the workaround with ctx.Device(), which is the same condition used
in the existing I18acac41aac885fc6635fbd55f96ba7c845eb5e7 workaround.
Bug: b/72706604
Bug: https://github.com/android-ndk/ndk/issues/498
Test: m libaudioflinger (on internal master); \
readelf -sW symbols/system/lib64/libaudioflinger.so | grep tlNBLogWriter
check that __emutls_t.tlNBLogWriter exists
check that no symbols of type TLS exist
Change-Id: I2cf65574c52476843cc017ee176a7d6777e2ce0b
Allow developers to disable LTO to reduce build time.
Test: m checkbuild
Test: DISABLE_LTO=true m checkbuild
Bug: 62839002
Change-Id: I30642e3cfff60c4874a2f8797d08105b4e07cc6d
Propagation of LTO flags from a main binary or library to its static
dependencies is necessary so that objects being linked with LTO are built as
bitcode rather than native objects. This propagation was broken when thin LTO
was added to the build system. This patch propagates the LTO mode (thin or full)
down to object and archive dependencies.
Test: Build libhwui with full and thin LTO and check that intermediate objects
are LLVM bitcode.
Change-Id: Ic57a80d75052d86940db076fbc0f5375a79c89a1
Also adds checks that the dependencies are android.Modules and
are not disabled.
Test: m checkbuild
Change-Id: I05e945f38915d49cd3c0ab72a86576949bc7eff2
ThinLTO achieves comparable performance to full LTO while taking much
less time for compilation.
Test: Build hwui with "thin" and "full" LTO
Change-Id: If3400b82af0d5e0226410c8b740999cdad746a59
In LTO mode the backend constructs a target exclusively from command line flags,
without opportunity to accept code gen arguments from Clang. Clang does not
currently pass the -emulated-tls parameter for Android, even though it defaults
to emulated TLS for Android in non-LTO builds. Until this is fixed upstream, we
can explicitly specify that the LTO output for Android should use emulated TLS.
Test: build libandroid with LTO, verified that it uses __emutls_get_address
rather than __tls_get_addr.
Change-Id: I18acac41aac885fc6635fbd55f96ba7c845eb5e7
Enabling the lto property for a module builds that module and all static
dependencies with LTO.
LTO (link-time optimization) allows the compiler to optimize and
generate code for the entire module at link time, rather than
per-compilation unit. LTO is required for Clang CFI and other
whole-program optimization techniques. LTO also allows cross-compilation
unit optimizations that should result in faster and smaller code, at the
expense of additional compilation time.
Test: make -j12 libc with lto: true for libc
Change-Id: Ib8baefedf60e02701d44673a7c473e0845730101
