Commit graph

71 commits

Author SHA1 Message Date
Yi Kong
950c17435d Remove GLOBAL_THINLTO build option
GLOBAL_THINLTO is now default, since it improves build speed, produces
more performant code and smaller binary. Remove the option to turn it
off since we no longer maintain that variation, and that simplifies the
logic (esp. for moving CFI builds to ThinLTO).

Also fixed a bug where ThinLTO is not propagated to its static deps on
non-default targets (32-bit or host).

Test: presubmit
Bug: 169004486
Change-Id: I31f41ba27c2b94a384d2ba5027049c307d6f4334
2023-10-09 10:56:14 +00:00
Yi Kong
577a73a22d Change lto_test to use GlobalThinLTO configuration
This is now the default, and we will be removing non-GlobalThinLTO configuration.

Also changed handling for lto: never, it doesn't need to create an
lto-none variant.

Change-Id: I971baa920df867fb762923d925eed22215a89c27
Test: presubmit
2023-10-05 05:05:40 +00:00
Yabin Cui
1aa29e6ca9 Revert "Disable Global ThinLTO by default"
This reverts commit 9c81c0cfd9.

Reason for revert: The ThinLTO problem has been fixed in https://android-review.git.corp.google.com/c/platform/bionic/+/2725997. So we can Re-enable Global ThinLTO.

Change-Id: I25dde4910692fd71124b48f45911da5b7b92a089
2023-09-01 17:22:58 +00:00
Yabin Cui
9c81c0cfd9 Disable Global ThinLTO by default
Because enabling ThinLTO makes some constructor functions not called
in arm64 executables.

Bug: 295944813
Test: build
Change-Id: Ic0cab5fd80aa86d0d9ca1131564b747a445c6e46
2023-08-14 15:47:36 -07:00
Yi Kong
d6ab48c660 Limit LTO inlining even when profile is present
Reduce the import instr limit from the LLVM default (100) to 40. This
helps reduce the binary size as well as improving performance slightly.

Size:
             Default(100)   40        30
libhwui.so   11059040       11058912  11060872
libart.so    10697576       10697160  10696568

Performance:
40 vs. default:  0.37% improvement
                 http://go/art-benchmark?p=BootImageProfileId:36054
30 vs. default:  0.36% improvement
                 http://go/art-benchmark?p=BootImageProfileId:36058

Test: presubmit
Change-Id: Id800ff7818cde908daab784bac0a312c6a71272d
2023-08-01 18:30:13 +00:00
Yi Kong
13beeedff7 Turn off Global ThinLTO for LP32 again
LP32 has many subtle issues and less test coverage.

Bug: 290859519
Bug: 169004486
Test: presubmit
Change-Id: I33fcf1a08b9e7a905ba81030250a9578532da5a0
2023-07-15 03:18:06 +09:00
Yi Kong
8f9ca23c1e Fix a typo. NFC.
Test upload from Cog...

Change-Id: I68964d14d0a01be2bdc0ad276bdfde9f879795f2
Test: N/A
2023-07-14 06:18:38 +00:00
Yi Kong
add6375a10 Turn on Global ThinLTO by default
Build C/C++ targets with ThinLTO with "--lto-O0" by default. This takes
advantage of better dead code elimination and CFG simplification due to
ThinLTO's global view of the program, but do not enable the costly LTO
optimizations. This also makes builds faster because we can avoid doing
codegen for the dead code.

Code size for panther-userdebug:
                   /system/bin  /system/lib  /system/lib64
Original           57648        86264        181700
Global ThinLTO     55764        84916        175728
                   -3.27%       -1.56%       -3.29%

Build time for aosp_arm64-userdebug:
                   clang time   linker time
Original           56993.87s    1712.36s
Global ThinLTO     52839.18s    3114.93s

saving 4.68% build time in total.

Bug: 169004486
Test: presubmit
Change-Id: Idb0f4675ca4750b8b12b24f4679579419d3448c8
2023-07-11 21:26:52 +09:00
Yi Kong
b9d5046d8b Simplify LTO flags handling
... and add more comments about the use of --lto-O0.

Test: presubmit
Change-Id: I2ef7bf4f100a8d7559a6b738e9211c596d579dc6
2023-07-03 17:16:42 +09:00
A. Cody Schuffelen
7188b90e69 Disable "-Wl,-plugin-opt,-import-instr-limit=5" on MacOS
This option is unsupported on the MacOS ld used in Android, leading to
compilation errors when LTO is enabled for binaries.

In particular the error includes:
```
ld64.lld: error: unknown argument '-plugin-opt'
ld64.lld: error: unknown argument '-import-instr-limit=5'
```

Test: m libc++ on a Mac OS machine
Bug: 288983988
Change-Id: I1873e0a77777832f833f211beb42633d462f04d5
2023-06-27 19:15:07 -07:00
Yi Kong
895d241013 Refactor LTO
We removed Full LTO support, now LTO is a binary choice. This allows us
to greatly simplify the LTO propagation logic.

Test: m
Test: GLBOAL_THINLTO=true m
Bug: 169004486
Change-Id: Ie4e453d7518c7069a7c755734dab1e776c16e656
2023-06-09 03:15:49 +00:00
Yi Kong
ed79fa382b Use a less hacky way to detect if a module is Fuzzer enabled
Bug: 169004486
Test: presubmit
Change-Id: I7fad86041622979b9befb7c86897a13f604bdf3d
2023-06-05 14:21:48 +09:00
Yi Kong
0713e336d9 Remove Full LTO support
We do not have any more projects directly depending on Full LTO (except
CFI which manages their own LTO flags), remove full LTO support from the
build system.

Test: presubmit
Bug: 169004486
Change-Id: I8d6b7999d716158b5d8fe34b2f197653d7bae8dd
2023-06-01 00:10:26 +00:00
Yi Kong
a7a345d55e Remove Use_clang_lld in lto
We have moved all LTO projects to LLD, this is unused.

Test: presubmit
Change-Id: I385b3302c04243be8fde80605b29df7646ec40ce
2023-05-30 15:19:06 +09:00
Aditya Kumar
efc36678a9 Enable LTO for riscv64
This partially reverts commit 1281b99495.

Bug: b/282584808, b/254713216

Reason for revert: LTO can be enabled as emultated tls is now globally disabled https://android-review.git.corp.google.com/c/platform/build/soong/+/2431153

Change-Id: I66c83b1116550179043490a11f66b8630c0e6b1b
2023-05-17 17:18:06 +00:00
Aditya Kumar
1281b99495 Disable LTO and CFI for riscv64
This reverts commit 24870216e9.

Reason for revert: b/282584808

Change-Id: I3808121b4437fdd01db5d305d8f0377e6ad7bd9e
2023-05-16 03:55:41 +00:00
Aditya Kumar
24870216e9 Revert "Disable LTO and CFI for riscv64"
This reverts commit ecf4e664e0.

Reason for revert: Not needed as emultated tls is now globally disabled https://android-review.git.corp.google.com/c/platform/build/soong/+/2431153

Change-Id: Idc3de2771afbbe2d5976f3c7974b3ffd15441b13
2023-05-12 18:46:42 +00:00
Aditya Kumar
4c660ae58e Revert "Build riscv64 with thin LTO"
This reverts commit 17c9c2a5b5.

Reason for revert: Wait until new toolchain with linker fix las landed (https://android-review.git.corp.google.com/c/toolchain/llvm_android/+/2424081)

Change-Id: I05b14c12ac93182dd93c5b490f260071c6d3a090
2023-02-15 01:32:22 +00:00
AdityaK
17c9c2a5b5 Build riscv64 with thin LTO
Depends on: https://android-review.git.corp.google.com/c/platform/system/libhwbinder/+/2336346

Bug: b/254713216
Change-Id: Id750cab796ddf8c180fa0570e46cab0161788974
2023-02-13 21:42:30 +00:00
Colin Cross
2c435a00ff Set -fno-emulated-tls for riscv64
Building for riscv64 fails with link errors on __thread variables.
Set -fno-emulated-tls to fix it.

Bug: 254713216
Test: lunch aosp_riscv64-userdebug && m ALLOW_MISSING_DEPENDENCIES=true ndk_sysroot
Change-Id: I3dca81dfd277d681b6c868a5e8385e3a37335a5f
2022-10-20 14:31:24 -07:00
Colin Cross
ecf4e664e0 Disable LTO and CFI for riscv64
Building with LTO and CFI enabled for riscv64 causes link failures:
ld.lld: error: lto.tmp: cannot link object files with different floating-point ABI
ld.lld: error: undefined symbol: guard variable for android::hardware::BufferedTextOutput::getBuffer() const::ts

Disable them for now.

Test: lunch aosp_riscv64-userdebug && m ALLOW_MISSING_DEPENDENCIES=true ndk_sysroot
Change-Id: I3489952abebeeb3f4de664fd3e436232aac298d7
2022-10-20 13:59:58 -07:00
Liz Kammer
729aaf4a10 Refactor lto mutator to not alter user-input props
Test: lto test
Change-Id: I700960c6fb6527123e08e8e3c90a15dea0987df3
2022-09-26 11:03:14 -04:00
Yi Kong
5e0f405abf Pass -flto to asflags as well
Although it has no effect, CFI mandates -flto in asflags.

Test: presubmit
Bug: 243476017
Change-Id: I33246488140c642894f3fbab65bc37bdd5de3024
2022-09-09 01:58:18 +08:00
Yi Kong
56fc1b6902 Do not enable global ThinLTO for tests
Tests are not shipped, no need to spend extra CPU cycles optimising them
during build.

Test: GLOBAL_THINLTO=true m
Bug: 169004486
Change-Id: I66ede9c01d43b574a7fe9f74f0bc6ba97f51be06
2022-09-06 16:24:00 +08:00
Yi Kong
b1a9aab987 Merge "Revert "Reland "Turn on Global ThinLTO by default""" 2022-08-23 11:36:02 +00:00
Yi Kong
7689c64f68 Revert "Reland "Turn on Global ThinLTO by default""
This reverts commit 4273c18a4f.

Bug: 243475788
Bug: 243490273
Change-Id: I5ea8c8e6a84408013bc7ed573776d8f5eebc091f
2022-08-23 09:24:48 +00:00
Treehugger Robot
8cab5906da Merge "Reland "Turn on Global ThinLTO by default"" 2022-08-23 00:47:28 +00:00
Yi Kong
4273c18a4f Reland "Turn on Global ThinLTO by default"
This reverts commit b90ff82772.

All the breakages are now fixed or mitigated.

Bug: 169004486
Change-Id: Ibec4b1f0a50c5ef6769bc6916a81721a4ce03c60
Test: presubmit
2022-08-22 08:18:03 +00:00
Yi Kong
c702ebd886 Do not enable ThinLTO for CFI enabled projects
CFI projects already use full LTO.

Test: GLOBAL_THINLTO=true m
Change-Id: I8a57efc237aed977f57dd4040500cc5f84843c3d
2022-08-19 16:05:14 +08:00
Yi Kong
b90ff82772 Revert "Turn on Global ThinLTO by default"
This reverts commit 406e7b725b.

Reason for revert: ndk_translation causes clang to crash

Change-Id: I61b262157f9f57c739391757d5a1f95299ddadca
2022-08-18 14:53:34 +00:00
Yi Kong
406e7b725b Turn on Global ThinLTO by default
Apply ThinLTO without optimization (--lto-O0) globally. This saves us
2.24% in binary size for system/lib64, as well as more than 4.68% build
time improvement.

Test: presubmit
Bug: 169004486
Change-Id: I1ed90c2058f9ff1fa8a2d3fd7ac1191c5d98dac1
2022-08-17 09:27:02 +08:00
Yi Kong
2121d16db8 Only enable global ThinLTO for LP64 targets
Due to the lack of engineering efforts into LP32 targets, do not enable
global ThinLTO for them by default.

Test: presubmit
Bug: 169004486
Change-Id: I79b77baecf16fda95ff5dccc5a401596a0557d5d
2022-08-17 09:22:07 +08:00
Treehugger Robot
5c8693f5d7 Merge "Revert "[cc_fuzz] Revert 'disable LTO' patches."" 2022-03-02 05:08:27 +00:00
Mitch Phillips
5007c4a06d Revert "[cc_fuzz] Revert 'disable LTO' patches."
Revert submission 1976512-revert-nolto-fuzzing

Reason for revert: b/222160662
Reverted Changes:
Iacee4fa29:[cc_fuzz] Revert 'disable LTO' patches.
Ic509b00a1:[cc_fuzz] Revert 'disable LTO' patches.
If85931f09:[cc_fuzz] Revert 'disable LTO' patches.

Change-Id: I84e01a4f84145e3331c2955836c8cc9bfa05c36b
2022-03-02 01:25:22 +00:00
Treehugger Robot
6c0de9938e Merge "[cc_fuzz] Revert 'disable LTO' patches." 2022-03-01 23:43:04 +00:00
Yi Kong
4ef5459f7d Don't reduce inline limit for afdo enabled projects
Test: presubmit
Change-Id: Id964165c7af50da9cd5e92c25ad2c4191eb2147a
2022-02-14 20:02:04 +08:00
Mitch Phillips
7fbac74dc9 [cc_fuzz] Revert 'disable LTO' patches.
The toolchain no longer has a problem with sancov + LTO.

Bug: 131771163
Test: make haiku
Change-Id: If85931f092c41336a8a5e0e7999ad39dd8ec3daf
2022-02-08 10:51:35 -08:00
Yi Kong
04e459d9ff Revert "Do not build tests with ThinLTO"
On second thought, this is actually counter-productive rather than an
optimisation. LTOing test code is usually quite cheap, however since
tests can pull in additional static libraries (e.g. to set up test
environment), we will have to create non-LTO variants for these
dependent libraries, which can be much more expensive.

This reverts commit 85d7297318.

Bug: 195134194
Bug: 203737712
Change-Id: I1a105136075133a5b5895d00847d8796c419964c
Test: m GLOBAL_THINLTO=true dist device-tests platform_tests
2021-10-29 17:53:27 +00:00
Yi Kong
85d7297318 Do not build tests with ThinLTO
There's not a great return on investment for spending additional
compilation/link time on tests that may only be run once or very few
times (as compared to actual library code that ships and is run orders
of magnitude more times).

Bug: 195134194
Bug: 203737712
Test: m GLOBAL_THINLTO=true dist device-tests platform_tests
Change-Id: I7149c4140f7c52b22bedbe322f1bf2c81f7ad66b
2021-10-24 17:01:43 +08:00
Yi Kong
8ea56f9da9 Add Global ThinLTO option (2nd try)
Instead of making everything into ThinLTO variant by default (it works
but many Soong tests don't like this, and got bit rot due to lack of
active builder for this configuration), let the default option be
ThinLTO and no LTO be a special variant.

Test: m GLOBAL_THINLTO=true
Test: m
Bug: 195134194
Change-Id: I2fd98061ba55eba1fdfdd056fb2f8c2051fd2553
2021-10-14 17:34:13 +08:00
Liz Kammer
b2fc4700de Delete local boolPtr function
This is defined in proptools, no need to duplicate.

Test: go test soong tests
Change-Id: I32d4417dc4ae73bff98f36360ed22aed572d8c18
2021-07-07 16:29:09 -04:00
Liz Kammer
3b606c8218 Delete unused functions
As they're unused, also removing them from the interface.

Test: go test soong tests
Change-Id: I76e7cbca12876395d8d7eaae4481c5e0d1350d76
2021-06-28 12:29:30 -04:00
Yi Kong
134161f7e5 Global ThinLTO: opt out vndk binaries as a workaround
With global ThinLTO enabled, vndk version has different symbol ordering
from the platform version. Opt out any binaries that has vndk enabled as
a temporary workaround.

Test: GLOBAL_THINLTO=true m
Bug: 169217596
Change-Id: I75b060cbe6c74421d283c6dfbd669af20f466d1f
2020-10-09 22:08:11 +08:00
Yi Kong
e2577141ba Do not implicitly turn on lto for static libraries
For global ThinLTO, don't implicitly turn on LTO for static libraries,
but instead rely on mutator to generate correct variants.

Bug: 169004486
Test: GLBOAL_THINLTO=true m
Change-Id: I9cdeea706ec6dd4ad31f55b9e12a96b42176aa89
2020-09-30 01:13:57 +08:00
Yi Kong
f43ff059ff lto: Rename Disabled to Never to reduce confusion
Having different name from the underlying property is confusing...

Test: build
Change-Id: Ia24b664d6cded68abed2966455193f0e20757e06
2020-09-28 17:18:36 +00:00
Yi Kong
93718e0d41 Global ThinLTO mode
Experimental global ThinLTO mode for compilation time evaluation.

Bug: 169004486
Test: GLBOAL_THINLTO=true m
Change-Id: I2ff06397be8b65c7972f1e5f09ab1496949567ee
2020-09-25 02:43:14 +08:00
Yi Kong
2f5f16d574 Tweak ThinLTO inling heuristics in absense of PGO profile
We previously disabled inlining and unrolling completely during ThinLTO
in absense of PGO profile. For global ThinLTO, we want to better balance
binary size and performance.

We evaluated a number of combination of heuristics with global ThinLTO
configuration:
                                binary size change
  no LTO                          baseline
  no inline, no unroll            -0.54%
  no inline, unroll               -0.50%
  import-instr-limit=5, unroll    +0.02%
  import-instr-limit=10, unroll   +0.13%

Loop unrolling does not contribute much to the binary size, therefore
it is re-enabled.

import-instr-limit=5 balances the binary size savings from ThinLTO and
size incrase due to additional optimisation.

Bug: 78485207
Bug: 169004486
Test: TreeHugger
Change-Id: I1c21153605e2ae42daa8857b06e27c081ee8ad85
2020-09-23 01:32:06 +08:00
Yi Kong
2d01fe28df Introduce LTO property for -fwhole-program-vtables cflag
-fwhole-program-vtables cflag requires LTO to be enabled. Some projects
add this flag unconditionally, breaking builds that turn off LTO.

Make this an LTO property so that build system will handle adding the
flag when LTO is on.

Test: build
Bug: 169004486
Change-Id: Ifce12c09e1d6b8c2807314c3bb982b076efcaaa4
2020-09-21 16:54:07 +00:00
Colin Cross
4fbb5e0234 Handle static unwinder in ltoDepsMutator
The implementation before libraryDependencyTag was added failed to
treat the libgcc_unwind dependency as a static library when traversing
dependencies for LTO because it didn't handle staticUnwinderDepTag.

Bug: 162437057
Test: m checkbuild
Change-Id: Iccc96c0daa0379f51eb975c51fd5e8c507a0daba
2020-07-29 13:22:33 -07:00
Colin Cross
6e511a9a9f Add libraryDependencyTag to track dependencies on static and shared libraries
dependencyTag uses a set of predefined tags to identify different types
of dependencies.  There are already multiple bits of metadata stored
in the dependency tag (Library, Shared, ReexportFlags), and supporting
them all requires a combinatorial explosion of predefined tags and
causes issues when using equality comparisons if a new bit of metadata
is added.

Add a new libraryDependencyTag type that will contain the metadata
bits, and replace the quality comparisons with checks on the metadata
bits.

There are 5 TODOs where modifying the checks identified problems with
the existing checks.  These were left in place to produce identical
build output and will be fixed separately.

Bug: 162437057
Test: no change to build.ninja or {Android,make_vars,late}-${TARGET_PRODUCT}.mk
Change-Id: I72d4207dcf381c07c92e00e5a03968ebb5ed8d30
2020-07-29 13:22:30 -07:00