Commit graph

81 commits

Author SHA1 Message Date
Pirama Arumuga Nainar
ada83ec0a6 Add PGO support to soong
Bug: http://b/63768402
Bug: http://b/65598278

Add support for the 'pgo' property to specify how a module is processed
under PGO.  A sample property is below:

pgo: {
    instrumentation: true, // could be "sampling: true" when supported
    profile_file: "pgo_simple.profdata",
    benchmarks: ["pgo_simple"],
}

1. Runtime profiles can be gathered using "sampling" or
"instrumentation".  Sampling is not supported initially.

2. If 'toolchain/pgo-profiles' project is found,
'toolchain/pgo-profiles/${profile_file}' is passed to the compiler and
linker when building this module.

3. If ANDROID_PGO_INSTRUMENT environment variable is set, and includes a
benchmark in the 'benchmarks' list, appropriate flags (for e.g.
-fprofile-generate for instrumentation) are passed to the compiler and
linker when building this module.

Test: Add example modules that specify the pgo property and verify
appropriate flags and dependencies in the Ninja file.  Some
tests/examples are in https://android-review.googlesource.com/474805

Change-Id: I6242e0c904497a115e367dea6927ba1c4b906355
2017-09-21 12:37:19 -07:00
Dan Willemsen
c77a0b3f9c Extract the linker and embed it into host bionic binaries
The linux kernel requires that the ELF interpreter (runtime linker)
that's referenced by PT_INTERP be either an absolute path, or a relative
path from the current working directory. We'd prefer a relative path
from the binary, similarly to how we handle looking up shared libraries,
but that's not supported.

Instead, extract the load sections from the runtime linker ELF binary
and embed them into each host bionic binary, omitting the PT_INTERP
declaration. The kernel will treat it as a static binary, and we'll use
a special entry point (linker_wrapper) to fix up the arguments passed by
the kernel before jumping to the embedded linker. From the linker's
point of view, it looks like the kernel loaded the linker like normal.

Bug: 31559095
Test: Enable host bionic,
      out/soong/host/linux_bionic-x86/nativetest64/libdemangle_test/libdemangle_test
Change-Id: I8d0aea9790b5e86fcc3ea6e2d00cfa33907e2853
2017-09-20 15:34:33 -07:00
dimitry
feda20b947 Support version_script for cc_binaries
Bug: http://b/65151367
Test: make
Change-Id: Ib9b2da45e3c748eeafa792dfd0447b756855f6ef
2017-08-29 15:00:01 +02:00
Colin Cross
36242850fd Refactor factories
Change module factories from returning a blueprint.Module and a list
of property structs to returning an android.Module, which holds the
list of property structs.

Test: build.ninja identical except for Factory: comment lines
Change-Id: Ica1d823f009db812c518f271a386fbff39c9766f
2017-06-30 21:08:36 +00:00
Jayant Chowdhary
6e8115acb1 Work-around for crt objects with sdk: current
Test: convert libcups from Android.mk->Android.bp, mm -j64.

Change-Id: I4fddcfa5806d0ce27a229c37a078b0d93e9082a8
2017-05-09 14:38:10 -07:00
Vishwath Mohan
e87b768200 Make PIC/PIE levels compatible for LTO (Soong).
The build system currently uses -fPIC for position independent
libraries, and -fpie for position independent executables. However,
these levels are incompatible and conflict when an executable includes
a static lib with the -fPIC flag - such as when building with
LTO. This CL changes the flag for executables to the compatible -fPIE
instead, which is equivalent for all architectures except PowerPC.

Bug: 30227045
Test: ENABLE_CFI=true m -j40 checkbuild # builds and boots
Change-Id: I95b585b553bc00bc1d4f52f4271c5e30e1007d9b
2017-04-17 16:21:41 -07:00
Dan Willemsen
4416e5db69 Split /system and /vendor modules, allow multi-installation
Nothing changes if BOARD_VNDK_VERSION isn't set.

When the VNDK is enabled (BOARD_VNDK_VERSION in Make), this will split
/system and /vendor modules into two different variant spaces that can't
link to each other. There are a few interfaces between the two variant
spaces:

The `llndk_library` stubs will be available in the /vendor variant, but
won't be installed, so at runtime the /system variant will be used.

Setting `vendor_available: true` will split a module into both variants.
The /system (or "core") variant will compile just like today. The
/vendor ("vendor") variant will compile against everything else in the
vendor space (so LL-NDK instead of libc/liblog/etc). There will be two
copies of these libraries installed onto the final device.

Since the available runtime interfaces for vendor modules may be
reduced, and your dependencies may not expose their private interfaces,
we allow the vendor variants to reduce their compilation set, and export
a different set of headers:

  cc_library {
      name: "libfoo",
      srcs: ["common.cpp", "private_impl.cpp"],
      export_include_dirs: ["include"],
      target: {
          vendor: {
	      export_include_dirs: ["include_vndk"],
	      exclude_srcs: ["private_impl.cpp"],
	      srcs: ["vendor_only.cpp"],
	  },
      },
  }

So the "core" variant would compile with both "common.cpp" and
"private_impl.cpp", and export "include".

The "vendor" variant would compile "common.cpp" and "vendor_only.cpp",
and export "include_vndk".

Bug: 36426473
Bug: 36079834
Test: out/soong/build.ninja, out/soong/Android- only changes due to _core addition and
      .llndk -> .vendor
Test: attempt to compile with BOARD_VNDK_VERSION:=current
Change-Id: Idef28764043bf6c33dc0d2e7e2026c38867ff769
2017-04-11 12:27:41 -07:00
Colin Cross
3edeee113a Add ctx.Windows()
Replace ctx.Os() == android.Windows with ctx.Windows()

Test: builds
Change-Id: I7c24c39bcf54a51ad152b21c0021e40a0c07f189
2017-04-04 15:39:04 -07:00
Dan Willemsen
01a405a481 Add LinuxBionic toolchain and switch
This adds a toolchain definition for LinuxBionic that only supports
Clang/64-bit. It pulls pieces from the x86_linux_host and x86_64_device
configs, and uses the android clang triple, with some manual overrides.

To enable building this, set your soong.config file to:

 {"Host_bionic": true}

Bug: 31559095
Test: out/soong/{Android,make-vars}-aosp_arm64.mk the same with or
      without host bionic turned on
Test: No change to out/soong/build.ninja before/after this change
Change-Id: Id97dda8bd9aa670c32aed31fbe6aaa8175e70b59
2017-03-15 14:18:42 -07:00
Dan Willemsen
581341d4f2 Native Coverage support in Soong (gcov)
This is configured the same as make -- a global NATIVE_COVERAGE=true
flag to allow native coverage, then COVERAGE_PATHS=path1,path2,... to
turn it on for certain paths.

There are .gcnodir files exported to Make and saved in $OUT/coverage/...
files which are `ar` archives containing all of the compiler-produced
.gcno files for a particular executable / shared library.

Unlike the Make implementation, this only passes links the helper
library (automatically through --coverage) when one of the object files
or static libraries being used actually has coverage enabled.

Host support is currently disabled, since we set -nodefaultlibs, which
prevents libclang_rt.profile-*.a from being picked up automatically.

Bug: 32749731
Test: NATIVE_COVERAGE=true COVERAGE_PATHS=system/core/libcutils m -j libbacktrace libutils tombstoned
      $OUT/coverage/system/lib*/libcutils.gcnodir looks correct (self)
      $OUT/coverage/system/lib*/libbacktrace.gcnodir looks correct (static)
      $OUT/coverage/system/lib*/libutils.gcnodir doesn't exist (shared)
      $OUT/coverage/system/bin/tombstoned.gcnodir looks correct (executable)
Test: NATIVE_COVERAGE=true COVERAGE_PATHS=external/libcxxabi m -j libc++
      Confirm that $OUT/coverage/system/lib*/libc++.gcnodir looks correct (whole_static_libs)
Change-Id: I48aaa0ba8d76e50e9c2d1151421c0c6dc8ed79a9
2017-02-14 13:05:48 -08:00
Colin Cross
7a108bccad Allow clang builds to disable -pie
-pie triggers a bug in glibc's linker when used with goma
(https://sourceware.org/bugzilla/show_bug.cgi?id=16381).  Allow
the clang build to disable -pie for host modules through the
DISABLE_HOST_PIE environment variable so it can produce a toolchain
that works with goma.

Bug: 15814177
Bug: 34722791
Change-Id: Ic664a1b821aaeaf2bde14b0afa1a1975e31300cb
2017-01-30 22:51:59 -08:00
Dan Albert
dc2597d25a Don't force libraries upon static executables.
The linker uses libc++_static but needs to avoid the libc.a
dependency. It does this by setting `stl: "none"` and manually
linking libc++. This behavior matches make.

A better approach would probably be to generalize system_shared_libs
to system_libs and apply those to static executables the same way we
do for dynamic ones, but that's a patch for another day.

Test: make checkbuild
Bug: http://b/34740564
Change-Id: Ie9da0d49a453a220593e8ec2ee721e9af9378007
2017-01-26 17:44:26 -08:00
Colin Cross
37047f1c7e Pass DepsContext to dependency methods
Pass a DepsContext that embeds android.BottomUpMutatorContext
instead of android.BaseContext so that dependency methods can
directly add dependencies.

Test: m -j
Change-Id: Id4c157975d3d6f03efd99785d217bef486a76139
2016-12-13 17:32:29 -08:00
Colin Cross
989c66e370 Use toolchain to get binary extension for symlinks
Get the binary extension directly from the toolchain instead of
computing it from the target.

Test: mmma -j external/clang
Change-Id: Ib181bc60d0617ad50529791e7b9fc61268376819
2016-12-08 09:44:58 -08:00
Colin Cross
9b09f248d8 Install symlinks with same suffix and extension as installed file
Move symlink installation into the binaryDecorator so that it can
access the suffix and extension when installing symlinks.  Also
consolidates the symlink and symlink_preferred_arch handling.

Test: m -j clang, ls -l out/host/windows-x86/bin/clang++.exe
Change-Id: I1204afb71fac87b276bd6b625b52ee21274855a0
2016-12-07 17:35:37 -08:00
Dan Willemsen
d2ede879b5 Add basic VNDK support in Soong
Export a list of libraries in the VNDK, which is currently empty.

Take in Make's global BOARD_VNDK_VERSION, and use that as the SDK
version for modules that specify use_vndk: true.

Modules that use the vndk have some configuration as if they were
building against the NDK (the absence of globally defined headers), but
in other cases look like platform modules (using the platform libc++,
for now).

This change does not attempt to enforce any linking constraints, that
will come in a later patch.

Test: out/soong/build.ninja doesn't change
Change-Id: I3be206b67015ac5177b7eef4a451c579e3dc903f
2016-11-28 13:45:24 -08:00
Dan Willemsen
2e47b34435 Add Toolchain.Bionic()
Move some cc ctx.Host() / ctx.Device() checks over to using
ctx.toolchain().Bionic(). There will be more changes, these are just the
obvious ones dealing with host_ldlibs / crt / system libraries.

Bug: 31559095
Test: out/soong/build.ninja is identical
Change-Id: Ibba60483b4ab0e01f6996eb7d883120e4acc1830
2016-11-17 01:50:43 -08:00
Dan Albert
ebedf678de Support sdk_version: "current" for the NDK.
We need to fall back to the old CRT objects since we aren't generating
those yet.

Test: Created a test module using "current", checked that it linked
      the libs from current.
Bug: None

Change-Id: I5fe170d7b26154da8877672ac2acb7da0262fe38
2016-11-10 14:01:08 -08:00
Dan Willemsen
a03cf6d322 Add clang-tidy support
For every file which we can run clang-tidy (C/C++ clang-built), we add a
new build node that depends on the object file (since clang-tidy does
not export a depfile), and is depended on by the link step. This is
better than how we're doing it in make, since calling tidy can be turned
on or off without needing to rebuild the object files.

This does not attempt to port WITH_TIDY_ONLY from Make, since the way
that it works is broken (due to the lack of a depfile).

Bug: 32244182
Test: WITH_TIDY=true mmma -j bionic/libc
Test: ./soong (Setting ClangTidy: true)
Change-Id: I40bbb5bb00d292d72bf1c293b93080b5f9f6d8ea
2016-10-31 16:21:01 -07:00
Dan Willemsen
5cb580f407 Start using "struct Objects" to store object Paths
So that we can represent other files that get generated along with the
objects, like the gcno coverage information, and per-file clang-tidy
runs.

Test: Soong's build.ninja identical before/after
Change-Id: I5c553a153c436d5403549f62c73fe79c5f101779
2016-10-31 16:18:49 -07:00
Colin Cross
ce75d2c6a2 Initial prebuilt support
Support prebuilt shared libraries as an initial proof-of-concept of
prebuilts.  Future changes will support binaries and static libraries,
and the ability to select which to use based on something besides
blueprint properties.

Test: TestPrebuilts run during m -j
Change-Id: I6e84da667e9005ae11844bad01d25cbe4ced1ce3
2016-10-13 14:52:47 -07:00
Colin Cross
798bfce9d0 Move registration into android package
Mutator registration is tightly coupled with the android package, move
all registration from the soong package to the android package.

Test: build.ninja identical
Change-Id: Ie183d0b52cc7431c9e05b231934d189208ef1efe
2016-10-12 14:30:44 -07:00
Colin Cross
26c34ede29 Add support for toc optimization in soong
Skip relinking against shared libraries whose interface hasn't changed.

Test: mmma -j frameworks/native/libs/gui
Test: touch frameworks/native/libs/gui/BufferItem.cpp
Test: mmma -j frameworks/native/libs/gui, see nothing relinks past libgui
Bug: 26014946
Change-Id: I4d4b8da6a35c682341ae51869f5c72b51e192053
2016-09-30 21:05:59 -07:00
Dan Willemsen
4aa75ca244 Expose HostToolPath on the cc module to fix genrule.tool
Bug: 31742855
Test: Use `tool` in a genrule, ensure it runs properly.
Change-Id: Ib9ff35125edd66cdba8ede4f28aa6b753de0689b
2016-09-28 17:38:02 -07:00
Colin Cross
8b74d17220 Fix binaries and symlinks for prefer32
Track the primary architecture selected for each class based on the
module's multilib setting and the global config.  Fixes building
binaries with multlib set to first and DevicePrefer32BitExecutables set,
and fixes symlinks to binaries with multilib set to prefer32.

Bug: 31452121
Test: mmma -j art HOST_PREFER_32_BIT=true
Change-Id: I75094df42f3273f6d613e4058eaa565957174c28
2016-09-13 10:01:45 -07:00
Colin Cross
522e373a22 Allow CustomizeProperties to override linker
Allow CustomizeProperties to override the dynamic linker by setting a
DynamicLinker property.  Needed by art, which overrides the linker for
device-on-host testing.

Change-Id: Ia4cb5a85b9a995d8138da33eb13543addf3b38cc
2016-09-07 13:14:06 -07:00
Colin Cross
20780c80a0 Remove HostPrefer32BitExecutables
HOST_PREFER_32_BIT was used during the switch to 64-bit host tools to
keep the SDK building as 32-bit, but is never set any more.

Change-Id: I7c2db269d3f7fa1f4e0c022cbced789755a62d81
2016-09-02 14:06:30 -07:00
Colin Cross
1e7d3706d6 Add support for preferred arch symlinks
Add a symlink_preferred_arch property to binaries to allow compiling the
binary for multiple architectures and then creating a symlink to the
preferred archicture, for example dalvikvm32 and dalvikvm64, with
dalvikvm symlinked to dalvikvm64.

Test: mmma -j art/dalvikvm
Change-Id: Ied15f2be9d52c01006fe8ac207c175b78558eab1
2016-08-25 22:42:02 +00:00
Colin Cross
b916a38233 Refactor cc modules to use decorators instead of inheritance
For example , instead of trying to have libraryLinker inherit from
baseLinker and libraryCompiler inherit from baseCompiler, create a
single decorator object that wraps both baseLinker and baseCompiler.

Test: Builds, no unexpected changes to build.ninja
Change-Id: I2468adaea8466c203a240259ba5694b8b1df7a52
2016-08-05 10:25:09 -07:00
Colin Cross
42742b8bb0 Rename compiler, linker and installer methods to be unique
compiler, linker, and installer interfaces may be implemented by a
single decorator object, rename their methods to be unique to avoid the
same method being called multiple times.

Test: out/soong/build.ninja unchanged
Change-Id: I1608c41cd68f614ba99c11bb9fcc7936f618d9aa
2016-08-01 13:47:15 -07:00
Colin Cross
4d9c2d17c3 Separate cc.go
Split cc.go into files per module type, plus files for the compiler,
linker, and installer stages.

Change-Id: Id44c03f42fcd180950ccd008d4de0c144ea3597b
2016-07-29 15:17:58 -07:00