Commit graph

9050 commits

Author SHA1 Message Date
Colin Cross
ef35448ac5 Allow cc_genrule srcs to vary by architecture
Add arch_variant to the genrule srcs property so that cc_genrule
modules can vary the srcs and outs by architecture.

Test: cc/genrule_test.go
Change-Id: Idb17013e137d7ca21ca4cfc3e4c1fa7d89977043
2018-10-23 18:40:37 +00:00
Treehugger Robot
5b9fde2d15 Merge "Add support for sysprop description files to c++" 2018-10-23 03:57:56 +00:00
Treehugger Robot
5429ed0f20 Merge "Add support for sysprop description files to Java" 2018-10-23 03:45:07 +00:00
Dan Willemsen
a0790e35c7 Rework how linux_bionic is built with LLD
In order to simplify the wrapper function, and stop using a linker
script, generate a set of flags to pass to LLD. Then run
host_bionic_inject on the linked binary in order to verify the
embedding, and give the wrapper function the address of the original
entry point (_start).

Bug: 31559095
Test: build host bionic with prebuilts/build-tools/build-prebuilts.sh
Change-Id: I53e326050e0f9caa562c6cf6f76c4d0337bb6faf
2018-10-22 15:46:03 -07:00
Dan Willemsen
9ff34c0ca8 Update linux_bionic configuration
Add a missing toolchain method, don't require modules to opt in to
linux_bionic.

Bug: 31559095
Test: attempt to build host bionic
Change-Id: Ia5c8e2f127b88f56191b7abc389d5329cfdb0720
2018-10-22 15:46:03 -07:00
Dan Willemsen
0db18c239a Add support to inject a uint64 symbol
Bug: 31559095
Test: m blueprint_tools
Change-Id: Ifb46e21f5cac413abdf1dfe7fd6d31e06a7dbca5
2018-10-22 15:46:03 -07:00
Dan Willemsen
2249dc892d Turn symbol_inject into a reusable package
Bug: 31559095
Test: m blueprint_tools
Change-Id: I0f7a9f14111af26d753db547c6de313a7079658a
2018-10-22 15:46:03 -07:00
Dan Willemsen
4f644da0ad Don't use arch-specific modules when we want a single module
This breaks when two host OSes are enabled for example.

Bug: 31559095
Test: enable host bionic as a second Host OS, attempt a build
Test: treehugger
Change-Id: If52f77d7d3b0755d768028cbddda320303a69e65
2018-10-22 15:46:03 -07:00
Dan Willemsen
0ef639bb59 Allow generating modules once per OS instead of once per Class
When building with two OSes of the same class (Host, etc) enabled,
independently produce variants of each module for each OS, instead of
making `compile_multilib: "first"` select just one of the OSes.

My use case this is for Linux + LinuxBionic, which are both considered
Host class, but this would also applied if we wanted to generate rules
for Linux+Darwin, or some combination of firmware bits (either in
addition to Android in the Device Class, or multiple firmwares in a new
Class)

Bug: 31559095
Test: enable host bionic, see a linux(_glibc) and linux_bionic
      definition for binaries
Change-Id: I55e7f046758b3dc2a31da2c4b20274ec68ece0a0
2018-10-22 15:46:03 -07:00
Rahul Chaudhry
1ffac8e636 Enable SHT_RELR sections to encode relative relocations.
Proposal for adding SHT_RELR sections in the generic-abi is at
https://groups.google.com/forum/#!topic/generic-abi/bX460iggiKg

This CL requires clang-r328903.

The following test results are on clang-r339409.

Size comparison for "aosp_marlin-userdebug":
$ du -B1 -s {none,pack,relr,both}/out/target/product/marlin/system
1014616064      none/out/target/product/marlin/system
1001074688      pack/out/target/product/marlin/system
1004273664      relr/out/target/product/marlin/system
1000099840      both/out/target/product/marlin/system

none contains a build with no packing (--pack-dyn-relocs=none).
pack contains a build with current settings (--pack-dyn-relocs=android).
relr contains a build with relr packing only (--pack-dyn-relocs=relr).
both contains a build with both options enabled (--pack-dyn-relocs=android+relr).

With --pack-dyn-relocs=android+relr, lld will encode the relative
relocations in a SHT_ANDROID_RELR section, and pack the rest of
the dynamic relocations in a SHT_ANDROID_REL(A) section.

--pack-dyn-relocs=android was saving us 13,541,376 bytes (1.35%) (none - pack).
--pack-dyn-relocs=relr will save us 10,342,400 bytes (1.03%) (none - relr).
--pack-dyn-relocs=android+relr is saving us 14,516,224 bytes (1.45%) (none - both).

Size comparison for the "system/bin" directory:
$ du -B1 -s {none,pack,relr,both}/out/target/product/marlin/system/bin
29413376        none/out/target/product/marlin/system/bin
28319744        pack/out/target/product/marlin/system/bin
28258304        relr/out/target/product/marlin/system/bin
28180480        both/out/target/product/marlin/system/bin

--pack-dyn-relocs=android was saving us 1,093,632 bytes (3.72%) (none - pack).
--pack-dyn-relocs=relr will save us 1,155,072 bytes (3.93%) (none - relr).
--pack-dyn-relocs=android+relr is saving us 1,232,896 bytes (4.19%) (none - both).

Bug: None
Test: Built aosp_marlin-userdebug image, boots on device.
Test: Built aosp_x86_64-userdebug image, boots in emulator.
Change-Id: Ibb726c3b9d87688f387a7f2974fdae9bfc24efc9
2018-10-22 20:29:39 +00:00
yelinhsieh
d30b940dff ATest: GTest auto gen config support run_test_as in Android.bp .
Feature request from developer, support setting uid in Android.bp.

Bug:113359343

Test: source build/envsetup.sh ; lunch
    vim platform_testing/tests/example/native/Android.bp
    add
    test_options: {
        run_test_as: "1234",
    },
    in cc_test
    make hello_world_test
    cat out/target/product/xxxx/testcases/hello_world_test/hello_world_test.config
    Will see  <option name="run-test-as" value="1234" />
    below <test class="com.android.tradefed.testtype.GTest" >

Change-Id: I5604af5f20c45728d19f4c01396a20a74997f8a8
2018-10-22 14:14:29 +08:00
Dan Willemsen
fa2aee1ca4 Remove USE_CLANG_LLD[=false]
This should always be true now. It will be a change for side branches
(build_tools, etc) that weren't setting UseClangLld in the
soong.variables file.

Test: treehugger
Change-Id: I9fd6157fda630bf8bb939677dbcb026f02685f19
2018-10-21 19:54:12 -07:00
Jiyong Park
46f78fb4c6 Don't check the link type from stubs libs.
getLinkType also tells whether the module in question is a stubs lib or
not. If it is a stubs lib, link type from it isn't checked beause
dependency from the stubs lib doesn't matter; the stubs lib is
build-time only artifact.

In addition, core-lambda-stubs is added to the list of stubs libs.

Bug: 117964170
Test: m with https://android-review.googlesource.com/c/platform/libcore/+/793518/4
Change-Id: Ibd63484a1cc48dfd0afed6e6de098b7a38db5e61
2018-10-22 09:29:32 +09:00
Dan Willemsen
6f03752422 Allow DIST_DIR to be read via dumpvars
DIST_DIR is controlled by soong_ui, and the make dumpvars code no longer
has access to it. So instead, handle some dumpvar requests inside
soong_ui itself.

Test: get_build_var DIST_DIR
Test: get_build_var OUT_DIR
Test: get_build_var BUILD_SYSTEM
Test: build/soong/soong_ui.bash --dumpvars-mode --vars="DIST_DIR
BUILD_SYSTEM"
Change-Id: Id3bcb8b0748db67c83d0e42d1ae5be564c5eb2f7
2018-10-21 09:26:55 -07:00
Dan Willemsen
2d31a44b8a Stop using DIST_DIR in Soong
We're only using it to distribute files in case of failure, which isn't
well supported currently, but can be handled for now by using the
DIST_DIR environment variable during the command execution.

This was at least one cause that we'd be re-running Soong during every
build server build, as the DIST_DIR values are unique.

Test: m dist
Change-Id: Ibd5e6b6c46695350de80b745bfb6a6aa685033a0
2018-10-20 21:33:41 -07:00
Dan Willemsen
d4ad1da3d7 Merge "Add a Kati-based packaging step" 2018-10-20 22:50:10 +00:00
Dan Willemsen
9565675616 Merge "Remove make and xmllint from the allowed tools list" 2018-10-20 21:29:48 +00:00
Dan Willemsen
41cabfbbf4 Remove make and xmllint from the allowed tools list
We've got a prebuilt of make in prebuilts/build-tools, and we build
xmllint from source.

Test: No logs on the build servers in the last 24 hours
Change-Id: Ib60a7b5feaa31097071b67ebf6beb7f0b9a9f81b
2018-10-19 12:29:46 -07:00
Bill Yi
a10cdc664c Merge pi-dr1-dev to aosp-master
Change-Id: I8075b08b0dc7b7af6ea87cbbc5f1a8c654ad6112
2018-10-19 12:20:09 -07:00
Dan Willemsen
fb1271a52b Add a Kati-based packaging step
The idea is that we'd move the installation and packaging tasks over to
it, using data from Soong & the Kati reading Android.mk files.

This would allow us to make more fundamental changes about how we
package things without having to adjust makefiles throughout the tree.
Possible use cases:

* Moving some information from Soong's Android.mk output to a file read
  by the packaging step may allow us to read the Android.mk files less
  often, speeding up builds.

* Refactoring our current two-stage ASAN builds to run the Kati build
  step twice, writing into different object directories, then have a
  single packaging step that reads both outputs. Soong already has the
  capability of writing out a single ninja file with all the asan
  combinations.

* Running two build steps, one building the system-related modules
  using a "generic" device configuration, and one building the vendor
  modules using a specific device configuration. This could enforce a
  GSI/mainline system vs vendor split in a single build invocation.

* If all installation is through this tool, it will be much easier to
  track what should no longer be installed on an incremental build,
  reducing the need for installclean.

* Changing PRODUCT_PACKAGES should be a much faster operation, which
  means we could keep track of local additions to the images. Then
  `mma` would be more persistent, instead of installing something once,
  then never updating it again.

Eventually we plan on switching from Kati to something Go-based, but
this is a more incremental approach while we clean up everything else.

Currently, this just moves the dist-for-goal handling over to the
packaging step, so that we don't need to read Android.mk files when
DIST_DIR changes, or we switch between dist vs not.

Bug: 116968624
Bug: 117463001
Test: m nothing
Change-Id: Idec5ac6f7c7475397ba0fb65bd3785128a7517df
2018-10-19 09:55:00 -07:00
Colin Cross
3e0eb95e53 Merge changes from topic "soong_instrumentation_for"
am: 1b4aa43303

Change-Id: I63cf2ced1604b024682749733db951b54d4057d3
2018-10-19 00:56:32 -07:00
Treehugger Robot
1b4aa43303 Merge changes from topic "soong_instrumentation_for"
* changes:
  Fix instrumentation_for to match LOCAL_INSTRUMENTATION_FOR
  Fix incremental build issue in aapt2
  Support main_class property in java_binary modules
  Always allow duplicates with identical CRC32 and size
2018-10-19 07:47:13 +00:00
Jiyong Park
b60205e869 Merge "APEX can be signed with different keys"
am: 796e9de93c

Change-Id: I5ea5dc769168390ccbaa56584942cd997d2cd8b8
2018-10-19 00:04:12 -07:00
Treehugger Robot
796e9de93c Merge "APEX can be signed with different keys" 2018-10-19 06:39:19 +00:00
Colin Cross
4b964c00a6 Fix instrumentation_for to match LOCAL_INSTRUMENTATION_FOR
The value from instrumentation_for should not go to
aapt2 link --rename-instrumentation-target-package, that should
be the equivalent of LOCAL_MANIFEST_INSTRUMENTATION_FOR.  That
property is never used in Make, so it is left unimplemented in
Soong.

Add the module listed in instrumentation_for as a shared library
for javac, but don't import its resources.

Bug: 117804211
Test: m checkbuild
Change-Id: I7f035dc0ecb964a3ca391ae1ca2b87cb0f6a7cec
2018-10-18 23:54:38 +00:00
Colin Cross
5cb7803037 Merge "Remove strip.sh intermediates"
am: 51250973e4

Change-Id: I28c18868e5ac2e321128aa3848ac843b10394173
2018-10-18 15:05:15 -07:00
Colin Cross
78e3cb096b Fix incremental build issue in aapt2
Clear the directory with generated java files before generating
new files into the directory.  Fixes an issue where old, no longer
generated files could remain and end up in R.jar.

Test: m checkbuild
Change-Id: I174030c336c5b7e6fe1198a43714af565babd7e6
2018-10-18 14:55:07 -07:00
Colin Cross
094054ab07 Support main_class property in java_binary modules
Add a main_class property that will be used to generate a manifest
containing a Main-Class entry.

Test: m checkbuild
Change-Id: I0a59bb2b93cad915afd82fba708fa0f7eda2fe7a
2018-10-18 14:55:07 -07:00
Treehugger Robot
51250973e4 Merge "Remove strip.sh intermediates" 2018-10-18 20:34:11 +00:00
Neil Fuller
a6d88abb54 Remove reference to core-oj/core-libart, etc.
am: 54c1d2f238

Change-Id: I692fe145391fdd9ac6a4a341be8bcde57a0fa17f
2018-10-18 13:06:15 -07:00
Colin Cross
3adb2ed734 Remove strip.sh intermediates
The intermediate files generated by strip.sh --keep-symbols can be
very large, and are not useful after the build.  Remove them once
the final output file has been generated.

Saves ~6GB on an aosp_sailfish-userdebug build.

Test: m checkbuild
Change-Id: I0413f16ac5f423bc1b010cc9b8538f19bdea561e
2018-10-18 18:07:44 +00:00
Neil Fuller
54c1d2f238 Remove reference to core-oj/core-libart, etc.
Various "core library" targets are no longer used in
soong so the tests do not have to create fake build
rules for them.

Bug: 113148576
Test: make droid
Change-Id: Ic754b4195f1597fb599e496c08940231ecbb13f7
2018-10-18 17:51:25 +01:00
Colin Cross
dc1e829b59 Always allow duplicates with identical CRC32 and size
Don't warn on duplicate files in merge_zips if they have identical
CRC32 and size values.

Test: m checkbuild
Test: merge_zips_test.go
Change-Id: I61336ca4d4d3b7402c24a7abd337bd350fe10930
2018-10-17 15:49:53 -07:00
Dan Willemsen
b2aba00619 Merge "Always ensure that DIST_DIR is set"
am: 251178f024

Change-Id: I8a303a7f5f36349a937003739dd1b7b00da68e0c
2018-10-17 10:37:43 -07:00
Treehugger Robot
251178f024 Merge "Always ensure that DIST_DIR is set" 2018-10-17 15:26:38 +00:00
Neil Fuller
043a41bbd9 Merge "Build platform with core library stubs by default"
am: ac857603ce

Change-Id: I7a0eb6a7f089e04fbac421e780a395ed32b60220
2018-10-17 08:10:26 -07:00
Neil Fuller
ac857603ce Merge "Build platform with core library stubs by default" 2018-10-17 14:59:48 +00:00
Nicolas Geoffray
ae42d21cf1 Merge "Define runfiles_path before try so that finally can use it."
am: 9605b3ac48

Change-Id: Id2daa19a36ba488e7f42a01798b455123c1d04f8
2018-10-17 06:23:03 -07:00
Nicolas Geoffray
9605b3ac48 Merge "Define runfiles_path before try so that finally can use it." 2018-10-17 13:11:02 +00:00
Martin Stjernholm
e608208a67 Merge "Improve messages for VNDK dependency errors."
am: c325edce18

Change-Id: I825ac5a438b13eb4c1edff75acfeafe317d2d72a
2018-10-17 05:10:03 -07:00
Martin Stjernholm
c325edce18 Merge "Improve messages for VNDK dependency errors." 2018-10-17 11:56:57 +00:00
Logan Chien
aceb86e345 Merge "Remove flags from ClangLibToolingUnknownCflags"
am: 0b6bcd4014

Change-Id: I7f7e9e80052e748c1d92f46fee4552c1b4a45a2b
2018-10-17 03:08:54 -07:00
Logan Chien
0b6bcd4014 Merge "Remove flags from ClangLibToolingUnknownCflags" 2018-10-17 09:46:26 +00:00
Jiyong Park
980b757b13 Merge "android.test.{base|mock} are added as optional libraries."
am: e8f2977f80

Change-Id: Ia185f939b28ac14ab9cabeb40a323549d5fa8759
2018-10-16 21:44:18 -07:00
Treehugger Robot
e8f2977f80 Merge "android.test.{base|mock} are added as optional libraries." 2018-10-17 04:29:07 +00:00
Dan Willemsen
d50e89f389 Always ensure that DIST_DIR is set
Set the default value if one isn't provided.

Test: `get_build_var DIST_DIR` after removing the default setting in envsetup.mk
Change-Id: I0cb310fc65f7747c36de14608b61786ef6863fd1
2018-10-16 17:49:25 -07:00
Dan Willemsen
04619c6ddf Merge "Unset BUILD_NUMBER in build_test.bash"
am: 08c3085998

Change-Id: I807043df12cdf7eb0072a3191cb62eaa0f251f70
2018-10-16 15:16:43 -07:00
Treehugger Robot
08c3085998 Merge "Unset BUILD_NUMBER in build_test.bash" 2018-10-16 21:49:22 +00:00
Dan Willemsen
5ffade3240 Merge "Remove perl from the allowed PATH tools list"
am: 1a968b91a6

Change-Id: I9c21e5bdfe4cde541057e743b81258b0861a21ba
2018-10-16 09:07:18 -07:00
Treehugger Robot
1a968b91a6 Merge "Remove perl from the allowed PATH tools list" 2018-10-16 15:39:45 +00:00