Commit graph

17282 commits

Author SHA1 Message Date
Kousik Kumar
0f095e1e65 Add additional variables to ninja variable whitelist
These variables control whether we accept cached-results / not and
whether we should update the cache with locally executed results or not.
I need accpet-cached whitelisted to purge the the RBE cache for the
invalid cache entry we have set for the failure we say yesterday.

Bug: b/148387048

Change-Id: I7344fc083f82e0b7bc11084376a267d19cf30bb8
2020-01-28 10:56:04 -08:00
Ulyana Trafimovich
911083e3e1 Merge "Fix --boot-image argument in dex2oat command for boot image extension." 2020-01-28 17:04:38 +00:00
Ulya Trafimovich
b0a2d37dcb Fix --boot-image argument in dex2oat command for boot image extension.
The primary image name should be "boot.art" in case of
"boot-framework.art", and "apex.art" in case of "apex-framework.art".
Although "boot.art" and "apex.art" are identical binaries, dex2oat
has hard-coded logic based on the image name.

This has been broken since boot image extension has been enabled for
JIT-zygote config in CL I5493e575ebf90bad1d5ad2850004d54590bbc079.

Test: compare boot-framework.art and apex-framework.art,
  they were identical before this CL, and differ after this CL.

Test: JIT-zygote config boots, steps 1-2:

  1. Temporarily enable Jit zygote in the product device config (in
    this case device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. Build and flash:
    $ lunch aosp_walleye-userdebug && m \
    && adb reboot bootloader && fastboot flashall -w

Change-Id: I98de271852ecc33feb9fd4c9b0addf0feba01856
2020-01-28 14:42:41 +00:00
Jiyong Park
7aa3f76696 Remove unnecessary symlink for non-flattend APEXes
This change fixes a bug that symlinks to the system partition are
created in /system/apex/<apex_name> directories even when the APEXes are
non-flattened. The symlinks are needed only for flattened APEX (of
course regardless of whether the APEX is a primary one or not).

Bug: N/A
Test: examine /system/apex directory manually
Change-Id: I00bb1423d0a2497408f05e49767b42437210bab8
2020-01-28 16:51:34 +09:00
Treehugger Robot
7c78b3c98e Merge "If input zip file cannot opened, print its name." 2020-01-28 01:48:32 +00:00
Treehugger Robot
057dd35669 Merge "Abstract sdk_version string using sdkSpec type" 2020-01-28 00:07:37 +00:00
Colin Cross
b7ec426e01 Merge "Simplify vendor conditionals" 2020-01-27 19:03:23 +00:00
Martin Stjernholm
699cf17f53 Merge changes from topic "revert-1211982-dex2oat-soong-dep-LLLKNULXYJ"
* changes:
  Revert "Separate dexpreopt.GlobalSoongConfig to allow independen..."
  Revert "Move the Once cache for dexpreopt.GlobalConfig into the ..."
  Revert "Get the dex2oat host tool path from module dependency on..."
2020-01-27 13:47:31 +00:00
Hans Boehm
e4b5342d43 Revert "Separate dexpreopt.GlobalSoongConfig to allow independen..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: Ice3990225635a737e49e9aed7373f06516fccea3
2020-01-27 13:44:24 +00:00
Hans Boehm
453bf09853 Revert "Move the Once cache for dexpreopt.GlobalConfig into the ..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I6b656afb5feaad70d958b9d38b6c6eab7b03fba1
2020-01-27 13:44:03 +00:00
Hans Boehm
7b2e6f3ed7 Revert "Get the dex2oat host tool path from module dependency on..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I0cf6fef04434fd3c0b7957ee9abdcaabeb20b9d8
2020-01-27 13:01:16 +00:00
Jiyong Park
6a927c4e6a Abstract sdk_version string using sdkSpec type
The value format that sdk_version (and min_sdk_version, etc.) can have
has consistently evolved and is quite complicated. Furthermore, with the
Mainline module effort, we are expected to have more sdk_versions like
'module-app-current', 'module-lib-current', etc.

The goal of this change is to abstract the various sdk versions, which
are currently represented in string and is parsed in various places,
into a type called sdkSpec, so that adding new sdk veresions becomes
easier than before.

The sdk_version string is now parsed in only one place 'SdkSpecFrom', in
which it is converted into the sdkSpec struct. The struct type provides
several methods that again converts sdkSpec into context-specific
information such as the effective version number, etc.

Bug: 146757305
Bug: 147879031
Test: m
Change-Id: I252f3706544f00ea71c61c23460f07561dd28ab0
2020-01-25 21:56:43 +09:00
Baligh Uddin
cf0bba7a30 Merge "Revert "Revert "Allow codename.fingerprint format for minSdkVersion""" 2020-01-25 04:12:30 +00:00
Treehugger Robot
6cc0515e82 Merge "Create a highmem pool and put metalava into it" 2020-01-25 01:09:43 +00:00
Colin Cross
9d34f35815 Simplify vendor conditionals
Support vendor conditionals with no Go code.

Test: TestSoongConfigModule
Change-Id: I42546e7f17324921ada80f4d8e1cd399830f8dfc
2020-01-24 16:43:40 -08:00
Treehugger Robot
cf52e102ce Merge changes from topic "dex2oat-soong-dep"
* changes:
  Get the dex2oat host tool path from module dependency on the binary module.
  Move the Once cache for dexpreopt.GlobalConfig into the dexpreopt package.
  Separate dexpreopt.GlobalSoongConfig to allow independent caching of it.
2020-01-25 00:38:19 +00:00
Baligh Uddin
f62013736a Revert "Revert "Allow codename.fingerprint format for minSdkVersion""
This reverts commit 230e241f58.

Reason for revert: This is a revert of a revert.  Downstream problem has been fixed and have been validated locally and via Forrest build.

Change-Id: I89c51d25b3adb818ea44a983d0ac681a88790d8c
2020-01-24 23:15:44 +00:00
Yifan Hong
a8c82cc218 Merge changes from topics "board_use_recovery_as_root_soong", "ramdisk"
* changes:
  libatomic / libgcc_stripped: ramdisk_available
  Add target.ramdisk
  Ramdisk modules install to correct location
  Add ramdisk image.
2020-01-24 22:39:41 +00:00
Martin Stjernholm
335d596384 Get the dex2oat host tool path from module dependency on the binary
module.

This uses the Once cache for GlobalSoongConfig to propagate the dex2oat
path from a module dependency to the singletons (both the one that
writes out dexpreopt_soong.config and the one that creates the
dexpreopted boot images). Unless dexpreopting is disabled altogether
through DisablePreopt in dexpreopt.config, that means:

- We must ensure at least one module registers a dex2oat tool
  dependency and resolves a GlobalSoongConfig using it, or else the
  singletons fail. That means we litter dex2oat dependencies in java
  modules even when they won't get dexpreopted and hence don't really
  need them.

- We still assume there's at least one java_library or android_app in
  the build.

Bug: 145934348
Test: m
  (check that out/soong/dexpreopt_soong.config points to dex2oatd64)
Test: env USE_DEX2OAT_DEBUG=false m
  (check that out/soong/dexpreopt_soong.config points to dex2oat)
Test: env OUT_DIR=out-tools prebuilts/build-tools/build-prebuilts.sh
  on the aosp-build-tools branch
Change-Id: I38317f2d5e5cf6822f87daae25745767c8a3656c
2020-01-24 22:18:42 +00:00
Martin Stjernholm
dae8a80426 Move the Once cache for dexpreopt.GlobalConfig into the dexpreopt
package.

Preparation for a future CL that will need to get the make-written
GlobalConfig from within dexpreopt.

Also rename the Load*Config functions to Parse*Config, since they don't
actually load the config files anymore.

Bug: 145934348
Test: m
Change-Id: I71df11c1e042ca8135d273a7263e9539ea3cd68f
2020-01-24 22:18:42 +00:00
Martin Stjernholm
be9d0d21d1 Separate dexpreopt.GlobalSoongConfig to allow independent caching of
it.

Introduce a Once cache for GlobalSoongConfig to allow it to get binary
tool paths from ordinary module dependencies (coming in a future CL)
that are then reused in singletons.

Bug: 145934348
Test: m
Change-Id: I440a09dba7d337965a196527566b0966a18e3653
2020-01-24 22:09:54 +00:00
Colin Cross
8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Jaewoong Jung
f0f747c949 Do not remove RRO resources
Resource configs should not be deduped when building RROs since it
would be impossible to override some resource configs with the same
value as the default config. Also, aapt2 removes resources that do not
have default configurations. If an overlay attempts to overlay a
non-default configuration without overlaying the default, the resource
will be removed and the value will not be overlaid at all.

Bug: 146227008
Fixes: 119811120
Test: app_test.go
Change-Id: I834a58b18d1e74a0f6b3de3d0523009788787e42
2020-01-24 10:35:52 -08:00
Treehugger Robot
8c94763bf7 Merge "Fix symlinks to system libs for flattened apex" 2020-01-24 08:32:05 +00:00
Treehugger Robot
84f71af41c Merge "Add runtime_resource_overlay." 2020-01-24 00:23:42 +00:00
Jaewoong Jung
9befb0c326 Add runtime_resource_overlay.
Fixes: 119811120
Test: app_test.go
Test: Converted and built IconPackFilledSystemUIOverlay
Change-Id: I71841148c25f820ba829f751a201d2c771c8bd20
2020-01-23 14:24:41 -08:00
Colin Cross
c61b38904e Merge changes from topic "filtershard"
* changes:
  Pass desired size proptools.FilterPropertyStructSharded
  Do a better job removing tags from arch structs
  Split the runtime arch property structs into 3
2020-01-23 21:26:39 +00:00
Jooyung Han
75de261056 Fix symlinks to system libs for flattened apex
Symlinks to system libs should be created for flattened apex regardless
that it is primary or not.

For example, GSI installs non-primary flattened apexes as well. These
flattened (non-primary) apexes could be activated on non-updatable
devices.

Bug: 148195518
Test: GSI runs on P
Change-Id: I238b226473d923e03280b1b28dd0d5d1f77ae74a
2020-01-24 02:02:45 +09:00
Treehugger Robot
7b2970536e Merge "Revert "Revert "apex_vndk uses vendor variants only""" 2020-01-23 13:16:52 +00:00
Ulyana Trafimovich
c18235cb86 Merge "For JIT-zygote config, add "apex" boot image files to the ART apex." 2020-01-23 10:27:09 +00:00
Vic Yang
641e87a02c Merge "Reland "Clean up no-vendor-variant VNDK whitelist"" 2020-01-23 06:07:58 +00:00
Jooyung Han
a57af4adc0 Revert "Revert "apex_vndk uses vendor variants only""
This reverts commit 5df3b11f78.

Reason for revert: re-land with a fix

Fix a broken soong test
Add implicit dependency (libprofile-clang-extra) to make a test pass.

Bug: n/a
Test: m
Change-Id: I0b179199bc032501354f8e24782837453781bd8c
2020-01-23 14:48:29 +09:00
Jooyung Han
7dfb229936 Merge "Revert "apex_vndk uses vendor variants only"" 2020-01-23 05:34:42 +00:00
Jooyung Han
5df3b11f78 Revert "apex_vndk uses vendor variants only"
This reverts commit 559ad01db9.

Reason for revert: broken builds

Change-Id: Ia8a09a2bdbdc8e76f2d4b9eb54e36ccc544e9397
Bug: b/148181875
2020-01-23 05:31:05 +00:00
Treehugger Robot
e4226c4d67 Merge "apex_vndk uses vendor variants only" 2020-01-23 04:47:05 +00:00
Jooyung Han
559ad01db9 apex_vndk uses vendor variants only
VNDK APEX is supposed to contain "vendor" variants of VNDK libraries.
This is different from normal APEXes which have "apex" variants.

Bug: 146758869
Test: build / flash / boot
Change-Id: I5e035678c337334092616b58d2e0e404788a6639
Exempt-From-Owner-Approval: Got ORV, but rebased with resolving merge conflicts.
2020-01-23 04:46:53 +00:00
Colin Cross
b1d8c99e12 Pass desired size proptools.FilterPropertyStructSharded
proptools.FilterPropertyStructSharded now takes a desired max size
of the name.

Fixes: 146234651
Test: all soong tests
Change-Id: Ifa10039e5f55a5d0f4a35172983d43f33af88e93
2020-01-22 18:30:56 -08:00
Colin Cross
b4fecbfef4 Do a better job removing tags from arch structs
Remove more android struct tags from runtime created arch structs
to reduce the size of their names, which hit the 64kB limit in
runtime.StructOf if they are too long.

Bug: 146234651
Test: m checkbuild
Change-Id: I6362765275b93c8932eb0b1abbcb4be47031d9b1
2020-01-22 18:30:56 -08:00
Yifan Hong
50b366584b libatomic / libgcc_stripped: ramdisk_available
Test: pass
Bug: 147347110
Change-Id: If32b1060a2a984f67082cf40fbd434e9cf671ebb
2020-01-22 16:54:18 -08:00
Yifan Hong
cf4832c8bc Add target.ramdisk
Add the following:
- exclude_shared_libs
- exclude_static_libs
- static_libs

Allow to customize linking for ramdisk variant.

Test: pass
Bug: 147347110
Change-Id: I6f55f11a6fdad8029d85833a3eedc2e95a712d51
2020-01-22 16:54:18 -08:00
Yifan Hong
82db735fbc Ramdisk modules install to correct location
Install to recovery/root/first_stage_ramdisk if BOARD_USES_RECOVERY_AS_BOOT,
otherwise ramdisk. In addition, append /system if not InstallInRoot().

Test: pass
Bug: 147347110
Change-Id: Ie5895ac61a3d5d85f5d958a27c7980d6c79d997e
2020-01-22 16:54:18 -08:00
Yifan Hong
1b3348d50a Add ramdisk image.
It is similar to recovery image.
Test: m nothing -j

Change-Id: I11389777c6bfb0c0d73bbb4c70091c1e70f44077
2020-01-22 16:54:18 -08:00
Treehugger Robot
59ee253b35 Merge "Export KYTHE_KZIP_ENCODING." 2020-01-23 00:29:25 +00:00
Sasha Smundak
f77ee745d3 Export KYTHE_KZIP_ENCODING.
Bug: 146224091
Test: manual
Change-Id: I998a51565419b86a9f183c0f3228ce09d7b035c0
2020-01-22 14:41:01 -08:00
Oliver Nguyen
b6aeb14150 Merge "Add coverage flags for Clang coverage builds." 2020-01-22 22:34:35 +00:00
Hadrien Zalek
8aecc11844 Merge "Revert "Default to pattern initialization for uninitialized variables."" 2020-01-22 22:16:18 +00:00
Hadrien Zalek
6fecbee4d5 Revert "Default to pattern initialization for uninitialized variables."
This reverts commit fae236f4ee.

Reason for revert: Breaks 32-bit app tests on Cuttlefish
Bug: 148123414

Change-Id: Ieeb0b09e711a55172e6faa0035d1eb84dd6c4471
2020-01-22 21:50:56 +00:00
Sasha Smundak
6172491b91 If input zip file cannot opened, print its name.
Test: touch empty.zip; merge_zips out.zip empty.zip
(output should contain `empty.zip: zip: not a valid zip file`)

Change-Id: If707876af2301d7ad7d065c004cffb0c2491c7df
2020-01-22 10:21:43 -08:00
Ulya Trafimovich
7eebb4f4e8 For JIT-zygote config, add "apex" boot image files to the ART apex.
JIT-zygote boot image was split into primary and framework extension
in CL I5493e575ebf90bad1d5ad2850004d54590bbc079, but that CL forgot to
add JIT-zygote boot image files to the ART apex. Add them now.

Test: Steps 1-3:

  1. Temporarily enable Jit zygote in the product device config (in
    this case device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. Build and flash:
    $ lunch aosp_walleye-userdebug && m \
    && adb reboot bootloader && fastboot flashall -w

  3. Ensure that apex.art and apex-framework.art are mapped in the
    JIT-zygote address space:

    $ adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex.art
    6ffde000-70102000 rw-p 00000000 00:00 0                                  [anon:dalvik-/apex/com.android.art/javalib/apex.art]
    701e5000-70262000 rw-p 00000000 00:00 0                                  [anon:dalvik-/apex/com.android.art/javalib/apex.art]
    72715cf000-72715d4000 r--p 000ba000 07:98 93                             /apex/com.android.art/javalib/arm64/apex.art

    $ adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex-framework.art
    7084b000-70cc2000 rw-p 00000000 00:00 0                                  [anon:dalvik-/system/framework/apex-framework.art]
    70fd8000-71109000 rw-p 00000000 00:00 0                                  [anon:dalvik-/system/framework/apex-framework.art]
    7271517000-7271526000 r--p 00295000 fc:00 1268                           /system/framework/arm64/apex-framework.art

Change-Id: I985e15de9f38f345ba3e99451c6d9b8e2cb5d698
2020-01-22 15:27:08 +00:00
Treehugger Robot
0c9c0f045f Merge "Remove unused java_sdk_library_import properties" 2020-01-22 06:03:25 +00:00