Commit graph

1940 commits

Author SHA1 Message Date
Automerger Merge Worker
fe46418b38 Merge "Parameterize java_sdk_library by api scope" am: b8e5c68fef am: 2f7d7215b1 am: d09578c368
Change-Id: I91b022d54c51dbd5880e7e6c19d776775b2a5584
2020-01-31 17:14:33 +00:00
Automerger Merge Worker
009b117b67 Merge "Use merge_csv python_binary." am: 44fc5d201f am: 17fc5e20c9 am: 10874f33a3
Change-Id: I94fd82782a4c2de9e0f052085dcc027c07e8bd3d
2020-01-31 16:32:23 +00:00
Paul Duffin
d1b3a92f05 Parameterize java_sdk_library by api scope
The java_sdk_library duplicated a lot of code related to api scopes and
also did a lot of switching on apiScope. This change eliminates that
duplication and switching by converting apiScope from an enum to a
structure containing api scope specific information.

It associates dependencies with corresponding scope by using a
scopeDependencyTag that contains a reference to the associated
apiScope.

Tested by:
1) Running the command below without the changes.
2) 'mv out/dist/apistubs $ANDROID_BUILD_TOP'
3) Applying these changes.
4) Running the command again.
5) Verifying that the out/dist/apistubs was identical to before the
   change using 'meld out/dist/apistubs $ANDROID_BUILD_TOP/apistubs'

Bug: 148080325
Test: m TARGET_PRODUCT=sdk TARGET_BUILD_VARIANT=userdebug dist sdk
      m droid
Change-Id: I0de3268b12254122e94ca83d09309b06ca2f9dbe
2020-01-30 17:26:18 +00:00
Treehugger Robot
44fc5d201f Merge "Use merge_csv python_binary." 2020-01-30 15:18:58 +00:00
Automerger Merge Worker
109cc43bc6 Merge "Set LOCAL_CERTIFICATE for flattened apex APKs" am: f9192060bd am: f9884fc658 am: c73e2393b4
Change-Id: Iacec9ac70ef24bc9f7874201aff069c743c1d0e7
2020-01-29 18:22:29 +00:00
Treehugger Robot
f9192060bd Merge "Set LOCAL_CERTIFICATE for flattened apex APKs" 2020-01-29 17:42:51 +00:00
Automerger Merge Worker
a3e117d183 Merge "Store uncompressed and aligned DEX files for java libs in APEXes." am: 28e36b1455 am: 6a517467aa am: 7bd4aa2853
Change-Id: I204fa801fa92b9603d4adb26f070310bc8b48955
2020-01-29 10:53:52 +00:00
Ulyana Trafimovich
28e36b1455 Merge "Store uncompressed and aligned DEX files for java libs in APEXes." 2020-01-29 10:25:33 +00:00
Colin Cross
503c1d06e8 Set LOCAL_CERTIFICATE for flattened apex APKs
Leaving out LOCAL_CERTIFICATE for flattened apex APKs causes
the apkcerts.txt to have empty keys for those APKs, which
confuses the signing tools.  Set LOCAL_CERTIFICATE for them.

Also refactor the Certificate support to avoid introducing
duplicated handling for presigned certificates.

Bug: 147765187
Test: m apkcerts-list
Change-Id: Ife07661761cd5a89c9f009b8ce041db4dff9ec54
2020-01-28 14:03:08 -08:00
Automerger Merge Worker
1bf59ab8ab Merge "Add support for experimentally enabling RBE support on specific rules." am: 5e0dbe4e3e am: 205edb5da3 am: a171ec4e95
Change-Id: I2922a3dd955162fa9f713ebfe9fe1852d76fd2c3
2020-01-28 20:31:14 +00:00
Ramy Medhat
5e0dbe4e3e Merge "Add support for experimentally enabling RBE support on specific rules." 2020-01-28 19:49:05 +00:00
Automerger Merge Worker
726644dc90 Merge "Fix --boot-image argument in dex2oat command for boot image extension." am: 911083e3e1 am: 00cc7b0b4d am: ab4c753ac7
Change-Id: Ib2b55daf80e463c744e85f6a197e7479541a68cd
2020-01-28 18:00:22 +00:00
Ramy Medhat
8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Ulyana Trafimovich
911083e3e1 Merge "Fix --boot-image argument in dex2oat command for boot image extension." 2020-01-28 17:04:38 +00:00
Ulya Trafimovich
b0a2d37dcb Fix --boot-image argument in dex2oat command for boot image extension.
The primary image name should be "boot.art" in case of
"boot-framework.art", and "apex.art" in case of "apex-framework.art".
Although "boot.art" and "apex.art" are identical binaries, dex2oat
has hard-coded logic based on the image name.

This has been broken since boot image extension has been enabled for
JIT-zygote config in CL I5493e575ebf90bad1d5ad2850004d54590bbc079.

Test: compare boot-framework.art and apex-framework.art,
  they were identical before this CL, and differ after this CL.

Test: JIT-zygote config boots, steps 1-2:

  1. Temporarily enable Jit zygote in the product device config (in
    this case device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. Build and flash:
    $ lunch aosp_walleye-userdebug && m \
    && adb reboot bootloader && fastboot flashall -w

Change-Id: I98de271852ecc33feb9fd4c9b0addf0feba01856
2020-01-28 14:42:41 +00:00
Automerger Merge Worker
f72855d247 Merge "Abstract sdk_version string using sdkSpec type" am: 057dd35669 am: 6ca47f29ce am: be91dedf1c
Change-Id: I1d5c190c71f53858ad81df4ca20029c870df30ba
2020-01-28 01:00:35 +00:00
Treehugger Robot
057dd35669 Merge "Abstract sdk_version string using sdkSpec type" 2020-01-28 00:07:37 +00:00
Automerger Merge Worker
cdcbcdd10f Merge changes from topic "revert-1211982-dex2oat-soong-dep-LLLKNULXYJ" am: 699cf17f53 am: 90e753d47e am: 546538926f
Change-Id: I63175eb2054119123fa6aa17231f617186f2d633
2020-01-27 15:32:02 +00:00
Martin Stjernholm
699cf17f53 Merge changes from topic "revert-1211982-dex2oat-soong-dep-LLLKNULXYJ"
* changes:
  Revert "Separate dexpreopt.GlobalSoongConfig to allow independen..."
  Revert "Move the Once cache for dexpreopt.GlobalConfig into the ..."
  Revert "Get the dex2oat host tool path from module dependency on..."
2020-01-27 13:47:31 +00:00
Hans Boehm
e4b5342d43 Revert "Separate dexpreopt.GlobalSoongConfig to allow independen..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: Ice3990225635a737e49e9aed7373f06516fccea3
2020-01-27 13:44:24 +00:00
Hans Boehm
453bf09853 Revert "Move the Once cache for dexpreopt.GlobalConfig into the ..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I6b656afb5feaad70d958b9d38b6c6eab7b03fba1
2020-01-27 13:44:03 +00:00
Hans Boehm
7b2e6f3ed7 Revert "Get the dex2oat host tool path from module dependency on..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I0cf6fef04434fd3c0b7957ee9abdcaabeb20b9d8
2020-01-27 13:01:16 +00:00
Ulya Trafimovich
f491dde8ea Store uncompressed and aligned DEX files for java libs in APEXes.
Test: aosp_walleye-userdebug boots.

Test: In internal, cherry-picked this CL, walleye-userdebug boots.

Test: In internal, ensure that DEX file wifi-service.jar in the
  com.android.wifi APEX is now stored uncompressed:

    $ zipinfo out/target/product/walleye/apex/com.android.wifi/javalib/wifi-service.jar
    Archive:  out/target/product/walleye/apex/com.android.wifi/javalib/wifi-service.jar
    Zip file size: 3086879 bytes, number of entries: 3
    -rw----     2.0 fat  3086304 b- stor 08-Jan-01 00:00 classes.dex
    -rw----     2.0 fat      339 b- defN 80-Feb-01 00:00 META-INF/proguard/androidx-annotations.pro
    -rw----     2.0 fat       45 b- defN 08-Jan-01 00:00 META-INF/MANIFEST.MF
    3 files, 3086688 bytes uncompressed, 3086480 bytes compressed:  0.0%

  Before the change it was:

    $ zipinfo out/target/product/walleye/apex/com.android.wifi/javalib/wifi-service.jar
    Archive:  out/target/product/walleye/apex/com.android.wifi/javalib/wifi-service.jar
    Zip file size: 1241938 bytes, number of entries: 3
    -rw----     2.0 fat  3086304 bl defN 08-Jan-01 00:00 classes.dex
    -rw----     2.0 fat      339 b- defN 80-Feb-01 00:00 META-INF/proguard/androidx-annotations.pro
    -rw----     2.0 fat       45 b- defN 08-Jan-01 00:00 META-INF/MANIFEST.MF
    3 files, 3086688 bytes uncompressed, 1241526 bytes compressed:  59.8%

Bug: 148099857
Bug: 146984232

Change-Id: Ia302967b280be85665032cc870c5d5d2419586fc
2020-01-27 10:19:10 +00:00
Jiyong Park
6a927c4e6a Abstract sdk_version string using sdkSpec type
The value format that sdk_version (and min_sdk_version, etc.) can have
has consistently evolved and is quite complicated. Furthermore, with the
Mainline module effort, we are expected to have more sdk_versions like
'module-app-current', 'module-lib-current', etc.

The goal of this change is to abstract the various sdk versions, which
are currently represented in string and is parsed in various places,
into a type called sdkSpec, so that adding new sdk veresions becomes
easier than before.

The sdk_version string is now parsed in only one place 'SdkSpecFrom', in
which it is converted into the sdkSpec struct. The struct type provides
several methods that again converts sdkSpec into context-specific
information such as the effective version number, etc.

Bug: 146757305
Bug: 147879031
Test: m
Change-Id: I252f3706544f00ea71c61c23460f07561dd28ab0
2020-01-25 21:56:43 +09:00
Automerger Merge Worker
e936d4072e Merge "Revert "Revert "Allow codename.fingerprint format for minSdkVersion""" am: cf0bba7a30 am: 0e7b913387 am: d786be38fc
Change-Id: Ic8fbbbefb3e2f8356d3d1220cdcdda0bce3b7872
2020-01-25 06:44:09 +00:00
Baligh Uddin
cf0bba7a30 Merge "Revert "Revert "Allow codename.fingerprint format for minSdkVersion""" 2020-01-25 04:12:30 +00:00
Automerger Merge Worker
ac19633b41 Merge "Create a highmem pool and put metalava into it" am: 6cc0515e82 am: 22a73155aa am: 23671f60c2
Change-Id: I4fac5e5cccbd73da406e384816cbc47a6b69ac10
2020-01-25 03:37:39 +00:00
Automerger Merge Worker
27efc54be4 Merge changes from topic "dex2oat-soong-dep" am: cf52e102ce am: fe568ba2a1 am: 1d1c492d20
Change-Id: I4797dc2b8837c1d982dd324aef619354fb61934e
2020-01-25 03:37:03 +00:00
Automerger Merge Worker
ad00cd0fcc Move the Once cache for dexpreopt.GlobalConfig into the dexpreopt package. am: dae8a80426 am: 0064a8888e am: 443178189c
Change-Id: I88bb622216b39dbe45c79deedf602042267ad632
2020-01-25 03:37:01 +00:00
Automerger Merge Worker
2ca3894edc Separate dexpreopt.GlobalSoongConfig to allow independent caching of it. am: be9d0d21d1 am: 852db0c1da am: 8e761f91c8
Change-Id: Ibf451ceb57c5770727ee1674634ef4d37e486213
2020-01-25 03:36:58 +00:00
Treehugger Robot
6cc0515e82 Merge "Create a highmem pool and put metalava into it" 2020-01-25 01:09:43 +00:00
Baligh Uddin
f62013736a Revert "Revert "Allow codename.fingerprint format for minSdkVersion""
This reverts commit 230e241f58.

Reason for revert: This is a revert of a revert.  Downstream problem has been fixed and have been validated locally and via Forrest build.

Change-Id: I89c51d25b3adb818ea44a983d0ac681a88790d8c
2020-01-24 23:15:44 +00:00
Martin Stjernholm
335d596384 Get the dex2oat host tool path from module dependency on the binary
module.

This uses the Once cache for GlobalSoongConfig to propagate the dex2oat
path from a module dependency to the singletons (both the one that
writes out dexpreopt_soong.config and the one that creates the
dexpreopted boot images). Unless dexpreopting is disabled altogether
through DisablePreopt in dexpreopt.config, that means:

- We must ensure at least one module registers a dex2oat tool
  dependency and resolves a GlobalSoongConfig using it, or else the
  singletons fail. That means we litter dex2oat dependencies in java
  modules even when they won't get dexpreopted and hence don't really
  need them.

- We still assume there's at least one java_library or android_app in
  the build.

Bug: 145934348
Test: m
  (check that out/soong/dexpreopt_soong.config points to dex2oatd64)
Test: env USE_DEX2OAT_DEBUG=false m
  (check that out/soong/dexpreopt_soong.config points to dex2oat)
Test: env OUT_DIR=out-tools prebuilts/build-tools/build-prebuilts.sh
  on the aosp-build-tools branch
Change-Id: I38317f2d5e5cf6822f87daae25745767c8a3656c
2020-01-24 22:18:42 +00:00
Martin Stjernholm
dae8a80426 Move the Once cache for dexpreopt.GlobalConfig into the dexpreopt
package.

Preparation for a future CL that will need to get the make-written
GlobalConfig from within dexpreopt.

Also rename the Load*Config functions to Parse*Config, since they don't
actually load the config files anymore.

Bug: 145934348
Test: m
Change-Id: I71df11c1e042ca8135d273a7263e9539ea3cd68f
2020-01-24 22:18:42 +00:00
Martin Stjernholm
be9d0d21d1 Separate dexpreopt.GlobalSoongConfig to allow independent caching of
it.

Introduce a Once cache for GlobalSoongConfig to allow it to get binary
tool paths from ordinary module dependencies (coming in a future CL)
that are then reused in singletons.

Bug: 145934348
Test: m
Change-Id: I440a09dba7d337965a196527566b0966a18e3653
2020-01-24 22:09:54 +00:00
Automerger Merge Worker
4efd8ba93c Do not remove RRO resources am: f0f747c949 am: 033eeada51 am: 3643cd99e7
Change-Id: I64254d6a94d49311da61022bc5a744ac26632bbf
2020-01-24 21:36:19 +00:00
Colin Cross
8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Jaewoong Jung
f0f747c949 Do not remove RRO resources
Resource configs should not be deduped when building RROs since it
would be impossible to override some resource configs with the same
value as the default config. Also, aapt2 removes resources that do not
have default configurations. If an overlay attempts to overlay a
non-default configuration without overlaying the default, the resource
will be removed and the value will not be overlaid at all.

Bug: 146227008
Fixes: 119811120
Test: app_test.go
Change-Id: I834a58b18d1e74a0f6b3de3d0523009788787e42
2020-01-24 10:35:52 -08:00
Automerger Merge Worker
1ae81e980a Merge "Add runtime_resource_overlay." am: 84f71af41c am: 51e926c45d am: 8f84cbc441
Change-Id: Ia827ca892800111c8530d617c2275f0b8ba62776
2020-01-24 01:01:36 +00:00
Treehugger Robot
84f71af41c Merge "Add runtime_resource_overlay." 2020-01-24 00:23:42 +00:00
Jaewoong Jung
9befb0c326 Add runtime_resource_overlay.
Fixes: 119811120
Test: app_test.go
Test: Converted and built IconPackFilledSystemUIOverlay
Change-Id: I71841148c25f820ba829f751a201d2c771c8bd20
2020-01-23 14:24:41 -08:00
Automerger Merge Worker
4acb8f82c8 Merge "For JIT-zygote config, add "apex" boot image files to the ART apex." am: c18235cb86 am: 4504c82639 am: e663326c09
Change-Id: Ifeef80c0aef719076ec869423906228b4ced26f3
2020-01-23 12:32:27 +00:00
Ulya Trafimovich
7eebb4f4e8 For JIT-zygote config, add "apex" boot image files to the ART apex.
JIT-zygote boot image was split into primary and framework extension
in CL I5493e575ebf90bad1d5ad2850004d54590bbc079, but that CL forgot to
add JIT-zygote boot image files to the ART apex. Add them now.

Test: Steps 1-3:

  1. Temporarily enable Jit zygote in the product device config (in
    this case device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. Build and flash:
    $ lunch aosp_walleye-userdebug && m \
    && adb reboot bootloader && fastboot flashall -w

  3. Ensure that apex.art and apex-framework.art are mapped in the
    JIT-zygote address space:

    $ adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex.art
    6ffde000-70102000 rw-p 00000000 00:00 0                                  [anon:dalvik-/apex/com.android.art/javalib/apex.art]
    701e5000-70262000 rw-p 00000000 00:00 0                                  [anon:dalvik-/apex/com.android.art/javalib/apex.art]
    72715cf000-72715d4000 r--p 000ba000 07:98 93                             /apex/com.android.art/javalib/arm64/apex.art

    $ adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex-framework.art
    7084b000-70cc2000 rw-p 00000000 00:00 0                                  [anon:dalvik-/system/framework/apex-framework.art]
    70fd8000-71109000 rw-p 00000000 00:00 0                                  [anon:dalvik-/system/framework/apex-framework.art]
    7271517000-7271526000 r--p 00295000 fc:00 1268                           /system/framework/arm64/apex-framework.art

Change-Id: I985e15de9f38f345ba3e99451c6d9b8e2cb5d698
2020-01-22 15:27:08 +00:00
Automerger Merge Worker
ed9176a259 Merge "Remove unused java_sdk_library_import properties" am: 0c9c0f045f am: 07e5b368c7 am: 1bb242bb7a
Change-Id: Ie82fa67604ef2cd46d28411a4b9c1db47a6bad74
2020-01-22 06:42:15 +00:00
Treehugger Robot
0c9c0f045f Merge "Remove unused java_sdk_library_import properties" 2020-01-22 06:03:25 +00:00
Paul Duffin
29f5576357 Remove unused java_sdk_library_import properties
The properties are just passed through to the java_import module that
it creates to import the stubs so could be used if set. However, none
of the current usages of java_sdk_library_import actually set them and
it is not clear why the following would ever be set on
java_sdk_library_import:
* installable
* exclude_files
* exclude_dirs

The above properties look as though they were simply copied from the
ImportProperties struct.

The libs property is also unused but it seems as though it should be
used to improve compatibility with the source module. e.g.
the source module for android.test.runner has android.test.base and
android.test.mock in its libs. However, the prebuilt version of it
does not.

Bug: 148080325
Test: m nothing
Change-Id: I710e3962b2e34b2b0cc81685d07d4948cef4e6ad
2020-01-21 18:37:58 +00:00
Automerger Merge Worker
1660ac1fb6 Merge "Revert "Allow codename.fingerprint format for minSdkVersion"" am: 070d69c74c am: 42a16b7d69 am: 7245755399
Change-Id: Ifcea8861733463b92a46de6ca08b2d6a8ec0a090
2020-01-21 14:51:42 +00:00
Pete Gillin
230e241f58 Revert "Allow codename.fingerprint format for minSdkVersion"
This reverts commit 014a85712d.

Reason for revert: Caused vendor/google/build/build_mainline_modules.sh to fail with `Error: minSdkVersion (10000) is greater than maxSdkVersion (30)`.

Bug: 130541924
Change-Id: Ifa233bf40a674481d21b61ee816c5fdde8201080
2020-01-21 13:44:50 +00:00
Automerger Merge Worker
52337edeb4 Merge "Allow codename.fingerprint format for minSdkVersion" am: 1cc755ac9f am: da726a802c am: c7caac1032
Change-Id: I9b6ee2bf0c0328bf496affd960fe67ef0b0fe30d
2020-01-21 02:33:59 +00:00
Artur Satayev
b01dd44266 Use merge_csv python_binary.
Instead of using .py file directly.

Bug: 145132366
Test: diff -w out/soong/hiddenapi/hiddenapi-greylist.csv
Change-Id: Ib79019234b4ae05486e831b30d27bd1be3bbba67
2020-01-20 17:59:02 +00:00