Commit graph

188 commits

Author SHA1 Message Date
Cole Faust
06ea531ab3 gofmt soong
Test: Presubmits
Change-Id: Ia76c35ba51685eca29df51738000eacd3f43ce20
2023-10-18 17:42:59 -07:00
Jiakai Zhang
4d90da29a8 Add a Make variable that overrides configured jar locations.
We had hardcoded logic to override "framework-minus-apex", but we need
to override more jars, so this CL adds a general solution for this.

Bug: 290583827
Test: m nothing
Change-Id: I211ebda7aa2803886ab6e6d081d26327665e49a6
2023-07-14 12:13:30 +01:00
Jiakai Zhang
cb13b5d1bd Refactor dexpreopt for boot jars to make it flexible to config changes.
In the past, dexpreopt for boot jars was very inflexible, and it was
incredibly hard to make a change that is as simple as adding a jar to a
boot image. Boot image generation was handled by
"platform_bootclasspath" and "bootclasspath_fragment" separately. This
caused not only code duplication but also the inflexiblity as such a
design did not fit today's use cases, where a boot image may take jars
from multiple mainline modules and the platform, and a mainline module
can contribute to multiple boot images. The design casued a huge
maintenance burden as any change to the boot image cost multi-week
efforts.

In recent years, efforts have been made to improve this a bit by a bit.
This change is another step towards making dexpreopt reasonable.

After this change, all boot images are generated by "dex_bootjars",
which is in build/soong and is therefore available on both the full
source tree and the thin manifest (master-art). The change decouples
profile generation/extraction from boot image generation. Profiles for
mainline modules are still handled by "bootclasspath_fragment"
because they need to be packed into APEXes when building mainline
modules and extracted from APEXes whem building the system image from
prebuilt modules. Boot images are not handled by
"bootclasspath_fragment" anymore.

Bug: 290583827
Test: m (all existing tests are still passing)
Test: Manually checked that the boot images are exactly the same as
  before.
Change-Id: Ib5a5f401bee334ffcab5c26618e0c8888b84575a
2023-07-14 12:13:28 +01:00
Jiakai Zhang
b898c112a2 Merge changes I0918f2fa,I3fc6ff91,I7adc97cb,I674a5fa1,I22c45cbf into main
* changes:
  Change the profile path on host.
  Extract duplicate code to common helper functions.
  Fix dumpOatRules.
  Remove Modules() from BootclasspathFragmentApexContentInfo.
  Fix some tests for dexpreopt and remove unnecessary tests.
2023-07-11 20:59:07 +00:00
Jiakai Zhang
3444e561dc Change the profile path on host.
This is to decouple profile generation from image configs. A boot image
profile is either for a mainline module or for the platform, which is
orthogonal to boot images.

Bug: 290583827
Test: m nothing
Change-Id: I0918f2fa945a2af1839f1a08ecede331c64d0317
2023-07-11 17:59:24 +01:00
Jiakai Zhang
b1639db53f Extract duplicate code to common helper functions.
Also, fall back to using a default name for the dexpreopt directory if
we are not building for Android.

Bug: 290583827
Test: m nothing
Change-Id: I3fc6ff9142a2dcdf995796f75891b242fe2848d0
2023-07-11 17:59:05 +01:00
Jiakai Zhang
da47d3c6f3 Fix dumpOatRules.
This method generates a build rule that runs oatdump for debugging
purposes.

- Pass "--runtime-arg -Xgc:CMC" to oatdump if UFFD GC is enabled.
- Build the phony name from the image name to avoid conflicts when using
  this method for multiple boot images.

Bug: 290583827
Test: m dump-oat-boot
Change-Id: I7adc97cb85b571486b78f173fb80ff7da2c4bfe4
2023-07-11 16:27:01 +01:00
Nicolas Geoffray
1ad8a13e47 Don't strip oat files on host.
We need the debugging info for perf.

Test: m
Change-Id: Iceaa72d8b1c7d27ea113ca72c8af1174ce6f05ba
2023-06-29 15:45:39 +01:00
Jiakai Zhang
225c4767ac Only pass "-Xgc:CMC" when building for Android.
The runtime always uses CC on host unless CMC is explicitly asked
(http://cs/android-internal/art/runtime/gc/collector/mark_compact.cc;l=212-214;rcl=50fe05b4f4650bc0888d2c218a1bfa476f811ea0).
This CL matches the build system behavior with the runtime.

Bug: 285228269
Test: m test-art-host-gtest-art_dex2oat_tests
Change-Id: I652ad6f754ab8c51283e5fb0e9e1ba644629fa11
2023-06-01 13:27:25 +00:00
LaMont Jones
0c10e4dcc0 Parallelize singleton execution
Bug: 281536768
Test: manual, presubmits
Change-Id: I57fdc76ba6b277e88e196b506af87127a530fd37
2023-05-19 20:31:32 +00:00
Jiakai Zhang
c08c162b5a Dexpreopt ART jars and framework jars together.
Bug: 280776428
Test: atest art_standalone_dexpreopt_tests
Test: -
  1. m
  2. Check .invocation file (http://gpaste/6498044089466880)
  3. Check files in $ANDROID_PRODUCT_OUT/system/framework/x86_64
Test: -
  1. m dist
  2. Check files in out/dist/boot.zip
Test: -
  1. art/tools/buildbot-build.sh --host
  2. m test-art-host-gtest
  3. art/test/testrunner/testrunner.py --host
Test: m build-art-target-golem
Change-Id: I89490252e56a05edab03fdddc6539fa4d7f79756
2023-05-11 19:04:17 +01:00
Jiakai Zhang
09d88df040 Unify installDirOnHost and installDirOnDevice.
These two fields never do what they are described to do. This CL unifies
them to avoid the confusion.

Bug: 280440941
Test: m
Change-Id: I3652d73a50832a2e494d9f5cae750f5fc38293b4
2023-05-11 18:25:18 +01:00
Jiakai Zhang
b47caccbc0 Remove the ability to install boot images by bootclasspath_fragments.
We don't need this anymore because we are going to compile ART jars and
framework jars together.

Bug: 280776428
Test: m
Change-Id: I070157530449a1bb5779e25984c367df3dde7b36
2023-05-11 18:25:18 +01:00
Jiakai Zhang
bc698cd28a Revert^2 "Generate boot image profiles even if dexpreopt is disabled."
Revert submission 2580631-revert-2574032-XXTWCJDTDQ

Reason for revert: Fixed build breakages

Reverted changes: /q/submissionid:2580631-revert-2574032-XXTWCJDTDQ

Bug: 280440941
Test: lunch aosp_cf_riscv64_minidroid-userdebug && m UNSAFE_DISABLE_HIDDENAPI_FLAGS=true dist
Test: Disable dex2oat on host (to simulate macOS) and build
Change-Id: I5f7f746ca1d4da660fe0c40115e6c71750dfdccc
2023-05-08 21:28:13 +01:00
Qiao Yang
7081aaf874 Merge changes from topic "revert-2574032-XXTWCJDTDQ"
* changes:
  Revert "Generate boot image profiles even if dexpreopt is disabled."
  Revert "Generate app profiles even if dexpreopt is disabled."
2023-05-05 16:43:45 +00:00
Qiao Yang
8d8c660710 Revert "Generate boot image profiles even if dexpreopt is disabled."
Revert submission 2574032

Reason for revert: DroidMonitor-triggered revert due to breakage <https://android-build.googleplex.com/builds/quarterdeck?branch=aosp-master&target=aosp_cf_riscv64_minidroid-userdebug&lkgb=10069333&lkbb=10075041&fkbb=10071083>, bug <b/280902279>

Reverted changes: /q/submissionid:2574032

Change-Id: I8e99f8231639198b149ea8d822ee7f9a5b391a89
BUG: <280902279>
2023-05-05 15:03:24 +00:00
Jiakai Zhang
dbfcf85c53 Merge changes I4e721b47,I1bf05ade
* changes:
  Generate app profiles even if dexpreopt is disabled.
  Generate boot image profiles even if dexpreopt is disabled.
2023-05-04 22:13:08 +00:00
Jiakai Zhang
b95f8345c8 Generate boot image profiles even if dexpreopt is disabled.
Bug: 280440941
Test: -
  1. Patch ag/22302622 to disable dexpreopt.
  2. See boot image profiles still generated.
Change-Id: I1bf05ade53fa83f3dba46f28a8f9246ba1fdf664
2023-05-03 15:25:18 +01:00
Jeongik Cha
4753b39cca Get rid of DeviceName() from path related to dexpreopt
As-is, dexpreopt config and bootjar is in the dir including
DeviceName(). It causes unnecessary dexpreopt invocation when target is
changed repeatly. To avoid it, rename dir to common name.

Bug: 278833696
Test: m
Change-Id: I2818d9ae09e6c22ca6989cc8efdb70b470ec502c
2023-04-28 11:50:18 +09:00
Jiakai Zhang
b879620ca8 Revert^2 "Generate a boot image extension for mainline BCP...
Revert submission 2465993-boot-image-mainline-revert

Reason for revert: Relanding the changes

Reverted changes: /q/submissionid:2465993-boot-image-mainline-revert

Change-Id: Ie611cf3a1ff4be7a7beab164f69b4186c474be5c
2023-03-16 08:46:29 +00:00
Cole Faust
18994c73f1 Replace SortedStringKeys with SortedKeys
Now that we have generics.

Bug: 193460475
Test: presubmits
Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-28 16:51:32 -08:00
Jiakai Zhang
63086d7af0 Omit "--compiler-filter" if the profile is disabled.
This fixes the Golem regression caused by aosp/2453067. On Golem, the
profile disabled and dex2oat uses the default "speed" compiler filter.

Bug: 269230245
Test: art/tools/golem/build-target.sh --machine-type=android-armv8 --golem=art-interpreter
Change-Id: I4fc0dbf9eac8839f7bacd264ddab177956ddd58e
2023-02-28 15:37:31 +00:00
Jiakai Zhang
8fe3a415b5 Refactor dexpreopt for boot jars to allow more complex dependencies.
After this change, the dependency hierachy can be arbitrarily deep. For
example, you can have one boot image that extends another boot image
that extends yet another boot image.

Bug: 269230245
Test: m
Change-Id: I096d0b57bda36b982ecc97378647f9c59071a3bf
2023-02-24 17:13:54 +00:00
Jiakai Zhang
8e9ea8bb83 Add a new option compilerFilter to bootImageConfig.
After this change, the compiler filter can be specified by the option,
and profiles will not be used if the compiler filter is not
a profile-guided one.

This change also allows preloadedClassesFile to be empty.

Bug: 269230245
Test: m
Change-Id: I65e6b7209d2f0510bcc784a62623ab402b7f96bb
2023-02-23 17:50:46 +00:00
Jiakai Zhang
616be0695e Pass "-Xgc:CMC" to dex2oat when userfaultfd GC is enabled.
This change is a no-op change. It doesn't enable userfaultfd GC by
default. ENABLE_UFFD_GC=true can be passed to the build system to enable
userfaultfd GC for testing purposes.

Bug: 242553398
Test: -
  1. lunch aosp_oriole-userdebug
  2. ENABLE_UFFD_GC=true m
  3. See "--runtime-arg -Xgc:CMC" in .invocation files.
Change-Id: I789b49a71b9604fd41bf1ef77d0ac5bac4cbdf25
2022-11-16 12:04:53 +00:00
Paul Duffin
20d90e3e51 Prevent bootImageVariant.licenseMetadataFile being set twice
Previously, in a build containing source and prebuilt art
bootclasspath_fragments, the bootImageVariant.licenseMetadataFile was
set twice with the source always being set after the prebuilt and
so winning.

This change only sets bootImageVariant.licenseMetadataFile for the
active module so it will use the prebuilt's license file if that is
preferred.

Bug: 245956352
Test: m nothing
Change-Id: I948c7e5123169452f67c85ad98c4bbdb90a5d2de
2022-10-06 11:06:53 +01:00
Paul Duffin
e10a9f2e5e Remove deviceInstalls from bootImageVariant
The use of this field to return information from buildBootImageVariant
up the call stack to one of the callers resulted in data races being
detected. This change simply passes the deviceInstalls up the call
stack.

Bug: 245956352
Test: m nothing
      go test -race ./sdk/... -run TestSnapshotWithBootclasspathFragment_ImageName -test.count 100
      # Run the previous command without this change and sometimes it
      # shows the data race around deviceInstalls. When run with this
      # change it reports no data races.
Change-Id: I3c73920dcb17a6c89a63c6a9c3a0bb049a98a690
2022-10-05 12:33:02 +01:00
Paul Duffin
9f6ac0bb42 Remove profilePathOnHost from bootImageConfig
The use of this field to return information from bootImageProfileRule
up the call stack to one of the users resulted in data races being
detected. This change simply returns the profile path back up the call
stack.

Bug: 245956352
Test: m nothing
      go test -race ./sdk/... -run TestSnapshotWithBootclasspathFragment_ImageName -test.count 100
      # Run the previous command without this change and sometimes it
      # shows the data race around profilePathOnHost. With this change
      # that data race is not reported. Although there is still another
      # data race.
Change-Id: I03b09e514cc94f2a6c9d5117d3b2f130cc2e4f5b
2022-10-05 12:33:02 +01:00
Paul Duffin
a8df7e1c5b Document bootImageConfig and bootImageVariant structs
Add warnings to the structs to explain how they are supposed to be used
and deprecate fields that are used incorrectly.

Bug: 245956352
Test: m nothing
Change-Id: I090698287b96fd37102b88beb5d7252977bddc54
2022-10-05 12:32:02 +01:00
Colin Cross
d079e0b270 Reformat build/soong for go 1.19
Test: none
Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-08-17 10:43:13 -07:00
Nicolas Geoffray
58d349cbe5 Add a new file for the boot image.
In that file, we can list methods which we want to compile but that our
boot image profiling implementation did not cover.

Test: m
Bug: 235557326
Change-Id: I839727c231a09b9208d00f3205996f2add8779bd
2022-07-11 17:47:58 +01:00
Ulya Trafimovich
442ec7fc07 Remove obsolete dexpreopt config option UseArtImage.
Bug: 170935728
Test: lunch aosp_cf_x86_64_phone-userdebug && m && lunch_cvd
Change-Id: Id83b9085794fd2ec338077ca909f84e04ccad041
2022-04-27 13:36:47 +01:00
Nicolas Geoffray
1a9df2604c Merge "Pass --preloaded-classes to boot image compilation." 2022-03-18 13:41:59 +00:00
Colin Cross
e7fe0962f4 Export the license metadata files for dexpreopted bootjars to Make
Track the license metadata file of the module used to dexpreopt
bootjars and pass it to Make.

Bug: 224612665
Test: m alllicensemetadata reportmissinglicenses
Change-Id: I8466c3d8edf7dc44976c2a343bd38799c6617db2
2022-03-15 19:56:45 -07:00
Nicolas Geoffray
b9a46fb624 Pass --preloaded-classes to boot image compilation.
Test: m
Bug: 162110941
Change-Id: I0df016c5d170ca101cd18cacc331499ae76677a6
2022-03-15 14:28:38 +00:00
Jiakai Zhang
6decef916c Allow installing boot images outside of APEX.
After this change, `bootImageConfig.installDirOnDevice` can be set to a
path outside of the APEX, in which case, the boot image will not be
installed in the APEX. Instead, it will be installed to the given path
by Make.

This is a no-op change. Current behavior is not affected.

Bug: 211973309
Test: m nothing
Test: -
  1. m com.android.art
  2. See the boot image still being installed in the ART APEX.
Test: -
  1. Change `installDirOnDevice` of the ART boot image config to
     `system/framework`.
  2. See the boot image being installed in `/system/framework/<arch>`.
Change-Id: Ib13b17cc9e94dc5754c9b51b04df3307323b8783
2022-01-13 14:50:05 +00:00
Martin Stjernholm
ed3df44363 Provide some guidance for the confusing "does not provide a dex boot
jar" message.

Test: n/a
Bug: 207778789
Change-Id: I720471568e0b57636fb979c93a689895e9beaa02
2021-12-10 16:38:23 +00:00
Jiakai Zhang
49b1eb6b04 Add ART boot image profile to the ART APEX.
We will need the profile when we generate the primary boot image on
device.

Bug: 203492478
Test: Run `banchan com.android.art x86_64 && m` and see
  `$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof`.
Test: Run `lunch aosp_cf_x86_64_phone-userdebug && m` and see both
  `$ANDROID_PRODUCT_OUT/apex/com.android.art/etc/boot-image.prof` and
  `$ANDROID_PRODUCT_OUT/system/etc/boot-image.prof`, in different
  sizes.
Test: Start Cuttlefish with the built image and see both
  `/apex/com.android.art/etc/boot-image.prof` and
  `/system/etc/boot-image.prof` on device.
Change-Id: Id879dc49b234133dfbb9563814328661a1f4a6c0
2021-12-03 20:02:10 +00:00
Martin Stjernholm
9fdd87d7e4 Do not pass --compiler-filter=speed-profile with --profile-file.
It is the default, and hence unnecessary to pass explicitly. This
allows some other compiler filter to be passed through BootFlags.

Test: m out/host/linux-x86/apex/art_boot_images/javalib/x86_64/boot.oat
  on master-art and check that it doesn't change
Test: m droid
  on master and check that
  out/soong/.intermediates/art/build/apex/com.android.art/android_common_com.android.art_image/image.apex/javalib/x86_64/boot.oat
  and out/target/product/vsoc_x86_64/system/framework/x86_64/boot-framework.oat
  are identical.
Bug: 174746397
Change-Id: Ie281e55aaa641d5fd8dcc5c7e201c7870cc934c1
2021-11-22 19:05:17 +00:00
Jiakai Zhang
46eb0fd501 Remove updatable-bcp-packages.txt.
Historically, this file was used by the `--updatable-bcp-packages-file`
flag for dex2oat. The flag is no longer needed and is being deprecated.

Unfortunately, we cannot remove `permitted_packages` and all the code
related to it because we still need it for checking the module
compatibility with Q and R.

Bug: 200241946
Test: m nothing
Change-Id: Ie3be54fd47a847ba0caf627d12da76b415d99466
2021-09-24 03:19:05 +00:00
Spandan Das
10ea4bf9d4 Skip dexbootjar check using AllowMissingDependencies flag
Soong checks that every module part of a bootclass_fragment provides a
dex boot jar file, even if the module is not a dependency of vendor.img

Using AllowMissingDepdencies() to skip this check allows vendors to be
more aggressive in removing projects from their source tree

Test: In build/soong, run go test ./...
Test: m nothing
Bug: 196306898
Bug: 192616764
Change-Id: I78b062afdc19a6a3251aa8552230f3fcf334b6fb
2021-08-27 02:54:07 +00:00
satayev
4102c0cc47 Merge "Rename UpdatableBootJars to ApexBootJars." 2021-07-26 17:59:36 +00:00
satayev
d604b210c4 Rename UpdatableBootJars to ApexBootJars.
Note that ART apex boot jars and core-icu4j are exceptions here as they
are not part of ApexBootJars. ART apex boot jars are defined in their
own variable, while core-icu4j is treated as a regular non-updatable
boot jar.

Bug: 191127295
Test: atest CtsClasspathsTestCases
Change-Id: I3cea3d82ef521655a1a5ffa8cae2258ab9d08bfc
2021-07-22 17:35:42 +01:00
Colin Cross
0c66bc615b Replace android.BuildOs with Config.BuildOS
Replace the android.BuildOs constant with Config.BuildOS so that it
can vary based on the product config.

Bug: 190084016
Test: all Soong tests
Change-Id: Ia67f872d8b2ab788747a22e3a9659dc21c9775cd
2021-07-20 12:46:48 -07:00
Martin Stjernholm
9fe17bc8d9 Merge "Document how APEXes are identified in the boot jar variables." 2021-07-15 18:57:00 +00:00
satayev
f208a5766e Update a comment about PRODUCT_UPDATABLE_BOOT_JARS.
Bug: 191127295
Test: n/a
Change-Id: I3d8e8689a79be79714b01cfd8abac34e7d963aee
2021-07-15 13:22:31 +01:00
Martin Stjernholm
1d51ea951e Document how APEXes are identified in the boot jar variables.
Test: n/a - comment changes only
Bug: 191269918
Change-Id: I5ccb5afb6bef5fa5af9d528dd1d6ffd555448d9a
2021-07-08 23:19:56 +01:00
Paul Duffin
a56be7d781 Generate boot images for host from prebuilts
Previously, when building from prebuilts boot no rules were created to
produce the boot image files for the host, i.e. the OS on which the
build was running. That caused problems with checkbuilds. No rules were
produced as there was no host variant of a prebuilt apex to provide
them.

This change restructures the code to allow the prebuilt bootclasspath
fragment to build the host variants of the files from the dex files
provided by the prebuilt APEX. The generated files will not be the same
as they would be if built from source as there is no boot image profile
to use but it should be sufficient to satisfy the checkbuild target and
allow for host side testing.

Bug: 192575099
Test: m SOONG_CONFIG_art_module_source_build=false droid dist checkbuild
Change-Id: I6af00f19bb71aa18dd462f5eac6aa38e3e721023
2021-07-06 13:45:46 +01:00
Paul Duffin
8fc51a8eb6 Switch boot image generation to use GetGlobalSoongConfig(ctx)
Previously, boot image generation used GetCachedGlobalSoongConfig(ctx)
in order to get access to the GlobalSoongConfig. That required that
some other part of the code had called GetGlobalSoongConfig(ctx) to
initialize the cached value. That was left over from when the boot
image generation was done in a singleton which could not call
GetGlobalSoongConfig(ctx) directly. That is no longer true.

This change switches the uses of GetCachedGlobalSoongConfig(ctx) to
GetGlobalSoongConfig(ctx) and removes the now unnecessary call to
GetGlobalSoongConfig(ctx) from outside the functions.

Bug: 192575099
Test: m nothing
Change-Id: I34b7b1526d072d8b09fda7caa96e381695888e16
2021-07-06 12:51:04 +01:00
Paul Duffin
56afb27fb0 Generate boot zip file from prebuilt_bootclasspath_fragment
Previously, the boot zip file, containing all the boot image files for
all the supported architectures, was only created from source. It was
not created when building from a prebuilt_bootclasspath_fragment. That
lead to build failures when building from ART prebuilts.

This change pulls the boot zip file creation out so that it can be done
for both source and prebuilt bootclasspath_fragment modules as well as
for the platform_bootclasspath module.

Bug: 192575099
Test: m out/target/product/generic_arm64/boot.zip
      m SOONG_CONFIG_art_module_source_build=false out/target/product/generic_arm64/boot.zip
      - Compare the output of the first command from before the change
        with the output from them both after and confirm that when the
        ART prebuilts are up to date with the source that there are no
        differences.
Change-Id: Ie7dd5e2ca4a865d06fd9ebf87320cf68c4d05bc3
2021-07-01 22:17:49 +01:00