Commit graph

73394 commits

Author SHA1 Message Date
Treehugger Robot
49c9d1867a Merge "Add TARGET_RECOVERY_FSTAB_GENRULE" 2022-01-12 20:55:38 +00:00
Treehugger Robot
35efbedb4f Merge "Updates OWNERS file" 2022-01-12 03:37:34 +00:00
Treehugger Robot
49380ebecf Merge changes Ia2ec1b81,Ibc831ae8
* changes:
  Performance and scale.
  Use struct{}
2022-01-12 00:27:41 +00:00
Treehugger Robot
b3f3e7b3f7 Merge "Fold annotations." 2022-01-12 00:27:35 +00:00
Treehugger Robot
d6bdfb3993 Merge "Remove win_sdk" 2022-01-11 22:25:02 +00:00
Bob Badour
103eb0f9bc Performance and scale.
Defer edge creation.

Don't create edges until the count is known to avoid repeated allocate+
copy operatios.

Limit resolutions.

Allow only a single resolution condition set per target, and overwrite
intermediate results. Reduces memory and obviates allocations.

Propagate fewer conditions.

Instead of propagating notice conditions to parents in graph during
initial resolve, leave them on leaf node, and attach to ancestors in
the final walk. Reduces copies.

Parallelize resolutions.

Use goroutines, mutexes, and waitgroups to resolve branches of the
graph in parallel. Makes better use of available cores.

Don't accumulate resolutions inside non-containers.

During the final resolution walk, only attach actions to ancestors from
the root down until the 1st non-aggregate. Prevents an explosion of
copies in the lower levels of the graph.

Drop origin for scale.

Tracking the origin of every potential origin for every restricted
condition does not scale. By dropping origin, propagating from top
to bottom can prune many redundant paths avoiding an exponential
explosion.

Conditions as bitmask.

Use bit masks for license conditions and condition sets. Reduces maps
and allocations.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ia2ec1b818de6122c239fbd0824754f1d65daffd3
2022-01-11 10:40:50 -08:00
Bob Badour
5446a6f8e1 Use struct{}
Using struct{}{} as the payload for set maps reduces memory use for
large sets.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ibc831ae80fc50f35e1000348fb28fc0167d0ebed
2022-01-11 10:40:50 -08:00
Bob Badour
67d8ae390a Fold annotations.
Use annotation slices looked up from map instead of read from disk to
reduce copies.

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Test: m systemlicense
Test: m listshare; out/soong/host/linux-x86/bin/listshare ...
Test: m checkshare; out/soong/host/linux-x86/bin/checkshare ...
Test: m dumpgraph; out/soong/host/linux-x86/dumpgraph ...
Test: m dumpresolutions; out/soong/host/linux-x86/dumpresolutions ...

where ... is the path to the .meta_lic file for the system image. In my
case if

$ export PRODUCT=$(realpath $ANDROID_PRODUCT_OUT --relative-to=$PWD)

... can be expressed as:

${PRODUCT}/gen/META/lic_intermediates/${PRODUCT}/system.img.meta_lic

Change-Id: Ibf6f7d319092e0d54ab451ffbbd35dcd7d0080e0
2022-01-11 10:40:39 -08:00
Cole Faust
fdc2cd300e Remove win_sdk
win_sdk is deprecated, if given on the command line
sdk will be build instead.

Bug: 212724080
Test: Presubmits
Change-Id: Ib3bd51923142b00100b49585ad88bc1292da2463
2022-01-11 10:31:11 -08:00
Treehugger Robot
7297032a8d Merge "Make com.android.sepolicy mandatory" 2022-01-11 14:11:37 +00:00
Yi-yo Chiang
27a309f627 Merge "Fix init_boot partition AVB signing" 2022-01-11 06:59:47 +00:00
Treehugger Robot
9d83209f12 Merge "Do not append .meta_lic to meta_lic" 2022-01-11 06:11:14 +00:00
Bowgo Tsai
221628f342 Updates OWNERS file
gsi_keys.mk has been removed on commit
I3714de75b94be43a1364b89b92d893b336aa014b. Removing it from
the OWNERS file and adds developer_gsi_keys.mk.

Bug: 176869478
Test: presubmit
Change-Id: I2eeee65a244c4987f217f004ec47a0b6dca9b564
2022-01-11 11:13:25 +08:00
Bob Badour
63a281cc29 Do not append .meta_lic to meta_lic
Test: manual
Change-Id: I8d7162dc55f057d542faaa7feb546050d3a872e7
2022-01-10 18:00:03 -08:00
Treehugger Robot
21e7285926 Merge "Embed host liblz4.so in target_files" 2022-01-10 22:50:51 +00:00
Pirama Arumuga Nainar
bd7e1a38ba Merge "Set a BIONIC_COVERAGE .mk variable" 2022-01-10 22:36:57 +00:00
Kelvin Zhang
6877e81467 Embed host liblz4.so in target_files
This file will be used by OTA generator to compress generated patches.
We need the host copy of LZ4 to ensure that compressed bytes matches
what the device would get if the compression were to be done on device.
For more details see go/lz4diff

Bug: 206729162
Test: m dist

Change-Id: I9c4ab772b001bf47da75cb87b028a1f27e2321a8
2022-01-10 12:12:21 -08:00
Victor Hsieh
70ca7c6add Merge "Add min_sdk_version to BuildManifest.apk" 2022-01-10 19:59:53 +00:00
Jiyong Park
efbb6ffa00 Add min_sdk_version to BuildManifest.apk
sign_target_files_apks demands it.

Bug: 213823227
Test: run com.google.android.tradefed.ota.SigningToolTest#Testcase1_Signingtest
Change-Id: I1bba2b25a71449a57d284024e77fb76d1b79a0ee
2022-01-10 20:14:05 +09:00
Jeff Vander Stoep
fe91d896ee Make com.android.sepolicy mandatory
Bug: 199914227
Test: aosp/1910032
Change-Id: I23599517b427363e758e6f06763079cc1482cb47
Merged-In: I23599517b427363e758e6f06763079cc1482cb47
2022-01-10 10:28:39 +00:00
Treehugger Robot
664dd48176 Merge "Add sepolicy compat versions to soong config" 2022-01-10 02:09:22 +00:00
Treehugger Robot
dcd7e283d2 Merge "Add option to enable zucchini" 2022-01-09 20:38:18 +00:00
Martin Stjernholm
76537be0a9 Merge "Add an option to specify the path to aapt2." 2022-01-09 19:11:49 +00:00
Yi-Yo Chiang
33f973fd2a Fix init_boot partition AVB signing
Typo: 'BOARD_AVB_INIT_BOOT_IMAGE_KEY_PATH' ->
  'BOARD_AVB_INIT_BOOT_KEY_PATH'

Record avb_init_boot_* arguments to misc_info.txt

Add INIT_BOOT_SECURITY_PATCH to support per-partition SPL.

Bug: 203698939
Test: avbtool info_image --image init_boot.img
Change-Id: I63e082b1dd675138a6ab5cceb8814b3673d26b13
2022-01-09 16:28:25 +08:00
Treehugger Robot
6a95477b0f Merge "Change ifneq(,$(A) $(B)) to ifneq(,$(strip $(A) $(B)))" 2022-01-08 18:20:12 +00:00
Yi-Yo Chiang
8193900f87 Change ifneq(,$(A) $(B)) to ifneq(,$(strip $(A) $(B)))
Else the branch condition would always evaluate to true, making the
other branch useless and wrong!

Bug: 211741246
Test: m dist
Change-Id: I4791712632cbe0d5a9e20962c8425dd20bbe0bfd
2022-01-08 15:59:02 +00:00
Treehugger Robot
555c3a7dc7 Merge "Write signature size even when it's empty" 2022-01-08 02:20:11 +00:00
Treehugger Robot
8ef44309b2 Merge "Fix GetCareMap for small images" 2022-01-08 02:14:02 +00:00
Martin Stjernholm
58472e8609 Add an option to specify the path to aapt2.
Thin manifests (e.g. master-art) may not have the aapt2 sources and
instead relies on the prebuilt in prebuilts/sdk/tools/linux/bin. In
that case there's no aapt2 in the search path, so it needs to be
specified.

Test: Heavy presubmit build on mainline_modules_bundles on git_master
Test: vendor/google/modules/ArtGoogle/build-art-module.sh
  with http://ag/16584845 on git_master-art, where only the prebuilt
  aapt2 is present
Bug: 212462945
Change-Id: I41a22e8146f5a88534c2721345b4d9d64f76698c
2022-01-07 23:08:53 +00:00
Kelvin Zhang
410bb389d8 Add option to enable zucchini
Change-Id: Id952826c9c5af912fba679af61f2a2e1645641dd
2022-01-07 14:59:01 -08:00
Kelvin Zhang
98ef7bb157 Fix GetCareMap for small images
Change-Id: Iac518e36dbb2fe9303e84d502cd7dfe74621c0af
2022-01-07 14:59:01 -08:00
Treehugger Robot
d541c6c898 Merge "Add missing dependencies of fs-verity manifest APK" 2022-01-07 22:28:25 +00:00
Cole Faust
64c2ddb619 Add TARGET_RECOVERY_FSTAB_GENRULE
This variable can be set instead of TARGET_RECOVERY_FSTAB
in order to specify a generated fstab file.

Bug: 201700692
Test: Presubmits
Change-Id: If0fae81ec53915e9ed4bab2e8b1a9f8376894537
2022-01-07 13:50:54 -08:00
Victor Hsieh
be1029af71 Add missing dependencies of fs-verity manifest APK
Otheriwse, the build complains about missing files when
PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA is enabled (aosp/1937717).

Bug: 206326351
Test: clobber, set PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true, build
Change-Id: I11d18914d63cab0b84ae711f2334747cb26ca234
2022-01-07 11:07:52 -08:00
Bill Yi
1ebd070c8d Merge "Merge sc-qpr1 to aosp-master - DO NOT MERGE" 2022-01-07 17:29:11 +00:00
Treehugger Robot
097f6bc1da Merge "Add android.hardware.bluetooth.audio-V1-ndk.so" 2022-01-07 13:07:25 +00:00
Treehugger Robot
1c2c284063 Merge "Add a.hardware.audio.common and a.media.audio.common" 2022-01-07 13:07:10 +00:00
Inseob Kim
934aaaffa6 Add sepolicy compat versions to soong config
Bug: 33691272
Test: build
Change-Id: Idef2efb8bb9ea6ea2b4bd072b4e75296b12294d1
2022-01-07 19:16:34 +09:00
Yi-yo Chiang
ec918431a5 Merge "Verbose warning message for PRODUCT_INSTALL_DEBUG_POLICY_TO_SYSTEM_EXT" 2022-01-07 05:30:23 +00:00
Yi-yo Chiang
5709c4a93f Merge "Unconditionally copy the system build.prop into target_files archive" 2022-01-07 05:27:35 +00:00
Treehugger Robot
7c88c9c1e6 Merge "Move fsverity metadata generation to Makefile" 2022-01-07 03:37:16 +00:00
Bill Yi
5a77fbf008 Merge sc-qpr1 to aosp-master - DO NOT MERGE
Merged-In: I017a5d9c3cd227e0b1a4deaa8dface646bc0a887
Merged-In: I71f542d511cf6397d84d40b1fb688c307cdd386e
Change-Id: Id5d2b7b2b18ae1a1919a53121c06fb309e5b2eee
2022-01-06 18:38:30 -08:00
Treehugger Robot
bd5fd25265 Merge "Avoid packing ramdisk into GKI boot-*.img" 2022-01-07 02:35:06 +00:00
Inseob Kim
3d5ae80f50 Write signature size even when it's empty
Normally this has no effect, but when we generate metadata for small
files (<=4KB), merkle tree isn't generated. In such case, writing zero
will make the metadata format simpler and unconditional.

Test: manual
Change-Id: Ibe18175b580af3409c896a8bb97323792ad9c459
2022-01-07 10:56:17 +09:00
Treehugger Robot
802c86f831 Merge "Export test data dependencies in module-info" 2022-01-07 00:36:54 +00:00
Inseob Kim
135c1f144f Move fsverity metadata generation to Makefile
build_image.py has been handling fsverity metadata generation in the
packing step, but it can cause issues because the metadata files are
missing in the $OUT directory, and they only exist in result system.img.
This change moves the generation logic into Makefile, and makes the
metadata tracked by ninja graph.

Bug: 206326351
Test: PRODUCT_SYSTEM_FSVERITY_GENERATE_METADATA := true and build
Change-Id: I1f910d8ac6e2cc3c54f35916871733c632f18e44
2022-01-07 09:16:42 +09:00
Spandan Das
13daf3d4a2 Merge "Move manifest generation logic into static_java_library.mk" 2022-01-06 22:21:03 +00:00
Yi-Yo Chiang
9b2584170e Unconditionally copy the system build.prop into target_files archive
releasetools preconditions requires build metadata defined in
SYSTEM/build.prop even when building a system.img-less product.
Copy the SYSTEM/build.prop into the build artifact archive whenever
possible to make add_img_to_target_files happy.

Bug: 212486689
Bug: 213028932
Test: Dist build a system.img-less product. Product properties are
  read from SYSTEM/build.prop and no build error occurred.
Change-Id: I2ce7f8b8ae981eaf06e2d8d2485b55b4e975caa9
2022-01-07 03:29:14 +08:00
Spandan Das
68684bea79 Move manifest generation logic into static_java_library.mk
Previously a manifest file would be generated for static_java_library.mk
and package_internal.mk. For pacakge_internal.mk, this would cause
cryptic errors like b/188612215#2

With this CL, a manifest file would only be generated for java_library
with need_compile_res == true. The advantages are
1. No redundant generation for java_library with need_compile_res == false
(i.e. modules that do not require aapt2)
2. Force devs to provide manifest for android_app, and not silently
generating one for them

Bug: 188652897
Test: TH
Test: m nothing in tv-dev, car-dev

Change-Id: I69a23e373f1b4ac1569ae5b1d36067bfc0dadc05
2022-01-06 18:47:57 +00:00
Treehugger Robot
e33e7e518c Merge "Fix build error on empty root folder" 2022-01-06 13:54:27 +00:00