Commit graph

2631 commits

Author SHA1 Message Date
Bob Badour
5e9e1fbe83 Add support for combining the notices from 2 roots.
Apparently needed for cts tests due to apks putting intermediate files under
TARGET_OUT even when going to HOST_OUT.

Test: m with and without change verified no differences in notice files

Change-Id: Iea50b6bd9af9452a2600c97e2f7cb89218b9be7f
2020-07-20 17:32:14 +00:00
Yifan Hong
997d227648 Merge changes from topic "odm_dlkm"
* changes:
  Create $OUT/{vendor,odm}/lib before symlink modules
  Add odm_dlkm/etc/build.prop
  Install ODM dlkm to appropriate place and symlink
  Add notice files for odm_dlkm
  Add odm_dlkm partition.
2020-07-17 22:05:26 +00:00
Joe Onorato
02fb89a4d7 Add mk2bp_catalog.py that outputs more data about makefiles to be converted to soong.
- Adds makefile and which files are installed to the SOONG_CONV CSV file
- Updates soong_to_convert.py to be able to parse that
- Adds new script that is more detailed.
- Outputs that file as part of the droidcore build to
  $(OUT_DIR)/target/product/$(TARGET_DEVICE)/mk2bp_remaining.html

Test: m out/target/product/$(get_build_var TARGET_DEVICE)/mk2bp_remaining.html
Change-Id: I7c380b6070754f4329bf3965595751e4dac794a0
2020-07-16 18:19:15 -07:00
Yifan Hong
1e6d21787a Create $OUT/{vendor,odm}/lib before symlink modules
Before creating symlinks $OUT/{vendor,odm}/lib/modules, make sure
parent directory exists first.

Test: build CF
Bug: 156020364
Change-Id: Id2a9e5273aae75944c1b31553fad3703540bf2f4
2020-07-16 12:34:30 -07:00
Yifan Hong
3d5e2c210a Install ODM dlkm to appropriate place and symlink
Test: builds
Bug: 156020364
Change-Id: Ib0e44fe827397d0d1ec6d7edfceba1e9a43c76a3
2020-07-16 12:34:30 -07:00
Yifan Hong
c83f9e922b Add notice files for odm_dlkm
Test: pass
Bug: 156020364
Change-Id: Ia7e562cb96b9f14fa7ab0b74c253bf75fb699f0b
2020-07-16 12:34:30 -07:00
Yifan Hong
f496f1b94f Add odm_dlkm partition.
Test: define odm_dlkm in CF.
Bug: 156020364
Change-Id: I5593d09a1cdf13e651a13e951336555a7475865e
2020-07-16 12:34:29 -07:00
Yifan Hong
e51dff25ec Install dlkm to appropriate place and symlink
Test: builds
Bug: 156020364
Change-Id: I5958021ab54a2f99140796c3ef915b5329808ce3
2020-07-13 14:20:56 -07:00
Yifan Hong
3d3f030c27 Add notice files for vendor_dlkm
Test: pass
Bug: 156020364
Change-Id: I87e8b57a7a7f212071de4e530424740103a29434
2020-07-13 14:20:56 -07:00
Yifan Hong
cfb917a1c7 Add vendor_dlkm partition.
Test: define vendor_dlkm for CF.
Bug: 156020364

Change-Id: I459059e1c20df7a285a377b8b3bbc8decc60b124
2020-07-13 14:20:56 -07:00
Colin Cross
c27d795b63 Use unzip -DD
We put reproducible timestamps in zip files so that the artifacts
are consistent, but that leads to old timestamps in the output
directory if they are unzipped as part of the build.  Use
unzip -DD when unzipping to update the timestamps.

Bug: 161015009
Test: touch -d 2020-01-01 ref; find $OUT/system -not -newer ref
Change-Id: I6f08ba8695d90a8225cfc04e679755e6296deed0
2020-07-12 05:38:03 +00:00
Jaewoong Jung
704b454ca1 Add LOCAL_APKCERTS_FILE support.
This is used when Soong and Make don't know the full output file list,
and so use a tool to generate a per-module apkcerts.txt file instead.

Fixes: 160119159
Test: m apkcerts-list
Merged-In: I48183bc8cbf9dcc521f791812734205ed3f28f4c
Change-Id: I48183bc8cbf9dcc521f791812734205ed3f28f4c
2020-07-01 10:01:16 -07:00
Treehugger Robot
6c9441a924 Merge "use first kernel when generating kernel configs file" 2020-06-30 17:36:01 +00:00
Steve Muckle
bbdfbdad50 use first kernel when generating kernel configs file
If multiple kernels are defined for a build, use the first one when
generating the kernel config file.

Bug: 159975506
Change-Id: Ia136264eb0d0b544bf0685bc22e600de187a9a9b
2020-06-26 16:39:34 -07:00
Luca Stefani
c158e421ec Remove empty new-space in dynamic_partition_list and super_$(group)_partition_list
* As a side product .strip() calls in releasetools are redunant,
  but since these scripts are also used out of tree we can't
  remove them to guarantee compatibility

Test: m dist
Change-Id: I5c513a4654e293c3d4eab98c8759ea094015fab1
2020-06-26 19:52:52 +02:00
Mark Salyzyn
46a9c028b8 depmod: pair in a modules.load.charger on BOARD_VENDOR_CHARGER_KERNEL_MODULES_LOAD
If the BoardConfig defined BOARD_VENDOR_KERNEL_MODULES_LOAD and
BOARD_VENDOR_CHARGER_KERNEL_MODULES_LOAD, generate both a
modules.load and modules.load.charger respectively on
/vendor/lib/modules.

Signed-off-by: Mark Salyzyn <salyzyn@google.com>
Bug: 159424228
Bug: 151950334
Test: build, confirm, change to charger mode, confirm set of modules loads
Change-Id: I55677ed74d80ff3909fe8cbe26e6766f9b5283d5
2020-06-25 08:57:36 -07:00
Tianjie Xu
be46f482e0 Merge "Add lz4 to otatools" 2020-06-24 23:38:49 +00:00
Tianjie
3031c29905 Add lz4 to otatools
A call to lz4 was added in http://go/aog/1329115. But the binary
isn't added to otatools, leading to a signing failure.

Bug: 159656873
Test: build otatools
Change-Id: I9cab00f36ce3c39cc174b4c9feffdf360f0364ef
2020-06-24 11:20:36 -07:00
J. Avila
dbbd970d18 Merge "Switch to using llvm-strip for modules" 2020-06-22 15:19:26 +00:00
Chris Gross
e08e40947b Merge "Use BUILD_*_IMAGE flags in add_img_to_target_files." 2020-06-19 22:32:37 +00:00
J. Avila
f7586025a2 Switch to using llvm-strip for modules
The gcc tools are deprecated; replace with the llvm equivalent.

Bug: 158484106
Change-Id: I9c3717bde281667eec82cc21dc63dc7fd5592a46
2020-06-19 21:32:45 +00:00
Steve Muckle
414dbe15ab adjust vendor ramdisk file extension based on compression
The extension for the generated vendor ramdisk should be updated based
on the compression scheme used.

Bug: 158433703
Change-Id: I25fff0b8a7879f6aa6d6fe2fce9f5a53f90eb636
2020-06-18 21:38:35 -07:00
Treehugger Robot
397972fa6a Merge "Include image descriptor of only the first boot image in vbmeta" 2020-06-16 00:05:28 +00:00
Dan Willemsen
d120ac68fa Fix dependencies on INSTALLED_RECOVERYIMAGE_TARGET
Instead of copying the logic (which led to divergences), use the
same list of dependencies for both users of build-recoveryimage-target

Change-Id: I8f8c006432f3b8120df1f39c98009155fdff32ad
Test: treehugger
2020-06-12 02:39:38 +00:00
J. Avila
b8d67f3722 Merge "Add LZ4 support to the release tools" 2020-06-11 14:56:38 +00:00
Martin Stjernholm
7a430fd4c8 Introduce TARGET_BUILD_UNBUNDLED.
Setting this flag enables unbundled building, i.e. without support for
building the system image and other platform targets. This
functionality was previously enabled by TARGET_BUILD_APPS, and setting
that still implies TARGET_BUILD_UNBUNDLED.

This helps unbundled builds that aren't apps, e.g. ART runtest builds.
Specifically, with the topic of the child CL
https://r.android.com/1324517 TARGET_BUILD_UNBUNDLED does not imply
disabling dexpreopting, unlike TARGET_BUILD_APPS.

TODO: There may still be app-specific conditions that are incorrectly
controlled by TARGET_BUILD_UNBUNDLED, in particular on the Soong side
through config.UnbundledBuild().

Test: Flash & boot
Test: TH, in particular builds green on ub-launcher3-master
Bug: 157549171
Change-Id: Ic09fc879117ee06cab5444edfc280ed2b52d2870
2020-06-11 01:09:07 +01:00
J. Avila
98cd4cc7a3 Add LZ4 support to the release tools
The core android build platform has changed to add support for LZ4
compression for ramdisks, but the release tools were not. Fix this.

Bug: 156129966
Change-Id: I39680b91930d2d3cbd0cd565beb78e1ee699397e
2020-06-10 21:51:19 +00:00
Roopesh Nataraja
3e15f6edb1 Include image descriptor of only the first boot image in vbmeta
Multiple boot images can be generated based on BOARD_KERNEL_BINARIES
but vbmeta would capture the image descriptor of only the first boot
image specified in BUILT_BOOTIMAGE_TARGET.

Bug : 158213759
Test: Compile multiple boot images and check if vbmeta.img
      includes image descriptor of only the first boot image
      specified in BUILT_BOOTIMAGE_TARGET

Change-Id: Ib70f65d98546a436da82656bc5c911d437618bb5
2020-06-10 07:44:12 -07:00
Elliott Hughes
9b824d5ceb Remove TARGET_PREFER_32_BIT.
Bug: https://issuetracker.google.com/138812821
Test: builds
Change-Id: I2ff7793c8b1eb56f6be63375cc7b70aa44254443
2020-06-08 16:36:52 -07:00
Jiyong Park
bb26c6f2bf Rule for system/build.prop is also from macro
Like other <partition>/build.prop files, the build rule for
system/build.prop is also created using the build-properties macro.

system/build.prop has a few number of auto-generate properties that the
macro doesn't create. Those properties are emitted to intermediate files
using manually crafted rules, the paths to the files are fed to the
macro, where their contents are copied to system/build.prop.

This change also refactors the code fragment that defines BUILD_*
variables that identifes the given build.

Bug: 117892318
Test: m

Change-Id: Ic218283b40fedeaddacb3fda93167edbec86094c
2020-06-08 10:09:12 +09:00
Steve Muckle
950d723b9d fix spurious / on toplevel kernel module installations
When kernel modules are being put directly into lib/modules/ (as opposed
to lib/modules/<subdir>/), the empty part of the destination path causes
a target containing a redundant /. If the target is also added elsewhere
in the build without the redundant /, then rule merging will miss it,
and a build error will occur.

Bug: 158158635
Change-Id: I797addce760f4886166efa283302a53783581625
2020-06-05 12:10:56 -07:00
Chris Gross
203191b5be Use BUILD_*_IMAGE flags in add_img_to_target_files.
Modify add_img_to_target_files.py to use the BUILDING_*_IMAGE flags when
deciding whether to create and add a given image to a target files
archive.

To do this, the BUILDING_*_IMAGE flags are now dumped to misc_info.txt.

The origin of this change was to use the BUILDING_USERDATA_IMAGE and
BUILDING_CACHE_IMAGE flags in add_img_to_target_files.py so that we
could reliably turn off the generation of the userdata and cache images.
The other image flags were added for symmetry.

Bug: 130307439
Test: m -j out/target/product/bonito/misc_info.txt
Test: m -j droid dist
Change-Id: I32d5a8d6c9ebb5f329d856030084d698ee8d271d
2020-06-04 17:53:04 +00:00
Pierre Couillaud
d99da8fa17 Strip debug symbols from modules (fixup)
allow to also skip the stripping of the modules present in vendor
ramdisk when the BOARD configuration is set to skip the strip step.

there is also a shortcoming of this functionality: it currently assumes
the modules to strip are all created equal against the same arch in
particular, however this does not work for kernel modules which are
for mixed mode platforms, as example {kernel,android} == {aarch64,armv7}
because it will select the "android" mode strip which is unaware of the
kernel mode architecture.

Bug: 156395749
Bug: 155193290
Change-Id: I35ef51845224e94e3e253acdeb6bb14b990a401b
2020-06-04 00:17:58 +00:00
Steve Muckle
2335d76ada refactor kernel module build, add version support
Refactor the build support for kernel modules. This refactoring is
leveraged to add support for directories of kernel modules in each
partition. To add a directory of kernel modules, define
BOARD_KERNEL_MODULE_DIRS to include the directory name, for example:

BOARD_KERNEL_MODULE_DIRS := 5.4

The build will then check all the usual kernel module related macros
with an extension of _5.4, for example you can define

BOARD_VENDOR_KERNEL_MODULES_5.4 := <modules to copy>
BOARD_VENDOR_KERNEL_MODULES_ARCHIVE_5.4 = <path to archive file>
BOARD_VENDOR_KERNEL_MODULES_LOAD_5.4 := <modules for load list>

Bug: 157645635
Change-Id: I1687d0ec85c1dcf21350350623b4689aba3f6717
2020-06-02 15:17:26 -07:00
Steve Muckle
c88a8c88e4 move BOARD_RECOVERY_KERNEL_MODULES_LOAD default
Moving this makes each of the partition's kernel module Makefile
generation more similar, enabling a forthcoming cleanup/refactoring.

If recovery-is-boot and there are kernel modules defined in
BOARD_GENERIC_RAMDISK_KERNEL_MODULES, those kernel modules will
now be included in the default assignment to
BOARD_RECOVERY_KERNEL_MODULES_LOAD. If this is not desired then
BOARD_RECOVERY_KERNEL_MODULES_LOAD may simply be set to
BOARD_RECOVERY_KERNEL_MODULES (or a custom list of recovery kernel
modules to load).

Bug: 157645635
Change-Id: Ia1392b7d20d07cee75420edd95dcee1dc825516e
2020-06-02 15:17:15 -07:00
Steve Muckle
1013d789de move module list creation outside conditionals
Currently the two special-case calls to create just a module load list
(as opposed to copying a set of modules and creating a module load list)
are inside checks that the related board macro for the module set is not
empty.

While it would be incorrect to create a non-empty module load list with
an empty set of modules, this error would quickly surface at runtime.
More importantly, moving these out will permit refactoring this code to
clean it up.

Bug: 157645635
Change-Id: I199788d49072d4c59737b3ef2c75546852cf3e36
2020-06-02 15:17:04 -07:00
Steve Muckle
f291097221 fix reference to BOARD_ODM_MODULES_LOAD
This check is meant to initialize BOARD_ODM_KERNEL_MODULES_LOAD to
BOARD_ODM_KERNEL_MODULES if BOARD_ODM_KERNEL_MODULES_LOAD is empty.

Bug: 157645635
Change-Id: I9c8ac0e64bb002e926d15b446a63801d7cfa9911
2020-06-02 15:16:54 -07:00
Treehugger Robot
6be7b57ce7 Merge "remove spurious comma from module-load-list-copy-paths call" 2020-05-29 22:08:11 +00:00
Steve Muckle
06b48162f4 remove spurious comma from module-load-list-copy-paths call
The module-load-list-copy-paths fn takes five parameters.

Merged-In: I7d1736da1599e265793173a741053d8a09f1622f
Change-Id: I9286c9e4ca81caf5392bc804dead718bd028b193
2020-05-29 13:54:49 -07:00
Baligh Uddin
ab25b2dcfa Add apksigner to otatools.zip
BUG: 157735036
Test: Local build.
Change-Id: I95c7001ad520d6b4a415d21ffdef55152b0e82b2
2020-05-29 19:01:32 +00:00
J. Avila
bc4d5bdb87 Merge "Add build support for LZ4 ramdisks" 2020-05-29 16:28:47 +00:00
Dan Willemsen
9767223dc5 Fix up recovery_text_res
First, make it safer for incremental builds. We used to just be
depending on the bootable/recovery/tools/recovery_l10n/res dir as a
dependency, but that would only trigger a rebuild if a direct child
file/directory was added or removed (so essentially, when a translation
was addded or removed). If a translation was updated, it wouldn't
re-trigger recovery_text_res.

So instead of depending on the directory, depend on the results from
running `find` to list all of the contents in that directory.

Next, move it out of PACKAGING. `m installclean`, which runs in between
incremental builds on our build servers, deletes the obj/PACKAGING
directory. So when recovery_text_res used that directory, we were having
to rebuild it on every incremental build, which isn't optimal when it
takes nearly 90s to build.

So with:
 $ lunch aosp_coral-eng
 $ m bootimage
 $ m installclean
 $ m bootimage

The second `m bootimage` went from executing 666 actions in 93s to
executing 658 actions in 6s.

Finally, remove the last uses of $(call include-path-for,recovery),
since they were all in this file (and they weren't using it for include
paths...). We'd like to remove all of these and switch them to header
libraries or other use cases, but in this case, it's just shorter to use
the real path.

Test: treehugger
Test: m bootimage;
      touch bootable/recovery/tools/recovery_l10n/res/values/strings.xml
      m bootimage
Test: m bootimage; m installclean; m bootimage

Change-Id: I005592e49443aab45ed039a2f0c63f7a69035565
2020-05-27 19:37:03 -07:00
J. Avila
3f6560e626 Add build support for LZ4 ramdisks
Ramdisks are currently compressed using gzip, which is generally
smaller, but slower to decompress than lz4. Provide an option for an
lz4 scheme to reduce the time taken to unpack initramfs during boot.

Bug: 156129966
Signed-off-by: J. Avila <elavila@google.com>
Change-Id: Iac9538e6ee6ec51e6b487de2101f53bb5d9c54c8
2020-05-27 23:44:33 +00:00
J. Avila
a2ceec665d Strip debug symbols from modules
The size of the ramdisk has greatly increased, which can be a source of
boot time regressions. Strip the modules of debug symbols for the vendor
ramdisk and vendor partitions; vendor stripping can be skipped via BOARD
config.

Bug: 156395749
Signed-off-by: J. Avila <elavila@google.com>
Merged-In: I248d4e2097992f00975b344b5da41aa5d8016812
Change-Id: I248d4e2097992f00975b344b5da41aa5d8016812
2020-05-26 23:35:26 +00:00
Treehugger Robot
a74d425017 Merge "depmod: pair in a modules.load.recovery on BOARD_VENDOR_RAMDISK_RECOVERY_KERNEL_MODULES_LOAD" 2020-05-26 21:10:54 +00:00
Treehugger Robot
8a87dae43b Merge "Add options to sign the prebuilt custom images." 2020-05-21 16:34:13 +00:00
Mark Salyzyn
4e20965769 depmod: pair in a modules.load.recovery on BOARD_VENDOR_RAMDISK_RECOVERY_KERNEL_MODULES_LOAD
If the BoardConfig defined BOARD_VENDOR_RAMDISK_MODULES_LOAD and
BOARD_VENDOR_RAMDISK_RECOVERY_KERNEL_MODULES_LOAD, generate both a
modules.load and modules.load.recovery respectively on the vendor
ramdisk.  BOARD_VENDOR_RAMDISK_MODULES must be the set, or super set,
of all the modules represented in both of *_LOAD BOARD variables.

Signed-off-by: Mark Salyzyn <salyzyn@google.com>
Bug: 151950334
Test: build, confirm, fastboot flashall, boot with split.
Change-Id: Ib2925aec6ea7a2a01a554c808d6697d8c85f9fc0
2020-05-20 14:44:01 -07:00
Yo Chiang
7c147f7e29 Merge "Resolve module bitness for PRODUCT_PACKAGES" 2020-05-20 04:55:22 +00:00
Hongguang Chen
f23364d280 Add options to sign the prebuilt custom images.
The custom images are any images owned by OEMs and SoCs, oem images
mounted on /oem is an example. The oem images can be used to customize
devices for different carriers, like wallpaper, ringtones, and
carrier-specific apks. OEMs can generate multiple oem images, like
oem.img, oem-carrier1.img and oem-carrier2.img and flash different oem
images for different carriers. The oem images are only one case, OEMs
and SoCs can add more custom images and mount them to custom partitions.

This change enables custom images to be vbmeta.img chained partitions.
The following configuration in BoardConfig.mk is an exmaple. It has two
custom partitions: oem and test. They will be signed by different keys.
And they will be chained by vbmeta.img. The custom images here are
prebuilts, which can be built by `make custom_images` separately.
BOARD_AVB_<CUSTOM_PARTITION>_IMAGE_LIST should include all custom images
to apply AVB signing. And to every custom partition, one image whose
name is partition name must be added in its
BOARD_AVB_<CUSTOM_PARTITION>_IMAGE_LIST.

BOARD_CUSTOMIMAGES_PARTITION_LIST := oem test

BOARD_AVB_OEM_KEY_PATH := external/avb/test/data/testkey_rsa4096.pem
BOARD_AVB_OEM_ALGORITHM := SHA256_RSA4096
BOARD_AVB_OEM_ADD_HASHTREE_FOOTER_ARGS :=
BOARD_AVB_OEM_ROLLBACK_INDEX_LOCATION := 1
BOARD_AVB_OEM_PARTITION_SIZE := 5242880
BOARD_AVB_OEM_IMAGE_LIST := \
    device/xxxx/yyyy/oem/oem.img \
    device/xxxx/yyyy/oem/oem1.img

BOARD_AVB_TEST_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
BOARD_AVB_TEST_ALGORITHM := SHA256_RSA2048
BOARD_AVB_TEST_ADD_HASHTREE_FOOTER_ARGS :=
BOARD_AVB_TEST_ROLLBACK_INDEX_LOCATION := 2
BOARD_AVB_TEST_PARTITION_SIZE := 10485760
BOARD_AVB_TEST_IMAGE_LIST := \
    device/xxxx/yyyy/test/test.img \
    device/xxxx/yyyy/test/test1.img

To resign the custom images in the target zip file, the
avb_extra_custom_image_key, avb_extra_custom_image_algorithms and
avb_extra_custom_image_extra_args options are added to the
sign_target_files_apks tool too. The following test cases list some
examples about how to use them.

BUG: 154171021
Test: 1) "atest --host releasetools_test releasetools_py3_test -c"
  2) Build images by 'make dist', sign and validate target files.
   a) Test on dist w/ chained vbmeta_system and ome custom images
   sign_target_files_apks -d certs \
     --avb_extra_custom_image_key oem=oem_rsa4096.pem \
     --avb_extra_custom_image_algorithm oem=SHA256_RSA4096 \
     xxx-target_xxx.zip signed.zip
   validate_target_files.py signed.zip
   Flash image and boot up.
   Verify the oem images and vbmeta images in OUT and target zips by
   avbtool.

   b) Test on dist w/ chained vbmeta_system and oem and test custom images
   sign_target_files_apks -d certs \
     --avb_extra_custom_image_key oem=oem_rsa4096.pem \
     --avb_extra_custom_image_algorithm oem=SHA256_RSA4096 \
     --avb_extra_custom_image_extra_args oem=--do_not_generate_fec \
     --avb_extra_custom_image_key test=test_rsa4096.pem \
     --avb_extra_custom_image_algorithm test=SHA256_RSA4096 \
     xxx-target_xxx.zip signed.zip
   validate_target_files.py signed.zip
   Verify the oem, test images and vbmeta images in OUT and target zips
   by avbtool.

   c) Test on dist w/o chained partition.
   sign_target_files_apks -d certs xxx-target_xxx.zip signed.zip
   validate_target_files.py signed.zip
   Flash image and boot up.
   Verify the vbmeta images in OUT and target zips by avbtool.

Change-Id: Ifccfee5e8909697eef6ccda0cc352fa16a9f6db6
2020-05-20 04:10:00 +00:00
Treehugger Robot
3dbbfde296 Merge changes I80341c2e,I0935f611,Ia805cb47
* changes:
  Deprecate ADDITIONAL_DEFAULT_PROPERTIES
  Move most of the sysprop generation rules to sysprop.mk
  Add PLATFORM_VERSION_LAST_STABLE
2020-05-20 03:49:26 +00:00