Substantially improves the performance of some BlockImageDiff steps on
large images. GenerateDigraph and AssertSequenceGood are now linear in
the number of blocks in the target image rather than quadratic in the
size of the partition. FindVertexSequence tracks the score of each
vertex, the set of sources and sinks, and maintains a heap for finding
the best vertex to use for the greedy step, instead of repeatedly
re-scanning the set of remaining vertexes.
Change-Id: Ifeea5053d80ef4f06a5aa239c9a1797dd5e47841
with both DAC and DevSite. Includes a variety of cleanup and redesign.
Now prints method parameters and return values even if not documented in source.
Now prints return value data type.
Now prints method name as heading without arguments, then the full signature below.
New heading styles with more spacing and removed background color.
Change-Id: I7a991df5e8d0d7ac8ff81c20b4b4692e67eb88aa
This is a follow-up to 6c41036bcf where
I forgot to update a section of OTA update ZIP code.
Bug: 26864066
Change-Id: Idbcde71d6377a16807e41c999120eeddd5b4d8a4
The bug has been fixed, .odex files are deterministic.
This reverts commit c4438d3d29.
Bug: 26956807
Change-Id: Iaa42dfba5cf79ee64ac20cf4f1f3a4598f4275f4
Add "ota-required-cache" into the metadata file in an OTA package,
which shows the minimum free space on /cache to apply the update.
Add "ota-type" into the metadata file, which shows the OTA type for
this package (i.e. one of FILE, BLOCK and AB).
Also add the cache free space check into updater-script when generating
block-based incremental OTAs (we only had such lines for file-based
incrementals before).
Bug: 26731903
Change-Id: Id6ff0fc4cdfb1443636b0b3800b0f8bddb5bb1d0
makeparallel inherits values for MAKEFLAGS and MAKELEVEL from make
through the environment, but they should not be propagated to the child
process in case the child process tries to run make again.
Change-Id: I4c5df10ea8055cd1f1f61a892d5b1a7acb287bbb
When building an A/B OTA package, include the payload.bin properties as
a key-value pairs text file, so it can easily be passed to
update_engine during payload application.
Bug: 26991255
TEST=`ota_from_target_files out/dist/${BOARD}-target_files.zip full-ota.zip` includes the properties.
Change-Id: I445c8a8e412a8e16b48b6ee626db8e27d48a38a9
The information is not useful. Javadoc hasn't included this
information for years.
Bug: 25767152
Change-Id: Ic0a398851b90fbf85785d5cf946e7f4c1aaa4be4
(cherry picked from commit a37bdc89b4)
They still are not deterministic -- only the boot images were made
deterministic (.oat/.art).
Bug: 26956807
Change-Id: I2e00bb6662b4c8d184c48ef9aa34b39142c94893
This also whitelists the ro.build.user property, which may change based
on host username.
Bug: 26687569
Change-Id: I8f3244c2325dd80a7961c474655f0c50670d1b46
am: 1cafe66be2
* commit '1cafe66be274a5a8bbbc3a0dcab9d4e8f6e5ae61':
Revert "Sign APKs using SHA-256 instead of SHA-1 when possible." This is breaking CTS.
am: 01ac26d942
* commit '01ac26d9422d8d54d3992ba9dd4506896c8556dd':
Revert "Sign APKs using SHA-256 instead of SHA-1 when possible." This is breaking CTS.
Brillo does not require Java. Add a JAVA_NOT_REQUIRED
flag to the build system to make the jdk requirment optional
Also don't build signapk for Brillo
BUG: 25281898
Change-Id: I31e68cc7d076bf6c234699c77c0ea1ea428be4f5
Previously, the timestamp was one hour ahead of NotBefore of the
signer's certificate, adjusted for the current timezone. With this
change the MS-DOS timestamp in output APK/ZIP files is
Jan 1 2009 00:00:00.
Bug: 26864066
Change-Id: Id6263c38ac7042489ab695454f8e0fb2d85a3958
This changes the build system to provide the signapk tool with the
minSdkVersion of the APK being signed. signapk in turn will then use
SHA-256 instead of SHA-1 if minSdkVersion is 18 (JB MR2) or higher
(see c2c49ed0c1).
To avoid increasing incremental OTA update package sizes for already
released platforms, release build scripts disable the above logic when
signing target files ZIPs for pre-N platforms.
Bug: 25643280
(cherry picked from commit de5bc04717)
Change-Id: I4b100750e47788ab6ed897a0a5abfd33542e8676
This changes the build system to provide the signapk tool with the
minSdkVersion of the APK being signed. signapk in turn will then use
SHA-256 instead of SHA-1 if minSdkVersion is 18 (JB MR2) or higher
(see c2c49ed0c1).
To avoid increasing incremental OTA update package sizes for already
released platforms, release build scripts disable the above logic when
signing target files ZIPs for pre-N platforms.
Bug: 25643280
Change-Id: I058393e0971d0d6559dbd69ea902754c6c7cab14
(cherry picked from commit de5bc04717)
This reverts commit de5bc04717 because it's breaking some builds. Turns out some APKs have multiple sdkVersion attributes returned by aapt.
Change-Id: I013d6ef5eac473dd3900e0b9edf4c32cdc838bab
This changes the build system to provide the signapk tool with the
minSdkVersion of the APK being signed. signapk in turn will then use
SHA-256 instead of SHA-1 if minSdkVersion is 18 (JB MR2) or higher
(see c2c49ed0c1).
To avoid increasing incremental OTA update package sizes for already
released platforms, release build scripts disable the above logic when
signing target files ZIPs for pre-N platforms.
Bug: 25643280
Change-Id: I048393e0971d0d6559dbd69ea902754c6c7cab14
APKs are now signed with the usual JAR signature scheme and then
with the APK Signature Scheme v2.
APK Signature Scheme v2 is a whole-file signature scheme which aims
to protect every single bit of the APK as opposed to the JAR signature
scheme which protects only the names and uncompressed contents of ZIP
entries.
The two main goals of APK Signature Scheme v2 are:
1. Detect any unauthorized modifications to the APK. This is achieved
by making the signature cover every byte of the APK being signed.
2. Enable much faster signature and integrity verification. This is
achieved by requiring only a minimal amount of APK parsing before
the signature is verified, thus completely bypassing ZIP entry
decompression and by making integrity verification parallelizable
by employing a hash tree.
Bug: 25794543
Change-Id: I275d2a6d0a98504891985309b9dfff2e0e44b878
This change makes signapk not reject the --disable-v2 command-line
flag which may be used by build scripts in some branches. The flag
is currently ignored.
This change is landed separately from the actual support for APK
Signature Scheme v2 because of unbundled branches which use prebuilt
versions of signapk.
Bug: 25794543
Change-Id: I900966244b8b6296b1f443bf98830cc7f7cc81a8
It calls brillo_update_payload to generate the payload for A/B update.
And packages the payload according to Android OTA package format.
Note that it only supports generating full/incremental OTAs with this
CL. Signing for release may not work properly at the moment.
Bug: 25715402
Change-Id: I4ac8505bacad28a572a9320dc8b52dd0f1ce47f5
The information is not useful. Javadoc hasn't included this
information for years.
Bug: 25767152
(cherry-picked from commit a37bdc89b4)
Change-Id: Id03a6e383118ab716ce870a1d7a3740656420fe7
Modify the number of max blocks per transfer to reduce splitting of
large files during patch generation. This will reduce the size of
final incremental OTA updater package. In specific, the size limit
for each file to transfer is set to 1/8 cachesize instead of a fix
number.
Bug: 26472126
Change-Id: Iffb135e378c4c7a91640f2fc4c40e9e93d1fe4bb
This uses far fewer hacks in order to build these host tools, but will
require more libraries(libc++,etc) from unbundled and stripped down
branches.
Once this becomes the only option, all builds will happen after soong
runs. Then the toolchain / global cflags information will be able to be
moved from make to soong (and exported back to make).
Change-Id: I270b1d92bcef28a96fbc3ec9910d9bd0c101ac0f
SHA-1 is deprecated, but the replacement SHA-256 is only supported
for JAR/APK and OTA update package signatures on API Level 18 and
newer. This change thus adds a --min-sdk-version command-line
parameter to signapk. When this parameter is set to 18 or higher,
SHA-256 is used instead of SHA-1. When the parameter is not provided,
SHA-1 is used same as before.
This change also removes any other digests from the MANIFEST.MF.
This is to ignore any MANIFEST.MF digests already there in the APK,
such as when re-signing an already signed APK.
Build scripts will be modified to provide the --min-sdk-version
parameter in a follow-up change. This is not done in this change
because of prebuilts which require a prebuilt version of signapk
to support this parameter before the build scripts can be modified.
Bug: 25643280
Change-Id: I6a2782e465600fe2a3ad0c10bd80db2b80a6fb76
Due to a bug introduced in 8562fd478d
SHA-1 digests of APK entries' contents were listed under wrong
attribute name. The effect is equivalent to not listing SHA-1
digests.
This change fix the issue by listing SHA-1 digests under the correct
attribute name. However, these digests are not that useful because:
(1) typically the digest of the MANIFEST.MF verifies and thus the
per-entry digests are ignored, and (2) per-entry digests of entries
with names longer than 64 characters are wrong in any case because the
digest generation code does not take into account that such names are
split over multiple lines.
An alternative to this change would be to completely omit outputting
per-entry sections of .SF files, thus saving space and speeding up APK
verification (.SF files would decompress faster).
Bug: 26513901
Change-Id: If95d58e9baa62b1113639fe70724e1e9c9f4e15c
If ninja is missing makeparallel prints an unhelpful error:
out/host/linux-x86/bin/makeparallel: exec failed: No such file or directory
which suggests that makeparallel is what is missing, not what is
reporting the error. Print the path passed to exec as well.
Change-Id: Ic04c1cde6da9c3a974b5c43e3fb3bdb6da6605f7
With BOARD_USES_RECOVERY_AS_BOOT = true, we skip building the
non-ramdisk boot.img but building the recovery image as boot.img. It
contains recovery's ramdisk (e.g. with /sbin/recovery). It depends on
the bootloader parameter (skip_initramfs) to determine the actual mode
to boot into.
Change-Id: Id6e2d0a2b94383944ca8f35bba688c6401745622
(cherry picked from commit d80bef2b9e)
With BOARD_USES_RECOVERY_AS_BOOT = true, we skip building the
non-ramdisk boot.img but building the recovery image as boot.img. It
contains recovery's ramdisk (e.g. with /sbin/recovery). It depends on
the bootloader parameter (skip_initramfs) to determine the actual mode
to boot into.
Change-Id: Id6e2d0a2b94383944ca8f35bba688c6401745622
Add a function check_first_block to read block0 and output a message
on screen if the device has been remounted. The function is called
for version >= 4 only; it executes after a failing block verification
and before recovery attempts.
Bug: 21124327
Change-Id: I49dc0b861c702698896a2495ca094215705d4650
(cherry picked from commit 9dac797013)
append2simg can be slow, so generate error correction data before verity
data is appended, and append all metadata to the image at the same time.
Needs matching changes from
I09349368925e5e2f2a7961a712a1e933eb0b9ec8
Bug: 26251929
Change-Id: Iea8cee0aee353b4d029d9581e8fbd68ec7b7dd4b
append2simg on a large image is fairly slow, so append verity metadata to
the hash tree to avoid two calls.
Bug: 26251929
Change-Id: I5ec84dcd6e8500e990af13b1f9cef5e809b41605
Add a function check_first_block to read block0 and output a message
on screen if the device has been remounted. The function is called
for version >= 4 only; it executes after a failing block verification
and before recovery attempts.
Bug: 21124327
Change-Id: I49dc0b861c702698896a2495ca094215705d4650
This also makes source files follow the standard directory structure
based on Java package names.
Bug: 25794543
Change-Id: Ie0b568057f836e56407f76d29eeacd28ab907ba8
This makes the signapk tool use Conscrypt (where possible) instead of
the platform-default JCA providers and the Bouncy Castle JCA provider.
This speeds up (by 10-30%) APK and OTA update signing because
Conscrypt's crypto primitives are backed by BoringSSL.
Previously, the signapk tool consisted only of the signapk.jar.
Because Conscrypt is backed by native code, signapk now consists of
signapk.jar and crypto_openjdk_jni shared library. This requires that
users of the tool be updated to provide a suitable -Djava.library.path
argument to the Java runtime. This change updates all known users of
the tool inside the Android source tree to do so.
Bug: 26097626
Change-Id: I8411b37d7f771ed99269751a3007dff103083552
This makes signapk align uncompressed .so entries to memory page
boundary (4096 bytes) to enable such libraries to be loaded at runtime
through memory-mapping the APK.
With this change in place, there should no longer be a need to run
zipalign after (or before) signapk.
Bug: 25794543
Change-Id: I74775af15a683791f57fcbd3497a79951b3f63a1
We used to silently ignore the return value from apply_patch() even if
it had failed. It gives us more trouble to investigate the failure when
the affected file/partition gets touched in subsequent OTAs. This CL
adds the checking of the return value and aborts the update accordingly.
Bug: 25893277
Change-Id: Ie5e1c563576e503343e6a5b28ed4d7039f6f919c
am: dbeab8de65
* commit 'dbeab8de6543db1d0393789cb18ee7fd7ed8fedd':
Use libstdc++ for ijar
Build ijar for apps build
Use .KATI_RESTAT to reduce unnecessary rebuilds of .jar files
am: bae8872294
* commit 'bae88722945130a0c8ebbb1ef27eaa845e9f03fd':
Use libstdc++ for ijar
Build ijar for apps build
Use .KATI_RESTAT to reduce unnecessary rebuilds of .jar files
am: bff3c9b4c1
* commit 'bff3c9b4c10dcb3ce3820d3a5e144e3df20313dc':
Use libstdc++ for ijar
Build ijar for apps build
Use .KATI_RESTAT to reduce unnecessary rebuilds of .jar files
Add an option "--log_diff <filename>" to ota_from_target_files.py
script. When enabled, it logs the differences between the source
and target builds into <filename> when generating incremental OTAs.
Also move target_files_diff.py into releasetools/ so that it can be
packed into otatools.zip.
Bug: 25372309
Change-Id: Ifd4ed0f2f12ef040ee377621ec8c35a873cec34f
For some reason ijar won't build against libc++ for TARGET_BUILD_APPS
builds, but does build with libstdc++.
(cherry picked from commit 718bab6aec)
Bug: 25904002
Change-Id: I1de103918faa5bb574af6f12cc954e7fed44dc62
The same optimization was done for binaries in
https://android-review.googlesource.com/#/c/175250/
To create a TOC file from .jar files, this change introduces
ijar, which is designed for this purpose. Only #include lines
were modified from the original version.
https://github.com/bazelbuild/bazel/tree/master/third_party/ijar
Performance:
$ m && touch
frameworks/base/core/java/com/google/android/util/Procedure.java && time
m
Before: 4m30s (1580 targets)
After: 3m57s (772 targets)
Unfortunately, the improvement is small yet, but local
experiments showed we can cut ~2 more minutes if the similar
optimization is done for .dex files.
(cherry picked from commit c1f5d9c203)
Bug: 24597504
Change-Id: Iec3b2b0b0e674bee5d80cce3c300dc8fad6e7c13
For some reason ijar won't build against libc++ for TARGET_BUILD_APPS
builds, but does build with libstdc++.
Change-Id: I8e900b0f764f0bb8f827705cb9173f07e4f33862
The packaging script is broken when we set TARGET_NO_RECOVERY and
SYSTEM_ROOT_IMAGE both. With TARGET_NO_RECOVERY set, we don't have
RECOVERY/RAMDISK/etc/recovery.fstab. It thus fails the assertion in
common.LoadRecoveryFSTab() that "/" must exist in fstab when enabling
SYSTEM_ROOT_IMAGE.
Change-Id: Id509d724f18be7cf7389dd786924ead894ac0cc1
The same optimization was done for binaries in
https://android-review.googlesource.com/#/c/175250/
To create a TOC file from .jar files, this change introduces
ijar, which is designed for this purpose. Only #include lines
were modified from the original version.
https://github.com/bazelbuild/bazel/tree/master/third_party/ijar
Performance:
$ m && touch
frameworks/base/core/java/com/google/android/util/Procedure.java && time
m
Before: 4m30s (1580 targets)
After: 3m57s (772 targets)
Unfortunately, the improvement is small yet, but local
experiments showed we can cut ~2 more minutes if the similar
optimization is done for .dex files.
Bug: 24597504
Change-Id: Id54953aea25e497c8ebbe783b03f13b488577c6a
Vendor-specific images (radio, bootloader and etc) used to stay in
RADIO/ in target_files zip. A/B updater expects them to be available
under IMAGES/. Make a copy for such images when calling
add_img_to_target_files.py.
We cannot move them directly from RADIO/ to IMAGES/, because we will
delete everything under IMAGES/ when signing for release.
Bug: 25674625
Change-Id: I977c879d0ff7e45c83b3f5c63844a5b135caa383
Set ro.build.ab_update if AB_OTA_UPDATER is true. Device will depend on
this property to call the proper update API.
Change-Id: I5916ef5b30ea3351d323382f7ce83a17df2b0a70
For system images that contain the root directory, we need to find the
root directory at ROOT/ instead of BOOT/RAMDISK/.
Conflicts:
tools/releasetools/common.py
Change-Id: Ica345c8b1b03475f8ac6c44fd576045fcf17c882
If BOARD_USES_FULL_RECOVERY_IMAGE == true, a full copy of recovery
image will be carried as /system/etc/recovery.img instead of a patch
at /system/recovery_from_boot.p.
Bug: 22641135
Change-Id: Ie271d3e2d55d7b003f667ac5b44203d69b23c63b
(cherry picked from commit 8beab69bd5)
When the root directory is included in system, the assumption that
system is mounted at /system breaks down. Redirect /system to /
in that case.
Change-Id: I8031b32d7cb9ca699c1cc22939e28afce9d1c847
Don't generate recovery.img when calling 'make dist' if
TARGET_NO_RECOVERY is set. The build system passes the flag to the
packaging script which then generates recovery.img conditionally.
Bug: 25329471
Change-Id: Ia7f08943834ee39e781d7e9b5beb1b31d95a73d9
(cherry picked from commit db45efa647)
For some old builds, we may not define cache partition size. Change the
exception into a warning to make the script backward compatible.
Change-Id: Ia90531e3455d4888cfac6510cab879c50883be53
(cherry picked from commit 575d68a48e)
check_prereq is a tool for recovery API v2, which has been obselete
since we moved to v3 in [1].
[1] commit e08991e02a7d678f2574e85289a34b2a9a537c82
Bug: 24621915
Change-Id: Ie96348fb56c5a3e029cac2417b59d8eb62dc085b
