We are already using O_SYNC and fsync() for the recursive case
(package_extract_dir()). Make it consistent for the single-file case.
Bug: 20625549
Change-Id: I487736fe5a0647dd4a2428845e76bf642e0f0dff
Output messages in log when recovery is attempted or succeeded during
incremental OTA update.
Change-Id: I4033df7ae3aaecbc61921d5337eda26f79164fda
(cherry picked from commit b686ba2114)
To accommodate new changes in N release, such as error correction [1]
and other potential changes to the updater.
[1]: commit 0a7b47397d
Change-Id: I4dd44417d07dd0a31729894628635a0aa1659008
Add block_image_recover function to rewrite corrupted blocks on the
partition. This can be attempted if block_image_verify fails.
Note that we cannot use libfec during block_image_update as it may
overwrite blocks required for error correction. A separate recovery
pass in case the image is corrupted is the only viable option.
Bug: 21893453
Change-Id: I6ff25648fff68d5f50b41a601c95c509d1cc5bce
We have the last line being empty as a result of
android::base::Split("a\nb\n"), which leads to "missing command"
warnings in the update. Just skip all the empty lines.
Bug: 24373789
Change-Id: I5827e4600bd5cf0418d95477e4592fec47bbd3a9
When processing ui_print commands in the updater, it misses a line break
when printing to the recovery log.
Also clean up uiPrintf() and UIPrintFn() with std::string's.
Change-Id: Ie5dbbfbc40b024929887d3c3ccd3a334249a8c9d
Replace C-string with std::string, pointers with references, and
variable-size arrays in struct with std::vector.
Change-Id: I57f361a0e58286cbcd113e9be225981da56721b2
There is an integer overflow when the size of system goes beyond the
signed int limits. Hence changing pos to size_t.
Change-Id: I6e5e1b2f0e72030b30a6df09a01642f4c82abc79
Currently the fsync() inside write_all() may be called multiple times
when performing a command. Move that to the outer loop and call it
only after completing the command.
Also remove the O_SYNC flag when writing a stash.
Change-Id: I71e51d76051a2f7f504eef1aa585d2cb7a000d80
This reverts commit b65f0272c8.
It slows down the update too much on some devices (e.g. increased
from 8 mins to 40 mins to take a full OTA update).
Bug: 22129621
Change-Id: I016e3b47313e3113f01bb4f8eb3c14856bdc35e5
(cherry picked from commit 7125f9594d)
A RangeSet has half-closed half-open bounds. For example, "3,5" contains
blocks 3 and 4. So "3,5" and "5,7" are actually not overlapped.
Bug: 22098085
Change-Id: I75e54a6506f2a20255d782ee710e889fad2eaf29
Due to observed BLKDISCARD flakiness, overwrite blocks that we want
to discard with zeros first to avoid later issues with dm-verity if
BLKDISCARD is not successful.
Bug: 20614277
Bug: 20881595
Change-Id: I4f6f2db39db990879ff10468c9db41606497bd6f
Due to observed BLKDISCARD flakiness, overwrite blocks that we want
to discard with zeros first to avoid later issues with dm-verity if
BLKDISCARD is not successful.
Bug: 20614277
Bug: 20881595
Change-Id: I0280fe115b020dcab35f49041fb55b7f8e793da3
Although stdout and stderr are both redirected to log file with no
buffering, we are seeing some outputs are mixed in random order.
This is because ui_print commands from the updater are passed to the
recovery binary via a pipe, which may interleave with other outputs
that go to stderr directly.
In recovery, adding ui::PrintOnScreenOnly() function to handle
ui_print command, which skips printing to stdout. Meanwhile, updater
prints the contents to stderr in addition to piping them to recovery.
Change-Id: Idda93ea940d2e23a0276bb8ead4aa70a3cb97700
In the block updater, if BLKDISCARD fails, the error is silently
ignored and some of the blocks may not be erased. This means the
target partition will have inconsistent contents.
If the ioctl fails, return an error and abort the update.
Bug: 20614277
Change-Id: I33867ba9337c514de8ffae59f28584b285324067
These are already getting libc++, so it isn't necessary. If any of the
other static libraries (such as adb) use new or delete from libc++,
there will be symbol collisions.
Change-Id: I55e43ec60006d3c2403122fa1174bde06f18e09f
Only trusted input is passed to parse_range, but check for invalid
input to catch possible problems in transfer lists.
Bug: 21033983
Bug: 21034030
Bug: 21034172
Bug: 21034406
Change-Id: Ia17537a2d23d5f701522fbc42ed38924e1ee3366
When automatically stashing overlapping blocks, should the stash
file already exist due to an explicit stash command, it's not safe
to remove the stash file after the command has completed.
Note that it is safe to assume that the stash file will remain in
place during the execution of the next command, so we don't have
take other measures to preserve overlapping blocks.
The stash file itself will be removed by a free command when it's
no longer needed.
Bug: 20297065
Change-Id: I8ff1a798b94086adff183c5aac03260eb947ae2c
I've added explanatory comments to mzExtractRecursive because
that function will live on as a utility even after we move the
zip format related logic to libziparchive.
bug: 19472796
(cherry-picked from commit c9ccdfd7a42de08c47ab771b94dc5b9d1f957b95)
Change-Id: I8b7fb6fa3eafb2e7ac080ef7a7eceb691b252d8a
Add support for transfer list version 3, which allows us to
verify the status of each command and resume an interrupted
block based OTA update. Notes on the changes:
- Move the previous BlockImageUpdateFn to a shorter and
reusable PerformBlockImageUpdate, which can be used also
in BlockImageVerifyFn for verification.
- Split individual transfer list commands into separate
functions with unified parameters for clarity, and use
a hash table to locate them during execution.
- Move common block reading and writing to ReadBlocks and
WriteBlocks to reduce code duplication, and rename the
readblock and writeblock to less confusing read_all and
write_all.
The coding style of the new functions follows the existing
style in the updater/edify code.
Needs matching changes from
Ia5c56379f570047f10f0aa7373a1025439495c98
Bug: 18262110
Change-Id: I1e752464134aeb2d396946348e6041acabe13942
When building for 32p, we need to be explicit that we wish to build
the 32bit version of the binaries that will be placed in the recovery
image. The recovery image doesn't actually care... but if we are not
explicit in this, the makefiles will ask for the 64bit binaries but the
Android.mk for the binaries will supply the 32bit images (causing the
build to fail).
Change-Id: Iea2d5f412740c082795da4358765751138a4b167
This allows tune2fs to be executed from within OTA scripts,
allowing for file system modifications without formatting the
partition
Bug: 18430740
Change-Id: I0c2e05b5ef4a81ecea043e9b7b99b545d18fe5e6
At the end of the OTA script, we walk through /system, updating
all the permissions on the filesystem, including the UID, GID,
standard UNIX permissions, capabilities, and SELinux labels.
In the case of a symbolic link, however, we want to skip most of
those operations. The UID, GID, UNIX permissions, and capabilities
don't meaningfully apply to symbolic links.
However, that's not true with SELinux labels. The SELinux label on
a symbolic link is important. We need to make sure the label on the
symbolic link is always updated, even if none of the other attributes
are updated.
This change unconditionally updates the SELinux label on the symbolic
link itself. lsetfilecon() is used, so that the link itself is updated,
not what it's pointing to.
In addition, drop the ENOTSUP special case. SELinux has been a
requirement since Android 4.4. Running without filesystem extended
attributes is no longer supported, and we shouldn't even try to handle
non-SELinux updates anymore. (Note: this could be problematic if
these scripts are ever used to produce OTA images for 4.2 devices)
Bug: 18079773
Change-Id: I87f99a1c88fe02bb2914f1884cac23ce1b385f91
In version 2 of block image diffs, we support a new command to load
data from the image and store it in the "stash table" and then
subsequently use entries in the stash table to fill in missing bits of
source data we're not allowed to read when doing move/bsdiff/imgdiff
commands.
This leads to smaller update packages because we can break cycles in
the ordering of how pieces are updated by storing data away and using
it later, rather than not using the data as input to the patch system
at all. This comes at the cost of the RAM or scratch disk needed to
store the data.
The implementation is backwards compatible; it can still handle the
existing version 1 of the transfer file format.
Change-Id: I4559bfd76d5403859637aeac832f3a5e9e13b63a
Otherwise, overflow problems can occur with images larger than
2G since the offsets will overflow a 32-bit off_t.
Change-Id: I05951a38ebeae83ad2cb938594e8d8adb323e2aa
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The computation of file offsets was overflowing for partitions larger
than 2 GB. The parsing of the transfer file could fail at the end if
the data happened to not be properly null-terminated.
Bug: 16984795
Change-Id: I3ce6eb3e54ab7b55aa9bbed252da5a7eacd3317a
These error messages include empty parens after each string
substition. Ill-advised cut and paste, probably.
Bug: 16467401
Change-Id: Ib623172d6228354afdcc2e33442cc53a07f0ecbc
Sometimes renames will move a file into a directory
that does not yet exist. This will create the
parent directories, using the same symlink logic,
to ensure that there is a valid destination.
Change-Id: Iaa005a12ce800c39f4db20f7c25a2a68cb40a52d
This adds F2FS support
- for wiping a device
- for the install "format" command.
Note: crypto data in "footer" with a default/negative length
is not supported, unlike with "ext4".
Change-Id: I8d141a0d4d14df9fe84d3b131484e9696fcd8870
Signed-off-by: JP Abgrall <jpa@google.com>
While executing syspatch and package_extract_file() calls with don't
care maps (both of which are used to rewrite the system image in
incremental and full block OTAs, respectively), pass a progress
callback in and use it to update the visible progress bar.
Change-Id: I1d3742d167c1bb2130571eb5103b7795c65ff371
The default recovery UI will reboot the device when the power key is
pressed 7 times in a row, regardless of what recovery is doing.
Disable this feature during package installation, to minimize the
chance of corrupting the device due to a mid-install reboot. (Debug
packages can explicitly request that the feature be reenabled.)
Change-Id: I20f3ec240ecd344615d452005ff26d8dd7775acf
The new build.prop for Sprout includes lines of the format:
import xxx.prop
These can be safely ignored when reading the property file.
Change-Id: Ia84a138e71461ffe8e591e88143b9787873def29
Make package_extract_file() take an optional third argument which is
the pathname (in the package zip) of a map of don't-care regions to
skip over when writing the file.
Modify syspatch() to take source and target don't-care maps and use
them when patching the system partition.
Add the wipe_block_device() function to do a discard of all data on
the partition.
Change-Id: I8c856054edfb6aab2f3e5177f16d9d78add20be4
updater now depends on the GPL'd libraries libsyspatch and libxdelta3,
so be careful when taking code from this directory.
Change-Id: Ib6f8c50ce7052912b9d81ff96d095f778bf9a3d0
Older versions of android supported an ASLR system where binaries were
randomly twiddled at OTA install time. Remove support for this; we
now use the ASLR support in the linux kernel.
Change-Id: I8348eb0d6424692668dc1a00e2416fbef6c158a2
Changes minzip and recovery's file signature verification to work on
memory regions, rather than files.
For packages which are regular files, install.cpp now mmap()s them
into memory and then passes the mapped memory to the verifier and to
the minzip library.
Support for files which are raw block maps (which will be used when we
have packages written to encrypted data partitions) is present but
largely untested so far.
Bug: 12188746
Change-Id: I12cc3e809834745a489dd9d4ceb558cbccdc3f71
In order to support multi-stage recovery packages, we add the
set_stage() and get_stage() functions, which store a short string
somewhere it can be accessed across invocations of recovery. We also
add reboot_now() which updater can invoke to immediately reboot the
device, without doing normal recovery cleanup. (It can also choose
whether to boot off the boot or recovery partition.)
If the stage string is of the form "#/#", recovery's UI will be
augmented with a simple indicator of what stage you're in, so it
doesn't look like a reboot loop.
Change-Id: I62f7ff0bc802b549c9bcf3cc154a6bad99f94603
* commit 'c64e76c75088b951f61a7f7bacf8af6eccd2ccfa':
Enable incremental builder to find files that moved, and try to process them via patch + rename, instead of delete + add.
set_perm and set_perm_recursive are no longer used. Delete.
(cherry picked from commit 08ef9a9570)
Change-Id: I1bcc90ae19af9df4f0705496c5876987159f75ac
Introduce two new updater functions:
* set_metadata
* set_metadata_recursive
Long term, these functions are intended to be more flexible replacements
for the following methods:
* set_perm
* set_perm_recursive
Usage:
set_metadata("filename", "key1", "value1", "key2", "value2", ...)
set_metadata_recursive("dirname", "key1", "value1", "key2", "value2", ...)
Description:
set_metadata() and set_metadata_recursive() set the attributes on a file/directory
according to the key/value pairs provided. Today, the following keys are
supported:
* uid
* gid
* mode (set_perm_extd only)
* fmode (set_perm_extd_recursive only)
* dmode (set_perm_extd_recursive only)
* selabel
* capabilities
Unknown keys are logged as warnings, but are not fatal errors.
Examples:
* set_metadata("/system/bin/netcfg", "selabel", "u:object_r:system_file:s0");
This sets the SELinux label of /system/bin/netcfg to u:object_r:system_file:s0.
No other changes occur.
* set_metadata("/system/bin/netcfg", "uid", 0, "gid", 3003, "mode", 02750, "selabel", "u:object_r:system_file:s0", "capabilities", 0x0);
This sets /system/bin/netcfg to uid=0, gid=3003, mode=02750,
selinux label=u:object_r:system_file:s0, and clears the capabilities
associated with the file.
* set_metadata_recursive("/system", "uid", 0, "gid", 0, "fmode", 0644, "dmode", 0755, "selabel", "u:object_r:system_file:s0", "capabilities", 0x0);
All files and directories under /system are set to uid=0, gid=0,
and selinux label=u:object_r:system_file:s0. Directories are set to
mode=0755. Files are set to mode=0644 and all capabilities are cleared.
Bug: 10183961
Bug: 10186213
Bug: 8985290
Change-Id: Ifdcf186a7ed45265511dc493c4036e1ac5e3d0af
Modify the OTA installer to understand SELinux filesystem labels.
We do this by introducing new set_perm2 / set_perm2_recursive
calls, which understand SELinux filesystem labels. These filesystem
labels are applied at the same time that we apply the
UID / GID / permission changes.
For compatibility, we preserve the behavior of the existing
set_perm / set_perm_recursive calls.
If the destination kernel doesn't support security labels, don't
fail. SELinux isn't enabled on all kernels.
Bug: 8985290
Change-Id: I99800499f01784199e4918a82e3e2db1089cf25b
Recovery currently has a random mix of messages printed to stdout and
messages printed to stderr, which can make logs hard to read. Move
everything to stdout.
Change-Id: Ie33bd4a9e1272e731302569cdec918e0534c48a6
The bonus data option lets you give an additional blob of uncompressed
data to be used when constructing a patch for chunk #1 of an image.
The same blob must be available at patch time, and can be passed to
the command-line applypatch tool (this feature is not accessible from
edify scripts).
This will be used to reduce the size of recovery-from-boot patches by
storing parts of the recovery ramdisk (the UI images) on the system
partition.
Change-Id: Iac1959cdf7f5e4582f8d434e83456e483b64c02c
* commit '3733d2185bbcedd9ef626907f1f32628986cc0f5':
Use the static version of libsparse
Multiple modules with the same name are going away.
host modules don't need LOCAL_MODULE_TAGS
Full OTAs currently fail if the build contains a directory containing
only symlinks, because nothing creates that directory. Change the
symlink() command to create any ancestor directories that don't exist.
They're created as owner root perms 0700 because we assume that in
practice subsequent set_perm_recursive() calls will fix up their
ownership and permissions.
Change-Id: I4681cbc85863d9778e36b924f0532b2b3ef14310
Extend minzip, recovery, and updater to set the security context on
files based on the file_contexts configuration included in the package.
Change-Id: Ied379f266a16c64f2b4dca15dc39b98fcce16f29
It's surprising if these fail, so abort the whole edify script to
catch any problems early.
Bug: 2284848
Change-Id: Ia2a0b60e7f086fc590b242616028905a229c9e05
Removes the retouch_binaries and undo_retouch_binaries from updater;
newly generated OTA packages should not call them any more.
Note that applypatch retains the ability to unretouch a file as it
reads it. This will be needed as long as we want to support OTAs from
devices that were installed with retouching.
Change-Id: Ib3f6baeae90c84ba85983f626d821ab7e436ceb2
libext4_utils now calls libselinux in order to determine the
file security context to set on files when creating ext4 images.
Change-Id: I09fb9d563d22ee106bf100eacd4cd9c6300b1152
Replace the device-specific functions with a class. Move some of the
key handling (for log visibility toggling and rebooting) into the UI
class. Fix up the key handling so there is less crosstalk between the
immediate keys and the queued keys (an increasing annoyance on
button-limited devices).
Change-Id: I698f6fd21c67a1e55429312a0484b6c393cad46f
updater now has a function "wipe_cache();" which causes recovery to
wipe the cache partition after the successful installation of the
package. Move log copying around a bit so logs and the last_install
flag file are copied to cache after it's wiped.
Bug: 5314244
Change-Id: Id35a9eb6dcd626c8f3a3a0076074f462ed3d44bd
write_raw_image() can now take either a blob or a filename as the
source. The blob format eliminates the need for a temp file.
Change-Id: I0c6effec53d47862040efcec75e64b7c951cdcf7
When formatting /data, if it's an ext4 filesystem, reserve the
last 16 Kbytes for the crypto footer.
Change-Id: I7b401d851ee87732e5da5860df0287a1c331c5b7
Make the mount and format functions take extra parameters describing
the filesystem type and add support for mounting and formatting ext4
filesystems on EMMC.
Change recovery to consistently use stdout for status messages instead
of mixing stdout and stderr.
Close the update package before invoking the binary, to allow the
installer to unmount /cache if it wants to. Add a function to allow
remounting of a mount as read-only.
Change-Id: Idfcc96c3da66083295177f729263560be58034e4
An extra parameter was added to the make_ext4fs() function, we these tools need
to be updated to match.
Change-Id: Id640a7f2b03153eb333b00337f0f991ff5332349
Separate files for retouch functionality are in minelf/*
ASLR for shared libraries is controlled by "-a" in ota_from_target_files.
Binary files are self-contained. Retouch logic can recover from crashes.
Signed-off-by: Hristo Bojinov <hristo@google.com>
Change-Id: I76c596abf4febd68c14f9d807ac62e8751e0b1bd
This CL removes the following line from the top of build logs:
"diff: out/target/product/*/obj/PACKAGING/updater_extensions_intermediates/register.inc.list: No such file or directory"
Change-Id: I79c15a69a0b1b0da0e45620b45a7a0fea5625250
Make the mount and format functions take extra parameters describing
the filesystem type and add support for mounting and formatting ext4
filesystems on EMMC.
Change recovery to consistently use stdout for status messages instead
of mixing stdout and stderr.
Change the applypatch function to take meaningful arguments instead of
argc and argv. Move all the parsing of arguments into main.c (for the
standalone binary) and into install.c (for the updater function).
applypatch() takes patches as Value objects, so we can pass in blobs
extracted from the package without ever writing them to temp files.
The patching code is changed to read the patch from memory instead of
a file.
A bunch of compiler warnings (mostly about signed vs unsigned types)
are fixed.
Support for the IMGDIFF1 format is dropped. (We've been generating
IMGDIFF2 packages for some time now.)
Change-Id: I217563c500012750f27110db821928a06211323f
- Move applypatch to this package (from build).
- Add a rudimentary type system to edify: instead of just returning a
char*, functions now return a Value*, which is a struct that can
carry different types of value (currently just STRING and BLOB).
Convert all functions to this new scheme.
- Change the one-argument form of package_extract_file to return a
Value of the new BLOB type.
- Add read_file() to load a local file and return a blob, and
sha1_check() to test a blob (or string) against a set of possible
sha1s. read_file() uses the file-loading code from applypatch so it
can read MTD partitions as well.
This is the start of better integration between applypatch and the
rest of edify.
b/2361316 - VZW Issue PP628: Continuous reset to Droid logo:
framework-res.apk update failed (CR LIBtt59130)
Change-Id: Ibd038074749a4d515de1f115c498c6c589ee91e5
Remove support for the HTC-specific "firmware" update command and the
corresponding edify function write_firmware_update(). This
functionality is now done by an edify extension library that lives in
vendor/htc.
Change-Id: I80858951ff10ed8dfff98aefb796bef009e05efb
Add a version of package_extract_file that returns the file data as
its return value (to be consumed by some other edify function that
expects to receive a bunch of binary data as an argument). Lets us
avoid having two copies of a big file in memory (extracting it into
/tmp, which is a ramdisk, and then having something load it into
memory) when doing things like radio updates.
Change-Id: Ie26ece5fbae457eb0ddcd8a13d74d78a769fbc70
The symlink() function should remove existing files before creating
symlinks, so scripts are idempotent. Log messages when various system
calls fail (but don't make the whole script fail).
updater (which is only needed in OTA packages) is getting included in
/system/bin, where it just takes up (quite a bit of) space. Use the
hack of including it only in eng builds so it's not there for user
builds.
Let recovery accept set_progress commands to control progress over the
'current segment' of the bar. Add a set_progress() builtin to the
updater binary.
Allow devices (in BoardConfig.mk) to define additional static
libraries to be linked in to updater, to make device-specific
functions available in edify scripts. Modify the updater makefile to
arrange for device libraries to register their edify functions.
Add a function to read a property from a ".prop"-formatted file
(key=value pairs, one per line, ignore # comment lines and blank
lines). Move ErrorAbort to the core of edify; it's not specific to
updater now that errors aren't stored in the app cookie.
To do a firmware-install-on-reboot, the update binary tells recovery
what file to install before rebooting. Let this file be specified as
"PACKAGE:<foo>" to indicate taking the file out of the OTA package,
avoiding an extra copy to /tmp. Bump the API version number to
reflect this change.
A few more changes to edify:
- fix write_raw_image(); my last change neglected to close the write
context, so the written image was corrupt.
- each expression tracks the span of the source code from which it
was compiled, so that assert()'s error message can include the
source of the expression that failed.
- the 'cookie' argument to each Function is replaced with a State
object, which contains the cookie, the source script (for use with
the above spans), and the current error message (replacing the
global variables that were used for this purpose).
- in the recovery image, a new command "ui_print" can be sent back
through the command pipe to cause text to appear on the screen.
Add a new ui_print() function to print things from scripts.
Rename existing "print" function to "stdout".
Adds more edify functions for OTAs:
is_mounted getprop apply_patch apply_patch_check apply_patch_space
write_raw_image write_firmware_image package_extract_file
This allows us to install radios, hboots, boot images, and install
incremental OTA packages.
Fixes a couple of dumb bugs in edify itself:
- we were doubling the size of the function table each time it was
*not* full, rather than each time it was full
- "no such function" errors weren't visible to the parser, so they
didn't prevent execution of the script.
Adds the following edify functions:
mount unmount format show_progress delete delete_recursive
package_extract symlink set_perm set_perm_recursive
This set is enough to extract and install the system part of a (full)
OTA package.
Adds the updater binary that extracts an edify script from the OTA
package and then executes it. Minor changes to the edify core (adds a
sleep() builtin for debugging, adds "." to the set of characters that
can appear in an unquoted string).
