sample_updater: create tools/gen_update_config.py

gen_update_config.py generates JSON config files
from given OTA image zip files.

README.md is updated.

Test: manually

Change-Id: Ifd09b49a73983a42752ee3842a566cecedb9cae0
Signed-off-by: Zhomart Mukhamejanov <zhomart@google.com>

updater_sample/.gitignore

@@ -7,3 +7,4 @@ Thumbs.db
 .idea/
 gen/
 .vscode
+local.properties

updater_sample/README.md

@@ -9,6 +9,39 @@ A/B (seamless) update is available since Android Nougat (API 24), but this sampl
 targets the latest android.
 
 
+## Workflow
+
+SystemUpdaterSample app shows list of available updates on the UI. User is allowed
+to select an update and apply it to the device. App shows installation progress,
+logs can be found in `adb logcat`. User can stop or reset an update. Resetting
+the update requests update engine to cancel any ongoing update, and revert
+if the update has been applied. Stopping does not revert the applied update.
+
+
+## Update Config file
+
+In this sample updates are defined in JSON update config files.
+The structure of a config file is defined in
+`com.example.android.systemupdatersample.UpdateConfig`, example file is located
+at `res/raw/sample.json`.
+
+In real-life update system the config files expected to be served from a server
+to the app, but in this sample, the config files are stored on the device.
+The directory can be found in logs or on the UI. In most cases it should be located at
+`/data/user/0/com.example.android.systemupdatersample/files/configs/`.
+
+SystemUpdaterSample app downloads OTA package from `url`. If `ab_install_type`
+is `NON_STREAMING` then app downloads the whole package and
+passes it to the `update_engine`. If `ab_install_type` is `STREAMING`
+then app downloads only some files to prepare the streaming update and
+`update_engine` will stream only `payload.bin`.
+To support streaming A/B (seamless) update, OTA package file must be
+an uncompressed (ZIP_STORED) zip file.
+
+Config files can be generated using `tools/gen_update_config.py`.
+Running `./tools/gen_update_config.py --help` shows usage of the script.
+
+
 ## Running on a device
 
 The commands expected to be run from `$ANDROID_BUILD_TOP`.
@@ -18,13 +51,6 @@ The commands expected to be run from `$ANDROID_BUILD_TOP`.
 3. Add update config files.
 
 
-## Update Config file
-
-Directory can be found in logs or on UI. Usually json config files are located in
-`/data/user/0/com.example.android.systemupdatersample/files/configs/`. Example file
-is located at `res/raw/sample.json`.
-
-
 ## Development
 
 - [x] Create a UI with list of configs, current version,
@@ -33,8 +59,8 @@ is located at `res/raw/sample.json`.
       update zip file
 - [x] Add `UpdateConfig` for working with json config files
 - [x] Add applying non-streaming update
-- [ ] Add applying streaming update
 - [ ] Prepare streaming update (partially downloading package)
+- [ ] Add applying streaming update
 - [ ] Add tests for `MainActivity`
 - [ ] Add stop/reset the update
 - [ ] Verify system partition checksum for package

updater_sample/tools/gen_update_config.py

@@ -0,0 +1,154 @@
+#!/usr/bin/env python3
+#
+# Copyright (C) 2018 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Given a OTA package file, produces update config JSON file.
+
+Example:  tools/gen_update.config.py \\
+            --ab_install_type=STREAMING \\
+            ota-build-001.zip  \\
+            my-config-001.json \\
+            http://foo.bar/ota-builds/ota-build-001.zip
+"""
+
+import argparse
+import json
+import os.path
+import sys
+import zipfile
+
+
+class GenUpdateConfig(object): # pylint: disable=too-few-public-methods
+    """
+    A class that generates update configuration file from an OTA package.
+
+    Currently supports only A/B (seamless) OTA packages.
+    TODO: add non-A/B packages support.
+    """
+
+    AB_INSTALL_TYPE_STREAMING = 'STREAMING'
+    AB_INSTALL_TYPE_NON_STREAMING = 'NON_STREAMING'
+    METADATA_NAME = 'META-INF/com/android/metadata'
+
+    def __init__(self, package, out, url, ab_install_type):
+        self.package = package
+        self.out = out
+        self.url = url
+        self.ab_install_type = ab_install_type
+        self.streaming_required = (
+            # payload.bin and payload_properties.txt must exist.
+            'payload.bin',
+            'payload_properties.txt',
+        )
+        self.streaming_optional = (
+            # care_map.txt is available only if dm-verity is enabled.
+            'care_map.txt',
+            # compatibility.zip is available only if target supports Treble.
+            'compatibility.zip',
+        )
+
+    def run(self):
+        """generate config"""
+        streaming_metadata = None
+        if self.ab_install_type == GenUpdateConfig.AB_INSTALL_TYPE_STREAMING:
+            streaming_metadata = self._gen_ab_streaming_metadata()
+
+        config = {
+            '__': '*** Generated using tools/gen_update_config.py ***',
+            'name': self.ab_install_type[0] + ' ' + os.path.basename(self.package)[:-4],
+            'url': self.url,
+            'ab_streaming_metadata': streaming_metadata,
+            'ab_install_type': self.ab_install_type,
+        }
+
+        with open(self.out, 'w') as out:
+            json.dump(config, out, indent=4, separators=(',', ': '), sort_keys=True)
+            print('Config is written to ' + out.name)
+
+    def _gen_ab_streaming_metadata(self):
+        """Open zip file and get metadata for files required for streaming update."""
+        with zipfile.ZipFile(self.package, 'r') as package_zip:
+            property_files = self._get_property_files(package_zip)
+
+            metadata = {
+                'property_files': property_files
+            }
+
+        return metadata
+
+    def _get_property_files(self, zip_file):
+        """Constructs the property-files list for A/B streaming metadata"""
+
+        def compute_entry_offset_size(name):
+            """Computes the zip entry offset and size."""
+            info = zip_file.getinfo(name)
+            offset = info.header_offset + len(info.FileHeader())
+            size = info.file_size
+            return {
+                'filename': os.path.basename(name),
+                'offset': offset,
+                'size': size,
+            }
+
+        property_files = []
+        for entry in self.streaming_required:
+            property_files.append(compute_entry_offset_size(entry))
+        for entry in self.streaming_optional:
+            if entry in zip_file.namelist():
+                property_files.append(compute_entry_offset_size(entry))
+
+        # 'META-INF/com/android/metadata' is required
+        property_files.append(compute_entry_offset_size(GenUpdateConfig.METADATA_NAME))
+
+        return property_files
+
+
+def main(): # pylint: disable=missing-docstring
+    ab_install_type_choices = [
+        GenUpdateConfig.AB_INSTALL_TYPE_STREAMING,
+        GenUpdateConfig.AB_INSTALL_TYPE_NON_STREAMING]
+    parser = argparse.ArgumentParser(description=__doc__,
+                                     formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument('--ab_install_type',
+                        type=str,
+                        default=GenUpdateConfig.AB_INSTALL_TYPE_NON_STREAMING,
+                        choices=ab_install_type_choices,
+                        help='A/B update installation type')
+    parser.add_argument('package',
+                        type=str,
+                        help='OTA package zip file')
+    parser.add_argument('out',
+                        type=str,
+                        help='Update configuration JSON file')
+    parser.add_argument('url',
+                        type=str,
+                        help='OTA package download url')
+    args = parser.parse_args()
+
+    if not args.out.endswith('.json'):
+        print('out must be a json file')
+        sys.exit(1)
+
+    gen = GenUpdateConfig(
+        package=args.package,
+        out=args.out,
+        url=args.url,
+        ab_install_type=args.ab_install_type)
+    gen.run()
+
+
+if __name__ == '__main__':
+    main()