Commit graph

5130 commits

Author SHA1 Message Date
Tianjie Xu
9989175c0e Merge "Generate care_map.txt when AVB is enabled" am: 65b98cd3b1 am: 1e0b14eb33
am: 72274464c4

Change-Id: I7549c4f1b66ee1bf8dfaab26e90a2022f9f19dea
2017-06-02 18:24:35 +00:00
Tianjie Xu
65b98cd3b1 Merge "Generate care_map.txt when AVB is enabled" 2017-06-02 18:09:58 +00:00
Tianjie Xu
6b2e155ee6 Generate care_map.txt when AVB is enabled
When AVB is enabled, generate care_map.txt and add it to the target
files. Also copy it into the OTA package where it will later be used
by the update_verifier.

Bug: 62208947
Test: \
1. Run add_img_to_target_files on the TF of a new pixel device,
and care_map.txt generates successfully.

2. Make dist in oc-dr1-release and find care_map.txt in the OTA package.

3. update_verifier succeeds in reading all the blocks on the care_map,
and fails to read out-of-bound blocks.

Change-Id: I2881711e6f87789cb7de150dbeca18b756fed68a
2017-06-01 21:36:43 -07:00
Tao Bao
c633ed0230 Support re-generating DTBO image from add_img_to_target_files.py.
This is a step to enable signing a given target_files zip with release
keys.

When calling sign_target_files_apks.py, we will delete all the entries
under IMAGES/ in order to re-generate them (with the proper release
keys). In order to support that, we need to pack everything in need into
TF.zip.

Steps to test the CL.
a) Choose a target that has both AVB and DTBO enabled.
 $ m dist

b) Check IMAGES/dtbo.img and PREBUILT_IMAGES/dtbo.img both exist in the
   generated out/dist/TF.zip.

c) Remove the entries under IMAGES/ from the generated TF.zip.
 $ zip -d TF.zip IMAGES/\*

d) Re-generate the images with TF.zip.
 $ build/make/tools/releasetools/add_img_to_target_files.py TF.zip

e) Check that IMAGES/dtbo.img is re-generated, and it's identical to the
   image in b). Note that by default the re-generated image will carry a
   different footer, because of the random salt. This CL is verified by
   specifying the same salt.

Bug: 38315721
Test: see above.
Change-Id: I0bdc4e1cd4800962dc3902ca550dad6a8ca56c78
2017-06-01 20:07:33 -07:00
Tao Bao
ffa864c5ed Merge "Pack avbtool into otatools.zip." am: db5fa05429 am: 308b234593
am: 0c2bc7a7b1

Change-Id: I1f15f62536e3030bee7fec0fdacb1ef8ee57a69e
2017-06-01 07:30:30 +00:00
Treehugger Robot
db5fa05429 Merge "Pack avbtool into otatools.zip." 2017-06-01 07:19:17 +00:00
Tao Bao
30295a396f Merge "Remove the obsolete logic in img_from_target_files.py." am: ba029141dc am: f37c956b6d
am: 464fed4f53

Change-Id: If990a48a06ba4b48e865e4098d58fed4a074d896
2017-05-31 23:30:59 +00:00
Tao Bao
2bb109709a Remove the obsolete logic in img_from_target_files.py.
img_from_target_files.py used to handle the case that a given TF.zip not
containing the image entries under IMAGES/. That is only the case for
pre-Lollipop releases.

Also unzip the needed files only since we know that for sure now.

Test: img_from_target_files.py with an existing bullhead-TF.zip gives
      the same bullhead-img.zip.
Change-Id: I892379ba388df80ae63be9d3ce647fbb77fd4753
2017-05-31 11:17:56 -07:00
Tao Bao
3cba374da2 Pack avbtool into otatools.zip.
Also pack the test keys for easier testing.

Bug: 38315721
Test: m otatools-package and avbtool is present in otatools.zip.
Change-Id: Ieb63bf3f4bc211ef1f48ab278cb01b70845d06da
2017-05-31 10:20:25 -07:00
Bowgo Tsai
b4c268eb7b Merge "Support signing custom images with AVB HASH or AVB HASHTREE" am: fe7aeb1de7 am: 16b64cbb58
am: 3989b738b8

Change-Id: I11393e1833db97301ffdd31ec3db0968e87c1a24
2017-05-31 03:38:24 +00:00
Treehugger Robot
fe7aeb1de7 Merge "Support signing custom images with AVB HASH or AVB HASHTREE" 2017-05-31 03:24:25 +00:00
Tianjie Xu
9307492d4e Merge "Remove the obsolete UnpackPackageDir() in edify generator" am: f7a76f9a30 am: cb2490c645
am: 932a9c8a61

Change-Id: I616920f452b7fda545d87dd285474f9f0acd5b14
2017-05-31 01:25:19 +00:00
Tianjie Xu
f7a76f9a30 Merge "Remove the obsolete UnpackPackageDir() in edify generator" 2017-05-31 01:08:15 +00:00
Tianjie Xu
3a97af0ca4 Merge "Move recovery building from sign_target_files_apk to add_img_to_target_files" am: 0996f815db am: d50b6e4cae
am: 38800822ef

Change-Id: Ib2a237c601142fa168f0f73f6c12fd28c3ac4b79
2017-05-30 21:02:27 +00:00
Tianjie Xu
0ba01a614a Merge "Update the recovery files under SYSTEM/ if recovery patch is rebuilt" am: ada0854169 am: 9080b10f3a
am: d153e86abd

Change-Id: Ic7e6731d8a5da15353c793acd33b6106e11382b2
2017-05-27 00:27:44 +00:00
Tianjie Xu
616fbeb144 Move recovery building from sign_target_files_apk to add_img_to_target_files
Currently we're building the boot/recovery image twice, which is
redundant. And b/38455129 shows a problematic case when the image
from two builds doesn't match. We should only build the recovery
image once in the add_img_to_target_files.

Bug: 62021378
Test: call sign_target_files_apk on an angler target file,
recovery-from-boot.p generates successfully; and SHA of recovery.img
matches the one in install-recovery.sh.

Change-Id: I01e033501d80c18a87cbb870300eee5c19a04441
2017-05-26 17:20:22 -07:00
Tianjie Xu
38af07ff20 Update the recovery files under SYSTEM/ if recovery patch is rebuilt
If we pass "rebuild_recovery" to add_img_to_target_files, the recovery
patch is rebuilt. But related files under SYSTEM/ (e.g.
SYSTEM/recovery-from-boot.p && SYSTEM/bin/install-recovery.sh) are not
updated.

This may cause a mismatch between system.img and SYSTEM/, and
may lead to a failure in validate_target_files.py.

Bug: 62096364
Test: Rebuild the system image in the TF and observe the recovery files
under SYSTEM/ get updated.

Change-Id: I7d679a612a86d02cf2eff81d1d120c0067138ed9
2017-05-26 22:40:23 +00:00
Bowgo Tsai
7ea994b21c Support signing custom images with AVB HASH or AVB HASHTREE
`make custom_images` supports to build different kinds of *non-droid* images,
e.g., odm.img. Adding the support of signing them with either AVB HASH footer
or AVB HASHTREE footer. The user can use HASH for small images and
HASHTREE for large images.

Sample signing configurations:
 * AVB HASH footer:
   - CUSTOM_IMAGE_AVB_HASH_ENABLE := true
   - CUSTOM_IMAGE_AVB_ADD_HASH_FOOTER_ARGS := --append_to_release_string my_odm_image

 * AVB HASHTREE footer:
   - CUSTOM_IMAGE_AVB_HASHTREE_ENABLE := true
   - CUSTOM_IMAGE_AVB_ADD_HASHTREE_FOOTER_ARGS := --fec_num_roots 8

 * Using custom signing key:
   - CUSTOM_IMAGE_AVB_ALGORITHM := SHA256_RSA2048
   - CUSTOM_IMAGE_AVB_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem

Bug: 36701014
Test: `make custom_images` with AVB HASH footer
Test: `make custom_images` with AVB HASHTREE footer
Test: `make droid` to check system.img is still properly signed with AVB HASHTREE
Test: `make droid` to check vendor.img is still properly signed with AVB HASHTREE

Change-Id: I8dc420e12e37e9a631345c0cd883339db05d489f
2017-05-24 14:21:03 +08:00
Tianjie Xu
6a324ba588 Remove the obsolete UnpackPackageDir() in edify generator
package_extract_dir is used in file based OTA only and should be killed.

Bug: 37959785
Test: code search shows no usage of this function in aosp.
Change-Id: Id3719b969c24b7ecef0c7f0e4a3af09a72be54d4
2017-05-23 17:15:14 -07:00
Tao Bao
4a0d1f2ece Fix the symlink detection when signing TF.zip.
am: 3eb353cdb0

Change-Id: I00c256d5eacab5e60c1292f9572eaaaee5a1c11e
2017-05-23 20:25:57 +00:00
Tao Bao
3eb353cdb0 Fix the symlink detection when signing TF.zip.
We used to check for 'attr >> 16 == 0xa1ff' (i.e. 0o120777) to detect
symlinks in the input target_files zip (TF.zip). This becomes broken
after we switch to soong_zip, which packs symlinks with 0o120700.

This CL fixes the issue by using stat.S_ISLNK() instead.

Note that we don't need to stage the files with the exact permission
bits as in the input TF.zip. Because this part is covered by mkbootfs
by using the canned or the compiled-in fs_config - as long as the
files/directories are accessible and the symlinks are created.

Bug: 38455129
Test: sign_target_files_apks.py on bullhead TF.zip. Check the
      checksums in SYSTEM/bin/install-recovery.sh.
Change-Id: I51c1fc9a257fb3f18c16c2ed71528abaa6f7d9c9
(cherry picked from commit 406050bdb6)
2017-05-23 11:18:26 -07:00
Tao Bao
78a41f6553 Merge "Fix the symlink detection when signing TF.zip." 2017-05-23 18:15:44 +00:00
Tao Bao
406050bdb6 Fix the symlink detection when signing TF.zip.
We used to check for 'attr >> 16 == 0xa1ff' (i.e. 0o120777) to detect
symlinks in the input target_files zip (TF.zip). This becomes broken
after we switch to soong_zip, which packs symlinks with 0o120700.

This CL fixes the issue by using stat.S_ISLNK() instead.

Note that we don't need to stage the files with the exact permission
bits as in the input TF.zip. Because this part is covered by mkbootfs
by using the canned or the compiled-in fs_config - as long as the
files/directories are accessible and the symlinks are created.

Bug: 38455129
Test: sign_target_files_apks.py on bullhead TF.zip. Check the
      checksums in SYSTEM/bin/install-recovery.sh.
Change-Id: I51c1fc9a257fb3f18c16c2ed71528abaa6f7d9c9
2017-05-22 23:33:11 -07:00
Mark Salyzyn
c2eb5566b6 Merge "Android.mk: clear variable at end of use" am: 5d9c1e7430 am: edf9c79e42
am: 05f9a1cec6

Change-Id: I628b58bc041c470ba6d4f426a0e09f6bca581132
2017-05-19 20:57:13 +00:00
William Roberts
6b19e495b1 Android.mk: clear variable at end of use
Clear fs_config_generate_extra_partition_list after use.

Test: build fs_config_files and fs_config_dirs targets.

Change-Id: Ie18bcd2df2ff1becb178e9ba45d226e2e9a8faab
Signed-off-by: William Roberts <william.c.roberts@intel.com>
2017-05-19 20:14:27 +00:00
Chih-Hung Hsieh
fd3b52addf Merge "Recognize new clang and make warnings." am: dd51cfc9a2 am: 6d8d482f79
am: e6a58345bd

Change-Id: I18c3a1d481846660c0d86dab7c1cb94ccd99b6d5
2017-05-18 23:35:21 +00:00
Chih-Hung Hsieh
8ef1aae6d3 Recognize new clang and make warnings.
Test: warn.py build.log
Change-Id: I078b3cc35a00590312208cb90af0a1f2431abf09
2017-05-18 14:29:59 -07:00
Yueyao Zhu
3a3d9c5837 Merge "add dtbo support for ota" am: 294d5c5f07 am: 793a9bb490
am: d2f336664c

Change-Id: I800efcf88ddb00d9624514b1c748e998f1ebd3e5
2017-05-15 17:01:55 +00:00
Treehugger Robot
294d5c5f07 Merge "add dtbo support for ota" 2017-05-15 16:49:46 +00:00
Julius D'souza
86568d4acc Merge "Add the option to reserve headroom for partition images. This is useful for devices with low disk space with different build variants." am: bba00b2cb9 am: e255054a08
am: b83a34bf9e

Change-Id: I5ae85fbf49d7cfae4c72d5cdec2ee5cfc28e8d8e
2017-05-13 01:48:23 +00:00
Treehugger Robot
bba00b2cb9 Merge "Add the option to reserve headroom for partition images. This is useful for devices with low disk space with different build variants." 2017-05-13 01:37:25 +00:00
Yueyao Zhu
889ee5e7d3 add dtbo support for ota
Test: make dist and locally push ota on a device
Change-Id: I920b98f20e248b437955b2a963eb69ed2ddb8d45
2017-05-12 17:50:46 -07:00
Tianjie Xu
ab40afa40d Merge "Add a script to parse an block-based ota package" am: f34d435925 am: ca3aad1943
am: 61ee5aecd0

Change-Id: Iccbe2c22c9091fb4e843e255242217e7299eef51
2017-05-11 22:33:46 +00:00
Tianjie Xu
f34d435925 Merge "Add a script to parse an block-based ota package" 2017-05-11 21:52:10 +00:00
Tao Bao
df8e605204 Merge "OTA Tools: Handle password encrypted keys" am: 4e1c113554 am: c3947ece97
am: 686c732c63

Change-Id: I67face061729bb0b25345ada85387f6e1bbefb57
2017-05-11 19:51:50 +00:00
Christian Oder
f63e2cd47b OTA Tools: Handle password encrypted keys
* Add logic to handle decrypted keys from common.GetKeyPassword in
WriteABOTAPackageWithBrilloScript.

* Get the keys passwords in main and store them in OPTIONS.key_passwords.
This allows accessing them in WriteABOTAPackageWithBrilloScript and SignOutput
so it's only required to ask for the password once, while allowing to use
decrypted signing keys.

Test: ota_from_target_files.py on marlin and angler respectively.
Change-Id: I7c9b0198855a4b630c52b8552e904f312f09c4ce
2017-05-10 16:54:42 -07:00
Tianjie Xu
ce9d78fcd9 Add a script to parse an block-based ota package
The script prints the meta info, new/patch data size. It also simulates
the transfer commands and calculates amount of I/O as well as cache
needed.

We can add the more options to parse system/vendor.map so that we can
analyze the location change of files.

Bug: 31514709
Test: parse a bullhead incremental/full ota.
Change-Id: I70ed4f4c15e595b1851109e9799d44ac4e815c16
2017-05-09 21:01:19 +00:00
Julius D'souza
001c676b81 Add the option to reserve headroom for partition images.
This is useful for devices with low disk space with different
build variants.

Bug: 37469715
Test: Regular image builds successfully, errors occur when
the headroom size is greater than available partition space.

Change-Id: I526cdd0f84981bbd16e3afcfe1cd7fc43dce98ef
2017-05-08 11:59:25 -07:00
Tao Bao
f5b5e5c378 Merge "releasetools: Add a verbose parameter to common.Run()." am: a149a83e24 am: 7d887baf42
am: 9464b0722d

Change-Id: I9f8aa72e347cbf744318242220b245e3a3ee308d
2017-05-05 23:21:57 +00:00
Tao Bao
39451582c4 releasetools: Add a verbose parameter to common.Run().
Caller can optionally specify the verbose flag which overrides
OPTIONS.verbose. The command line won't be outputed with verbose=False.
This is useful for cases that a) those command lines are less useful
(but will spam the output otherwise); b) sensitive info is part of the
invocation.

'verbose=False' will be consumed by common.Run() only, instead of being
passed to subprocess.Popen().

Test: ota_from_target_files.py on a block based OTA.
Change-Id: I7d5b4094d756a60f84f89c6a965e7ccc68e435f8
2017-05-04 11:18:56 -07:00
Jaekyun Seok
9e7eab53c1 Split vendor license notices into vendor partition
Xml files will be generated instead of html files, and then Settings
will generate html file on runtime from xml files of partitions.

Test: build succeeded and tested on sailfish
Bug: 37099941
Merged-In: Id7899381a537b1947cbb6164ed2b4a98492334af
Change-Id: Id7899381a537b1947cbb6164ed2b4a98492334af
(cherry picked from commit 3b7560bf88)
2017-05-01 22:07:50 +00:00
Jaekyun Seok
035a49f301 Split vendor license notices into vendor partition
am: 3b7560bf88

Change-Id: Ib2926d1b97a5bdc9ef59738bd9e993487da0730d
2017-05-01 21:26:51 +00:00
Jaekyun Seok
3b7560bf88 Split vendor license notices into vendor partition
Xml files will be generated instead of html files, and then Settings
will generate html file on runtime from xml files of partitions.

Test: build succeeded and tested on sailfish
Bug: 37099941
Change-Id: Id7899381a537b1947cbb6164ed2b4a98492334af
2017-04-28 16:29:17 +09:00
Chih-Hung Hsieh
ee0ffb596b Merge "Get Android TOP from build.log file." am: 72a662a282 am: 5bc24ca63a
am: a463931292

Change-Id: Ibd67280ffe97bf5ab3ea988f862331a145681da5
2017-04-27 23:00:21 +00:00
Chih-Hung Hsieh
ef21d1458b Get Android TOP from build.log file.
find_android_root works only when warn.py is found
so it works for build bot, but not when warn.py is
run against a stand alone build.log file.
Now get the TOP directory path from build.log
if it is found.

Bug: 37725427
Test: warn.py --gencsv build.log
Change-Id: I7b776c2e71a8d8d769f0600b08abe37acbd6ce7c
2017-04-27 10:25:37 -07:00
Tao Bao
f9937b5c24 Merge "Revert "Revert "Add compatibility metadata to OTA package""" 2017-04-27 16:12:57 +00:00
Mark Salyzyn
c1b0993552 Merge "fs_config_dirs and fs_config_files miscalculates LOCAL_REQUIRED_MODULES" am: d0020fafe2 am: 151e856015
am: de12e39af7

Change-Id: I4352677bec833b32c6c6a2acf86276d5fa7ed72a
2017-04-27 15:49:35 +00:00
Mark Salyzyn
eb8dc72724 fs_config_dirs and fs_config_files miscalculates LOCAL_REQUIRED_MODULES
Misspelled fs_config_generate_extra_partition_list when used to
calculate LOCAL_REQUIRED_MODULES dependencies.  This resulted in
fs_config_dirs and fs_config_files failing to generate the vendor,
oem or odm variants (ie: fs_config_files_vendor)

Test: build generates correct fs_config override files.
Bug: 37721445
Bug: 36071012
Change-Id: I053dd5cdb357c7cea05a5ff63b0686bd0bc99df6
2017-04-27 08:01:27 -07:00
Tao Bao
4bede622dc Merge "Revert "Revert "Add compatibility metadata to OTA package""" into oc-dev
am: cc2bfa484e

Change-Id: Ief6654d9b1f6d4f355b792806b809e9378a32b4a
2017-04-26 21:30:53 +00:00
Tao Bao
21803d3574 Revert "Revert "Add compatibility metadata to OTA package""
This reverts commit b1e59b8a73.

This CL enables generating compatibility zip entry in an A/B OTA
package. We're not enabling the support for non-A/B OTAs for now until
they have the necessary support (e.g. /proc/config.gz exists).

Bug: 36810913
Test: Generate full and incremental packages for marlin and applly under
      normal boot and sideload respectively.

Change-Id: I69f0a3feb7a0efc7f727f92f31c9e63f47ee6530
(cherry picked from commit b63c952cc7)
2017-04-26 14:28:46 -07:00
Mark Salyzyn
0934979502 Merge changes I44289a3d,I0537453b,I23a655b1 am: 7928afb83d am: b6650076ad
am: 6d63c8a7d5

Change-Id: I2165562709fdebf98c90958299ed23079b41f080
2017-04-26 14:47:57 +00:00
Mark Salyzyn
1fd588cfb2 fs_config_generator: Add fs_config_(dirs|files)_(vendor|oem|odm)
Add build targets to generate the vendor, oem and odm slices of the
fs_config_dirs and fs_config_files targets for each of the respective
filesystems, and automatically trigger the individual and enabled
partition build objects based as dependents of fs_config_dirs and
fs_config_files.

Sensitive to the following board definition pairs:

BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE  BOARD_USES_VENDORIMAGE
BOARD_OEMIMAGE_FILE_SYSTEM_TYPE     BOARD_USES_OEMIMAGE
BOARD_ODMIMAGE_FILE_SYSTEM_TYPE     BOARD_USES_ODMIMAGE

Which when defined and set means the images are generated for vendor,
oem or odm respectively.

Test: manually build w/ m 'fs_config_(dirs|files)_(vendor|oem|odm)'
      and check the expected products, or fails if not pertinent.
Test: build for product that produces split to system and vendor and
      subsequently boots with vendor products correctly configured.
Bug: 36071012
Change-Id: I44289a3dd50b7657367c826ad813c671a3de6afc
2017-04-25 15:09:09 -07:00
Mark Salyzyn
256d339413 fs_config_generate: add -P <partitionlist> option
Add a -P option to fs_config_generate that allows us to filter on
a comma separated partition whitelist or minus prefixed -blacklist.
A partition is defined as accessible as <partition>/ or
system/<partition>/.

No change for legacy, where no -P flag is specified.

For system.img, but not vendor, oem or odm file references:
fs_config_generate -P -vendor,-oem,-odm

For vendor.img file references:
fs_config_generate -P vendor

For oem.img file references:
fs_config_generate -P oem

For odm.img file references:
fs_config_generate -P odm

Add and fortify tests for each of these cases.

SideEffects: none
Test: gTest host fs_config-unit-test
Bug: 36071012
Change-Id: I0537453bb289d41ef98ad76e69f78ae49358a1b3
2017-04-25 15:09:01 -07:00
Mark Salyzyn
5649b31a17 fs_config_generate: unit test
Test -D and -F flag of fs_config_generate

Test: gTest host fs_config-unit-tests
Bug: 36071012
Change-Id: I23a655b17b7c55817f2aacc760700ff1b0728db3
2017-04-25 15:08:56 -07:00
Tao Bao
b63c952cc7 Revert "Revert "Add compatibility metadata to OTA package""
This reverts commit b1e59b8a73.

This CL enables generating compatibility zip entry in an A/B OTA
package. We're not enabling the support for non-A/B OTAs for now until
they have the necessary support (e.g. /proc/config.gz exists).

Bug: 36810913
Test: Generate full and incremental packages for marlin and applly under
      normal boot and sideload respectively.

Change-Id: I69f0a3feb7a0efc7f727f92f31c9e63f47ee6530
2017-04-25 15:04:41 -07:00
George Burgess IV
ce662c40f4 Merge "warn.py: have URLs open in new tabs" am: 7ffb8c8fa8 am: 2f6f02d812
am: 763e59213d

Change-Id: I7ec4e89370c007d28dd0db5ceafd80ae9f15a94f
2017-04-25 17:17:25 +00:00
George Burgess IV
169f5f1c6c warn.py: have URLs open in new tabs
Bug: None
Test: Ran warn.py on a build log. Code links in resultant HTML open in
new tabs when clicked.

Change-Id: Ia3def5ea14fe216700cb2ffabe4a39b98e69d55a
2017-04-21 15:19:36 -07:00
TreeHugger Robot
dd4cdfd30a Merge changes Ieb58dd9a,I8ada38df into oc-dev
* changes:
  AVB: support BOARD_BUILD_SYSTEM_ROOT_IMAGE
  AVB: support signing vendor.img
2017-04-18 04:10:58 +00:00
Tao Bao
f6d073e487 Merge "Revert "Add compatibility metadata to OTA package"" into oc-dev
am: 533183dc68

Change-Id: Iacede7d5a6357c91e86a8bc8dbf113b991e2765c
2017-04-17 20:44:46 +00:00
Tao Bao
b1e59b8a73 Revert "Add compatibility metadata to OTA package"
This reverts commit 853cd306b0.

Temporarily reverting the CL to work around the bug in recovery code.

Bug: 37413730
Change-Id: I8e77112ffafcf32332794739ee1c855cc83b32b0
2017-04-17 18:40:18 +00:00
Bowgo Tsai
16ba563b71 AVB: support BOARD_BUILD_SYSTEM_ROOT_IMAGE
Passing --setup_rootfs_from_kernel to avbtool when
BOARD_BUILD_SYSTEM_ROOT_IMAGE is true

Bug: 33590159
Test: 'make' sailfish with BOARD_AVB_ENABLE := true
Test: 'make dist' with BOARD_AVB_ENABLE := true
Change-Id: Ieb58dd9ae6be1eceb90a33c739b85cff5cbc6e0a
Merged-In: Ieb58dd9ae6be1eceb90a33c739b85cff5cbc6e0a
(cherry picked from commit 9b3776017f)
2017-04-15 13:57:42 +08:00
Bowgo Tsai
3f0fd3d085 AVB: support signing vendor.img
Uses avbtool to sign vendor.img if BOARD_AVB_ENABLE is set.
It also allows appending additional arguments to avbtool via
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS.

e.g.,
  BOARD_AVB_ENABLE := true
  BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS := --generate_fec

Bug: 35415839
Test: "make" with the above variables and use avbtool to check vbmeta is
      appended to vendor.img
Test: "make dist" with the above variables

Change-Id: I8ada38dff3def6d34613e77c67944def8a49f464
Merged-In: I8ada38dff3def6d34613e77c67944def8a49f464
(cherry picked from commit 8ee4a3db8c)
2017-04-15 13:57:21 +08:00
Bowgo Tsai
1c6d3e1b87 Merge changes Ieb58dd9a,I8ada38df am: 9ea5c1aad0 am: 225a18d07d
am: 13de7b5f0b

Change-Id: Ied184158f06af1fae79240299bad732e067e5fcd
2017-04-15 05:47:28 +00:00
Bowgo Tsai
9b3776017f AVB: support BOARD_BUILD_SYSTEM_ROOT_IMAGE
Passing --setup_rootfs_from_kernel to avbtool when
BOARD_BUILD_SYSTEM_ROOT_IMAGE is true

Bug: 33590159
Test: 'make' sailfish with BOARD_AVB_ENABLE := true
Test: 'make dist' with BOARD_AVB_ENABLE := true
Change-Id: Ieb58dd9ae6be1eceb90a33c739b85cff5cbc6e0a
2017-04-15 08:53:40 +08:00
Bowgo Tsai
8ee4a3db8c AVB: support signing vendor.img
Uses avbtool to sign vendor.img if BOARD_AVB_ENABLE is set.
It also allows appending additional arguments to avbtool via
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS.

e.g.,
  BOARD_AVB_ENABLE := true
  BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS := --generate_fec

Bug: 35415839
Test: "make" with the above variables and use avbtool to check vbmeta is
      appended to vendor.img
Test: "make dist" with the above variables

Change-Id: I8ada38dff3def6d34613e77c67944def8a49f464
2017-04-15 08:53:06 +08:00
Michael Schwartz
140ba703b6 Merge "Add compatibility metadata to OTA package" into oc-dev
am: 5a933fc8d5

Change-Id: I69efaa4a920ddf13bbdd0de98dbabc0075480b3b
2017-04-14 21:04:49 +00:00
Michael Schwartz
853cd306b0 Add compatibility metadata to OTA package
Store the metadata in a archive within the OTA package so the collection
can be retrieved as a single unit by the device.

Bug: 36810913
Test: `make dist` marlin (ab) and angler (non-AB) for incremental and full OTA.
Change-Id: I94118e48a3c6ed5ff890b67d7dafbca02346aeea
2017-04-13 21:32:05 -07:00
Sam Saccone
32f9483eb5 Merge "Add the ability to save off the CSV to a file." am: c5b172b1eb am: 4b22df2b4e
am: 340b9610e2

Change-Id: I7dabc9900ee039aca124b6efc2cbc50858d5d895
2017-04-13 15:00:18 +00:00
Sam Saccone
03aaa7e2d8 Add the ability to save off the CSV to a file.
While we are generating the html file also expose a way for a user to
save off a csv file. This prevents having to run the tool twice.

* Add a new flag called csvpath that takes a file path for the csv dump.
* Use csv.writer to simply print formats.
* Modify the dump_csv and count_severity to take a csv writer.

Test: run with a build.log file, compare before and after html/csv dumps.
Change-Id: I27ed730625a352fdc3fb2928433eb5e8b2ea229c
2017-04-12 18:58:53 -07:00
Tao Bao
f859165bd1 Merge "releasetools: Add validate_target_files.py." am: 6cb177f15c am: c23d275d02
am: d5244a593f

Change-Id: I2f4e4473ffbe1e1802ec31a0e95d1247634ff2ea
2017-04-06 21:07:01 +00:00
Tao Bao
6cb177f15c Merge "releasetools: Add validate_target_files.py." 2017-04-06 20:50:09 +00:00
Chih-hung Hsieh
badde95384 Merge "Encode special characters in email addresses." am: c4a7ed3530 am: 1124a1bd01
am: f754ef73a5

Change-Id: I1eb192492a53263182eddce2f1ab86a015400cce
2017-04-06 00:53:15 +00:00
Chih-hung Hsieh
c4a7ed3530 Merge "Encode special characters in email addresses." 2017-04-06 00:40:51 +00:00
Chih-Hung Hsieh
bee0dec6c4 Encode special characters in email addresses.
Now accept email addresses with '+' character.

Bug: 33166666
Test: checkowners.py -v -c `find . -name OWNERS`
Change-Id: I7792551e6be74e942906a75344233439d4bfeca3
2017-04-05 14:16:49 -07:00
Tao Bao
afaa0a638b releasetools: Add validate_target_files.py.
Bug: 35408446
Test: validate_target_files.py on existing target_files zips.
Change-Id: I140ef86533eee5adb93c2546510fdd7e9ce4e81a
2017-04-05 09:04:13 -07:00
Alex Deymo
4775eb67db Merge "Don't include the dev-key on IOT products." am: 41febee0cc am: b20d1d7a74
am: b83d6d50ab

Change-Id: I09c22951828599f7d0b3f290d0297bc06455528c
2017-04-04 00:57:10 +00:00
Alex Deymo
cec578c916 Don't include the dev-key on IOT products.
This patch updates the logic to use the PRODUCT_IOT variable instead of
BRILLO.

Bug: 36702887
Test: `make`; Image doesn't have the dev key.
Change-Id: I1751e97d8cdfeba83c7e4720a017a5f4dcfd49da
2017-04-03 15:04:35 -07:00
Elliott Hughes
b2fedccb26 Merge "fsconfig: Use defined macro for masking" am: 35eaf52ee2 am: d83bb4193a
am: ced2e48998

Change-Id: I9819032f3d0473cb307e96b96331cd753b871e69
2017-04-03 21:35:42 +00:00
doheon1.lee
5cd3bcab86 fsconfig: Use defined macro for masking
Mask opeation with CAP_ prefixed name can be wrapped with
CAP_MASK_LONG macro.

Change-Id: Ide31ad62fac0d28e7a02e45059fe6d2461da7e27
2017-04-03 15:17:06 +09:00
Jaekyun Seok
39f97ae22a Enforce RROs for all the build-time ROs
This CL is to generate every static RRO package for its target package
automatically at build-time.

BOARD_ENFORCE_RRO build variable is added to specify whether enforcing
RRO is required or not.

BOARD_ENFORCE_RRO_EXEMPT_SOURCES build variable is added to specify
the module list of which item should be exempt from enforcing RRO.

Test: tested on bullhead and sailfish
Bug: 34097942
Change-Id: I455b2ce34e66c57a540c299b5e378b7c4e78d5b8
(cherry picked from commit 3070610b72)
2017-03-30 10:43:38 +09:00
Makoto Onuki
517d32d4da Add USE_FILESLIST_GO to switch to fileslist.go
fileslist.go is still disabled by default.  To enable, use:
USE_FILESLIST_GO=true m -j

Bug: 36274890
Test: Manual, with m -j

Change-Id: I722e17fa7fb6ba91e2b2c44cd7524d974cbe4a56
(cherry picked from commit 9fe97e231b)
2017-03-28 13:19:03 -07:00
Makoto Onuki
c99ba3fb96 Merge "Add USE_FILESLIST_GO to switch to fileslist.go" am: d625ef2894 am: 0a069a74c2
am: 6a05fb3d33

Change-Id: I9d49341e4e9145151a728160944276b43f83d76e
2017-03-28 16:58:23 +00:00
Makoto Onuki
d625ef2894 Merge "Add USE_FILESLIST_GO to switch to fileslist.go" 2017-03-28 16:41:05 +00:00
Mark Salyzyn
f8b86e2651 Merge "fs_config_generate: README" am: 058e2f7b19 am: ccf3fcaba0
am: 4f254445e7

Change-Id: I334f9f5183ae93ba2db01bd82d10528967e024c0
2017-03-27 23:38:52 +00:00
Mark Salyzyn
058e2f7b19 Merge "fs_config_generate: README" 2017-03-27 23:29:02 +00:00
Scott Main
803e7d0e86 Delete all of templates-sdk/.
These files are now located in external/doclava/res/assets/
Related CLs:
https://googleplex-android-review.git.corp.google.com/#/c/1524290/
https://googleplex-android-review.git.corp.google.com/#/c/1928220/

Change-Id: Ia7dba884897c5b69243055190b6283537d35717f
2017-03-27 17:23:36 +00:00
Tianjie Xu
7224a8ff83 Merge "Fix unittest for blockimgdiff" am: 36bcb78eb9 am: 0e6a7f1838
am: 1b7e203272

Change-Id: Id6db83ef492aec3aba8ef8bf4c00658963ffaef3
2017-03-26 05:35:59 +00:00
Tianjie Xu
a011dec9b2 Fix unittest for blockimgdiff
The test broke due to api change in constructor of Transfer.

Bug: 36456381
Test: blockimgdiff test passes
Change-Id: Ic73b66f8fb6141ff6f50c113b0d8cc55d34e0b5d
2017-03-24 16:28:16 -07:00
Makoto Onuki
9fe97e231b Add USE_FILESLIST_GO to switch to fileslist.go
fileslist.go is still disabled by default.  To enable, use:
USE_FILESLIST_GO=true m -j

Bug: 36274890
Test: Manual, with m -j

Change-Id: I722e17fa7fb6ba91e2b2c44cd7524d974cbe4a56
2017-03-24 16:12:16 -07:00
Mark Salyzyn
87ba0144f8 fs_config_generate: README
Test: none
Bug: 36071012
Change-Id: If30461fb9c522f705874fdfdc2b7cd54a5c3732e
2017-03-24 08:57:27 -07:00
Tao Bao
15d816bfc5 Merge "Add a missing default value for OPTIONS.extracted_input." am: 2c280271ac am: 3918b311f5
am: 5114910040

Change-Id: Ia52560941ffe86d08ffeb20728e6aea6afff583d
2017-03-22 07:43:10 +00:00
Tao Bao
5f8ff9319b Add a missing default value for OPTIONS.extracted_input.
Test: ota_from_target_files.py without --extracted_input_target_files.
Change-Id: Ieaba43fa557b544b24f3eeccd68ac289d1fe3d31
2017-03-21 22:36:20 -07:00
Dan Willemsen
ea8f68ed6c Merge "Speed up non-AB ota_from_target_files in the build system" am: 47ba4239c8 am: 02e29c5b48
am: 62fb821faf

Change-Id: Idb3a0c991efe712865db7343861212a74ea56b0a
2017-03-22 02:26:50 +00:00
Dan Willemsen
cea5cd210e Speed up non-AB ota_from_target_files in the build system
We've just created the target files package and still have the extracted
directory, so pass that to ota_from_target_files so that it doesn't have
to re-extract what we just compressed.

This saves a little bit of time -- for bullhead-userdebug on internal
master, this brings the time to build an OTA from ~340s -> ~310s. Much
of the time is still spent generating and signing the OTA.

Test: lunch bullhead-userdebug; m -j otapackage
Test: bullhead-ota-*.zip is identical before/after
Change-Id: Ib51cf6b02123e9c49280e9957c6590645470a52c
2017-03-21 15:34:27 -07:00
Tao Bao
9deb61d782 Merge "releasetools: Remove a dead line." am: 83a6b953c9 am: 9a7201408f
am: 84161e9820

Change-Id: I8712c79112953830b0db4ff034caf552fbe3a921
2017-03-16 22:44:10 +00:00
Tao Bao
2f80e83e98 releasetools: Remove a dead line.
system_root_image has been dead since commit
2ce63edab7.

Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Change-Id: Id791747b7313923b82279a21d264a998455a92d4
2017-03-16 11:59:24 -07:00
Tianjie Xu
5aa9a7b6b6 Merge "Provide more info for bsdiff/imgdiff exceptions." am: 53ba93d615 am: e742de1c4a
am: a9453d915d

Change-Id: I8eadeebba3a5e2c9778464ee7d6e97abe202a078
2017-03-16 18:57:25 +00:00
Tianjie Xu
b59c17fc8f Provide more info for bsdiff/imgdiff exceptions.
When bsdiff/imgdiff fails, dump the output along with the src/tgt
filenames and block ranges. Also, exit the script gracefully instead
of raising an exception about iterating over 'NoneType'.

Bug: 31381530
Test: Package generates successfully for angler, and error outputs
      correctly with error injection.

Change-Id: I06a2dfe545fbdff7043de05fee34b378453a9291
2017-03-14 18:51:21 -07:00
Jaekyun Seok
3070610b72 Enforce RROs for all the build-time ROs
This CL is to generate every static RRO package for its target package
automatically at build-time.

BOARD_ENFORCE_RRO build variable is added to specify whether enforcing
RRO is required or not.

BOARD_ENFORCE_RRO_EXEMPT_SOURCES build variable is added to specify
the module list of which item should be exempt from enforcing RRO.

Test: tested on bullhead and sailfish
Bug: 34097942
Change-Id: I455b2ce34e66c57a540c299b5e378b7c4e78d5b8
2017-03-14 23:55:50 +09:00
Tao Bao
aa769126c2 Merge "releasetools: Fix the diff_worker progress report." am: e5b2ff2a97 am: 81c01a9adc
am: e2ba14c9c5

Change-Id: Ifcdbab431082648476b75b0083b7eb4f6aab5f3a
2017-03-13 19:27:46 +00:00
Tao Bao
e5b2ff2a97 Merge "releasetools: Fix the diff_worker progress report." 2017-03-13 19:12:26 +00:00
Tianjie Xu
99fd91eacb Merge "Switch command to subprocess in build_image.py" am: 36e612b753 am: 82e63d3037
am: 44bc991757

Change-Id: I17c40dc6daa30097bf376617cb91fff65f87fa45
2017-03-13 18:34:13 +00:00
Tianjie Xu
36e612b753 Merge "Switch command to subprocess in build_image.py" 2017-03-13 18:23:08 +00:00
Tao Bao
33635b1f32 releasetools: Fix the diff_worker progress report.
Test: Observe the progress update during an incremental generation.
Change-Id: Ib0629a0fd9f925076fd20a040345c4f169133c30
2017-03-12 13:02:51 -07:00
Tianjie Xu
e3ad41bb2f Switch command to subprocess in build_image.py
Bug: 36012162
Test: image builds successfully for bullhead and sailfish

Change-Id: Iaa83034b39d392ff8a2154c7b32b21bf33ef552f
2017-03-10 15:16:24 -08:00
Tao Bao
86839ca639 Merge "releasetools: Drop the support for BBOTA v1 and v2." am: c3e3406125 am: a592924505
am: 03c64e425c

Change-Id: I634bc6d8cbfa4940b1d107248810e50beb8a9390
2017-03-10 19:39:19 +00:00
Tao Bao
c3e3406125 Merge "releasetools: Drop the support for BBOTA v1 and v2." 2017-03-10 19:25:06 +00:00
Tao Bao
1da7162cd0 Merge "releasetools: Fix the missing 'post-build' in full OTAs." am: 1e8364a8be am: b538502353
am: d10f5a71a3

Change-Id: If23f78f8d80b6a9e41c511d545dcf45dbcf8c375
2017-03-10 00:45:07 +00:00
Tao Bao
39f3eaf221 releasetools: Fix the missing 'post-build' in full OTAs.
The line was unintentionally removed in commit 7f804ba.

Test: ota_from_target_files.py generates a full OTA. Check the package
      metadata.

Change-Id: Icae88e2a9bb2bfc450a3d0d7ab524d6a6eac9df5
2017-03-09 15:01:11 -08:00
Tao Bao
85bfe4ac8c Merge "releasetools: Remove dead functions in EdifyGenerator." am: c87b38f547 am: debedc3343
am: dc880f5122

Change-Id: I23189ca7272094c65dd3d8aa7ed28ad8b5a2606d
2017-03-09 20:36:48 +00:00
Tao Bao
8fad03e771 releasetools: Drop the support for BBOTA v1 and v2.
BBOTA v1 and v2 (introduced in L and L MR1 respectively) don't support
resumable OTA. We shouldn't generate packages using v1/v2 at the risk of
bricking devices.

BBOTA v3 (since M) and v4 (since N) both support resumable OTAs. BBOTA
v4 additionally supports using FEC to possibly recover a corrupted
image.

Bug: 33694730
Test: Generate full and incremental OTAs w/ and w/o the CL. They should
      give identical packages (in v4).
Change-Id: Ib89d9cd63ba08e8e9aa4131bed18876b89d244c0
2017-03-09 12:25:46 -08:00
Tao Bao
f388104eaa releasetools: Remove dead functions in EdifyGenerator.
Remove the following functions that are needed for file-based OTAs only:
 - SetPermissions()
 - SetPermissionsRecursive()
 - MakeSymlinks()
 - DeleteFiles()
 - DeleteFilesIfNotMatching()
 - RenameFiles()
 - SkipNextActionIfTargetExists()

Bug: 35853185
Test: Verified there's no reference to these functions.
Change-Id: Iff24a9d705476211effaef28eed2a147fa5fcbce
2017-03-08 16:37:07 -08:00
Tao Bao
afb8c5255c Merge "releasetools: Drop the support for file-based OTA generation." am: 9bca43858f am: aac8098d61
am: 3b848d039d

Change-Id: I9b6cb2b6502f9c9997cf0730710b9585cab4e0a5
2017-03-08 21:13:20 +00:00
Tao Bao
9bca43858f Merge "releasetools: Drop the support for file-based OTA generation." 2017-03-08 21:00:03 +00:00
Dan Willemsen
266b2c2f55 Merge changes Ia5ce6870,I155654cd am: c619f99e2f am: 6ffe13f8e7
am: 53d64224ab

Change-Id: Id25a30a8616d50a661fdd978e68e485e6ed81633
2017-03-08 20:57:48 +00:00
Dan Willemsen
c619f99e2f Merge changes Ia5ce6870,I155654cd
* changes:
  Zip after add_img_to_target_files
  Allow add_img_to_target_files to work on a directory
2017-03-08 20:45:56 +00:00
Tao Bao
25643f313a Merge "releasetools: Reduce memory footprint for BBOTA generation." am: 3b1f260205 am: cf82616047
am: a56e7c81e2

Change-Id: Ie0717b0b71b791c1dca826735d31c03a7f4728b1
2017-03-08 17:55:34 +00:00
Tao Bao
3b1f260205 Merge "releasetools: Reduce memory footprint for BBOTA generation." 2017-03-08 17:43:34 +00:00
Dan Willemsen
2ee00d5f66 Allow add_img_to_target_files to work on a directory
In addition to the current behavior of add_img_to_target_files working
on an existing zip file, allow passing in a directory where the target
files have already been extracted. When in this mode, it writes the
images out to that directory instead of the zip file.

This allows us to call add_img_to_target_files on the temp directory
used during the build to create the target files package, saving the
time and space of unzipping what we just zipped. This also allows us to
use the parallel soong_zip, which compresses the images much faster.

Test: aosp_marlin target_files zip is the same before/after this change
Test: marlin target_files zip is the same before/after this change
Test: bullhead target_files zip is the same before/after this change
Change-Id: I155654cdc7ad7754ba4ef76ec69c31f504a58198
2017-03-07 14:37:52 -08:00
Tao Bao
457cbf6a8a releasetools: Drop the support for file-based OTA generation.
We have stopped shipping devices with file-based OTAs, and are not
actively maintaining the support. Devices using file-based OTAs
should be moved to block-based, if not A/B OTAs.

We will also need to clean up EdifyGenerator class, which will be
handled in follow-up CLs.

Bug: 35853185
Test: Generate full and incremental OTAs w/ and w/o the CL, and they
      give identical packages.
Test: Not specifying --block also generates block-base OTAs.
Change-Id: I3b0fc8ce5600e109f3251fe41f655534aaa298c7
2017-03-07 00:17:58 -08:00
Tao Bao
b357b83936 Merge "releasetools: Remove the obsolete path in GetImage()." am: 7dc54e9761 am: 438753de24
am: 657ed318d8

Change-Id: Iac897cdeb013ae72b9e0930b2234881280b142dc
2017-03-07 06:30:58 +00:00
Tao Bao
7dc54e9761 Merge "releasetools: Remove the obsolete path in GetImage()." 2017-03-07 06:20:04 +00:00
Tao Bao
958c691a0e Merge "releasetools: Fix two minor issues with _LoadOemDicts()." am: 0c9e8d87c4 am: cedbcb634f
am: e319d34b21

Change-Id: Ie653cbb3bc5bf4e746a28d6e9620f3a55a9f784f
2017-03-07 02:45:34 +00:00
Tao Bao
7e0f160f30 releasetools: Remove the obsolete path in GetImage().
We must have created the images (system.img, system.map etc) prior to
calling ota_from_target_files.py (added by commit
2e0d8fcf08, into Lollipop).

Remove the obsolete suppport for handling "old" target_files zips that
don't have such images. This simplies the logic for BuildSystem() and
BuildVendor(), which now would only be called by
add_img_to_target_files.py itself.

Test: Generating full and incremental OTAs give the same results, w/ and
      w/o this CL.
Change-Id: I0ea243d09d3378249d7982701ae4ec339b04b7b8
2017-03-06 16:22:44 -08:00
Tao Bao
ebce697429 releasetools: Fix two minor issues with _LoadOemDicts().
The first one in WriteVerifyPackage() is mismatching function parameters
that can be trivially fixed.

The other one is in WriteABOTAPackageWithBrilloScript(), where we don't
have edify script instance.

Test: `pylint --rcfile=pylintrc ota_from_target_files.py`.
Change-Id: Ie238ef5b296dfe9e725b61376992447b662d2376
2017-03-06 10:29:13 -08:00
Tao Bao
da12392738 Merge "Revert "Revert "releasetools: Use pattern filter when unzipping.""" am: 8e02284362 am: 85f8de467a
am: 654e4f4c77

Change-Id: I75e8a5111c321b8ecda183f758060a4b59bd1e62
2017-03-06 16:57:37 +00:00
Tao Bao
183e56e83d releasetools: Reduce memory footprint for BBOTA generation.
The major issue with the existing implementation is unnecessarily
holding too much data in memory, such as HashBlocks() which first reads
in *all* the data to a list before hashing. We can leverage generator
functions to stream such operations.

This CL makes the following changes to reduce the peak memory use.
 - Adding RangeSha1() and WriteRangeDataToFd() to Image classes. These
   functions perform the operations on-the-fly.
 - Caching the computed SHA-1 values for a Transfer instance.

As a result, this CL reduces the peak memory use by ~80% (e.g. reducing
from 5.85GB to 1.16GB for the same incremental, as shown by "Maximum
resident set size" from `/usr/bin/time -v`). It also effectively
improves the (package generation) performance by ~30%.

Bug: 35768998
Bug: 32312123
Test: Generating the same incremental w/ and w/o the CL give identical
      output packages.
Change-Id: Ia5c6314b41da73dd6fe1dbe2ca81bbd89b517cec
2017-03-06 08:46:30 -08:00
Tao Bao
6b0b2f9db0 Revert "Revert "releasetools: Use pattern filter when unzipping.""
This reverts commit a7316ce094.

This CL differs from the original CL by not unzipping RADIO/*. This is
because: a) AOSP targets don't have RADIO/ entries in the TF.zip; b)
we're not using the unzipped RADIO files (but reading them from the zip
files directly) - checked all the device-specific releasetools for
angler, bullhead, ryu, shamu, volantis, fugu, marlin and sailfish.

Test: `m dist` with AOSP targets (tested fugu and bullhead).
Change-Id: I4d0c67214ddd6202fc27c71bb79f52b5f4d40c64
2017-03-05 13:11:42 -08:00
Tao Bao
f9566f38e2 Merge "Revert "releasetools: Use pattern filter when unzipping."" am: 0c3e31c69b am: 6c98ad6c4e
am: a4170833d1

Change-Id: I529f9eb380fc06b08d02be80635f9ac9a9317654
2017-03-05 17:56:19 +00:00
Tao Bao
0c3e31c69b Merge "Revert "releasetools: Use pattern filter when unzipping."" 2017-03-05 17:44:58 +00:00
Tao Bao
a7316ce094 Revert "releasetools: Use pattern filter when unzipping."
This reverts commit aa3a04f19d.

Reason for revert: Some AOSP targets don't include RADIO/ in the TF.zip. We may possibly skip unzipping RADIO/, or by always creating a dummy RADIO folder in TF.zip. Revert this CL for now.

Change-Id: I8e90d322706a4fe82129bdfab5ffa1eab989c648
2017-03-05 17:43:33 +00:00
Tao Bao
ee5f10acab Merge "Change the default parameter values in common.MakeTempFile()." am: cc63bc3a91 am: 2b7a043a06
am: a21cff6280

Change-Id: I048ca3c7b4bcd5fa24cb1b7b903ebb60cc7fac47
2017-03-05 17:12:45 +00:00
Tao Bao
220bdee570 Merge "releasetools: Use pattern filter when unzipping." am: 76f038a63c am: e2309ba525
am: adf0881dc5

Change-Id: Icb68cb9205f9da4fd8fdb8e334c819d69a5abb6c
2017-03-05 17:01:44 +00:00
Tao Bao
cc63bc3a91 Merge "Change the default parameter values in common.MakeTempFile()." 2017-03-05 17:00:38 +00:00
Tao Bao
76f038a63c Merge "releasetools: Use pattern filter when unzipping." 2017-03-05 16:49:21 +00:00
TreeHugger Robot
039af59636 Merge "fallout of cleaning-up libutils includes" 2017-03-04 00:52:45 +00:00
Tao Bao
aa3a04f19d releasetools: Use pattern filter when unzipping.
When building BBOTAs, it only needs *some* unzipped entries in the given
target_files zip(s). In particular, it needs 'IMAGES/*', 'META/*',
'RADIO/*'. (It also reads 'SYSTEM/build.prop' and 'OTA/bin/updater', but
directly from the zip file.)

This CL specifies the entries to unzip. It saves the I/O cost, as well as
the temporary storage.

Test: ota_from_target_files.py gives the same package w/ and w/o the CL.
Test: check_target_files_signatures.py still works.
Change-Id: I728428aa0e138879e49f9efbdb46a85892fc7038
2017-03-03 16:38:17 -08:00
Sen Jiang
67b7aaab7d Merge "Remove DBus support." am: cf858d873b am: efcda5173d
am: 3d6b408aea

Change-Id: I183f58049ef63ce8f6bfbd67554199878b7a69cb
2017-03-03 21:02:37 +00:00
Mathias Agopian
319524ba33 fallout of cleaning-up libutils includes
Test: run & compile
Bug: n/a
Change-Id: Ia213c25f166805a203d472761d024fa6b6efdc24
2017-03-03 12:37:34 -08:00
Sen Jiang
d1f397e8b4 Remove DBus support.
external/dbus and external/dbus-binding-generator was removed from
the manifest.

Bug: 31602715
Test: make checkbuild
Change-Id: Iea0277720acad8ac0fa630b8745f90fb3c3b5f00
2017-03-02 18:00:21 -08:00
Tianjie Xu
d9880a0246 Merge "Do not read the block device when generating the care map" 2017-03-02 21:36:12 +00:00
Tao Bao
b116368c3f Merge "releasetools: Drop the support for fstab_version 1." am: afc0ea14d8 am: 39809e3800
am: 126d7b178d

Change-Id: Ibe90e941c85d0b56abf39b2d48608c736c6ab1e5
2017-03-02 19:26:37 +00:00
Treehugger Robot
afc0ea14d8 Merge "releasetools: Drop the support for fstab_version 1." 2017-03-02 19:19:09 +00:00
Tianjie Xu
955629b4aa Do not read the block device when generating the care map
This remove the fstab dependency when building the OTA package for
marlin/sailfish.

Bug: 35811655
Test: OTA package builds successfully for sailfish.
Change-Id: If223d11dddca396c47262042c576f9e7d0cb5b33
(cherry picked from commit 7d051adc3b)
2017-03-02 11:07:12 -08:00
TreeHugger Robot
e627078740 Merge "Do not read the block device when generating the care map" 2017-03-02 19:04:15 +00:00
Tao Bao
d1de6f326e releasetools: Drop the support for fstab_version 1.
fstab_version is defined by RECOVERY_FSTAB_VERSION in
bootable/recovery. We have moved to fstab_version 2 since commit
f35d1cef7c19db975a1295e8c23c7fb8bd2489f9 (landed into JB MR2).

Drop the support for fstab_version 1, since we won't run the latest OTA
script over a JB target_files zip.

Test: No impact on building full/incremental OTAs.
Change-Id: Ia87c4e7da6c5e71ce0908fca2e4f1ad1c06ba869
2017-03-01 16:56:45 -08:00
Tao Bao
d2de5e9ba8 Merge "releasetools: Add support for --override_timestamp." am: e889893776 am: 19e700aaf9
am: 44a544bc59

Change-Id: I92afbb0d8a4ee74c8b5b9ce0bce359bd07043550
2017-03-02 00:21:40 +00:00
Tao Bao
e889893776 Merge "releasetools: Add support for --override_timestamp." 2017-03-02 00:00:59 +00:00
Tianjie Xu
7d051adc3b Do not read the block device when generating the care map
This remove the fstab dependency when building the OTA package for
marlin/sailfish.

Bug: 35811655
Test: OTA package builds successfully for sailfish.
Change-Id: If223d11dddca396c47262042c576f9e7d0cb5b33
2017-03-01 11:55:02 -08:00
Tao Bao
fb8d22de03 Merge "releasetools: Merge two identical conditional branches." am: 59978ab6e3 am: 514e6e4488
am: 81d1ad1267

Change-Id: Ia106d318e77b954eadcfdcd80e13e85f89e43187
2017-03-01 18:08:17 +00:00
Tao Bao
9beea2a4a3 releasetools: Merge two identical conditional branches.
The merged two branches have become identical since commit
fc3422ad36 (landed into Nougat).

Test: Get identical incremental packages w/ and w/o the CL.
Change-Id: Id1183f8ed83f684a0dac1a4af87b6e075b08aabc
2017-02-28 20:20:27 -08:00
Dan Willemsen
aa7e67c871 Merge changes I5e684409,I4fa35540,I24015ef0 am: 93de77745c am: d1378dec05
am: ead8af8580

Change-Id: I0608060da7307b9bd9a5bd15e601b1787c510dd4
2017-02-28 21:41:33 +00:00
Dan Willemsen
93de77745c Merge changes I5e684409,I4fa35540,I24015ef0
* changes:
  Improve module tags warnings
  Improve warnings for package-modules.mk
  Fix all_named_products, remove kati_all_products
2017-02-28 21:30:55 +00:00
Tao Bao
3e6161a3b3 releasetools: Add support for --override_timestamp.
We use the timestamps in builds to determine a downgrade, which might
not be always the truth. For examples, two builds cut from different
branches may carry timestamps in a reverse order. An incremental package
won't be able to be pushed nor applied, based on the timestamp
comparison.

We used to handle such a case with manual work, by setting the
post-timestamp to (pre-timestamp + 1) in the package metadata. This CL
automates the process by adding a new flag --override_timestamp.

Note that it doesn't change anything in the installed image, but only
affects the assertions for pushing / installing the package.

With the change in this CL:
 - If it's a downgrade without any extra flag, fail the package
   generation (we only print warnings prior to this CL);
 - If it's a downgrade with --downgrade flag, generate a downgrade
   package with forced data wipe (same as before);
 - If it's a downgrade with --override_timestamp, generate a normal
   incremental with hacked timestamp (pre-timestamp + 1) (new in this CL
   to avoid the manual change);
 - If it's not a downgrade but with any of the above two flags specified,
   fail the package generation.

Bug: 33744169
Test: Generate an incremental from builds with reversed timestamps.
Change-Id: I8b187d32708b4a7c3e20f8c6adb8f9527b73b965
2017-02-28 13:01:13 -08:00
Elliott Hughes
41232ebd5c Merge "There's no longer a limit on property names." am: dda91dbef8 am: daac92eb51
am: 319fcc7668

Change-Id: Idb24042310fa9de8f534e9988285a16b714daf10
2017-02-28 20:22:56 +00:00
Treehugger Robot
dda91dbef8 Merge "There's no longer a limit on property names." 2017-02-28 20:13:14 +00:00
Tao Bao
0e03a0b168 Merge "Remove the backwards compatibility support for misc_info.txt." am: 042d5a846f am: ba765d7c47
am: 1f4831830e

Change-Id: I61d552561842f6b4f27d19c0f3924aab5b8f41c8
2017-02-28 18:15:17 +00:00
Elliott Hughes
05c1a2a55c There's no longer a limit on property names.
Bug: http://b/33926793
Test: boots
Change-Id: I12cdae782090fb0856171e5c90a268e91ba2ae1a
2017-02-28 10:04:23 -08:00
Tao Bao
6cd54739a4 Remove the backwards compatibility support for misc_info.txt.
We introduced META/misc_info.txt to hold the misc info since Gingerbread
(commit 37974731fc). Remove the backwards
compatibility support for building pre-G TF zips.

Test: `m dist` works.
Change-Id: Ibff7aaf69cc7e460634c049d11a004f7196f8f73
2017-02-27 17:00:13 -08:00
Jaekyun Seok
4e3fcca9fd Split board-specific system properties into vendor partition
Test: building succeeded and tested on sailfish-userdebug
Bug: 35696968
Change-Id: Ia1e8013411da36583fbf0d2115c54ca5d5980e20
2017-02-27 13:35:37 +09:00
Tao Bao
7e48403012 Merge "releasetools: Fix an equality check bug when asserting stash size." am: 2af09af86b am: db5463d8dd
am: 9def0e5609

Change-Id: I16d0bb1a16e135eaf48e18f445d6a257834c23b7
2017-02-26 21:42:54 +00:00
Tao Bao
9def0e5609 Merge "releasetools: Fix an equality check bug when asserting stash size." am: 2af09af86b
am: db5463d8dd

Change-Id: I198b9be0d250cabb25c0ea9bae11b47f29856873
2017-02-26 21:39:23 +00:00
Treehugger Robot
2af09af86b Merge "releasetools: Fix an equality check bug when asserting stash size." 2017-02-26 21:31:24 +00:00
Tao Bao
e8c68a03c8 releasetools: Fix an equality check bug when asserting stash size.
Otherwise the comparison is inconsistent between ReviseStashSize() and
WriteTransfers().

Bug: 35775675
Test: Successfully generate a previously failed incremental.
Change-Id: I554a51a210bf322cb5c79e28cf85607a417b094a
2017-02-26 10:58:22 -08:00
Chih-Hung Hsieh
eb27d0ef2e Merge "Add a script to check OWNERS file syntax." am: d84721a2cc am: a5b0fec8b6
am: edef957aeb

Change-Id: I45128f06c8e90c2d7800857d9ce047bd8e827bb7
2017-02-25 01:11:08 +00:00
Chih-Hung Hsieh
edef957aeb Merge "Add a script to check OWNERS file syntax." am: d84721a2cc
am: a5b0fec8b6

Change-Id: I9cb8eb3737daa56621adcf237bcbb43b354497ef
2017-02-25 01:09:09 +00:00
Treehugger Robot
d84721a2cc Merge "Add a script to check OWNERS file syntax." 2017-02-25 01:00:18 +00:00
Dan Willemsen
594c3fc3d7 Fix all_named_products, remove kati_all_products
all_named_products was giving the file path instead of product name for
the name:path format of PRODUCT_MAKEFILES.

kati_all_products has been replaced with multiproduct_kati

Test: get_build_var all_named_products
Test: multiproduct_kati
Change-Id: I24015ef0778ac7cd45201aa55c1737b0553f09fe
2017-02-24 15:49:46 -08:00
Chih-Hung Hsieh
adaed14ff3 Add a script to check OWNERS file syntax.
* -v option to print found files and email addresses.
* -c option to check found email addresses.

Bug: 33166666
Test: ./checkowners.py -v -c `find . -name OWNERS`
Change-Id: I32f23f19e904055e421ddec713536c8a5c970af4
2017-02-24 15:28:29 -08:00
Tao Bao
4f30f7cfa1 Revert "Revert "releasetools: Allow using futility-host to sign old TF.zip.""
This reverts commit d6145b6a16.

The cause of the earlier breakage has been identified and fixed by
commit 4b577413ff.

Bug: 35467608
Test: lunch ryu-userdebug; m dist otatools-package
2017-02-23 11:06:42 -08:00
Dan Willemsen
3b06556f3a Merge "Wait for children to exit in makeparallel" am: 9b1a185029 am: c08e57ce4d
am: d04de52560

Change-Id: Ia6c4cb26f2d5d8c485f1139ac128e7482b6235a0
2017-02-22 03:38:56 +00:00
Dan Willemsen
d04de52560 Merge "Wait for children to exit in makeparallel" am: 9b1a185029
am: c08e57ce4d

Change-Id: I2bc1fe1b9834ab56575db0c2e6fa96b968550382
2017-02-22 03:33:27 +00:00
Dan Willemsen
f0e2986cbf Wait for children to exit in makeparallel
This is a partial revert to the last makeparallel change, since now we
were being killed by the signal before waiting for our child to exit. So
instead of not installing the handlers, only pass the signal along if
it's a SIGTERM.

Bug: 35214134
Test: Ensure that we're still only getting one signal for SIGINT
Test: Ctrl-C, ensure that all the soong_ui lines are before the make
      error line.
Change-Id: I26fff9483a3abfd79ceb5a9ea47e3f7572d9e923
2017-02-21 17:53:35 -08:00
Dan Willemsen
06faf20877 Merge "Prevent duplicate signals from makeparallel" am: 70aac88a61 am: 52ac801739
am: 06fbe5bec5

Change-Id: I1b162f02dad89e781662ad3a1e9b3f5d654760d0
2017-02-21 23:31:38 +00:00
Dan Willemsen
06fbe5bec5 Merge "Prevent duplicate signals from makeparallel" am: 70aac88a61
am: 52ac801739

Change-Id: I8b779c85df3864115f5894c0cdd597c790417386
2017-02-21 23:29:40 +00:00
Treehugger Robot
70aac88a61 Merge "Prevent duplicate signals from makeparallel" 2017-02-21 23:22:32 +00:00
Dan Willemsen
c6a900bee1 Prevent duplicate signals from makeparallel
As part of the soong_ui effort, we noticed that we'd sometimes see
multiple SIGINTs from a single ctrl-c from a user. ctrl-c sends a SIGINT
to the entire process group, so make, makeparallel, soong_ui, and all of
its children would get a signal. Since makeparallel was passing it along
to it's child, soong_ui would get two signals.

So instead, follow what Make does and only pass along SIGTERM. Assume
that all other signals went to the entire process group.

Bug: 35214134
Test: Send SIGINT to process group, check in makeparallel's child for
more than one signal received.
Change-Id: I5b2a77ad0fcebbaa5087439948e71bf3b541061a
2017-02-21 13:50:12 -08:00
Tao Bao
4412ff48b4 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac am: 983c979c25 am: bd6d03ab18 am: e64f52ebcd am: 55f185e0a4
am: 4f0a147b69

Change-Id: Id274c51c31d9a05248aa1a49bc776b7fec43010e
2017-02-21 20:40:55 +00:00
Tao Bao
4f0a147b69 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac am: 983c979c25 am: bd6d03ab18 am: e64f52ebcd
am: 55f185e0a4

Change-Id: Ic47af33906d2ead19cc3b6346f1aa55e5de5952d
2017-02-21 20:39:56 +00:00
Tao Bao
55f185e0a4 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac am: 983c979c25 am: bd6d03ab18
am: e64f52ebcd

Change-Id: I5ccd6ba60e5e9eb3e0875e99630db583a178a569
2017-02-21 20:37:24 +00:00
Tao Bao
e64f52ebcd Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac am: 983c979c25
am: bd6d03ab18

Change-Id: I488b164ae8c3a93858a293e2a538ff6b8742736c
2017-02-21 20:35:25 +00:00
Tao Bao
bd6d03ab18 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac
am: 983c979c25

Change-Id: I938ee86611ea8c452c07324e2f3c6a9b0180cb7e
2017-02-21 20:33:25 +00:00
Tao Bao
983c979c25 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev
am: b0df1ef7ac

Change-Id: I3924b44e75aa0f8f18129212dd80fd843f6a6a47
2017-02-21 20:31:25 +00:00
Tao Bao
d6145b6a16 Revert "releasetools: Allow using futility-host to sign old TF.zip."
Broke ryu targets.

This reverts commit 31a0de00a4.

Change-Id: I3ffcdc604a1cb92c86c574b5cd358d0148430d55
2017-02-21 20:27:27 +00:00
Alain Vongsouvanh
0a5773ba67 Merge "releasetools: allow for multiple OEM property values." am: 9a3049d681 am: b98cde50aa
am: 5c9f21da04

Change-Id: Icd07d36155b5d835acba42f938de114701769558
2017-02-21 18:34:21 +00:00
Tao Bao
ea88a3f3fa releasetools: Allow using futility-host to sign old TF.zip. am: 31a0de00a4 am: a9bdf8716f am: ae2f5abf06
am: 79d719b313

Change-Id: Ib858be8c430acf21d6aa175481ee68bb3e477caa
2017-02-21 17:50:18 +00:00
Tao Bao
79d719b313 releasetools: Allow using futility-host to sign old TF.zip. am: 31a0de00a4 am: a9bdf8716f
am: ae2f5abf06

Change-Id: I514c69dca1aede0fe047682dab9b11e7263d7634
2017-02-21 17:48:16 +00:00
Tao Bao
ae2f5abf06 releasetools: Allow using futility-host to sign old TF.zip. am: 31a0de00a4
am: a9bdf8716f

Change-Id: I411b26baf0cd7d74ec6c8c80a12f0499d017a360
2017-02-21 17:46:45 +00:00
Tao Bao
a9bdf8716f releasetools: Allow using futility-host to sign old TF.zip.
am: 31a0de00a4

Change-Id: I46041e6e5e37c451f8f23dca773850c025aa18b2
2017-02-21 17:45:15 +00:00
Alain Vongsouvanh
7f804ba71f releasetools: allow for multiple OEM property values.
This CL changes the --oem_settings flag to allow a comma seperated list of
property files. All property values will be used when asserting properties such
as ro.product.name.

For example, if two property files are provided with ro.product.name values of
"sprout" and "sprout_a", the resulting otapackage will check that the device's
ro.product.name property matches at least one of them.

Bug: 34191373
Test: manual

Change-Id: I954673511be8f0929982235cc9cbfbd85a9ee1f4
2017-02-21 16:09:34 +00:00
Tao Bao
b2b5b5933f Merge "releasetools: Allow using futility-host to sign old TF.zip." am: 48a3ccbf1b am: aa778c7af4
am: 45d092498a

Change-Id: Iffb49096716087c9da0b2ec829ec4fdee75c8072
2017-02-18 23:03:26 +00:00
Tao Bao
31a0de00a4 releasetools: Allow using futility-host to sign old TF.zip.
Commit e98fb7a8d3 switched to using
futility-host instead of the prebuilt futility. This CL adds support to
handle signing old TF.zip that still says "futility=prebuilt/..." in
META/misc_info.txt.

Bug: 35467608
Test: Generate otatools.zip and sign an old ryu TF.zip.
Change-Id: I48a9cc918c7afce361e1ec9bc4f85f74fa92566e
(cherry picked from commit 4f104d1b94)
2017-02-17 23:36:00 -08:00
Tao Bao
4f104d1b94 releasetools: Allow using futility-host to sign old TF.zip.
Commit e98fb7a8d3 switched to using
futility-host instead of the prebuilt futility. This CL adds support to
handle signing old TF.zip that still says "futility=prebuilt/..." in
META/misc_info.txt.

Bug: 35467608
Test: Generate otatools.zip and sign an old ryu TF.zip.
Change-Id: I48a9cc918c7afce361e1ec9bc4f85f74fa92566e
2017-02-17 23:27:44 -08:00
Narayan Kamath
c85327b679 Merge "zipalign: remove support for adding members from gzip sources." am: 96a4bdd9a6 am: 080f68acf5 am: 8778583000
am: 6e5855f9ed

Change-Id: I7fec7afe032fd8b28ede1b64bb125f72370d98d8
2017-02-17 14:29:52 +00:00
Narayan Kamath
b46507ff9e zipalign: remove support for adding members from gzip sources.
This was vestigial code and never actually called anywhere.

Test: make
Bug: 35246701
Change-Id: I9297ac5fa030d369946857414155450635b65410
2017-02-16 10:53:09 +00:00
Tao Bao
6f9f65d195 Merge "releasetools: Fix a bug in ReviseStashSize()." am: 0ee0fc5898 am: f158f2d12b am: d88725b457
am: 42967b19a9

Change-Id: I2a3090546e8094087874f4ff4e11b38b25a7594d
2017-02-13 22:43:07 +00:00
Tao Bao
0ee0fc5898 Merge "releasetools: Fix a bug in ReviseStashSize()." 2017-02-13 22:29:40 +00:00
Alex Deymo
613f408e7f Merge "fat16copy: Allow to copy files to an existing directory." am: 34fd8a168a am: bf6b5e8452 am: f8b5a14884
am: 25fa2d387e

Change-Id: I8fd13023b8960298276dd9ee513c766373111e34
2017-02-13 21:04:57 +00:00
Tao Bao
3c5a16d49c releasetools: Fix a bug in ReviseStashSize().
We check the needed stash size in ReviseStashSize(), and may not
generate a stash command if it would exceed the max allowed size. This
CL fixes a bug when skipping a stash operation: we shouldn't update the
'stashes' map if a stash command won't be generated.

Bug: 35313668
Test: Successfully generate the package that was failing due to the bug.
Change-Id: If0a3a5fadda9b4a4edad66a2a5826b5f978400ae
2017-02-13 12:31:42 -08:00
Alex Deymo
9a535b5188 fat16copy: Allow to copy files to an existing directory.
This patch re-uses existing directories when copying directories,
allowing to copy files to a directory previously created.

Bug: 34811400
Test: fat16copy.py fat.bin a/b/foo; fat16copy.py fat.bin c/d/foo

(cherry-picked from 0af24a27a8d0dc7e1782f36a5a9963021ff8b3f6)

Change-Id: I283184da18f3c4951d6c84aafa4185074791acd2
2017-02-13 16:00:07 +00:00