Commit graph

5228 commits

Author SHA1 Message Date
Tao Bao
e54c2150ea Merge "releasetools: Always create IMAGES/ directory." am: db7dfacb96 am: a41e00784b
am: 32c91c9c31

Change-Id: I2aeb55b993bbc90ea2945ae6c00128c0edd033ca
2017-11-20 22:57:27 +00:00
Tao Bao
32c91c9c31 Merge "releasetools: Always create IMAGES/ directory." am: db7dfacb96
am: a41e00784b

Change-Id: I91f7bc071f531f10ed8d49e3eab06ca1f1abd9b1
2017-11-20 22:52:50 +00:00
Tao Bao
a41e00784b Merge "releasetools: Always create IMAGES/ directory."
am: db7dfacb96

Change-Id: Iaede3e35fdd64dc748a98f625d390f0f38cd7e4a
2017-11-20 22:50:01 +00:00
Tao Bao
ae396d9b0c releasetools: Always create IMAGES/ directory.
AddImagesToTargetFiles() takes either a zip file, or a zip root as
input. We used to create IMAGES/ directory only when working with zip
root input. Commit 262bf3f0b5 has changed
to also stage boot / recovery images there when working with a zip file.
This CL makes sure the directory is always available under both modes.

Bug: 63456822
Test: zip -d target_files.zip IMAGES/\*;
      add_img_to_target_files.py target_files.zip
Test: sign_target_files_apks.py target_files.zip signed-target_files.zip
Change-Id: Iea91d0403cdec1b16bb93bb71d3ed06856b8f7c3
2017-11-20 12:44:18 -08:00
Tao Bao
a0e521ed16 Merge "releasetools: Fix the rebuilding of vbmeta.img." am: 2f7fab9ab9 am: 4727930bff
am: 497862c757

Change-Id: I7bb84a65a9ef6e3d4bbe98244f51819b569ecbc9
2017-11-18 00:53:15 +00:00
Tao Bao
497862c757 Merge "releasetools: Fix the rebuilding of vbmeta.img." am: 2f7fab9ab9
am: 4727930bff

Change-Id: If73915a15fa675ca635588c99bc4e6ed56e4be05
2017-11-18 00:44:28 +00:00
Tao Bao
4727930bff Merge "releasetools: Fix the rebuilding of vbmeta.img."
am: 2f7fab9ab9

Change-Id: Ie6c6536be7da1e85167c22de36832b4145255f19
2017-11-18 00:32:23 +00:00
Tao Bao
2f7fab9ab9 Merge "releasetools: Fix the rebuilding of vbmeta.img." 2017-11-18 00:22:14 +00:00
Colin Cross
a80df4238e Merge changes from topics "soong_logtags", "droiddoc_srcjars" am: f1f66974c6 am: 0b1fc880e5
am: 99dc51da38

Change-Id: I711c56f11c785ff7d2ccadbf912d96b6854c94b7
2017-11-17 21:48:22 +00:00
Colin Cross
99dc51da38 Merge changes from topics "soong_logtags", "droiddoc_srcjars" am: f1f66974c6
am: 0b1fc880e5

Change-Id: I7cc4970a5be00762c0127db0502ab03cbdfb13e7
2017-11-17 21:42:51 +00:00
Colin Cross
0b1fc880e5 Merge changes from topics "soong_logtags", "droiddoc_srcjars"
am: f1f66974c6

Change-Id: Ic47da3913b34da288c258d61540892427ddf4506
2017-11-17 21:36:51 +00:00
Colin Cross
f1f66974c6 Merge changes from topics "soong_logtags", "droiddoc_srcjars"
* changes:
  Allow converting logtags to java without merged logtags file
  Add support for srcjars to droiddoc
2017-11-17 21:32:27 +00:00
Chih-hung Hsieh
dfa1a5b3dc Merge "Recognize new clang-tidy android-* warnings." am: 315761ace1 am: dafe61c817
am: c982ae3728

Change-Id: Icbdb34606e0bcebf5733904fe20cbd50e52abecb
2017-11-17 17:19:07 +00:00
Chih-hung Hsieh
c982ae3728 Merge "Recognize new clang-tidy android-* warnings." am: 315761ace1
am: dafe61c817

Change-Id: Iebf015a5351d46ba6331908c7a945fd661d1de6e
2017-11-17 17:16:58 +00:00
Chih-hung Hsieh
dafe61c817 Merge "Recognize new clang-tidy android-* warnings."
am: 315761ace1

Change-Id: I516702a4bad35c6a4ec34f33bf42fa004400a860
2017-11-17 17:13:27 +00:00
Chih-Hung Hsieh
2cd467b36c Recognize new clang-tidy android-* warnings.
Test: warn.py build.log
Change-Id: I9bcd1fffcb567346a3d959f98821e2cf58e87ba2
2017-11-16 15:42:11 -08:00
Colin Cross
0bb782bd4e Allow converting logtags to java without merged logtags file
Most logtags files only have entries with hardcoded numbers.
Allow these files to be converted to java without needing the
merged logtags file.

Test: m checkbuild
Change-Id: I4cd38aa502fc9b8e50af92ff8296d3ad6abfb56e
2017-11-16 14:58:05 -08:00
Tomasz Wasilczyk
ca4224fc7c Merge "Silence superfluous build messages." am: 4dbdb60698 am: a62800487e
am: 463dcf37f8

Change-Id: I69b560b725724152cf8fb87cf27d9c0d5c7c2e45
2017-11-15 22:42:25 +00:00
Tomasz Wasilczyk
463dcf37f8 Merge "Silence superfluous build messages." am: 4dbdb60698
am: a62800487e

Change-Id: I1c2f39812f1be01940887aed70c630f7e9b64a0f
2017-11-15 22:39:21 +00:00
Tomasz Wasilczyk
a62800487e Merge "Silence superfluous build messages."
am: 4dbdb60698

Change-Id: I3574184c22b8fca41fd85e09ceacc2b598643c67
2017-11-15 22:35:48 +00:00
Tomasz Wasilczyk
4dbdb60698 Merge "Silence superfluous build messages." 2017-11-15 22:27:30 +00:00
Tao Bao
7cc46f8f40 Merge "releasetools: Fix the size check for AVB images." am: 905c84cada am: 029b3b6619
am: 29472b1848

Change-Id: I00ae52779aad8ef7ad4fdf874e13548776ec1ec3
2017-11-15 19:05:07 +00:00
Tao Bao
29472b1848 Merge "releasetools: Fix the size check for AVB images." am: 905c84cada
am: 029b3b6619

Change-Id: I60768378bfaa57fa47ad5a89d235a060d06e887c
2017-11-15 18:45:40 +00:00
Tomasz Wasilczyk
29ec06b7ba Silence superfluous build messages.
Bug: 69315492
Test: it builds
Change-Id: Ic02484b80fa27c53786204402930ed5f33ec7fda
2017-11-15 10:34:01 -08:00
Tao Bao
029b3b6619 Merge "releasetools: Fix the size check for AVB images."
am: 905c84cada

Change-Id: I4321bb226c8e77536fd0896980c64ac4ed680e37
2017-11-15 18:32:38 +00:00
Tao Bao
262bf3f0b5 releasetools: Fix the rebuilding of vbmeta.img.
If only vbmeta.img is missing from the target_files.zip,
'add_img_to_target_files.py -a' will fail to re-create vbmeta.img.

++++ vbmeta ++++

Traceback (most recent call last):
  File "./build/make/tools/releasetools/add_img_to_target_files.py", line 693, in <module>
    main(sys.argv[1:])
  File "./build/make/tools/releasetools/add_img_to_target_files.py", line 687, in main
    AddImagesToTargetFiles(args[0])
  File "./build/make/tools/releasetools/add_img_to_target_files.py", line 584, in AddImagesToTargetFiles
    boot_contents = boot_image.WriteToTemp()
AttributeError: 'NoneType' object has no attribute 'WriteToTemp'

Because it has skipped loading the boot.img, which leads to the above error.

Bug: 63456822
Test: As follows:
  $ zip -d target_files.zip vbmeta.img
  $ add_img_to_target_files.py -a target_files.zip
Test: `m dist` on aosp_arm64-userdebug
Change-Id: Ieba5f0b6848c3ec5d8bcfd24d48cc5af8b39f06c
2017-11-14 17:05:57 -08:00
Tao Bao
9dd909e6db releasetools: Fix the size check for AVB images.
When generating full OTAs for AVB-enabled targets, it fails the size
assertion for boot.img.

  ERROR: boot size (31457280) is 100.00% of limit (31457280)

The size assumption is no longer true for AVB-signed images, because
the actual image size should be always identical to the limit.

Bug: 67841177
Test: Enable AVB on bullhead. `m dist`.
Change-Id: I9bd432ccd88094628eb71308554839549eccb6d9
2017-11-14 15:43:36 -08:00
Tianjie Xu
15b728b8ed Merge "Add multithread support to call imgdiff with block-limit" am: 5d60cd2d3e am: 13680814dc
am: fd0011a9bb

Change-Id: Id3356c0511b8296181925598258655cedefd080a
2017-11-14 20:02:37 +00:00
Xin Li
44dd4881de Merge commit '66b8b530187b129934a685bf0c4349eef00e4b0c' from
oc-mr1-dev-plus-aosp-without-vendor into stage-aosp-master.

Change-Id: I7594bb72dd7e847292eb502edd918e41318aac29
Merged-In: Ide82473d358719f7e01cd2a4a85db954f3722f14
2017-11-14 11:36:22 -08:00
Tianjie Xu
fd0011a9bb Merge "Add multithread support to call imgdiff with block-limit" am: 5d60cd2d3e
am: 13680814dc

Change-Id: I772ccc6540bdf2fe41e70ae8e959094918717c29
2017-11-14 18:59:06 +00:00
Tianjie Xu
5d60cd2d3e Merge "Add multithread support to call imgdiff with block-limit" 2017-11-14 18:54:08 +00:00
Alex Deymo
b22db671d0 Merge "Update binary name to "brotli"." am: cbb4ec6cf0 am: ea26c8104e
am: a1c5d47708

Change-Id: I57092f1bcb427db14b7b6e0a21ff16e56ef7781b
2017-11-13 23:03:30 +00:00
Alex Deymo
a1c5d47708 Merge "Update binary name to "brotli"." am: cbb4ec6cf0
am: ea26c8104e

Change-Id: I330a67434c9d69c3c9d9aadae495e92d9788928c
2017-11-13 22:58:16 +00:00
Tianjie Xu
2536607d90 Add multithread support to call imgdiff with block-limit
With the new implementation of handling large apks, we need to call
imgdiff with block-limit to split the apk and generate the patch at
the same time. The call to imgdiff would significantly increase the
time consumption of the "FindTransfers" function which we used to
execute sequentially. This cl addresses this issue and speeds up the
process by making the imgdiff call parallel.

Bug: 34220646
Test: Create and sideload an incremental package for angler
Change-Id: Id62e348418fc1d22e32ea6c8ac16d9ab3ec92d7b
2017-11-13 14:58:05 -08:00
Alex Deymo
cbb4ec6cf0 Merge "Update binary name to "brotli"." 2017-11-13 22:43:22 +00:00
Alex Deymo
b10e07aea7 Update binary name to "brotli".
With the update of brotli to version 1.0.1 the new binary tool is now
called "brotli". This patch updates the scripts to use the new name.

Bug: 34220646
Test: `make dist`; ota_from_target_files on non-A/B device.
Change-Id: Ie8da4333388d029d5960a723a55efd875d8ee820
2017-11-13 18:11:45 +01:00
Tao Bao
d8bb564f99 Merge "releasetools: Reduce the memory use in test_common.py." am: 44cb0db6a7 am: 4fa1453f31
am: b3f0a951cf

Change-Id: I2abed40806a26bcbbf7bab9f5f25e6c2c26e6e7e
2017-11-11 01:48:27 +00:00
Tao Bao
b3f0a951cf Merge "releasetools: Reduce the memory use in test_common.py." am: 44cb0db6a7
am: 4fa1453f31

Change-Id: I16286bfdb209d3497acd161b1aafd0f756a686bb
2017-11-11 01:44:17 +00:00
Tao Bao
44cb0db6a7 Merge "releasetools: Reduce the memory use in test_common.py." 2017-11-11 01:32:31 +00:00
Colin Cross
30ca7be207 Merge "Convert signapk to Android.bp" am: 3563ac582a am: 4666a08608
am: 14409571c7

Change-Id: Idab0f1f5550324af17e7f7b402d3000b8b23f21c
2017-11-10 19:37:28 +00:00
Colin Cross
14409571c7 Merge "Convert signapk to Android.bp" am: 3563ac582a
am: 4666a08608

Change-Id: I9cbe96a55927b22ac897632260b3cd38edc7ff82
2017-11-10 19:28:29 +00:00
Colin Cross
3563ac582a Merge "Convert signapk to Android.bp" 2017-11-10 19:17:26 +00:00
Tao Bao
5155990a68 Merge "releasetools: Write back default_system_dev_certificate." am: ff87a9f9d9 am: b55251898d
am: 3b71247a2e

Change-Id: Ib9b1c936037b66ad98bffb53d5f3ee2b764749fa
2017-11-10 00:08:15 +00:00
Tao Bao
3b71247a2e Merge "releasetools: Write back default_system_dev_certificate." am: ff87a9f9d9
am: b55251898d

Change-Id: Ia2b4dc9f853d60f272b494b0d71be4a91383cfb6
2017-11-09 23:58:03 +00:00
Colin Cross
ad44dd4f3c Convert signapk to Android.bp
See build/soong/README.md for more information

Test: m checkbuild
Change-Id: I89a7fa7647e7081ca49720fae3be23ce84ab38e5
2017-11-09 15:11:34 -08:00
Tao Bao
f718f90212 releasetools: Write back default_system_dev_certificate.
When signing a target_files.zip, the OTA certificate specified by
default_system_dev_certificate could be replaced with a mapped key. When
that happens, we must explicitly specify --package_key when generating
OTA packages with ota_from_target_files.py. Otherwise the OTA package
will be signed with the wrong key, which leads to verification failures.

This CL updates the default_system_dev_certificate value in
misc_info.txt accordingly.

Test: Sign a target_files.zip and replace the OTA key. Check
      META/misc_info.txt in the generated target_files.zip.
  $ ./build/make/tools/releasetools/sign_target_files_apks.py -v \
      --replace_ota_keys \
      -k build/target/product/security/testkey=build/target/product/security/platform \
      out/dist/aosp_marlin-target_files-eng.tbao.zip \
      signed-marlin-target_files-test.zip

Change-Id: I093234b5add3e27c5b3887cefeffd74e6f0a3e98
2017-11-09 10:23:50 -08:00
Tao Bao
31b0807310 releasetools: Reduce the memory use in test_common.py.
test_common constructs a few 2GiB strings in memory, which leads to huge
memory footprint (18GiB). This CL moves away from in-memory strings to
generators, which reduces the memory use down to 41MiB. It also reduces
the time cost from 294s to 139s as an extra benefit for free.

The CL addresses some trivial pylint warnings as well.

* Before

$ /usr/bin/time -v python -m unittest -v test_common
...
----------------------------------------------------------------------
Ran 11 tests in 294.986s

OK
	Command being timed: "python -m unittest -v test_common"
	User time (seconds): 110.51
	System time (seconds): 109.34
	Percent of CPU this job got: 74%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 4:55.06
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 18894172
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 1
	Minor (reclaiming a frame) page faults: 20774908
	Voluntary context switches: 48
	Involuntary context switches: 3241
	Swaps: 0
	File system inputs: 184
	File system outputs: 8406424
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

* After

$ /usr/bin/time -v python -m unittest -v test_common
...
----------------------------------------------------------------------
Ran 11 tests in 139.100s

OK
	Command being timed: "python -m unittest -v test_common"
	User time (seconds): 59.00
	System time (seconds): 4.73
	Percent of CPU this job got: 45%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 2:19.17
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 41252
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 106569
	Voluntary context switches: 44
	Involuntary context switches: 103
	Swaps: 0
	File system inputs: 8
	File system outputs: 8422808
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Fixes: 68988396
Test: See above.
Change-Id: I00f16603a4ee59fb085b189c6f5b5ee9d2378690
2017-11-08 17:10:33 -08:00
Tianjie Xu
b2f60c0611 Merge "Fix the imgdiff failure when running test_common unittest" am: 7136100be6 am: 2b8489cf45
am: 929c7c6c81

Change-Id: I3af15e4e5d2b05e35ea51e9718bae40fc57df55b
2017-11-08 00:03:55 +00:00
Tianjie Xu
929c7c6c81 Merge "Fix the imgdiff failure when running test_common unittest" am: 7136100be6
am: 2b8489cf45

Change-Id: I6b94e092bd980ccd4266cd47af6cf53a0cc44e46
2017-11-07 23:53:42 +00:00
Tianjie Xu
7136100be6 Merge "Fix the imgdiff failure when running test_common unittest" 2017-11-07 23:40:20 +00:00