Commit graph

4640 commits

Author SHA1 Message Date
Chih-Hung Hsieh
edef957aeb Merge "Add a script to check OWNERS file syntax." am: d84721a2cc
am: a5b0fec8b6

Change-Id: I9cb8eb3737daa56621adcf237bcbb43b354497ef
2017-02-25 01:09:09 +00:00
Treehugger Robot
d84721a2cc Merge "Add a script to check OWNERS file syntax." 2017-02-25 01:00:18 +00:00
Chih-Hung Hsieh
adaed14ff3 Add a script to check OWNERS file syntax.
* -v option to print found files and email addresses.
* -c option to check found email addresses.

Bug: 33166666
Test: ./checkowners.py -v -c `find . -name OWNERS`
Change-Id: I32f23f19e904055e421ddec713536c8a5c970af4
2017-02-24 15:28:29 -08:00
Dan Willemsen
d04de52560 Merge "Wait for children to exit in makeparallel" am: 9b1a185029
am: c08e57ce4d

Change-Id: I2bc1fe1b9834ab56575db0c2e6fa96b968550382
2017-02-22 03:33:27 +00:00
Dan Willemsen
f0e2986cbf Wait for children to exit in makeparallel
This is a partial revert to the last makeparallel change, since now we
were being killed by the signal before waiting for our child to exit. So
instead of not installing the handlers, only pass the signal along if
it's a SIGTERM.

Bug: 35214134
Test: Ensure that we're still only getting one signal for SIGINT
Test: Ctrl-C, ensure that all the soong_ui lines are before the make
      error line.
Change-Id: I26fff9483a3abfd79ceb5a9ea47e3f7572d9e923
2017-02-21 17:53:35 -08:00
Dan Willemsen
06fbe5bec5 Merge "Prevent duplicate signals from makeparallel" am: 70aac88a61
am: 52ac801739

Change-Id: I8b779c85df3864115f5894c0cdd597c790417386
2017-02-21 23:29:40 +00:00
Treehugger Robot
70aac88a61 Merge "Prevent duplicate signals from makeparallel" 2017-02-21 23:22:32 +00:00
Dan Willemsen
c6a900bee1 Prevent duplicate signals from makeparallel
As part of the soong_ui effort, we noticed that we'd sometimes see
multiple SIGINTs from a single ctrl-c from a user. ctrl-c sends a SIGINT
to the entire process group, so make, makeparallel, soong_ui, and all of
its children would get a signal. Since makeparallel was passing it along
to it's child, soong_ui would get two signals.

So instead, follow what Make does and only pass along SIGTERM. Assume
that all other signals went to the entire process group.

Bug: 35214134
Test: Send SIGINT to process group, check in makeparallel's child for
more than one signal received.
Change-Id: I5b2a77ad0fcebbaa5087439948e71bf3b541061a
2017-02-21 13:50:12 -08:00
Tao Bao
4f0a147b69 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac am: 983c979c25 am: bd6d03ab18 am: e64f52ebcd
am: 55f185e0a4

Change-Id: Ic47af33906d2ead19cc3b6346f1aa55e5de5952d
2017-02-21 20:39:56 +00:00
Tao Bao
55f185e0a4 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac am: 983c979c25 am: bd6d03ab18
am: e64f52ebcd

Change-Id: I5ccd6ba60e5e9eb3e0875e99630db583a178a569
2017-02-21 20:37:24 +00:00
Tao Bao
e64f52ebcd Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac am: 983c979c25
am: bd6d03ab18

Change-Id: I488b164ae8c3a93858a293e2a538ff6b8742736c
2017-02-21 20:35:25 +00:00
Tao Bao
bd6d03ab18 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev am: b0df1ef7ac
am: 983c979c25

Change-Id: I938ee86611ea8c452c07324e2f3c6a9b0180cb7e
2017-02-21 20:33:25 +00:00
Tao Bao
983c979c25 Merge "Revert "releasetools: Allow using futility-host to sign old TF.zip."" into nyc-dev
am: b0df1ef7ac

Change-Id: I3924b44e75aa0f8f18129212dd80fd843f6a6a47
2017-02-21 20:31:25 +00:00
Tao Bao
d6145b6a16 Revert "releasetools: Allow using futility-host to sign old TF.zip."
Broke ryu targets.

This reverts commit 31a0de00a4.

Change-Id: I3ffcdc604a1cb92c86c574b5cd358d0148430d55
2017-02-21 20:27:27 +00:00
Tao Bao
ea88a3f3fa releasetools: Allow using futility-host to sign old TF.zip. am: 31a0de00a4 am: a9bdf8716f am: ae2f5abf06
am: 79d719b313

Change-Id: Ib858be8c430acf21d6aa175481ee68bb3e477caa
2017-02-21 17:50:18 +00:00
Tao Bao
79d719b313 releasetools: Allow using futility-host to sign old TF.zip. am: 31a0de00a4 am: a9bdf8716f
am: ae2f5abf06

Change-Id: I514c69dca1aede0fe047682dab9b11e7263d7634
2017-02-21 17:48:16 +00:00
Tao Bao
ae2f5abf06 releasetools: Allow using futility-host to sign old TF.zip. am: 31a0de00a4
am: a9bdf8716f

Change-Id: I411b26baf0cd7d74ec6c8c80a12f0499d017a360
2017-02-21 17:46:45 +00:00
Tao Bao
a9bdf8716f releasetools: Allow using futility-host to sign old TF.zip.
am: 31a0de00a4

Change-Id: I46041e6e5e37c451f8f23dca773850c025aa18b2
2017-02-21 17:45:15 +00:00
Alain Vongsouvanh
7f804ba71f releasetools: allow for multiple OEM property values.
This CL changes the --oem_settings flag to allow a comma seperated list of
property files. All property values will be used when asserting properties such
as ro.product.name.

For example, if two property files are provided with ro.product.name values of
"sprout" and "sprout_a", the resulting otapackage will check that the device's
ro.product.name property matches at least one of them.

Bug: 34191373
Test: manual

Change-Id: I954673511be8f0929982235cc9cbfbd85a9ee1f4
2017-02-21 16:09:34 +00:00
Tao Bao
31a0de00a4 releasetools: Allow using futility-host to sign old TF.zip.
Commit e98fb7a8d3 switched to using
futility-host instead of the prebuilt futility. This CL adds support to
handle signing old TF.zip that still says "futility=prebuilt/..." in
META/misc_info.txt.

Bug: 35467608
Test: Generate otatools.zip and sign an old ryu TF.zip.
Change-Id: I48a9cc918c7afce361e1ec9bc4f85f74fa92566e
(cherry picked from commit 4f104d1b94)
2017-02-17 23:36:00 -08:00
Tao Bao
4f104d1b94 releasetools: Allow using futility-host to sign old TF.zip.
Commit e98fb7a8d3 switched to using
futility-host instead of the prebuilt futility. This CL adds support to
handle signing old TF.zip that still says "futility=prebuilt/..." in
META/misc_info.txt.

Bug: 35467608
Test: Generate otatools.zip and sign an old ryu TF.zip.
Change-Id: I48a9cc918c7afce361e1ec9bc4f85f74fa92566e
2017-02-17 23:27:44 -08:00
Narayan Kamath
b46507ff9e zipalign: remove support for adding members from gzip sources.
This was vestigial code and never actually called anywhere.

Test: make
Bug: 35246701
Change-Id: I9297ac5fa030d369946857414155450635b65410
2017-02-16 10:53:09 +00:00
Tao Bao
0ee0fc5898 Merge "releasetools: Fix a bug in ReviseStashSize()." 2017-02-13 22:29:40 +00:00
Tao Bao
3c5a16d49c releasetools: Fix a bug in ReviseStashSize().
We check the needed stash size in ReviseStashSize(), and may not
generate a stash command if it would exceed the max allowed size. This
CL fixes a bug when skipping a stash operation: we shouldn't update the
'stashes' map if a stash command won't be generated.

Bug: 35313668
Test: Successfully generate the package that was failing due to the bug.
Change-Id: If0a3a5fadda9b4a4edad66a2a5826b5f978400ae
2017-02-13 12:31:42 -08:00
Alex Deymo
9a535b5188 fat16copy: Allow to copy files to an existing directory.
This patch re-uses existing directories when copying directories,
allowing to copy files to a directory previously created.

Bug: 34811400
Test: fat16copy.py fat.bin a/b/foo; fat16copy.py fat.bin c/d/foo

(cherry-picked from 0af24a27a8d0dc7e1782f36a5a9963021ff8b3f6)

Change-Id: I283184da18f3c4951d6c84aafa4185074791acd2
2017-02-13 16:00:07 +00:00
Tao Bao
b31892e5de releasetools: Support generating downgrade A/B OTAs.
We already support generating downgrade OTAs for non-A/B devices (with
mandatory data wipe), but we have missed the --downgrade flag in A/B OTA
path.

This CL factors out the function that writes the downgrade metadata, and
fixes the path for generating A/B OTAs.

Bug: 35094540
Test: Generate incrementals with --downgrade for A/B and non-A/B OTAs.
Change-Id: I30b9bf83e69e8aba3be666507681b555db6ab743
2017-02-07 12:25:59 -08:00
Tao Bao
b09fef9158 Merge "releasetools: Pack the offset/length for metadata." 2017-02-07 04:16:20 +00:00
Tao Bao
82bdf6c1d7 releasetools: Fix the bug when generating care_map.txt.
Commit f1a13180db intended to remove the
verity blocks from care_map.txt, but it added new code without removing
the old one. This leads to a malformed care_map.txt and causes
update_verifier failure.

Bug: 34391662
Test: 'm -j dist' gives a TF.zip with 4-line META/care_map.txt (as
      opposed to a 6-line file).

Change-Id: I7ff1aa525795c4b049af54c1755b0f0ea84f7e0e
2017-02-05 14:31:44 -08:00
Tao Bao
2dd1c48377 releasetools: Pack the offset/length for metadata.
For streaming OTAs, we will also need the info in the metadata entry
(META-INF/com/android/metadata). Compute and pack its offset/length
values into 'ota-streaming-property-files'.

Bug: 34986195
Test: Create an OTA package and check the offset/length values.

Change-Id: Id150700f2bc9bff02467cda9fe8927c8a374412a
2017-02-03 16:50:25 -08:00
Tianjie Xu
3a79c6d784 Merge "Remove verity blocks from care map" 2017-02-03 21:50:07 +00:00
Tao Bao
bfdcb1257c releasetools: Prepend 'ota-' to 'streaming-property-files'.
'streaming-property-files' is a property related to the OTA package
itself. Prepend 'ota-' to make it consistent with others like
'ota-type' and 'ota-required-cache'.

Bug: 34852392
Test: Generate an A/B OTA package and check METADATA entry.
Change-Id: Ia681e6e19ff509e6da0d8718933b42aac997e1cf
2017-01-31 15:53:20 -08:00
Tianjie Xu
f1a13180db Remove verity blocks from care map
When reading /dev/block/dm-X, update_verifier isn't able to access the
verity meta blocks at the end of the system/vendor partition. So we need
to remove these block ranges from the care_map.

Bug: 34391662
Test: care_map generated successfully without verity meta blocks
Change-Id: Id57c602b7e5fd1b0c9d1e1fe5fcdd74e85b6b255
2017-01-25 23:01:29 -08:00
Tao Bao
c96316c89b Revert "Revert "releasetools: Generate streamable A/B OTA packages.""
This reverts commit ea4325baf8 to re-land
commit ef1bb4360f. It fixes the bug when
handling a package without care_map.txt (e.g. dm-verity not enabled).

In order to support streaming A/B OTA packages, we pack
payload_properties.txt and care_map.txt in ZIP_STORED mode. These two
entries along with payload.bin (already in ZIP_STORED prior to this CL)
can be fetched directly based on the offset and length info.

We write the offset and length info into the package metadata entry
(META-INF/com/android/metadata), which can be parsed by the OTA server.

payload_properties.txt and care_map.txt are usually less than 1-KiB. So
the change only incurs marginal size increase.

Bug: 33382114
Test: Generate an A/B OTA package. Verify the 'streaming-property-files'
      entry in the metadata file.
Test: Generate an A/B OTA package on a device with dm-verity not enabled.

Change-Id: I3469c8b62385a1fc58b4fb82e3f9d4690aef52ba
2017-01-24 22:27:02 -08:00
Tao Bao
ea4325baf8 Revert "releasetools: Generate streamable A/B OTA packages."
This reverts commit ef1bb4360f.

It has broken the builds that don't have care_map.txt entry.

Change-Id: I343455e66be5e79457968dfc2813c7c1c234b6dc
2017-01-25 01:41:13 +00:00
Tao Bao
ef1bb4360f releasetools: Generate streamable A/B OTA packages.
In order to support streaming A/B OTA packages, we pack
payload_properties.txt and care_map.txt in ZIP_STORED mode. These two
entries along with payload.bin (already in ZIP_STORED prior to this CL)
can be fetched directly based on the offset and length info.

We write the offset and length info into the package metadata entry
(META-INF/com/android/metadata), which can be parsed by the OTA server.

payload_properties.txt and care_map.txt are usually less than 1-KiB. So
the change only incurs marginal size increase.

Bug: 33382114
Test: Generate an A/B OTA package. Verify the 'streaming-property-files'
      entry in the metadata file.

Change-Id: I04504e834eb36e18876c5f5a5a09289ee05c6f9a
2017-01-19 22:25:12 -08:00
Colin Cross
524c868107 Don't pass empty arguments to ninja
Passing an empty argument in argv to ninja results in an error:
ninja: error: empty path

Don't add jarg to the argument list if it is empty.

Bug: 34392351
Test: m -j
Change-Id: I39c8998c79b09835650628e1413ae7625cdbc555
2017-01-18 21:12:28 -08:00
Colin Cross
ba55c623cb Convert -j to a reasonable parallelism for kati
When running makeparallel in non-ninja mode, which is used when running
kati with USE_GOMA=true, convert -j to a reasonable parallelism value
the same way ninja does.

Bug: 34392351
Test: make -C build/make/tools/makeparallel makeparallel_test
Change-Id: I9aee4dd2a3b0f2b1c2c10087be83f7b2b06f4368
2017-01-18 14:50:50 -08:00
Connor O'Brien
cf4f7eac85 Merge "Pass flash erase & logical block size to mke2fs" 2017-01-17 20:07:11 +00:00
Tao Bao
4a0b494eea Merge "releasetools: Replace print stmt with print()." 2017-01-14 15:36:41 +00:00
Tao Bao
89fbb0f6d5 releasetools: Replace print stmt with print().
So that it's compatible with Python 3.

Test: pylint --pylint=pylintrc

Change-Id: If06c135a492c94bedd713c8cbdf03155a502d5cd
2017-01-13 14:55:14 -08:00
Tao Bao
2a0d1da4d9 releasetools: Delete the obsolete --aslr_mode/-a flag.
It was added in commit 96be7205dc
("Working ASLR implementation.") in 2010, and removed in commit
1807e700a5 ("don't generate retouch
commands in OTA scripts") in 2012.

Remove the obsolete --aslr_mode flag.

Test: ota_from_target_files.py still works (by generating incremental
      and full OTAs respectively).

Change-Id: I6d8e62730ac192f3574d484c4a4b9b43b4ee0a9e
2017-01-13 22:46:27 +00:00
Connor O'Brien
20f08c3e16 Pass flash erase & logical block size to mke2fs
This information can be used to tune ext4 stripe and stride in the
userdata partition for better performance

Test: Build & flash userdata, confirm correct stripe & stride values
Bug: 33243520
Merged-In: Ia97cdd2d0239c3484b895fce49299f692ef911d8
Change-Id: Ia97cdd2d0239c3484b895fce49299f692ef911d8
Signed-off-by: Connor O'Brien <connoro@google.com>
2017-01-13 12:17:05 -08:00
Patrick Tjin
37a73e0f09 build_image: add support for specifying number of inodes [DO NOT MERGE]
Add support for specifying number of inodes when creating
system, vendor, oem partitions.  These are all read-only
and have no use for extra inodes.  Removing extra inodes
saves a lot of space.

Bug: 32246383
Bug: 33252070
Bug: 33428604
Change-Id: I13f1d4614b64a4abc752c42a1c65d3d151481c21
2017-01-05 11:19:43 -08:00
William Roberts
4165c635b8 fsconfig: fix pylint warning
Fix pylint warning:
W:292,47: Cell variable lineno defined in loop (cell-var-from-loop)

For the closure function that prints the error message, we actually
want the value of modified lineno within the closure.

Test: run unit tests

Change-Id: I7d08184563d347c826ca1af7558fb8e378f151e1
Signed-off-by: William Roberts <william.c.roberts@intel.com>
2017-01-04 18:22:05 +00:00
William Roberts
820421c9b5 fsconfig: fix style formating with yapf
A few changes got introduced that yapf re-styles. To make it easy,
keep this file formated to:

yapf -i --style=google fs_config_generator.py

Test: run unit tests

Change-Id: I3160b92d7fd07a4d315dde59f1d20fe4c7587da1
Signed-off-by: William Roberts <william.c.roberts@intel.com>
2017-01-04 18:21:28 +00:00
Elliott Hughes
ce6c8e629f Merge "fs_config: Fix oem range parsing" 2017-01-04 00:37:58 +00:00
Johan Redestig
1552a28e5f fs_config: Fix oem range parsing
Addresses the following build error:
  "value" not in valid range [], got: 2901 for: "AID_ABC" file:\
  "device/somc/abc/config.fs"

that happens when parsing device/somc/abc/config.fs:
  [AID_ABC]
  value: 2901

The problem is that the AID.PREFIX is 'AID_' so the _OEM_RANGE
expression is searching for AID__OEM_RESERVED, which never
exists. Drop the extra '_'.

Change-Id: I8a1556731e14e49cef66e3e91121e4be23a308e0
Test: ./build/tools/fs_config/fs_config_generator.py fsconfig --aid-header=system/core/include/private/android_filesystem_config.h device/somc/abc/config.fs
2017-01-03 09:45:33 +01:00
Tao Bao
3a2e350239 blockimgdiff.py: Clean up stash id computation in BBOTA v3+.
Only BBOTA v2 needs to maintain a pool of available 'stash slot id'.
BBOTA v3+ uses the hash of the stashed blocks as the slot id, which
doesn't need the id pool anymore.

Bug: 33694544
Test: Generate v2 and v4 incrementals w/ and w/o the CL. They produce
      the same packages respectively.
Change-Id: I8121af5b6b1bee98c3639d54a00b06fd12e378e8
2016-12-28 09:26:46 -08:00
Elliott Hughes
5f60b74297 Merge "Revert "Revert "Merge changes from topic 'fsconfig-2'""" 2016-12-27 20:51:07 +00:00
Tao Bao
304ee27e88 blockimgdiff.py: Add a test case for ReviseStashSize().
Bug: 33687949
Test: python -m unittest test_blockimgdiff
Change-Id: I31e42648d2c0e60b941908c098ce5ccc967861ca
2016-12-19 11:17:12 -08:00