Commit graph

32676 commits

Author SHA1 Message Date
Treehugger Robot
e65af5ce72 Merge "Include file_contexts conditionally elsewhere" 2017-03-10 23:54:18 +00:00
Tao Bao
c3e3406125 Merge "releasetools: Drop the support for BBOTA v1 and v2." 2017-03-10 19:25:06 +00:00
Treehugger Robot
1e8364a8be Merge "releasetools: Fix the missing 'post-build' in full OTAs." 2017-03-10 00:30:55 +00:00
Tao Bao
39f3eaf221 releasetools: Fix the missing 'post-build' in full OTAs.
The line was unintentionally removed in commit 7f804ba.

Test: ota_from_target_files.py generates a full OTA. Check the package
      metadata.

Change-Id: Icae88e2a9bb2bfc450a3d0d7ab524d6a6eac9df5
2017-03-09 15:01:11 -08:00
Tao Bao
8fad03e771 releasetools: Drop the support for BBOTA v1 and v2.
BBOTA v1 and v2 (introduced in L and L MR1 respectively) don't support
resumable OTA. We shouldn't generate packages using v1/v2 at the risk of
bricking devices.

BBOTA v3 (since M) and v4 (since N) both support resumable OTAs. BBOTA
v4 additionally supports using FEC to possibly recover a corrupted
image.

Bug: 33694730
Test: Generate full and incremental OTAs w/ and w/o the CL. They should
      give identical packages (in v4).
Change-Id: Ib89d9cd63ba08e8e9aa4131bed18876b89d244c0
2017-03-09 12:25:46 -08:00
Tao Bao
c87b38f547 Merge "releasetools: Remove dead functions in EdifyGenerator." 2017-03-09 20:24:19 +00:00
Jeff Vander Stoep
f57a154f2e Include file_contexts conditionally elsewhere
Test: Marlin and Bullhead build and boot with no new denials.
Test: Marlin and Bullhead recovery boots with no new denials.
Test: Bullhead boots with file_contexts.bin in /
Test: Marlin boot with /system/etc/selinux/plat_file_contexts and
      /vendor/etc/selinux/nonplat_file_contexts.
Bug: 36002414

Change-Id: I25af653e060c0c72ea6ad08db8869132885adff8
2017-03-08 21:12:47 -08:00
Tao Bao
f388104eaa releasetools: Remove dead functions in EdifyGenerator.
Remove the following functions that are needed for file-based OTAs only:
 - SetPermissions()
 - SetPermissionsRecursive()
 - MakeSymlinks()
 - DeleteFiles()
 - DeleteFilesIfNotMatching()
 - RenameFiles()
 - SkipNextActionIfTargetExists()

Bug: 35853185
Test: Verified there's no reference to these functions.
Change-Id: Iff24a9d705476211effaef28eed2a147fa5fcbce
2017-03-08 16:37:07 -08:00
Treehugger Robot
df58de6303 Merge "hardware/interfaces: Android.mk always -Werror" 2017-03-08 23:16:35 +00:00
Tao Bao
9bca43858f Merge "releasetools: Drop the support for file-based OTA generation." 2017-03-08 21:00:03 +00:00
Dan Willemsen
c619f99e2f Merge changes Ia5ce6870,I155654cd
* changes:
  Zip after add_img_to_target_files
  Allow add_img_to_target_files to work on a directory
2017-03-08 20:45:56 +00:00
Tao Bao
3b1f260205 Merge "releasetools: Reduce memory footprint for BBOTA generation." 2017-03-08 17:43:34 +00:00
Steven Moreland
324c436084 hardware/interfaces: Android.mk always -Werror
Bug: 35840847
Test: pass
Change-Id: I6ebf3f5d279ff405446592cdabaeca9d1fdf6cf9
2017-03-08 16:29:50 +00:00
Treehugger Robot
3838ae7e9b Merge "Put system_version in target_files.zip" 2017-03-08 03:21:43 +00:00
Sen Jiang
9b8e4ca518 Put system_version in target_files.zip
Bug: 35364984
Test: m dist
Change-Id: I309acb41eb85de239c4c1718e14d9442ca1b0408
(cherry picked from commit cea4d52d2fee19281e660943129e1ad3bdc51fef)
2017-03-07 17:40:27 -08:00
Dan Willemsen
2a5a34f6bd Zip after add_img_to_target_files
For aosp_marlin (on AOSP), this speeds up target files creation from
~57 seconds to ~15 seconds.

For marlin-userdebug (on internal master), this speeds up target files
creation from ~300 seconds to ~95 seconds.

This does make some minor changes to the resulting target_files zip:

* All of the META files are actually at the beginning of the zip now,
  previously the ones created by add_img_to_target_files would be at
  the end.
* The images are slightly larger. Go's zip implementation in soong_zip
  isn't quite as good as the `zip` tool.
* vendor_filesystem_config.txt isn't present if we're not building a
  vendor image.

Test: aosp_marlin target_files zip changes look reasonable
Test: marlin target_files zip changes look reasonable
Test: bullhead target_files zip changes look reasonable
Change-Id: Ia5ce6870b85559d65107c3b94332391b4b43a0ea
2017-03-07 15:28:42 -08:00
Dan Willemsen
2ee00d5f66 Allow add_img_to_target_files to work on a directory
In addition to the current behavior of add_img_to_target_files working
on an existing zip file, allow passing in a directory where the target
files have already been extracted. When in this mode, it writes the
images out to that directory instead of the zip file.

This allows us to call add_img_to_target_files on the temp directory
used during the build to create the target files package, saving the
time and space of unzipping what we just zipped. This also allows us to
use the parallel soong_zip, which compresses the images much faster.

Test: aosp_marlin target_files zip is the same before/after this change
Test: marlin target_files zip is the same before/after this change
Test: bullhead target_files zip is the same before/after this change
Change-Id: I155654cdc7ad7754ba4ef76ec69c31f504a58198
2017-03-07 14:37:52 -08:00
Treehugger Robot
2b33e46593 Merge "Add vnod." 2017-03-07 21:54:07 +00:00
Steven Moreland
a2e734d786 Add vnod.
Short name for vendorimage-nodeps like snod.

Bug: 33356516
Test: works
Change-Id: Iad62aa5a021d603fa9cebe90f4e28cf09d5d49ae
2017-03-07 12:17:51 -08:00
Tao Bao
457cbf6a8a releasetools: Drop the support for file-based OTA generation.
We have stopped shipping devices with file-based OTAs, and are not
actively maintaining the support. Devices using file-based OTAs
should be moved to block-based, if not A/B OTAs.

We will also need to clean up EdifyGenerator class, which will be
handled in follow-up CLs.

Bug: 35853185
Test: Generate full and incremental OTAs w/ and w/o the CL, and they
      give identical packages.
Test: Not specifying --block also generates block-base OTAs.
Change-Id: I3b0fc8ce5600e109f3251fe41f655534aaa298c7
2017-03-07 00:17:58 -08:00
Tao Bao
7dc54e9761 Merge "releasetools: Remove the obsolete path in GetImage()." 2017-03-07 06:20:04 +00:00
Tao Bao
0c9e8d87c4 Merge "releasetools: Fix two minor issues with _LoadOemDicts()." 2017-03-07 00:24:44 +00:00
Tao Bao
7e0f160f30 releasetools: Remove the obsolete path in GetImage().
We must have created the images (system.img, system.map etc) prior to
calling ota_from_target_files.py (added by commit
2e0d8fcf08, into Lollipop).

Remove the obsolete suppport for handling "old" target_files zips that
don't have such images. This simplies the logic for BuildSystem() and
BuildVendor(), which now would only be called by
add_img_to_target_files.py itself.

Test: Generating full and incremental OTAs give the same results, w/ and
      w/o this CL.
Change-Id: I0ea243d09d3378249d7982701ae4ec339b04b7b8
2017-03-06 16:22:44 -08:00
Mathieu Chartier
4d42fef1db Merge "Fix profile dependency" 2017-03-06 19:45:00 +00:00
Mathieu Chartier
fb6c6b38c3 Fix profile dependency
Otherwise it doesn't work for incremental builds.

Bug: 34929159
Bug: 35997300

Test: repo sync && m

Change-Id: I78ead57cf89a9d0f26dbf4f3a225a9357bdca1b3
2017-03-06 11:38:49 -08:00
Tao Bao
ebce697429 releasetools: Fix two minor issues with _LoadOemDicts().
The first one in WriteVerifyPackage() is mismatching function parameters
that can be trivially fixed.

The other one is in WriteABOTAPackageWithBrilloScript(), where we don't
have edify script instance.

Test: `pylint --rcfile=pylintrc ota_from_target_files.py`.
Change-Id: Ie238ef5b296dfe9e725b61376992447b662d2376
2017-03-06 10:29:13 -08:00
Tao Bao
183e56e83d releasetools: Reduce memory footprint for BBOTA generation.
The major issue with the existing implementation is unnecessarily
holding too much data in memory, such as HashBlocks() which first reads
in *all* the data to a list before hashing. We can leverage generator
functions to stream such operations.

This CL makes the following changes to reduce the peak memory use.
 - Adding RangeSha1() and WriteRangeDataToFd() to Image classes. These
   functions perform the operations on-the-fly.
 - Caching the computed SHA-1 values for a Transfer instance.

As a result, this CL reduces the peak memory use by ~80% (e.g. reducing
from 5.85GB to 1.16GB for the same incremental, as shown by "Maximum
resident set size" from `/usr/bin/time -v`). It also effectively
improves the (package generation) performance by ~30%.

Bug: 35768998
Bug: 32312123
Test: Generating the same incremental w/ and w/o the CL give identical
      output packages.
Change-Id: Ia5c6314b41da73dd6fe1dbe2ca81bbd89b517cec
2017-03-06 08:46:30 -08:00
Tao Bao
8e02284362 Merge "Revert "Revert "releasetools: Use pattern filter when unzipping.""" 2017-03-06 16:45:26 +00:00
Tao Bao
6b0b2f9db0 Revert "Revert "releasetools: Use pattern filter when unzipping.""
This reverts commit a7316ce094.

This CL differs from the original CL by not unzipping RADIO/*. This is
because: a) AOSP targets don't have RADIO/ entries in the TF.zip; b)
we're not using the unzipped RADIO files (but reading them from the zip
files directly) - checked all the device-specific releasetools for
angler, bullhead, ryu, shamu, volantis, fugu, marlin and sailfish.

Test: `m dist` with AOSP targets (tested fugu and bullhead).
Change-Id: I4d0c67214ddd6202fc27c71bb79f52b5f4d40c64
2017-03-05 13:11:42 -08:00
Tao Bao
0c3e31c69b Merge "Revert "releasetools: Use pattern filter when unzipping."" 2017-03-05 17:44:58 +00:00
Tao Bao
a7316ce094 Revert "releasetools: Use pattern filter when unzipping."
This reverts commit aa3a04f19d.

Reason for revert: Some AOSP targets don't include RADIO/ in the TF.zip. We may possibly skip unzipping RADIO/, or by always creating a dummy RADIO folder in TF.zip. Revert this CL for now.

Change-Id: I8e90d322706a4fe82129bdfab5ffa1eab989c648
2017-03-05 17:43:33 +00:00
Tao Bao
cc63bc3a91 Merge "Change the default parameter values in common.MakeTempFile()." 2017-03-05 17:00:38 +00:00
Tao Bao
76f038a63c Merge "releasetools: Use pattern filter when unzipping." 2017-03-05 16:49:21 +00:00
Treehugger Robot
2f6c2b44d0 Merge "Support kernel modules in vendor, recovery images" 2017-03-04 03:35:00 +00:00
Howard Chen
adbb9c30d1 Support kernel modules in vendor, recovery images
Support BOARD_VENDOR_KERNEL_MODULES which accepts a list of the
kernel module binaries. Binaries listed there are copied to
$(TARGET_OUT_VENDOR)/lib/modules and 'depmod' is executed to generate
the dependencies. BOARD_RECOVERY_KERNEL_MODULES is similarly
implemented to provide a way to specify kernel modules to be copied
to the recovery image.

Bug: 33305644
Test: android master build on marlin and load dummy modules
Merged-In: I0a58a270dbb0d2d38fd089f4abff66e62e14fac1
Change-Id: Ia5f429508ca3d7c77448358e42af6efa98fa8624
2017-03-03 18:35:30 -08:00
Mathieu Chartier
b05880e311 Merge "Move profile logic to dex_preopt_odex_install" 2017-03-04 02:21:24 +00:00
Alex Deymo
a57a2fd118 Merge "Update product_id and product_version location." 2017-03-04 01:46:21 +00:00
Alex Deymo
e90295014f Merge "Add PRODUCT_IOT to the list of product vars." 2017-03-04 01:46:06 +00:00
Tao Bao
aa3a04f19d releasetools: Use pattern filter when unzipping.
When building BBOTAs, it only needs *some* unzipped entries in the given
target_files zip(s). In particular, it needs 'IMAGES/*', 'META/*',
'RADIO/*'. (It also reads 'SYSTEM/build.prop' and 'OTA/bin/updater', but
directly from the zip file.)

This CL specifies the entries to unzip. It saves the I/O cost, as well as
the temporary storage.

Test: ota_from_target_files.py gives the same package w/ and w/o the CL.
Test: check_target_files_signatures.py still works.
Change-Id: I728428aa0e138879e49f9efbdb46a85892fc7038
2017-03-03 16:38:17 -08:00
Mathieu Chartier
2ac264fe13 Move profile logic to dex_preopt_odex_install
It is not necessary to have it in setup_one_odex since the profile is
the same for multilib.

Prevents errors related to using ../ in profile path resulting in not
having a directory.

Bug: 34929159
Test: m

Change-Id: Ifdcba119f3599b0c80e27594781db5e4d64bd473
2017-03-03 16:23:14 -08:00
Mathieu Chartier
c2159fd5b6 Merge "Avoid copying profile" 2017-03-03 22:36:46 +00:00
Sen Jiang
cf858d873b Merge "Remove DBus support." 2017-03-03 20:54:38 +00:00
Mathieu Chartier
62e1ed7a63 Avoid copying profile
We can just use the source profile.

Bug: 34929159
Test: m

Change-Id: Ibbc0b392052f2a7f76bada13493ca62dc0334ace
2017-03-03 12:47:30 -08:00
Mathieu Chartier
4fdf2189b0 Merge "Use ANDROID_LOG_TAGS to filter spammy profman output" 2017-03-03 17:47:08 +00:00
Alex Deymo
b4309d4f54 Update product_id and product_version location.
These files are used by the update console to determine the product
associated with a given target_files.zip.

Bug: 35804180
Bug: 33211225
Test: `make dist` generates a target_files.zip with these files.
Change-Id: I6066da07d76749ae64c26d5f661c7a6d06f3fb3d
2017-03-03 17:32:42 +00:00
Alex Deymo
8fe63c3899 Add PRODUCT_IOT to the list of product vars.
This variable tells whether the product is an Android Things variant of
Android.

Bug: 32549448
Test: `make dist` on an iot product.
Change-Id: I0875951ae57867bc8bf47885e77e494d6284acdb
2017-03-02 22:08:41 -08:00
Vijay Venkatraman
69e8252aec Merge "Fix for compile error when BUILD_HEADER_LIBRARY is used" 2017-03-03 04:10:50 +00:00
Treehugger Robot
3e78704825 Merge "Include secilc and sepolicy conditionally, elsewhere" 2017-03-03 03:50:48 +00:00
Sen Jiang
d1f397e8b4 Remove DBus support.
external/dbus and external/dbus-binding-generator was removed from
the manifest.

Bug: 31602715
Test: make checkbuild
Change-Id: Iea0277720acad8ac0fa630b8745f90fb3c3b5f00
2017-03-02 18:00:21 -08:00
Mathieu Chartier
fea31c0b9e Merge "Add local preopt options for generating app image and profile" 2017-03-03 01:14:03 +00:00