Commit graph

41408 commits

Author SHA1 Message Date
Tao Bao
ba557707d8 releasetools: Support validating Verified Boot images.
For a given (signed) target-files.zip, this CLs allows verifying the
Verified Boot related images. It works with both of VB 1.0 and VB 2.0
images.

As part of the CL, it also moves validate_target_files.py to argparse,
which is more flexible than the traditional getopt module.

Also add unittests for the VB 1.0 path. VB 2.0 tests will be added in
follow-up CL.

Example usage:

- Run the script on aosp_bullhead target-files.zip.

$ ./build/make/tools/releasetools/validate_target_files.py \
    --verity_key build/target/product/security/verity.x509.pem \
    --verity_key_mincrypt build/target/product/security/verity_key \
    aosp_bullhead-target_files-4522605.zip

- Run the script on aosp_walleye target-files.zip.

$ ./build/make/tools/releasetools/validate_target_files.py \
    --verity_key external/avb/test/data/testkey_rsa4096.pem \
    aosp_walleye-target_files-4627254.zip

Bug: 63706333
Bug: 65486807
Test: Run validate_target_files.py on target_files.zip files.
Test: PYTHONPATH=build/make/tools/releasetools python -m unittest \
          test_validate_target_files
Change-Id: I170f14d5828d15f3687d8af0a89a816968069057
2018-03-12 00:17:09 -07:00
Treehugger Robot
32dfa4914d Merge "releasetools: Create StreamingPropertyFiles class." 2018-03-09 19:56:18 +00:00
Treehugger Robot
d29f3e7026 Merge "Rename stubsjar to stubsSrcjar for droiddoc" 2018-03-09 09:42:29 +00:00
Anton Hansson
5a49a36a55 Merge "Enforce specifying sdk/private for apps. DO NOT MERGE" 2018-03-09 09:18:06 +00:00
Treehugger Robot
1cfafdd9fc Merge "Support a pre-extracted PDK zip" 2018-03-09 04:26:48 +00:00
Nan Zhang
89f12b00f0 Rename stubsjar to stubsSrcjar for droiddoc
Test: m -j core-docs
Bug: b/70351683
Change-Id: I90e4df3ba002c7488f3a80066bf3723b6f299b04
2018-03-08 17:29:16 -08:00
Treehugger Robot
0b4b321565 Merge "Remove unnecessary link-type check" 2018-03-09 00:21:21 +00:00
Tao Bao
ae5e4c30fe releasetools: Create StreamingPropertyFiles class.
This CL breaks down ComputeStreamingMetadata() into mutiple member
functions of StreamingPropertyFiles class, which correspond to the
two-pass logic when generating streaming property files (aka streaming
metadata).

StreamingPropertyFiles.Compute() does the work for the first pass, by
putting placeholders before doing initial signing. Finalize()
corresponds to the second pass, where the placeholders get replaced with
actual data. Verify() can be optionally called to assert the correctness
of the work.

The separation between Compute() and Finalize() is to allow having
multiple StreamingPropertyFiles instances (in coming up CLs). This way
we can call Compute() multiple times for each instance, followed by only
one call to SignOutput(). And similarly for Finalize().

Bug: 74210298
Test: Generate an A/B OTA package. Check the METADATA entry.
Test: python -m unittest test_ota_from_target_files
Change-Id: I45be0372a4863c4405e6d8e20bcb9ccdc29e7e11
2018-03-08 14:59:42 -08:00
Dan Willemsen
0663f685c2 Support a pre-extracted PDK zip
Pre-extracting the zip files are more sustainable for git, so that
objects can be shared if unchanged, rather than the 900MB zip file
changing on every build.

This also has the advantage that we could put an Android.bp file inside
the PDK, and Soong would just pick it up.

Bug: 68767391
Test: Build mini_arm64 PDK with platform.zip
Test: Build mini_arm64 PDK with extracted zip file
Change-Id: I16db030a731aea55b69c1d6e2260dbd70b167544
2018-03-08 13:50:21 -08:00
Ivan Lozano
b5399007ca Merge "Add minimal-runtime support for integer overflows." 2018-03-08 20:12:10 +00:00
Baligh Uddin
1617db36af Merge "releasetools: Fix the broken validate_target_files.py." 2018-03-08 15:50:15 +00:00
Anton Hansson
62c7de3792 Enforce specifying sdk/private for apps. DO NOT MERGE
All apps in master comply with this now, but new offenders
are being added daily. This will ensure compliance going
forward.

Test: build/soong/build_test.bash
Bug: 73535841

Change-Id: If28049750a9787bf455f72402e624dbddf842253
2018-03-08 13:19:36 +00:00
Nicolas Geoffray
2bff9a7c79 Merge "Revert "Remove art targets from PARSE_TIME_MAKE_GOALS"" 2018-03-08 08:57:44 +00:00
Jiyong Park
98a4f52c28 Remove unnecessary link-type check
Modules from Soong do not have dependency specified (via
LOCAL_JAVA_LIBRARIES). So, link-type check 'from' them doesn't make
sense. Resetting my_warn_types and my_allowed_types

Bug: 69899800
Test: m -j checkbuild on aosp_walleye, aosp_sailfish
Test: build/soong/build_test.bash --dist
Change-Id: I2a409e64f388bd8cb9336be589bf6af33e48a6f4
2018-03-08 15:23:08 +09:00
Tao Bao
c63626b4a3 releasetools: Fix the broken validate_target_files.py.
The file was broken due to earlier touches:
 - Missing 'import zipfile';
 - Mismatching arguments when calling GetSparseImage().

Bug: 73996151
Test: Run validate_target_files.py with a walleye-target_files.zip.
Test: pylint --rcfile=pylintrc validate_target_files.py
Change-Id: I3692bd51fb27a3da698e06b75155e84502549f66
2018-03-07 21:45:40 -08:00
Treehugger Robot
f74c027301 Merge "build/make: Add default suite for all native tests/benchmarks" 2018-03-08 01:38:04 +00:00
Mathieu Chartier
afc270b066 Merge "For user and userdebug generate error if DEXPREOPT is not enabled" 2018-03-07 22:34:14 +00:00
Colin Cross
78d5ea8e9b Merge changes I31ea4954,Ia02f8160,I5cfea62e,If484e9eb
* changes:
  Remove missed PRODUCT_AAPT_FLAGS
  Allow soong java libraries to include exported resources and proguard flags
  Silence more duplicate path warnings
  Remove framework_res_package_export_deps
2018-03-07 22:25:30 +00:00
Mathieu Chartier
2594230d83 For user and userdebug generate error if DEXPREOPT is not enabled
This prevents cases where system_server is running in interpreter
only mode.

Removed unused flag in product.mk

(cherry-picked from commit 27f4287406)

Bug: 74209329
Test: WITH_DEXPREOPT=false make
Merged-In: I4ab3afed95a5baf77d0cd089dafaa18bcc1913e5
Change-Id: I4ab3afed95a5baf77d0cd089dafaa18bcc1913e5
2018-03-07 22:21:43 +00:00
Simran Basi
d63a8153da build/make: Add default suite for all native tests/benchmarks
Not all native tests have the "tests" tag so adjust the logic to be:
NATIVE_TESTS||NATIVE_BENCHMARKS||(APPS&&tags==tests)

Bug: 74339384
Test: Rebuilt module-info.json, verified native tests now have
      their suite set to null-suite and have auto_test_config=True
      Double checked a frameworks App unittest as well.

Change-Id: Idea00adcebfc1a78b0c6d8ade40feddb1e90ed83
2018-03-07 12:36:55 -08:00
Colin Cross
8dad6c62d4 Remove missed PRODUCT_AAPT_FLAGS
PRODUCT_AAPT_FLAGS was removed by Iacc914114616b5bd19d9a1011802f4f9bca9bc19,
remove one more that it missed.

Test: m checkbuild
Change-Id: I31ea4954f5af5af37f13210925ed5d622625d71b
2018-03-07 09:48:33 -08:00
Colin Cross
7874138a8a Allow soong java libraries to include exported resources and proguard flags
Test: m checkbuild
Change-Id: Ia02f816084d0eb1c0e0f20e1ecd57aaf3778874c
2018-03-07 09:48:33 -08:00
Colin Cross
f6799e859a Silence more duplicate path warnings
Silence warnings when merging header jars that were missed by
Ifb67dbf7734b5f53941d110f578c4fe642e36005.

Bug: 69316739
Test: m checkbuild
Change-Id: I5cfea62e46d99f9b315a589d8915bf62307220aa
2018-03-07 09:47:41 -08:00
Colin Cross
6e136922f3 Remove framework_res_package_export_deps
framework_res_package_export is now always either the prebuilt
android.jar from an SDK or a package-export.apk generated by
Soong, so the dependency on R.stamp is never useful.  Remove
all assignments to framework_res_package_export_deps, and replace
usages with frameworks_res_package_export.

Test: m checkbuild
Change-Id: If484e9eb08061cb0ed0697755f13db71d741aaab
2018-03-07 09:47:41 -08:00
Roland Levillain
209b96e152 Revert "Remove art targets from PARSE_TIME_MAKE_GOALS"
This reverts commit 86a596c711.

Reverting as that commit breaks ART tests' results (see
b/73804944).

Test: m test-art-host
Bug: 73804944
Bug: 26275726
Change-Id: I06af4b170d845d3e55e044c28ec22d4b3ca3dede
2018-03-07 16:25:55 +00:00
Treehugger Robot
f99efebc75 Merge "Add ld.config.txt for wearables" 2018-03-07 07:55:40 +00:00
Justin Yun
61c412ed05 Add ld.config.txt for wearables
When BOARD_VNDK_VERSION is set, the required VNDK libs for vendor
modules are installed in vndk directory.
The default namespace configuration cannot reach those directories
and fails to link the required libs.
By installing ld.config.txt, the namespace configurations for each
build target will be selected automatically. This does not affect
the devices that requires default namespace configurations.

Bug: 74242105
Test: lunch sdk_gwear_x86-userdebug; make -j; emulator
Change-Id: Ib1097846fcd9c123184b13b6fbcfcb9a72ca3d97
2018-03-07 13:27:24 +09:00
Tao Bao
737bc2b082 Merge "releasetools: Separate streaming metadata computation into functions." 2018-03-07 04:24:13 +00:00
Treehugger Robot
1ac4e3c424 Merge "Conditionally generate META/boot_filesystem_config.txt." 2018-03-06 23:32:40 +00:00
Treehugger Robot
0d6a25c73f Merge "Clarify error message for bootclasspath whitelist." 2018-03-06 18:24:11 +00:00
Ivan Lozano
911cb99bc1 Add minimal-runtime support for integer overflows.
Adds Make support for -fsanitze-minimal-runtime when using
the integer overflow sanitizers. This makes the crashes due to these
sanitizers less mysterious. This also cleans up the handling of the
integer_overflow flag.

Bug: 64091660
Test: Compiled and checked the generated compiler commands
Test: Checked program that overflows for the abort reason
Test: Checked integer_overflow flag still emits overflow checks

Change-Id: I11012ed0cbbf51935f549a08bd17109b5ce6f330
2018-03-06 09:20:21 -08:00
Anton Hansson
53876001bf Merge "Add more control to the java sdk enforcement." 2018-03-06 11:19:38 +00:00
Tao Bao
f5110498c0 releasetools: Separate streaming metadata computation into functions.
And add unittests for ComputeStreamingMetadata().

This prepares for the changes that add additional property-files (for
both of A/B and non-A/B).

Bug: 74210298
Bug: 72751683
Test: python -m unittest test_ota_from_target_files
Test: Generate A/B OTA package. Check the ota-streaming-property-files
      in the METADATA entry.
Change-Id: Ib4b069f61c2c06c035c0cff73a55112f3936b969
2018-03-06 00:06:23 -08:00
Treehugger Robot
9d08769406 Merge "Fix the incorrect turbine flag." 2018-03-06 01:22:39 +00:00
Nan Zhang
78c19aa9ef Fix the incorrect turbine flag.
Test: m -j java TURBINE_ENABLED=false
Change-Id: Id6ca380c97bbcd3731c815e37513b7e949cccdfb
2018-03-05 13:35:45 -08:00
Treehugger Robot
8d36d4b8ea Merge "Remove ApiDemos and other packages modules in /data/app" 2018-03-04 01:04:27 +00:00
Weilun Du
1eb1568ce3 Remove ApiDemos and other packages modules in /data/app
Since API 27, PackageManager would not install random APKs that reside
in /data/app folder. They must be explicitly installed by "adb install"
instead.

BUG: 72750759
Change-Id: Ibc8e4e55e020439f5cf950d61fb7a815aa12fa6f
Signed-off-by: Weilun Du <wdu@google.com>
2018-03-04 00:37:57 +00:00
Tao Bao
395bf274b8 Merge "releasetools: Fix a mismatching close." 2018-03-02 23:32:11 +00:00
Treehugger Robot
903be914bb Merge "releasetools: Call Cleanup() in check_target_files_signatures.py." 2018-03-02 22:06:57 +00:00
Tao Bao
a652c00c8e releasetools: Fix a mismatching close.
There's a mismatch in WriteABOTAPackageWithBrilloScript().

  temp_zip_file = tempfile.NamedTemporaryFile()
  ...
  common.ZipClose(temp_zip_file)

It's benign since common.ZipClose() happens to be calling
"temp_zip_file.close()". This CL moves the use of tempfile to
common.MakeTempFile(), so that the tempfile will be cleaned up
automatically as part of the call to common.Cleanup(). (Not fixing the
close() directly, since the nearby lines will be refactored into another
function shortly.)

Also remove one assert in the same function, which trivially holds in
the current code.

Test: Generate an A/B OTA.
Change-Id: I53b375d1150820de718dec0ead55abf5f4951071
2018-03-02 13:20:43 -08:00
Jayant Chowdhary
1fa15d7491 Merge "Collect all lsdump paths and write them to $(OUT_DIR)." 2018-03-02 17:34:33 +00:00
Treehugger Robot
5c93279fbe Merge "releasetools: Don't return ZipFile from common.UnzipTemp()." 2018-03-01 22:21:25 +00:00
Treehugger Robot
e706929161 Merge "Build: Add some dex2oat boot image error message" 2018-03-01 20:46:05 +00:00
Treehugger Robot
16596052cb Merge "Build: Factor out ANDROID_LOG_TAGS" 2018-03-01 19:50:05 +00:00
Tao Bao
da01b211f8 Conditionally generate META/boot_filesystem_config.txt.
if BOARD_BUILD_SYSTEM_ROOT_IMAGE != true:  # case A
  - BOOT/RAMDISK corresponds to the / under normal boot, with matching
    fs_config in META/boot_filesystem_config.txt.
  - RECOVERY/RAMDISK corresponds to the / under recovery, with fs_config
    in META/recovery_filesystem_config.txt.
else:
  if BOARD_USES_RECOVERY_AS_BOOT == true:  # case B
    - ROOT/ corresponds to the / under normal boot, with fs_config in
      META/root_filesystem_config.txt.
    - BOOT/RAMDISK corresponds to the / under recovery, with fs_config
      in META/boot_filesystem_config.txt.
  else:  # case C
    - ROOT/ corresponds to the / under normal boot, with fs_config in
      META/root_filesystem_config.txt.
    - RECOVERY/RAMDISK corresponds to the / under recovery, with fs_config
      in META/recovery_filesystem_config.txt.
    - BOOT/RAMDISK doesn't exist.

This CL fixes case C, where we shouldn't try to generate
'META/boot_filesystem_config.txt' for BOOT/RAMDISK. It wouldn't be fatal
without this fix, but would wrongly scan the current directory and
include a large fs_config output into target-files.zip.

Bug: 72731506
Test: `lunch aosp_bullhead-userdebug` and `m dist`. (case A)
Test: `lunch aosp_marlin-userdebug` and `m dist`. (case B)
Test: Define 'BOARD_BUILD_SYSTEM_ROOT_IMAGE := true' for angler. `m
      dist` and check the generated target-files.zip. (case C)
Change-Id: I5582ce8cca464d535af0718be0fd8e65791bd6c2
2018-03-01 10:53:38 -08:00
Tao Bao
767543a659 releasetools: Call Cleanup() in check_target_files_signatures.py.
Also minor cleanups to make it pylint clean.

Test: Run check_target_files_signatures.py with a target-files.zip.
Test: pylint --rcfile=pylintrc check_target_files_signatures.py
Change-Id: Ife3b54c7805c2f2562e87e91ab4b4de355782012
2018-03-01 10:12:39 -08:00
Tao Bao
dba59eeabd releasetools: Don't return ZipFile from common.UnzipTemp().
In addition to the unzipping work, common.UnzipTemp() kindly bundles an
open ZipFile object as part of the return value. It doesn't look very
helpful to the callers though. It also looks less obvious that the
caller needs to properly close the handle (missing the close here is
benign though). This CL just removes the ZipFile object out of the
return value, and leaves the work to callers.

Test: `m dist` on both of A/B and non-A/B target.
Test: python -m unittest test_add_img_to_target_files
Test: python -m unittest test_common
Test: python -m unittest test_ota_from_target_files
Test: Check the callers to common.UnzipTemp() in code search.
Change-Id: Id47da3fd42a0e76d6ae8851f05780db319ee48cf
2018-03-01 10:03:31 -08:00
Andreas Gampe
5147749d1a Build: Add some dex2oat boot image error message
Typical failure case for boot image dex2oat is an inconsistent boot
classpath left over from incomplete build dependencies. Give advice
to run a top-level build.

Bug: 73749543
Test: m
Change-Id: I81c4ce9d02b2b360fe867b594d0a2b21c763e473
2018-03-01 08:47:44 -08:00
Andreas Gampe
fc2fefcd83 Build: Factor out ANDROID_LOG_TAGS
Factor out ANDROID_LOG_TAGS for boot image compilation. Do not use
the setting when ART_BOOT_IMAGE_EXTRA_ARGS is set.

Bug: 73749543
Test: m
Test: ART_BOOT_IMAGE_EXTRA_ARGS="--runtime-arg -verbose:verifier" m art-boot-image
Change-Id: Ia599381991f74f243fee966184715b0172742e78
2018-03-01 08:47:44 -08:00
Anton Hansson
f5f29aa976 Add more control to the java sdk enforcement.
I plan on turning the error on for APPS in AOSP soon, and in preparation for
that I'm introducing a finer granularity of warning/error control.

Also add an almost-empty whitelist, which will likely need to be expanded
in the future.

Bug: 73535841
Test: make
Change-Id: I2fc6700a504b7af50aa7bde727047bc56b167937
2018-03-01 14:28:47 +00:00