Commit graph

41818 commits

Author SHA1 Message Date
Tao Bao
f3d0d80d81 Merge "releasetools: Fix an issue in GetMinSdkVersion."
am: f18ba63095

Change-Id: I43cf88b9394a077fa76166a2b94734f6bd3b7147
2018-03-23 18:18:43 +00:00
Treehugger Robot
f18ba63095 Merge "releasetools: Fix an issue in GetMinSdkVersion." 2018-03-23 18:08:40 +00:00
Isaac Chen
ac70cd2b8c Merge "Move emulator fstab to vendor for "system as root""
am: 1b8a60d4e8

Change-Id: Ie4a8524aae4cf89d13e764dbd1c286cb8667b34b
2018-03-23 17:48:17 +00:00
Rahul Chaudhry
a715ace74e Merge "Disable relocation_packer and migrate to SHT_RELR sections."
am: 38633b6ad1

Change-Id: I05d76464df767a8b6da43d8c8954c02eeae72045
2018-03-23 17:47:23 +00:00
Treehugger Robot
1b8a60d4e8 Merge "Move emulator fstab to vendor for "system as root"" 2018-03-23 17:38:54 +00:00
Treehugger Robot
38633b6ad1 Merge "Disable relocation_packer and migrate to SHT_RELR sections." 2018-03-23 17:28:08 +00:00
Tao Bao
49df32de92 Merge "releasetools: Capture and dump stdout/stderr outputs on errors."
am: f3e39e5878

Change-Id: I5b63680a176f465c3dbbc0d80e6000e7b415f96a
2018-03-23 01:18:09 +00:00
Treehugger Robot
f3e39e5878 Merge "releasetools: Capture and dump stdout/stderr outputs on errors." 2018-03-23 01:05:27 +00:00
Martijn Coenen
f12ffc445f Merge "Products launching with API 28 need 64-bit binder interface."
am: c0811e291c

Change-Id: I58419bfd193f098273e2bdea35c3bc82dd12817c
2018-03-22 22:15:42 +00:00
Rahul Chaudhry
dbf3bfffc6 Disable relocation_packer and migrate to SHT_RELR sections.
Proposal for adding SHT_RELR sections in the generic-abi is at
https://groups.google.com/forum/#!topic/generic-abi/bX460iggiKg

Comparison with relocation_packer for aosp_marlin-userdebug:
$ du -B1 -s */out/target/product/marlin/system
 996958208      base/out/target/product/marlin/system
1008340992      no_pack/out/target/product/marlin/system
 997801984      sht_relr/out/target/product/marlin/system

base contains a build with current settings.
no_pack contains a build with relocation_packer disabled.
sht_relr contains a build with SHT_RELR sections enabled.

relocation_packer was saving 11,382,784 bytes (no_pack - base).
SHT_RELR sections are saving 10,539,008 bytes (no_pack - sht_relr).

Bug: None
Test: Built aosp_marlin-userdebug image, boots on device.
Test: Built aosp_x86_64-userdebug image, boots in emulator.
Change-Id: Ic753abecb77863c64996157c543c344e66f0f4dd
2018-03-22 15:14:40 -07:00
Treehugger Robot
c0811e291c Merge "Products launching with API 28 need 64-bit binder interface." 2018-03-22 21:52:26 +00:00
Tao Bao
2a8e13f75d Merge "Remove the hard-coded path of build_verity_metadata.py."
am: 0793683e86

Change-Id: I5e181e5473e1727b5dec5951bdcfe12af701ec33
2018-03-22 18:55:12 +00:00
Tao Bao
0793683e86 Merge "Remove the hard-coded path of build_verity_metadata.py." 2018-03-22 18:41:06 +00:00
Tao Bao
f47bf0fecf releasetools: Fix an issue in GetMinSdkVersion.
The following is a buggy pattern that won't capture anything into err.
The issue is benign, since a failed run would be eventually captured by
a subsequent check.

  p = Run(["aapt", ...], stdout=subprocess.PIPE)
  output, err = p.communicate()
  if err:
    raise ...

This CL changes the error detection to be based on the return code from
aapt. It also adds some sanity test to ensure the call to aapt works.
The test app is built from AOSP com.android.cts.ctsshim (chosen mostly
because of its small size).

Test: python -m unittest test_common
Change-Id: I337f141bd0fc5f0801dfc628c601b88b7640789c
2018-03-22 10:27:04 -07:00
Tao Bao
80921986d1 releasetools: Capture and dump stdout/stderr outputs on errors.
For the functions in common.py, capture the stdout/stderr outputs when
shelling out to external tools. Dump the outputs on errors.

Bug: 76123422
Test: Inject errors to signapk.jar / brotli / unzip. Check the outputs.
Change-Id: Ib2d4272528b83a50bb727788cf8a5211e2ddade8
2018-03-22 00:09:33 -07:00
Tao Bao
b4ec6d752a Remove the hard-coded path of build_verity_metadata.py.
build_image.py used to invoke build_verity_metadata.py with a hard-coded
path of 'system/extras/verity/build_verity_metadata.py', which makes it
hard to run unittests from non-$(ANDROID_BUILD_TOP) directory.

This CL adds the dependency on the tool, so that it gets installed to
$(HOST_OUT_EXECUTABLES), then removes the hard-coded path.

Bug: 74544459
Bug: 76015688
Test: `m dist`
Test: python -m unittest test_validate_target_files
Change-Id: I0dcf4eb067a0db6f099cb589eb99a151a05c7f2b
2018-03-21 21:15:59 +00:00
Tao Bao
1824427572 Merge "releasetools: Handle two edge cases in FinalizeMetadata()."
am: 2ebcf419e4

Change-Id: I469411e3870432c8b00a62d94919671d5177beef
2018-03-21 20:39:38 +00:00
Treehugger Robot
2ebcf419e4 Merge "releasetools: Handle two edge cases in FinalizeMetadata()." 2018-03-21 20:29:25 +00:00
Colin Cross
275e8caf92 Merge changes I54b8ec9c,Ibef28507,Ia858de22,Ide3050ec,Ib6e00be4, ...
am: 2a63439a31

Change-Id: Ied8afd9274082eaa3405412421f40d4ab70719a3
2018-03-21 20:26:14 +00:00
Colin Cross
2a63439a31 Merge changes I54b8ec9c,Ibef28507,Ia858de22,Ide3050ec,Ib6e00be4, ...
* changes:
  Add asan-specific CleanSpec
  Use a srcjar for aapt and aapt2 output
  Move R.txt and make aapt2 generate it
  Move aapt2.mk before java.mk
  Remove redundant renderscript_target_api computation
  Move java renderscript support to java_renderscript.mk
  Use srcjar and resource zip as output of renderscript rules
2018-03-21 19:59:41 +00:00
Bowgo Tsai
6c73723b3d Merge "Includes odm SELinux contexts files for recovery"
am: d39eeb44f1

Change-Id: Icc7dd3b0a3e1f4078411fa0953ba5a136ff8e068
2018-03-21 07:18:13 +00:00
Treehugger Robot
d39eeb44f1 Merge "Includes odm SELinux contexts files for recovery" 2018-03-21 07:02:36 +00:00
Isaac Chen
4cf3e65be7 Move emulator fstab to vendor for "system as root"
In P, the root of the system image is used as the root of the
device, so all device (or vendor) spicific files need to be
moved to vendor partition. Relevant configuration files need to
be modified accordingly.

Bug: 76037265
Test: m -j; emulator # booted to home screen and
                     # checked fstab.ranchu is under vendor/etc

Change-Id: I88bfdbc1d7791f6afb64edf352b4898a50b41190
2018-03-21 10:40:44 +08:00
Colin Cross
96b0c0d9d7 Add asan-specific CleanSpec
ASAN builds run as two consecutive builds.  The cleanspec is applied by
the first one, which cleans the obj directory, and ignored by the
second.  This can result in old entries in the obj_asan directory.

Test: obj_asan dirs are deleted
Change-Id: I54b8ec9cff581fcd9ec2a843aa7126805340c387
Merged-In: I54b8ec9cff581fcd9ec2a843aa7126805340c387
(cherry picked from commit 4e2da3277a)
2018-03-20 15:44:43 -07:00
Colin Cross
f6d68cf6fe Use a srcjar for aapt and aapt2 output
Dumping files in the *_intermediates/src causes incremental build
problems, because we don't know when to delete them if they are
no longer generated.  Switch to outputting a srcjar instead, and
pass it in to the turbine and javac compiles.

Bug: 73885582
Test: m checkbuild
Change-Id: Ibef28507522339d930c09426d0c98372eb864cb6
Merged-In: Ibef28507522339d930c09426d0c98372eb864cb6
(cherry picked from commit 2d1cddd567)
2018-03-20 15:26:27 -07:00
Colin Cross
2ae710c29f Move R.txt and make aapt2 generate it
Move R.txt from $(intermediates.COMMON)/src to $(intermediates.COMMON)
in preparation for moving aapt/aapt2 java files out of src.  Also
make aapt2 generate R.txt, and make it an implicit output of the rule
and and input to the aar rule.

Bug: 73885582
Test: m out/target/common/obj/JAVA_LIBRARIES/SettingsLib_intermediates/javalib.aar
Test: m checkbuild
Change-Id: Ia858de226355965b69794c002f841333df16a612
Merged-In: Ia858de226355965b69794c002f841333df16a612
(cherry picked from commit 7c24f8ec85)
2018-03-20 15:26:27 -07:00
Colin Cross
7a28158b90 Move aapt2.mk before java.mk
aapt2.mk will export a srcjar that will be compiled by java.mk, so
it needs to be evaluated first.  Also move java_renderscript.mk
before aapt2.mk so that the generated resoures are available.

Bug: 73885582
Test: m checkbuild
Test: rm -rf out/target/common/obj/APPS/RsHelloCompute_intermediates &&
      m out/target/common/obj/APPS/RsHelloCompute_intermediates/src/R.stamp
Change-Id: Ide3050ec993a945f7077b75e952c3b3532306ede
Merged-In: Ide3050ec993a945f7077b75e952c3b3532306ede
(cherry picked from commit 8528eabbba)
2018-03-20 15:26:27 -07:00
Colin Cross
39772fedde Remove redundant renderscript_target_api computation
renderscript_target_api is set by java_renderscript.mk, it doesn't
need to be computed again in static_java_library.mk and
package_internal.mk.

Bug: 73885582
Test: no change to ninja file
Change-Id: Ib6e00be425c385f4911def19ed5042a740f2c79b
Merged-In: Ib6e00be425c385f4911def19ed5042a740f2c79b
(cherry picked from commit 55cce23360)
2018-03-20 15:26:27 -07:00
Colin Cross
1e047d39a5 Move java renderscript support to java_renderscript.mk
Move java renderscript support to java_renderscript.mk in
preparation to moving it before aapt.  Filter *.rs files
out of LOCAL_SRC_FILES and forbid them in java_common.mk
so that using *.rs files is an error if java_renderscript.mk
was not included.

Bug: 73885582
Test: m checkbuild
Change-Id: Ib37ce89daee58b025dec98f733c5a08149641afd
Merged-In: Ib37ce89daee58b025dec98f733c5a08149641afd
(cherry picked from commit 71822d6fb9)
2018-03-20 15:26:27 -07:00
Colin Cross
dc1e028994 Use srcjar and resource zip as output of renderscript rules
Replace RenderScript.stamp with a srcjar, with a resource zip as
an implicit output.  This makes the renderscript rules self contained,
which will make it easier to move them before aapt.

Bug: 73885582
Test: m checkbuild
Change-Id: I12d33e52019aebac6ea33271939228e4690a4173
Merged-In: I12d33e52019aebac6ea33271939228e4690a4173
(cherry picked from commit e00e2fae52)
2018-03-20 15:25:51 -07:00
Nan Zhang
4db0821c0d Merge "Add dist-for-goals for droiddoc docs.zip"
am: 550a3b6dd9

Change-Id: I214ca3aef755f961723421062b1ace6ebf1b225c
2018-03-20 22:03:45 +00:00
Treehugger Robot
550a3b6dd9 Merge "Add dist-for-goals for droiddoc docs.zip" 2018-03-20 20:04:47 +00:00
Nan Zhang
0aa96a05fb Add dist-for-goals for droiddoc docs.zip
Old droiddoc will install the *-docs.zip & API files to the dist.

And also remove INTERNAL_PLATFORM_*API_FILE variables from config.mk and
get rid of the dependencies on api-stubs, system-api-stubs, test-api-stubs
timestamp.

Test: m -j api-stubs-docs
Bug: b/70351683
Change-Id: I08923bb5f62d2f55961d9a2865723d7b3fae7206
Merged-In: I08923bb5f62d2f55961d9a2865723d7b3fae7206
2018-03-20 11:42:23 -07:00
Julien Desprez
6d948a6cf5 Merge "Add loganalysis to cts"
am: ee9ea1d601

Change-Id: I0f10664f9798757df04a4992d3be1cf8f5ec4cd6
2018-03-20 03:02:35 +00:00
Julien Desprez
ee9ea1d601 Merge "Add loganalysis to cts" 2018-03-20 02:44:20 +00:00
Tao Bao
40feb95108 Merge "Use bsdiff to generate recovery-from-boot.p for system-root-image."
am: ad29fe419e

Change-Id: I064cf6ae89b547dd18061f438362c0d1152b88a7
2018-03-20 00:19:13 +00:00
Treehugger Robot
ad29fe419e Merge "Use bsdiff to generate recovery-from-boot.p for system-root-image." 2018-03-20 00:00:34 +00:00
jdesprez
4619031efe Add loganalysis to cts
Tradefed depends on Loganalysis so it should also be part
of cts. Luckily there was no overlap (cts never called a
part of TF that use loganalysis) so it never failed
with classNotFound before.

Test: build cts
Bug: 75979908
Change-Id: I2542ecd077149f7165aff120d67054943df20b54
2018-03-19 16:47:22 -07:00
Tao Bao
6d5d623987 Use bsdiff to generate recovery-from-boot.p for system-root-image.
This CL handles a path that uses system-root-image on non-A/B device.
For this path, we can't generate recovery-from-boot patch with imgdiff,
because boot/recovery images contain different number of entries (only
recovery iamge has ramdisk image now).

Using BOARD_USES_FULL_RECOVERY_IMAGE can work around the issue, at the
cost of extra size. Compared to carrying full recovery image, this CL
saves the cost of the kernel size, by putting a patch that's roughly the
size of the recovery ramdisk.

The applypatch executable already detects and handles a bsdiff patch
automatically. No change required to that end.

Note that it won't further reduce the patch size by handling that
ramdisk entry specially, because (a) that's the only difference between
the two images; and (b) there's no corresponding data in boot image to
be diff'd against.

Bug: 72731506
Test: `m dist` with aosp_angler-userdebug. Check the device can install
      recovery image successfully (after intentionally corrupting the
      recovery image).
Test: Build aosp_angler-userdebug with BOARD_BUILD_SYSTEM_ROOT_IMAGE set.
      Verify the generated patch files.
Test: Run validate_target_files.py with the target_files.zips.
Change-Id: I69c06f51ba8c39ae059c5e9a6872a9f10600cf17
2018-03-19 14:13:02 -07:00
Colin Cross
70e4074aa8 Merge "Provide testcases directory for host cross out (windows) modules"
am: 238bb91678

Change-Id: Ia79d4709be1426636527da40fa7a0f39d3f521d9
2018-03-19 20:55:51 +00:00
Colin Cross
f434e35a47 Merge "Replace extract_srcjars.sh with zipsync"
am: c9907cc8f6

Change-Id: Ib99ea1ef3a51cf34a3494453f5ebefe7f2059203
2018-03-19 20:54:49 +00:00
Colin Cross
238bb91678 Merge "Provide testcases directory for host cross out (windows) modules" 2018-03-19 20:40:50 +00:00
Tao Bao
3bf8c65029 releasetools: Handle two edge cases in FinalizeMetadata().
In FinalizeMetadata and PropertyFiles, we need to reserve space between
the calls to Compute() and Finalize(). We used to put a 10-byte
placeholder, in the hope of covering the 'offset:length' space for the
metadata entry, as well as the possible value changes in other entries.

However, this could fail in two possible cases: (a) metadata entry
itself has a large offset (e.g. staying near the end of a 1-GiB package,
where the offset itself has 10-digit); or (b) the offsets for other
entries change substantially due to entry reordering. Note that for case
(b), it's space inefficient to always reserve 15-byte for _each_ token
in the property-files.

This CL handles both of these two cases. For (a), we bump up the 10-byte
to 15-byte, which is large enough to cover a package size up to 10-digit
number (i.e. ~9GiB) with a metadata entry size of 4-digit. All these
15-byte will be used for the metadata token alone.

For (b), we add a fallback flow that would retry one more time, but
based on the already signed package that has entries in desired order.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate aosp-bullhead full OTA with '--no_signing' flag.
Change-Id: If20487602d2ad09b3797465c01972f2fa792a1f1
2018-03-19 13:35:38 -07:00
Colin Cross
c9907cc8f6 Merge "Replace extract_srcjars.sh with zipsync" 2018-03-19 20:33:08 +00:00
Tao Bao
f13591bbb3 Merge "releasetools: Add NonAbOtaPropertyFiles for non-A/B OTA packages."
am: 93a84dd9a6

Change-Id: I0636d2d29410f980681e463aff9eacc4e4f71678
2018-03-19 19:59:50 +00:00
Tao Bao
93a84dd9a6 Merge "releasetools: Add NonAbOtaPropertyFiles for non-A/B OTA packages." 2018-03-19 19:42:24 +00:00
Søren Gjesse
85ac8c6e39 Merge "Enable R8 by default (second attempt for Q)"
am: d04f28db7b

Change-Id: Ib700a57f75ad3e7ce780faa0fcf74684aee59816
2018-03-19 07:51:47 +00:00
Søren Gjesse
d04f28db7b Merge "Enable R8 by default (second attempt for Q)" 2018-03-19 07:27:20 +00:00
Tao Bao
c0746f4e94 releasetools: Add NonAbOtaPropertyFiles for non-A/B OTA packages.
This CL exposes ota-property-files flag for non-A/B OTA packages.
Currently the line only contains the info for the METADATA entry, for
example "ota-property-files=metadata:69:286". This allows system updater
to just download the METADATA entry, as opposed to downloading the
entire package, to learn about the info regarding the OTA package (e.g.
post-OTA build fingerprint). Note that this requires the OTA server-side
support to pass down the flag along with the update URL.

Bug: 74210298
Test: python -m unittest test_ota_from_target_files
Test: Generate a non-A/B package and check the property-files string.
Change-Id: I1482c587e18ea7101c8328777ea988c2d8ca06ac
2018-03-18 11:59:38 -07:00
Colin Cross
e887ccbf12 Provide testcases directory for host cross out (windows) modules
If HOST_CROSS_OUT_TESTCASES is not defined, then a module that
builds for windows and has a test config will attempt to install
the config file to /<module>/<module>.config.

Fixes:
[ 54% 99/183] Copy: /simpleperf_unit_test/x86/simpleperf_unit_test.exeninja: error: mkdir(/simpleperf_unit_test): Permission denied

Test: m simpleperf_unit_test
Change-Id: I07dcecbce9c710b29fec5fbdedc462be3245b578
2018-03-17 05:14:13 +00:00