Commit graph

5763 commits

Author SHA1 Message Date
Tao Bao
a57ab9f210 releasetools: Add test cases for common.LoadInfoDict().
Fixes: 113073663
Test: python -m unittest test_common
Change-Id: I352a2731849d175f94e0f23dd68b6cf28e554f64
2018-08-27 18:18:10 -07:00
Tao Bao
71142ea2c9 Merge "releasetools: Use int value of FIXED_FILE_TIMESTAMP." 2018-08-27 19:39:03 +00:00
Tao Bao
e30a6a6df9 releasetools: Use int value of FIXED_FILE_TIMESTAMP.
Otherwise it gives a floating-point value (i.e. 1230768000.0). e2fsdroid
(i.e. the user of the value) didn't complain though, because it calls
strtoul(3) without checking endptr.

Bug: 80600931
Test: `m dist` with aosp_taimen-userdebug, check the build log.
Change-Id: I7467a60c947134b42f079c7a57fb8a9a9d4277da
2018-08-27 11:03:55 -07:00
Tao Bao
687da3b0a5 releasetools: Fix the path to the OTA keys in recovery image.
The recovery image will be packed under BOOT/RAMDISK only if
system_root_image and recovery_as_boot both are true (e.g. non-A/B
devices launched since P).

Bug: 113191245
Test: Run sign_target_files_apks.py on a target_files file that uses
      system-as-root but not recovery-as-boot.
Change-Id: I262a268055c6b5078d21694b5094a1c393d0d37c
2018-08-26 14:39:58 -07:00
Tao Bao
22f28aa1b5 Merge "A/B update: Replace the zip FileHeader mechanism for update package" 2018-08-24 19:54:10 +00:00
Tao Bao
b4adc06732 releasetools: Look for recovery.fstab at both locations.
The change in [1] moved the recovery etc files from /etc to /system/etc.
However, we may use the latest OTA tools to build incremental OTAs for
old target_files zips. This CL adds a workaround to look at both of the
old and new locations.

[1] commit 696bb33676

Bug: 113073663
Test: Build a previously failing incremental OTA for taimen.
Change-Id: Ie07aa1713e616d523838b1260a992b20f5a11612
2018-08-22 18:41:49 -07:00
Shashikant Baviskar
338856f92b A/B update: Replace the zip FileHeader mechanism for update package
When the update package gets larger than 2 GiB, payload.bin offset
mentioned in metadata file for ota-streaming-property-files gets
shifted (CrAU of payload.bin) because ZipInfo FileHeader() returns
incorrect value. To solve the issue, offset is re-calculated from
fixed bytes of central directory file header, filename length and
extra length.

This patch is to sync with update_device.py script.

Test: manually create an A/B update package and run it using
      update_device.py
Bug: 111198589

Change-Id: I9bf5a5ca24938cad3206d04af529f70d45e992c0
2018-08-23 10:13:20 +09:00
Tao Bao
696bb33676 Install recovery etc files to recovery/root/system/etc.
They used to be installed under recovery/root/etc. This CL moves the
files to the new location and creates a symlink from /etc to /system/etc
(done by the rule in system/core/rootdir). This gives similar layout
between normal boot and recovery, and allows installing prebuilt_etc
files with Soong (`recovery_available: true`).

As part of the change, we no longer need the whitelisting rule for
mke2fs.conf.

Bug: 112780007
Test: Build with other changes in the topic (aosp_taimen-userdebug).
      Check the generated files under recovery (/etc being a symlink to
      /system/etc).
Test: Boot into recovery. Verify basic functionalities (`adb shell` and
      `adb sideload`, factory reset).
Test: `build/soong/build_test.bash --dist`
Change-Id: Ibb6dea6f179a339f0c2d0fd8ba05ec0085b79a12
2018-08-21 10:30:49 -07:00
Yifan Hong
d35fdd8b3d Merge changes from topic "product-service-remove-hyphen"
* changes:
  s/product-services/product_services/g
  Reland "s/PRODUCT-SERVICES/PRODUCT_SERVICES/g"
  Update CleanSpec to remove generated files from IVold.aidl.
2018-08-21 01:07:56 +00:00
Chih-Hung Hsieh
f77c7f75e5 Use multiple globs/emails in per-file syntax
Bug: 112259427
Test: tools/checkowners.py -c -v OWNERS */OWNERS */*/OWNERS
Change-Id: I0e2c2c64a23df360fbe7691c4336ca04099fba13
2018-08-20 14:16:04 -07:00
Dario Freni
924af7d4dd s/product-services/product_services/g
Attempting to reduce the number of different spellings we have for
"product services" partition in the codebase.

Bug: 112431447
Test: m
Change-Id: I6debd3f5f5abc4bd1e22f2ef11c5fe131915976c
Merged-In: I6debd3f5f5abc4bd1e22f2ef11c5fe131915976c
2018-08-20 10:44:59 -07:00
Yifan Hong
35be6ca8f2 Reland "s/PRODUCT-SERVICES/PRODUCT_SERVICES/g"
Test: builds
This reverts commit 87021065b9.

Reason for revert: reland CL

Change-Id: Ic88c2bd612ad7a19d88686b2907c812bd586ee35
Merged-In: Ic88c2bd612ad7a19d88686b2907c812bd586ee35
2018-08-20 10:44:59 -07:00
Treehugger Robot
68883e0717 Merge "Revert "s/PRODUCT-SERVICES/PRODUCT_SERVICES/g"" 2018-08-17 20:32:18 +00:00
Shawn Yang
87021065b9 Revert "s/PRODUCT-SERVICES/PRODUCT_SERVICES/g"
This reverts commit 77c778c040.

Reason for revert: git-master break on 4960355

Change-Id: I2fd272c63ef496067f12852bcf98ffc1194c54bf
2018-08-17 19:02:08 +00:00
Tianjie Xu
ccbae48ac5 Generates the care_map.txt in protobuf format
Call the host binary care_map_generator to generate the care_map in
protobuf format. The parsing part of proto messages has already been
supported in the update_verifier.

Bug: 77867897
Test: unittests pass; run add_image_to_target_files
Change-Id: I40d3184b4b5c48a6dd55203afc84cca73d5765e9
2018-08-17 17:39:00 +00:00
Dario Freni
77c778c040 s/PRODUCT-SERVICES/PRODUCT_SERVICES/g
Attempting to reduce the number of different spellings we have for
"product-services" in the codebase.

Bug: 112431447
Test: m
Change-Id: Iec531122297cf4b86ff8601732d4ab7ccf5c2183
2018-08-17 00:28:17 +01:00
Treehugger Robot
a5985593c4 Merge "findleaves: Fix typo for --dir argument" 2018-08-16 04:39:42 +00:00
Treehugger Robot
c272a99b5c Merge "Add support for /odm partition" 2018-08-16 03:37:36 +00:00
Paul Keith
8368d23d47 findleaves: Fix typo for --dir argument
Change-Id: I9b78c304d27dd652ef49f72fa6ad9260e4ce605b
2018-08-16 02:30:45 +02:00
Christopher Ferris
9f7e0cff7f Merge "Remove micro_bench." 2018-08-13 22:11:46 +00:00
Chih-Hung Hsieh
9b2bf2d55b Fix print format error.
Bug: 112478836
Test: build with WITH_TIDY=1
Change-Id: I0b9239be7c73cceb0b525c091a9b3a25add31204
Exempt-From-Owner-Approval: fix broken build on mac
2018-08-11 11:33:43 -07:00
Chih-Hung Hsieh
0c0d92887c Fix or suppress some google-runtime-int warnings.
Bug: 112478838
Test: build with WITH_TIDY=1
Change-Id: Ie3b15a85770b5806cddfd49a475aaf0553f9ec78
2018-08-10 15:14:26 -07:00
Bowgo Tsai
d624fa63d5 Add support for /odm partition
This CL is largely an adaptation of Change-Id
I5d229f6ac729ea6df9ff1f14cee2e28972cd9b4d

tools/vendor_buildinfo.sh is also renamed to tools/device_buildinfo.sh.
The caller then can invoke device_buildinfo.sh "vendor" | "odm" to
generate properties for vendor.img and odm.img, respectively.

It adds the following variables:

- BOARD_AVB_ODM_KEY_PATH
- BOARD_AVB_ODM_ALGORITHM
- BOARD_AVB_ODM_ROLLBACK_INDEX_LOCATION
- BOARD_ODMIMAGE_FILE_SYSTEM_TYPE
- BOARD_ODMIMAGE_EXTFS_INODE_COUNT
- BOARD_ODMIMAGE_EXTFS_RSV_PCT
- BOARD_ODMIMAGE_PARTITION_SIZE
- BOARD_ODMIMAGE_JOURNAL_SIZE
- BOARD_ODMIMAGE_SQUASHFS_BLOCK_SIZE
- BOARD_ODMIMAGE_SQUASHFS_COMPRESSOR
- BOARD_ODMIMAGE_SQUASHFS_COMPRESSOR_OPT
- BOARD_ODMIMAGE_SQUASHFS_DISABLE_4K_ALIGN
- BOARD_PREBUILT_ODMIMAGE
- BOARD_USES_ODMIMAGE
- LOCAL_ODM_MODULE
- PRODUCT_ODM_BASE_FS_PATH
- PRODUCT_ODM_VERITY_PARTITION
- PRODUCT_ODM_PROPERTIES
- TARGET_COPY_OUT_ODM
- TARGET_OUT_ODM
- TARGET_OUT_ODM_*

Bug: 64195575
Test: boot a Taimen with existing images

Test: `make odmimage` with
      - BOARD_AVB_ENABLE := true
      - BOARD_ODMIMAGE_PARTITION_SIZE := 62914560
      - BOARD_ODMIMAGE_FILE_SYSTEM_TYPE := ext4
      - TARGET_COPY_OUT_ODM := odm
      - PRODUCT_ODM_PROPERTIES += odm.test.build=success

Test: `make odmimage` with
      - BOARD_ODMIMAGE_PARTITION_RESERVED_SIZE := 10485760
      - BOARD_ODMIMAGE_FILE_SYSTEM_TYPE := ext4
      - BOARD_AVB_ENABLE := true
      - TARGET_COPY_OUT_ODM := odm
      - PRODUCT_ODM_PROPERTIES += odm.test.build=success
      - PRODUCT_USE_DYNAMIC_PARTITION_SIZE := true

Change-Id: I4dea7b567ec49a766c7a4683decaf81c7e921d55
2018-08-08 18:59:44 +08:00
Christopher Ferris
e027dd4aef Remove micro_bench.
micro_bench has been deprecated, so remove it from the list of executables
to be built by default.

Bug: 26928775

Test: Builds.
Change-Id: I701b3077ec00c7e75145aeb52463f47bf8ff2f09
2018-08-07 19:57:56 +00:00
Neil Fuller
4f2051b1bb Merge "Make host targets use non-repackaged targets"
am: 09c81c303a

Change-Id: I5606474b66c13dba18987163e7ff0dc184f3f497
2018-08-05 08:12:39 -07:00
Treehugger Robot
09c81c303a Merge "Make host targets use non-repackaged targets" 2018-08-05 15:05:48 +00:00
Isaac Chen
13d3426f3c Merge "Work around e2fsck issue for building sparse GSIs"
am: a0d3277b8d

Change-Id: I53b45da39145cb10f37bd7bcede55f13cfed801b
2018-08-03 17:07:18 -07:00
Treehugger Robot
a0d3277b8d Merge "Work around e2fsck issue for building sparse GSIs" 2018-08-03 23:29:33 +00:00
Isaac Chen
ec7fa1cd09 Work around e2fsck issue for building sparse GSIs
When trying to support sparse format for GSIs, aosp/720949, a bug in
e2fsck was triggered, and hence failed the build for sparse GSI.

This workaround disables the detection of ASAN ODR so we can build
sparse GSIs successfully like we previously did for the vendors,
some of which can't handle unsparse images.

Bug: 112062612
Test: $ lunch aosp_x86_64-eng
      $ m SANITIZE_HOST=address ASAN_OPTIONS=detect_odr_violation=0 -j
      $ emulator

Change-Id: Ifafd27aa1e57d785b914f78a4d9bfef6a0c33fc9
2018-08-03 11:51:08 +08:00
Yifan Hong
635887d5ac Merge "target_files: copy to PRODUCT-SERVICES instead"
am: 567df6214f

Change-Id: I95bf48eba07daa2b0f6d5b84c45de9b22f7aac0b
2018-08-02 15:31:56 -07:00
Yifan Hong
567df6214f Merge "target_files: copy to PRODUCT-SERVICES instead" 2018-08-02 21:53:57 +00:00
Bryan Henry
a2bf1e5c5c Merge "Fix 2009-01-01 timestamps in releasetools to always be UTC"
am: e118d5bc4a

Change-Id: I12c8cc0664306ba116076b3f6d43e1c01fb9120f
2018-08-01 17:24:41 -07:00
Bryan Henry
e118d5bc4a Merge "Fix 2009-01-01 timestamps in releasetools to always be UTC" 2018-08-02 00:16:06 +00:00
Yifan Hong
4d6b38c47a target_files: copy to PRODUCT-SERVICES instead
The add_img_to_target_files.py script uses upper() to determine
the name of sub-directory under target_files_intermediates. The
name of the image is product-services.img but the sub-directory
name was PRODUCT_SERVICES. Change it to PRODUCT-SERVICES so that
'm dist' works.

Test: m dist
Change-Id: I8cad841c674b5d62c1d803d0383fc3aee14f272a
2018-08-01 16:46:16 -07:00
Yifan Hong
9a7d525849 Merge "Split PRODUCT_USE_LOGICAL_PARTITIONS into sub-flags."
am: d1c308bf40

Change-Id: I834e75dcfc04af0dd8655027bfc251c3d0162815
2018-08-01 11:39:59 -07:00
Bryan Henry
e6d547d53a Fix 2009-01-01 timestamps in releasetools to always be UTC
The usage of datetime.fromtimestamp previously resulted in the build or
signing machine's local timezone affecting the Unix timestamp ultimately
applied to images generated by add_img_to_target_files. The go/ab build
outputs would use 2009-01-01 00:00 UTC, for example, but local builds
and the release signed images (generated through go/ab-sign) would use
2009-01-01 00:00 PST. This change makes the timestamps always use UTC.

Bug: 80600931
Bug: 80093599
Test: 'm -j droid dist' and verified timestamps in resulting
target_files zip.

Change-Id: Ic2a19591519850c249f78254e1464aa6839bfc6c
2018-07-31 18:32:00 -07:00
Yifan Hong
2dae5721d4 Split PRODUCT_USE_LOGICAL_PARTITIONS into sub-flags.
Split the flag into PRODUCT_USE_DYNAMIC_PARTITION_SIZE
and PRODUCT_BUILD_SUPER_PARTITION. More sub-flags can be
added with the same way.

Also change some checks with USE_LOGICAL_PARTITIONS with
the appropriate sub-flags.

This allows easier device bring-up to fulfill the requirements
separately.

This also enables dynamic partition size without logical patition.
Originally, to enable BOARD_*_PARTITION_RESERVED_SIZE must also
enable PRODUCT_USE_LOGICAL_PARTITIONS. The patch fix the rule to
let dynamic partition size is able to be used independently.

Fixes: 111966003

Test: make superimage with PRODUCT_USE_LOGICAL_PARTITIONS
Change-Id: I45b29c87bf94356d8416bbd58a14c8982039c222
2018-08-01 00:39:13 +00:00
Hung-ying Tyan
3868a7fd9a Merge "Add more ro.vendor.build.*"
am: 2e0fb33af8

Change-Id: I9357d2e338198eb59ad7a9472bd82ca75177412a
2018-07-31 01:13:20 -07:00
Treehugger Robot
2e0fb33af8 Merge "Add more ro.vendor.build.*" 2018-07-31 08:07:57 +00:00
Yifan Hong
8883a739b3 Merge changes I1459d62f,I4c2b975e
am: bb937a6b69

Change-Id: Ifdedee1b0251bc9c2a8baf30d28b5d315b5c9e20
2018-07-30 11:35:33 -07:00
Yifan Hong
37c0c7c82d Build image for super partition
... for bootstrapping / initializing the device.
Image is built to $(PRODUCT_OUT)/super.img when running
`m dist`. For A/B devices, the image contains other
partitions in the _a slot.

Change-Id: I1459d62f02b95f142dfb3b7608f88ec6801dbf37
Fixes: 111758129
Test: m superimage -j
2018-07-27 13:45:43 -07:00
Yifan Hong
ebc041af56 productservices => product_services
Make names consistent.
Some things that does not change:
- productservicesimage target and intermediates

Test: build product services image
Bug: 111609632
Change-Id: I4c2b975e194577aad3d51b908b103880625deb5e
2018-07-26 17:19:02 -07:00
Tao Bao
0adc598d82 Merge "releasetools: Fix an issue in image size computation."
am: d578232c95

Change-Id: Idd139fe594e95cd92926da42beb670be239ee113
2018-07-26 12:53:51 -07:00
Tao Bao
4f35d47160 Merge "releasetools: Return the actual image size when building logical partitions."
am: f42fc1a7c3

Change-Id: I714fbd9458c2d6708ade24b36cbfd965a135b840
2018-07-26 10:09:53 -07:00
Tao Bao
c2606eb548 releasetools: Fix an issue in image size computation.
When building a system image with system_root_image enabled, the size
computation should include files under both of in_dir (i.e. /system
files) and root (pointed by 'root_dir'). Because files from both
locations will end up into the built image. The files under root are
usually only a few MiBs, but should be accounted for especially in the
context of logical partitions (where the partition size will be
allocated based on the actual need). Note that we will still need some
"reserved space" (defined via BOARD_*_PARTITION_RESERVED_SIZE) to cover
the cost for filesystem and/or verity metadata.

This CL moves the combination of the two dirs up, before parsing and
computing other properties. This doesn't affect anything for a
successful image building path. It may however increase the time to
error out in certain error path, since it copies the files earlier now.

Test: python -m unittest test_build_image
Test: `make dist`
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
      BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Build system
      image successfully.
Test: Setup a target with PRODUCT_USE_LOGICAL_PARTITIONS == true and
      BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE == 20MiB. Write a large
      file to root dir (PRODUCT_OUT_ROOT). The image building fails, but
      reporting a size that accounts for both of /system and root.
Change-Id: Idfb26b8e259626ba57ec3bd4f85d357c30e56163
2018-07-26 16:59:57 +00:00
Tao Bao
f42fc1a7c3 Merge "releasetools: Return the actual image size when building logical partitions." 2018-07-26 16:59:36 +00:00
Hung-ying Tyan
c565f5c2ed Add more ro.vendor.build.*
to facilitate independent updates of vendor/odm images.

Bug: 78600951 (#56,#57)
Bug: 111491184
Test: build aosp_marlin-userdebug and check vendor/build.prop

Change-Id: I798c13822695259375fbc8d5c7fe08d43a540e46
2018-07-26 17:39:47 +08:00
Tao Bao
4251fe97aa releasetools: Return the actual image size when building logical partitions.
When verity is enabled, `partition_size` has the "adjusted" partition
size for holding the filesystem files (i.e. excluding verity hashtree,
and FEC metadata if applicable), whereas `original_partition_size` has
the one for the actual image returned by build_image.py.
`partition_size` must be smaller than `original_partition_size`. As a
result, the later assertion in Makefile (e.g. assert-max-image-size) may
fail to hold when using `partition_size`.

This CL addresses the issue by returning the value in
`original_partition_size` instead, if available. It also changes to
include both values when dumping the size info on image creation
failure.

Fixes: 79106666
Test: `m dist` with aosp_walleye-userdebug
Test: Set up walleye to use logical partitions, with a reserved size of
      20MiB. `m systemimage` no longer fails.
Test: Set up walleye that doesn't use verified boot. Use `m systemimage`
      to build an oversized image. Check the error message.
Change-Id: I41d77bffa85e8eadfbf4a2a7dcd2e798fadfdab7
2018-07-24 11:57:10 -07:00
Tao Bao
fdcb3b5829 Merge "releasetools: Support verifying AVB signed images with chained partitions."
am: 78078646e9

Change-Id: If7656208f92fde69dad358d298cd1c9eb1002ecd
2018-07-24 11:53:57 -07:00
Tao Bao
02a0859b3c releasetools: Support verifying AVB signed images with chained partitions.
For example, verify a target_files.zip that has system AVB-signed as a
chained partition.

  $ build/make/tools/releasetools/validate_target_files.py \
      signed-target_files-4904652.zip \
      --verity_key verifiedboot_pub.pem \
      --avb_system_key_path system_pub.pem

Note that verifiedboot_pub.pem should be the key (either public or
private) to verify vbmeta.img, and 'system_pub.pem' should be the key
(either public or private) for the chained partition of system.

testdata/testkey.key is the private key converted from
testdata/testkey.pk8 for testing purpose (`openssl pkcs8 -in
testdata/testkey.pk8 -inform DER -out testdata/testkey.key -nocrypt`).

Bug: 63706333
Test: python -m unittest test_common
Test: python -m unittest test_add_img_to_target_files
Test: `m dist` on aosp_walleye-userdebug; Run validate_target_files.py
      on the generated target_files.zip.
Test: Set up walleye with chained system partition; `m dist`; Run
      validate_target_files.py on the generated target_files.zip.
Change-Id: I38517ab39baf8a5bc1a6062fab2fe229b68e897d
2018-07-23 15:20:14 -07:00