Commit graph

85 commits

Author SHA1 Message Date
dianlujitao
33ee59662f extract_utils: Drop string after semicolon when parsing destination
This fixes parsing when arguments contain colons, a typical usecase
would be:
-vendor/app/TimeService/TimeService.apk;:timeservice_app_cert

Change-Id: I7500ae09632632ddc10734d9b1df267e28286b67
2020-01-10 15:19:47 +01:00
Mohd Faraz
d1d72351b2 extract_utils: Added Suffix uses COMMON_SUFFIX
* COMMON_JAVA_PACKAGE_SUFFIX for jar

 * COMMON_ANDROID_PACKAGE_SUFFIX for apk

Change-Id: I812405dac12ef7183985c66a6e43b0ea5f85989c
Signed-off-by: Mohd Faraz <mohd.faraz.abc@gmail.com>
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
34b5cdc1e3 extract_files: Implement blueprint file generation
Switch to blueprint on:
 - shared objects
 - $partiton/etc/ files
 - JARs
 - executable binaries and scripts
 - APKs

Only /sbin binaries are still in Android.mk because blueprint
doesn't handle sbin installation yet

Change-Id: I1dfd7e8bb575367b2a7fa9e333c4c6fa3aa68180
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
42752d4d4e extract_files: Add initial blueprint creation
Start the copyrights for these at 2019 because the blueprints won't
have existed before then.

Change-Id: I5300109949785dd19555e35d17a1e80daef498b0
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
8aae50df3c extract_files: Match non-treble path prefixes
Some devices put stuff on /system, /system/vendor or even
/system/vendor/odm. Search for these paths too when generating
TARGET_COPY_OUT_$partition variables.

Change-Id: Ie2c087e57aaca02d5ea93f290d5fc50d1315a600
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
8de76b204b extract_files: Support system/ prefixes
With support for 4 independent partitions now, we seriously
need to start putting /system blobs in their own directory.
Add support for file lists with system/ prefixes while
maintaining support for old file lists without it.

Also, TARGET_COPY_OUT_SYSTEM is a thing now, and all devices,
regardless of treble or not, set TARGET_COPY_OUT_$partition
so let's get rid of the treble compat option and default it
to true.

Change-Id: I5b798d293768d7c1e16db3ba01e2de3e083088d7
2019-10-27 11:24:06 +00:00
PIPIPIG233666
17d7dcba43
extract_utils: Fix typo
* ./../../xiaomi/sm6150-common/../../../vendor/lineage/build/tools/extract_utils.sh: line 1: /#!/bin/bash: No such file or directory.

Signed-off-by: PIPIPIG233666 <2212848813@qq.com>
Change-Id: I178f745d4ecb818c38706ff100611df19221065d
2019-09-29 12:49:44 -04:00
Bruno Martins
8194b8e583 extract_utils: Source the prebuilts from the new repo
The prebuilt tools now live in prebuilts/tools-lineage.

Change-Id: I4022210c52e1002aae0c6e622a480e5c7e10c366
2019-09-29 03:06:49 +02:00
Rashed Abdel-Tawab
08e3a27135
extract_files: Add ODM partition blob support
Change-Id: Id94bbef879b6f4bcc20645c259981f9d163649c9
2019-09-27 06:46:44 -07:00
razorloves
b5c2c96a56 extract_utils: Support extracting from product partition
Change-Id: I2d0d01253fdf93d63a26556208d45cd69e76329f
(cherry picked from commit ea15be6e92e3be0c103948d220de64fd2e1d781f)
2019-09-10 11:28:05 +02:00
Rashed Abdel-Tawab
19c36cd7a6 extract_utils: Support multidex
Co-authored-by: Erfan Abdi <erfangplus@gmail.com>
Change-Id: I4881658d303b6c5c7f0b141beb50c1e9100af611
2019-07-18 07:31:36 +02:00
Rashed Abdel-Tawab
1c29c373dd extract_utils: Add functions to extract vendor blobs from vendor.img
Change-Id: I6f37adc955856fa78e719eb30249800abe80020c
2019-06-26 04:37:44 +02:00
Vladimir Oltean
4818c230b3 extract_utils: introduce support for executing blob fixups
* Traditionally, the task of hex-editing blobs has been approached in 2 ways:
  (1) Do it out-of-band, commit the modified blob, and record its edited
      sha1sum in proprietary-files.txt (aka pin it).
  (2) Do it in-band, by adding code to the device-level extract-files.sh
      (usually this performs patchelf or sed). This code runs after the
      extract_utils functions were invoked.
* Problems of approach (1):
    - It relies on verbal (basically commit message) documentation of
      the hex-editing that was done. Makes it more difficult to reproduce.
    - Each time blobs are updated, pinning needs to be temporarily removed,
      hex-editing done again manually and new hash put back.
* Problems of approach (2):
    - It is incompatible with the concept of pinning, which is useful
      for kanging blobs from another device. A pinned blob would either:
        - Match the hash, get hex-edited, then it won't match the hash
          next time around.
        - Not match the hash (because of, say, hex-editing), then the
          extraction script would use an unwanted blob version instead of the
          pinned one (either that, or say "!! file not found in source").

* In summary, this patch adds system-wide support for approach (2) in order
  to address the aforementioned shortcomings.

* At device level, users of extract_utils who wish to perform blob
  fixups can override a blob_fixup() Bash function in their
  extract-files.sh immediately after running "source ${HELPER}". The
  blob_fixup() function will be called by the common extract() function
  after extracting every individual blob, giving the user the
  opportunity to hook custom code after this operation takes place.

* In proprietary-files.txt, the line corresponding to this blob which
  needs fixups can look in one of 2 ways:

(a) vendor/lib64/vendor.qti.gnss@1.0_vendor.so

    Do this if you are taking the blob from the stock ROM. The fixup
    script will always run after the blob is extracted.

(b) vendor/lib64/vendor.qti.gnss@1.0_vendor.so|249c76153f8de014bf2dd2ab623ee3d87741fbc8|f7e9ee8e3804887a2f3939128e860767e6f27258

    Do this if you are kanging the blob from somebody else. The pinning
    logic now applies for both the pre- and the post-fixup hashes. The
    fixup script will only run if the blob doesn't match the hex-edited blob,
    although the fixup script should really be idempotent.

Change-Id: Ifdd73c885d995c645f6210597537693d1a2f903f
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:44:53 +02:00
Vladimir Oltean
5238ba8fe5 extract_utils: make --section match words anywhere in the line
Change-Id: Ia3fe2160f0f4bc7da1b5bd31a21080af51b41fab
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
bec92ebaec extract_utils: introduce -k (kang mode) option
* The use case is easier updating of pinned blobs. When --kang is set,
  pinning is automatically ignored, and the script will print lines at
  its output that can be directly copied back into the
  proprietary-files.txt.
* Best served together with --section ${SECTION}, and proper grouping
  of the proprietary-files.txt.

Change-Id: I648fbcbd4580a4a002b00828bcfee18d1e265d7b
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
c503446ec3 extract_utils: cleanup variable names and arguments passed to extract() function
* This also makes the --section argument non-positional, since otherwise
  it is not possible to easily support more than one optional positional
  argument. This is in preparation of one more optional argument to come
  in a follow-up patch: --kang.

Change-Id: Ieb142e0854319defb9a278ab68cd4aeefd0fbdd5
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
78d690d12c extract_utils: be compatible with system-as-root layouts
* The use case is that if you have the following layout:
  $TOP --- system.img
       |
       +-- vendor.img
  you should be able (from $TOP) to:
    mkdir system; mount -o ro,loop system.img system
    mkdir vendor; mount -o ro,loop vendor.img vendor
  and then (from device tree)
    ./extract-files.sh $TOP

  But this doesn't work if system.img is SAR and contains another
  "system" dir inside. This patch makes sure it searches for a "system"
  dir in the provided path as well, if it couldn't find the blob
  anywhere else.

Change-Id: Ib49cd5b587b3a57478a66ff69cf840270c2b1403
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
4fe79b11c1 extract_utils: drop the "/system/" prefix from prints of blob names
* This makes the printed output closer to the proprietary-files.txt syntax

Change-Id: I81b844bb6bb1d1a2f91a39151a892fbfc0bed20b
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
b2c382193d extract_utils: fix mismatch with non-lowercase pinned sha1sum
Change-Id: I86421ca69007ad1992aa67deedf7d2e32bbe4dba
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-01-17 11:26:34 +01:00
Vladimir Oltean
b8084ec8a5 extract_utils: avoid doing stuff on files that don't exist
* If an apk/jar doesn't exist, the script would still try to deodex it.
* If an xml doesn't exist, the script would still try to "fix" it.
* Take it easier, man, it's not your fault.

Change-Id: I3061fb48b403da5121e3c17dd9ecdb6cd148bf97
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-12-24 01:32:53 +02:00
Luca Stefani
99a66bffdc extract_utils: Redirect vdexextractor log to /dev/null
Change-Id: I55028997c0396d57c024e6c42a53907f6d1510c8
2018-11-13 22:06:17 +01:00
Luca Stefani
f6096e989a Add CDEX support for boot JAR files
Change-Id: I0039256c883a58c5a5125fd9bff6e6ec3b98688d
2018-10-21 10:28:37 +02:00
TheStrix
5aad07422e extract_utils: Fix output dex file name generated by vdexExtractor
- Reflect the changes required by:
   ff58255706
 - Also fix JARVDEX file path for deodexing jars

Change-Id: Ief355b0bd9e69a67b552d0bd2060f43adb39fb5f
2018-10-07 12:17:35 +02:00
codeworkx
1c29bf66dd extract_utils: add CDEX support via compact_dex_converter
Source: https://github.com/anestisb/vdexExtractor#compact-dex-converter

Change-Id: I63664cae02e51a05e5a62c0c5c0595eead6c6513
2018-10-05 18:25:46 +02:00
Luca Stefani
a9a9cc730c extract_utils: Newline love for adb extraction
Change-Id: Ibbf7a03ed6920d556ed7658c277f37b02d8876b8
2018-07-12 21:28:21 +02:00
Vladimir Oltean
d577325ae2 extract_utils: make get_file() able to search paths with and w/o /system prefix
Change-Id: I75eba627646caa8ceb8a30575124c57e5ee9445c
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
5f15e3e63c extract_utils: extract(): fix src not being searched in "src:dst;args" spec
* Root cause of the issue is improper naming of variables. Turns out,
  there was no variable to even denote where the blob should have been
  searched for, at "src".
* Previously there was one such variable, suggestively called "TARGET", that
  was desperately trying to serve as both, depending on who +2d hacks harder.
* One such example is "c982836 extract_utils: Fix makefile generation issues".
  That patch deliberately trimmed the "src:" from a spec (therefore
  obviously breaking the search at src) but enabling the searching at
  dst, via the good-for-all TARGET variable.
* This patch introduces the following variables:
  - SRC_FILE: absolute path corresponding to SPEC_SRC_FILE in the
    Android filesystem.
  - DST_FILE: absolute path corresponding to SPEC_DST_FILE in the
    Android filesystem. Somewhat analogous to the old TARGET variable,
    but actually contains the leading / as well (/system/bin/adsprpcd
    instead of system/bin/adsprpcd).
* Use existing common get_file() function (which previously was
  impossible to use, because it was impossible to determine calling
  arguments properly) to reduce complexity of handling adb and disk
  image as blob sources.
* Via the new SRC_FILE and DST_FILE variables, search for a blob in all
  possible locations (including paths stripped of "/system" which
  transforms an absolute path in the Android filesystem into a proper
  relative path to that file in a disk image).

Change-Id: Ic40fb4dc93541d8b3f33fde586b773199cf4ded2
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
d674771760 extract_utils: extract(): rename DEST variable to VENDOR_REPO_FILE
* This denotes the path of the file that results from the extraction
  process, relative to the "proprietary" directory.
* This is a cleanup patch.

Change-Id: I38e759bb6ed697f0a31ca35a7aa9b9b92f8b6793
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
d652a0675c extract_utils: extract(): rename ARGS variable to SPEC_ARGS
* This is a cleanup patch.

Change-Id: I2a4aac840c06c4f9b94b77b32ead33352ce395ff
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
da3b64413d extract_utils: extract(): rename FILE variable to SPEC_SRC_FILE
* This is a cleanup patch.

Change-Id: I971b2b9eba5c1f303c0d5f902feaaad25b3b26ed
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
411e0695f2 extract_utils: extract(): rename FROM variable to SPEC_DST_FILE
* This is a cleanup patch.

Change-Id: I308b0478484ccf9a594078825731606fce5f5507
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
9564328417 extract_utils: create src_file() function to mirror behavior of target_file()
* This is a cleanup-only patch.

Change-Id: I144252de765686ac3469652994a04820ecc96d4d
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
2654eaa7ef extract_utils: prefix_match(): do not strip target_args from its output
* The write_product_copy_files() and write_product_packages() functions
  rely on its undocumented behavior of keeping target_args in the
  returned list, because they are users of target_args (such as
  ";PRESIGNED" etc).
* Make the behavior documented.

Change-Id: If71595dca32abd40039706d4fed2d7f12e005365
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
6a7946baeb extract_utils: target_file() and target_args() cleanup
* Strip target_args from target_file at callee instead of at caller
* WARNING! Changes (improperly documented) behavior of prefix_match()
  function, which is expected to not strip target_args(), and the root
  cause why stripping target_args was currently done at caller. Will be
  addressed in next patch.

Change-Id: I820d2350aa64ff41374809fcb22f812257132652
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
59074e6832 Revert "extract_files: Add support for paths without system/"
This reverts commit 8b87b09ef7.
This will be addressed cleaner in the cleanup commit to follow.

Change-Id: Iec013bfafdbb334ab2657b10c433158002809b62
2018-07-08 20:14:12 +02:00
Vladimir Oltean
a8b4f887ea Revert "extract_utils: Fix makefile generation issues"
This reverts commit c982836ca6.
This breaks extracting from src in "src:dst", such as from a disk image.
Will be addressed in the cleanup commit that follows.

Change-Id: Iff84a926f0c3bf908320b43ba40235e0a89db644
2018-07-08 20:14:12 +02:00
Vladimir Oltean
a48b9fe9b6
write_makefiles: fix paths in BUILD_PREBUILT with destination
* proprietary-files.txt entries such as
   "-app/TimeService/TimeService.apk:priv-app/TimeService/TimeService.apk" should
   generate a "LOCAL_SRC_FILES := proprietary/priv-app/TimeService/TimeService.apk"
   in the Makefile definition.
 * However, currently, the prefix_match function is being called on the whole
   PRODUCT_PACKAGES_LIST entry (whole line, including ":"), and therefore,
   TimeService.apk would be included in the APPS list instead of PRIV_APPS.
 * Furthermore, because of the incorrect prefix_match, the generated
   LOCAL_SRC_FILES is "proprietary/app/priv-app/TimeService/TimeService.apk",
   which breaks the build because there is no file at that wrong path.
 * The fix is to make the extract function match up with write_product_packages
   by applying the target_file function on all BUILD_PREBUILT source files.

Change-Id: Ib4859b8854db0f2142bb3f28cce8dd25f7141732
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-04-09 12:55:31 -07:00
Joe Maples
9be579f8d7 extract_utils: Use vdexExtractor for extracting oreo classes.dex
vdexExtractor is a tool made by anestisb that is written in C++
with code taken directly from art. However, anestisb has also added
a quicken decompiler to oatdump that was merged upstream, so we only
need vdexExtractor for 8.1 and 8.0.

Change-Id: Ic2cf2dc627a1ad2fa4d500d02d9eac8b8a9577b5
Signed-off-by: Joe Maples <joe@frap129.org>
2018-03-30 20:46:07 +00:00
Michael Bestas
a3f97c75bf extract_utils: Fix vendor framework makefile generation
* Fix copy-pasta

Change-Id: I9b2f61702985cb59af34ad146fd61bedba043f81
2018-02-27 22:32:50 +02:00
Ethan Chen
5bc3c8426b extract-utils: Use LOCAL_VENDOR_FILE instead of LOCAL_PROPRIETARY_FILE
* Oreo expects VNDK compatible files to be listed as LOCAL_VENDOR_FILE,
  not LOCAL_PROPRIETARY_FILE.

Change-Id: Ia2384c4f3ab3a99b79df52c796c53dc25a0f4a88
2018-02-26 02:17:37 +00:00
Paul Keith
4e75f576f9 extract_utils: Fix regressions from 3f9b94cb67
Change-Id: I78fc47c63e6bfbd5ee98a56a1d80f763e96ac266
2018-01-29 23:33:57 +02:00
Michael Bestas
3f9b94cb67 extract_utils: Split optional args with semicolon instead of colon
* Fix optional flags like PRESIGNED
  after I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed

Change-Id: I889f995d7f19b09908cfbaf7474442ce48bfabdb
2018-01-27 00:42:57 +00:00
Michael Bestas
c982836ca6 extract_utils: Fix makefile generation issues
* Fix makefile generation for packages that have set
  a different target destination

* Thanks to rashed and javelinanddart for their help
  in debugging and solving this issue.

Change-Id: I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed
2018-01-27 00:42:46 +00:00
conbold
575c63567e extract_utils: Use temporary folder for system dump
* This way it's automatically deleted at the end of the extraction

Change-Id: I77b3357875b8070d370f04c5a245f9aa3ca1939c
2017-12-03 22:44:11 +00:00
Gabriele M
e6df25b3d9 extract_utils: Fix pinning when not cleaning vendor dir
Skip the extraction of pinned files if the ones currently available
have the expected sha1. If we don't, we will overwrite pinned files
with potentially incorrect files when the current vendor files are
not moved to a temporary directory (i.e. when not cleaning vendor).

Change-Id: I640d6bf2ed98eb366a4df17f0ebeaec81cb5274b
2017-11-13 12:27:13 +00:00
Gabriele M
6c3c2c0bca extract_utils: Allow to interrupt the extraction
We want to cleanup our temporary files independently on the signal,
so just execute a trap on 0. This will ensure temporary files are
always removed and doesn't require any extra care when trapping
signals such as SIGINT that require an explicit exit call.

Change-Id: Ieff4f15c44a9ac9d5a543d14c140ebd72c0e7344
2017-10-14 22:55:22 +01:00
Rashed Abdel-Tawab
00c0766852
extract_files: Mark vendor etc files as proprietary
Change-Id: Iad8d87a18e5b1dc68c7f2265c0e35a9313804408
2017-10-08 17:34:12 -04:00
Christian Oder
c16f327f51
extract_utils: cover vendor/frameworks
Change-Id: If60b9b6c47cfd9c32477f92ad87011eddb41753b
2017-10-08 23:17:18 +02:00
Rashed Abdel-Tawab
0ca7643957 extract_files: Add support for treble compatible makefiles
To be honest, this name is a little misleading, this is how it should
have been done in the first place. This allows devices to copy vendor
files to the proper location depending on TARGET_COPY_OUT_VENDOR rather
than hardcoding system/vendor. This allows devices with dedicated vendor
partitions to copy directly to vendor. The only reason it's optional
is that some nexi set TARGET_COPY_OUT_VENDOR to system which would cause
some weird breakage.

Change-Id: Ic46bc1086737835340abef9f61693d386bc6a5dc
2017-10-07 21:24:15 +00:00
Rashed Abdel-Tawab
11186d6c83 extract_files: Adjust deodex path and tmpdir
* Make a tempdir using the mktemp command rather than just making a
  dir in /tmp to accomodate for systems that don't set proper perms
  or dont have /tmp
* Fix the deodex procedure to pull files from the right path

Change-Id: I181863599b6670e3a149069dbb7b13ebf73bae8e
2017-10-07 21:24:15 +00:00