Commit graph

70 commits

Author SHA1 Message Date
Vladimir Oltean
c503446ec3 extract_utils: cleanup variable names and arguments passed to extract() function
* This also makes the --section argument non-positional, since otherwise
  it is not possible to easily support more than one optional positional
  argument. This is in preparation of one more optional argument to come
  in a follow-up patch: --kang.

Change-Id: Ieb142e0854319defb9a278ab68cd4aeefd0fbdd5
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
78d690d12c extract_utils: be compatible with system-as-root layouts
* The use case is that if you have the following layout:
  $TOP --- system.img
       |
       +-- vendor.img
  you should be able (from $TOP) to:
    mkdir system; mount -o ro,loop system.img system
    mkdir vendor; mount -o ro,loop vendor.img vendor
  and then (from device tree)
    ./extract-files.sh $TOP

  But this doesn't work if system.img is SAR and contains another
  "system" dir inside. This patch makes sure it searches for a "system"
  dir in the provided path as well, if it couldn't find the blob
  anywhere else.

Change-Id: Ib49cd5b587b3a57478a66ff69cf840270c2b1403
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
4fe79b11c1 extract_utils: drop the "/system/" prefix from prints of blob names
* This makes the printed output closer to the proprietary-files.txt syntax

Change-Id: I81b844bb6bb1d1a2f91a39151a892fbfc0bed20b
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00
Vladimir Oltean
b2c382193d extract_utils: fix mismatch with non-lowercase pinned sha1sum
Change-Id: I86421ca69007ad1992aa67deedf7d2e32bbe4dba
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-01-17 11:26:34 +01:00
Vladimir Oltean
b8084ec8a5 extract_utils: avoid doing stuff on files that don't exist
* If an apk/jar doesn't exist, the script would still try to deodex it.
* If an xml doesn't exist, the script would still try to "fix" it.
* Take it easier, man, it's not your fault.

Change-Id: I3061fb48b403da5121e3c17dd9ecdb6cd148bf97
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-12-24 01:32:53 +02:00
Luca Stefani
99a66bffdc extract_utils: Redirect vdexextractor log to /dev/null
Change-Id: I55028997c0396d57c024e6c42a53907f6d1510c8
2018-11-13 22:06:17 +01:00
Luca Stefani
f6096e989a Add CDEX support for boot JAR files
Change-Id: I0039256c883a58c5a5125fd9bff6e6ec3b98688d
2018-10-21 10:28:37 +02:00
TheStrix
5aad07422e extract_utils: Fix output dex file name generated by vdexExtractor
- Reflect the changes required by:
   ff58255706
 - Also fix JARVDEX file path for deodexing jars

Change-Id: Ief355b0bd9e69a67b552d0bd2060f43adb39fb5f
2018-10-07 12:17:35 +02:00
codeworkx
1c29bf66dd extract_utils: add CDEX support via compact_dex_converter
Source: https://github.com/anestisb/vdexExtractor#compact-dex-converter

Change-Id: I63664cae02e51a05e5a62c0c5c0595eead6c6513
2018-10-05 18:25:46 +02:00
Luca Stefani
a9a9cc730c extract_utils: Newline love for adb extraction
Change-Id: Ibbf7a03ed6920d556ed7658c277f37b02d8876b8
2018-07-12 21:28:21 +02:00
Vladimir Oltean
d577325ae2 extract_utils: make get_file() able to search paths with and w/o /system prefix
Change-Id: I75eba627646caa8ceb8a30575124c57e5ee9445c
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
5f15e3e63c extract_utils: extract(): fix src not being searched in "src:dst;args" spec
* Root cause of the issue is improper naming of variables. Turns out,
  there was no variable to even denote where the blob should have been
  searched for, at "src".
* Previously there was one such variable, suggestively called "TARGET", that
  was desperately trying to serve as both, depending on who +2d hacks harder.
* One such example is "c982836 extract_utils: Fix makefile generation issues".
  That patch deliberately trimmed the "src:" from a spec (therefore
  obviously breaking the search at src) but enabling the searching at
  dst, via the good-for-all TARGET variable.
* This patch introduces the following variables:
  - SRC_FILE: absolute path corresponding to SPEC_SRC_FILE in the
    Android filesystem.
  - DST_FILE: absolute path corresponding to SPEC_DST_FILE in the
    Android filesystem. Somewhat analogous to the old TARGET variable,
    but actually contains the leading / as well (/system/bin/adsprpcd
    instead of system/bin/adsprpcd).
* Use existing common get_file() function (which previously was
  impossible to use, because it was impossible to determine calling
  arguments properly) to reduce complexity of handling adb and disk
  image as blob sources.
* Via the new SRC_FILE and DST_FILE variables, search for a blob in all
  possible locations (including paths stripped of "/system" which
  transforms an absolute path in the Android filesystem into a proper
  relative path to that file in a disk image).

Change-Id: Ic40fb4dc93541d8b3f33fde586b773199cf4ded2
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
d674771760 extract_utils: extract(): rename DEST variable to VENDOR_REPO_FILE
* This denotes the path of the file that results from the extraction
  process, relative to the "proprietary" directory.
* This is a cleanup patch.

Change-Id: I38e759bb6ed697f0a31ca35a7aa9b9b92f8b6793
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
d652a0675c extract_utils: extract(): rename ARGS variable to SPEC_ARGS
* This is a cleanup patch.

Change-Id: I2a4aac840c06c4f9b94b77b32ead33352ce395ff
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
da3b64413d extract_utils: extract(): rename FILE variable to SPEC_SRC_FILE
* This is a cleanup patch.

Change-Id: I971b2b9eba5c1f303c0d5f902feaaad25b3b26ed
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
411e0695f2 extract_utils: extract(): rename FROM variable to SPEC_DST_FILE
* This is a cleanup patch.

Change-Id: I308b0478484ccf9a594078825731606fce5f5507
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
9564328417 extract_utils: create src_file() function to mirror behavior of target_file()
* This is a cleanup-only patch.

Change-Id: I144252de765686ac3469652994a04820ecc96d4d
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
2654eaa7ef extract_utils: prefix_match(): do not strip target_args from its output
* The write_product_copy_files() and write_product_packages() functions
  rely on its undocumented behavior of keeping target_args in the
  returned list, because they are users of target_args (such as
  ";PRESIGNED" etc).
* Make the behavior documented.

Change-Id: If71595dca32abd40039706d4fed2d7f12e005365
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
6a7946baeb extract_utils: target_file() and target_args() cleanup
* Strip target_args from target_file at callee instead of at caller
* WARNING! Changes (improperly documented) behavior of prefix_match()
  function, which is expected to not strip target_args(), and the root
  cause why stripping target_args was currently done at caller. Will be
  addressed in next patch.

Change-Id: I820d2350aa64ff41374809fcb22f812257132652
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-07-08 20:14:12 +02:00
Vladimir Oltean
59074e6832 Revert "extract_files: Add support for paths without system/"
This reverts commit 8b87b09ef7.
This will be addressed cleaner in the cleanup commit to follow.

Change-Id: Iec013bfafdbb334ab2657b10c433158002809b62
2018-07-08 20:14:12 +02:00
Vladimir Oltean
a8b4f887ea Revert "extract_utils: Fix makefile generation issues"
This reverts commit c982836ca6.
This breaks extracting from src in "src:dst", such as from a disk image.
Will be addressed in the cleanup commit that follows.

Change-Id: Iff84a926f0c3bf908320b43ba40235e0a89db644
2018-07-08 20:14:12 +02:00
Vladimir Oltean
a48b9fe9b6
write_makefiles: fix paths in BUILD_PREBUILT with destination
* proprietary-files.txt entries such as
   "-app/TimeService/TimeService.apk:priv-app/TimeService/TimeService.apk" should
   generate a "LOCAL_SRC_FILES := proprietary/priv-app/TimeService/TimeService.apk"
   in the Makefile definition.
 * However, currently, the prefix_match function is being called on the whole
   PRODUCT_PACKAGES_LIST entry (whole line, including ":"), and therefore,
   TimeService.apk would be included in the APPS list instead of PRIV_APPS.
 * Furthermore, because of the incorrect prefix_match, the generated
   LOCAL_SRC_FILES is "proprietary/app/priv-app/TimeService/TimeService.apk",
   which breaks the build because there is no file at that wrong path.
 * The fix is to make the extract function match up with write_product_packages
   by applying the target_file function on all BUILD_PREBUILT source files.

Change-Id: Ib4859b8854db0f2142bb3f28cce8dd25f7141732
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2018-04-09 12:55:31 -07:00
Joe Maples
9be579f8d7 extract_utils: Use vdexExtractor for extracting oreo classes.dex
vdexExtractor is a tool made by anestisb that is written in C++
with code taken directly from art. However, anestisb has also added
a quicken decompiler to oatdump that was merged upstream, so we only
need vdexExtractor for 8.1 and 8.0.

Change-Id: Ic2cf2dc627a1ad2fa4d500d02d9eac8b8a9577b5
Signed-off-by: Joe Maples <joe@frap129.org>
2018-03-30 20:46:07 +00:00
Michael Bestas
a3f97c75bf extract_utils: Fix vendor framework makefile generation
* Fix copy-pasta

Change-Id: I9b2f61702985cb59af34ad146fd61bedba043f81
2018-02-27 22:32:50 +02:00
Ethan Chen
5bc3c8426b extract-utils: Use LOCAL_VENDOR_FILE instead of LOCAL_PROPRIETARY_FILE
* Oreo expects VNDK compatible files to be listed as LOCAL_VENDOR_FILE,
  not LOCAL_PROPRIETARY_FILE.

Change-Id: Ia2384c4f3ab3a99b79df52c796c53dc25a0f4a88
2018-02-26 02:17:37 +00:00
Paul Keith
4e75f576f9 extract_utils: Fix regressions from 3f9b94cb67
Change-Id: I78fc47c63e6bfbd5ee98a56a1d80f763e96ac266
2018-01-29 23:33:57 +02:00
Michael Bestas
3f9b94cb67 extract_utils: Split optional args with semicolon instead of colon
* Fix optional flags like PRESIGNED
  after I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed

Change-Id: I889f995d7f19b09908cfbaf7474442ce48bfabdb
2018-01-27 00:42:57 +00:00
Michael Bestas
c982836ca6 extract_utils: Fix makefile generation issues
* Fix makefile generation for packages that have set
  a different target destination

* Thanks to rashed and javelinanddart for their help
  in debugging and solving this issue.

Change-Id: I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed
2018-01-27 00:42:46 +00:00
conbold
575c63567e extract_utils: Use temporary folder for system dump
* This way it's automatically deleted at the end of the extraction

Change-Id: I77b3357875b8070d370f04c5a245f9aa3ca1939c
2017-12-03 22:44:11 +00:00
Gabriele M
e6df25b3d9 extract_utils: Fix pinning when not cleaning vendor dir
Skip the extraction of pinned files if the ones currently available
have the expected sha1. If we don't, we will overwrite pinned files
with potentially incorrect files when the current vendor files are
not moved to a temporary directory (i.e. when not cleaning vendor).

Change-Id: I640d6bf2ed98eb366a4df17f0ebeaec81cb5274b
2017-11-13 12:27:13 +00:00
Gabriele M
6c3c2c0bca extract_utils: Allow to interrupt the extraction
We want to cleanup our temporary files independently on the signal,
so just execute a trap on 0. This will ensure temporary files are
always removed and doesn't require any extra care when trapping
signals such as SIGINT that require an explicit exit call.

Change-Id: Ieff4f15c44a9ac9d5a543d14c140ebd72c0e7344
2017-10-14 22:55:22 +01:00
Rashed Abdel-Tawab
00c0766852
extract_files: Mark vendor etc files as proprietary
Change-Id: Iad8d87a18e5b1dc68c7f2265c0e35a9313804408
2017-10-08 17:34:12 -04:00
Christian Oder
c16f327f51
extract_utils: cover vendor/frameworks
Change-Id: If60b9b6c47cfd9c32477f92ad87011eddb41753b
2017-10-08 23:17:18 +02:00
Rashed Abdel-Tawab
0ca7643957 extract_files: Add support for treble compatible makefiles
To be honest, this name is a little misleading, this is how it should
have been done in the first place. This allows devices to copy vendor
files to the proper location depending on TARGET_COPY_OUT_VENDOR rather
than hardcoding system/vendor. This allows devices with dedicated vendor
partitions to copy directly to vendor. The only reason it's optional
is that some nexi set TARGET_COPY_OUT_VENDOR to system which would cause
some weird breakage.

Change-Id: Ic46bc1086737835340abef9f61693d386bc6a5dc
2017-10-07 21:24:15 +00:00
Rashed Abdel-Tawab
11186d6c83 extract_files: Adjust deodex path and tmpdir
* Make a tempdir using the mktemp command rather than just making a
  dir in /tmp to accomodate for systems that don't set proper perms
  or dont have /tmp
* Fix the deodex procedure to pull files from the right path

Change-Id: I181863599b6670e3a149069dbb7b13ebf73bae8e
2017-10-07 21:24:15 +00:00
M1cha
15f226c23a
extract_utils: implement LOCAL_MODULE_RELATIVE_PATH
Change-Id: Ia97633652ff9647a6e82013b208097be5c5575a9
2017-09-20 22:12:36 -04:00
Rashed Abdel-Tawab
54b5d5e1e5
extract_utils: Add VDEX deodex capabilities for oreo
Oreo introduces new optimized dex files (or vdex). While smali/baksmali
have supported vdex since 2.2.1, you actually have to have the .vdex in
the same location as the .oat or else the smali will fail due to not
finding the .vdex. Copy it (if it exists) and echo a warning since vdex
deodexing is still experimental.

Change-Id: Ic612751be45a8b6e54b8794b426a4d5adbbf13d2
2017-09-20 22:12:36 -04:00
Luca Stefani
5c60e4fd24
lineage: Complete the rebrand
* TODO: Rebrand cmsdk when it's in a working state

Change-Id: I0d8d573f9e0032110ae3a0f54ebe11a877b83137
2017-08-23 17:33:22 +03:00
Dan Pasanen
91f7620c4d
vendor: change vendor to lineage
Change-Id: I2df13091f5d62b552ef434aef1a389f4fe1249e0
2017-08-23 03:26:28 +03:00
Harry Youd
9c5d129d90 Fix extract_utils when not using blob sections
"$3" evaluates as "" when $3 is empty, and so $# is still 3
When $3 is empty, do not use it

Change-Id: I87df7dd8ed23cac3d4cff16dce0996bbe0a3e6cc
2017-08-05 08:21:38 +00:00
Rashed Abdel-Tawab
8b87b09ef7
extract_files: Add support for paths without system/
Sometime blob directories are contained in dirs not named system
(ex. extracting blobs from another vendor dir) so check if the file
exists without system/ before appending "system/" to filenames.

Change-Id: I4ba946e178260b4f4804b1ac4422645e3085c665
2017-07-30 12:48:18 -04:00
Rashed Abdel-Tawab
855fbdd4bc
extract_files: Add support for specifying blob sections
Sometimes the need arises to update one specific subset of blobs
while holding others steady. Introduce a way to specify the "section"
of blobs to update (obviously requires the proprietary-files to be
properly split up first)

Requires additional changes in device tree extract_files.sh to support
passing the additional arguments

Change-Id: I98feab56f8bfade2818b1b5264e019440d4c57c5
2017-07-30 12:48:18 -04:00
Gabriele M
b6effb3ebb extract_utils: Fix cleanup variables assigment
The current assignment is wrong, we cleanup the files when we tell
not to do so.

Change-Id: I73a9a17f5ebae6f2dbdc3d6d9aaa28cd75fb4f87
2017-07-29 16:46:53 +00:00
Dobroslaw Kijowski
65f03f1408 extract_utils: fix extracting XML files containing binary characters
When a line from the XML file contains a character which grep classifies
as a binary one, it will omit this line at all. Moreover it will append
'Binary file matches' to the stdout. This behavior breaks some XML files.
To fix this issue, let's tell grep to process a binary file as if it were
text.

Change-Id: I6ec5689f73a0926870e502f3f8ee77346b47d05f
2017-05-18 12:37:08 +02:00
Dan Pasanen
7dc287f4b9 extract_utils: support extracting directly from an ota zip
* Specify the ota zip name as the only parameter to extract-files.sh
  * Will extract to $CM_ROOT/system_dump
  * Bail out on A/B OTA zips. We cannot support these.
  * Handles block based OTA zips by using sdat2img.py
  * Store the zip's MD5 and check if its already extracted. If so, don't
    bother extracting again

Change-Id: I03038e38dac51e6cb60d493c7e6362754d1daf02
2017-03-27 08:46:48 -05:00
Paul Keith
85cf0bb8c6 extract_utils: Fix locale issue with sort
* On some machines, LC_ALL=C is set, causing the sort
  order of makefiles to be different than those without
  this locale change that is set
* Set LC_ALL=C on all machines for consistent ordering
* From 'man sort':
  The locale specified by the environment affects sort order.
  Set LC_ALL=C to get the traditional sort order that uses native byte values.
* Thanks to haggertk and rashed for verifying this for me

Change-Id: I5bbb94b0dfe599a67036de4312e8af913e9b8d30
Signed-off-by: Paul Keith <javelinanddart@aidenswann.com>
2017-03-08 15:03:51 -06:00
Christopher R. Palmer
052d9d9384 extract-utils: Fix handling of pulling src:dest pairs from dumps
Currently, the priority of the src and dest is inverted between
pulling from adb and pulling from a system dump.

Assume that we have a camera wrapper and write the proprietary-files.txt

lib/hw/camera.msm8996.so:lib/hw/camera.vendor.msm8996.so

If we pull from a phone running Lineage that has both files, we get

camera.vendor.msm8996.so

as the pulled blob.  If we take the exact same build and pull it
from the system dump (aka, your own $OUT directory that built the
installed software) you get

camera.msm8996.so

pull instead!

Make both paths follow the same logic so that you get the same
file independent of the source.

Change-Id: I479e0ae765339cc38fa05fcaad7943c528129463
2017-03-04 05:16:28 -05:00
Adrian DC
3c6bdac74b extract_utils: Detect output existence before moving it for backup
* Avoids:
    Cleaning output directory (./../../../vendor/.../.../proprietary)..
    mv: cannot stat './../../../vendor/.../.../proprietary/*':
    No such file or directory

Change-Id: I3258e64f3997f139d8db4fb4e07d14964a469c78
2017-01-18 23:20:47 +00:00
Matt Mower
8945f5e8c7 extract_utils: Introduce variable INITIAL_COPYRIGHT_YEAR
Let a device specify INITIAL_COPYRIGHT_YEAR=XXXX in setup-makefiles.sh
so that the correct copyright(s) is/are included in vendor makefiles.
The CyanogenMod copyright is retained when INITIAL_COPYRIGHT_YEAR<2017
and the LineageOS copyright is now included for >=2017.

Change-Id: I6895b5b69bc7ba399042ac3c29e17f3209d15f1b
2017-01-09 01:26:44 +00:00
Gabriele M
4cf635a5ad extract_utils: Update backsmali command
-e no longer exists starting from v2.2b3, so update the
command. While at it, add the logic to deal with Android N
separated boot oat files.

Change-Id: Iab8e3946d6421824fea74e26dc43d2db38573022
2017-01-06 10:23:46 +00:00