There've been reports of issues where, when a volume is ejected, the
MediaProvider process gets killed. This happens because the
MediaProvider has a file open on the volume (eg, during a scan). We do
abort the scan when the volume is ejected, however this could take some
time. So, we give MediaProvider a bit more time before getting killed,
by only looking for files open on the pass_through paths last. This
order anyway seems to make more sense - ideally we kill apps using
external storage before we unmount the pass_through path underlying it.
Bug: 171367622
Test: atets AdoptableHostTest
Change-Id: Ie8eacaa72a80ff8161ecf1e8c0243afcd890ee39
Remove the error-prone 'keepOld' parameter, and instead make begin()
(renamed to BeginKeymasterOp()) do all the key upgrade handling.
Don't handle /data and /metadata differently anymore. Previously, when
a checkpoint is active, key blob files were replaced on /data
immediately; only the actual Keymaster key deletion was delayed until
checkpoint commit. But it's easier to just delay the key blob file
replacement too, as we have to implement that for /metadata anyway.
Also be more vigilant about deleting any leftover upgraded keys.
Test: Tested on bramble using an OTA rvc-d1-release => master. In OTA
success case, verified via logcat that the keys were upgraded and
then were committed after the boot succeeded. In OTA failure
case, verified that the device still boots -- i.e., the old keys
weren't lost. Verified that in either case, no
keymaster_key_blob_upgraded files were left over. Finally, also
tried 'pm create-user' and 'pm remove-user' and verified via
logcat that the Keymaster keys still get deleted.
Change-Id: Ic9c3e63e0bcae0c608fc79050ca4a1676b3852ee
These ioctls take a pointer to an 'int' (or an 'unsigned int', it
doesn't matter), not an 'unsigned long'. See 'man ioctl_iflags'.
Presumably it happened to work anyway because Android only runs on
little endian platforms.
Bug: 146419093
Bug: 163453310
Change-Id: I73099dafd4ee8d497c0a754149271871a37454f6
Signed-off-by: Eric Biggers <ebiggers@google.com>
We previously only set +F for /data/media, but adopted storage needs
this as well. Instead we add support for adding attrs to PrepareDir.
Bug: 163453310
Test: sm set-virtual-disk true
follow UI setup and confirm +F on /mnt/expand/*/media
Change-Id: I08f13b57a4de3538e88b38eb95b0ac115a5a5ce8
Merged-In: I08f13b57a4de3538e88b38eb95b0ac115a5a5ce8
By default FUSE filesystems have a max_ratio of 1%, meaning only 1% of
dirty pages on the system can belong to a FUSE filesystem before we
start writing back pages (and throttling, if writeback can't keep up).
This limit is useful for untrusted filesystems, but in our case, we
trust the FUSE filesystem. Since FUSE writes result in writes to the
lower filesystem, FUSE should take at most 50%. Let's start with
changing max_ratio to 40%, to avoid needless throttling.
Bug: 159254170
Bug: 159770752
Test: inspect /sys/class/bdi manually after boot
Change-Id: I467e3770fc4afba0a08fa480c0b86aa054c8b875
For fuse read ahead can be configured by writing a value to the
/sys/class/bdi/{MAJOR}:{MINOR}/read_ahead_kb file.
There are several different ways of getting {MAJOR}:{MINOR} values of
the filesystem:
* Look at st_dev of stat("/mnt/user/0/emulated").
* Parse /proc/self/mountinfo.
Stat'ing approach is used since it's easier to implement.
Bug: 157982297
Test: atest vold_tests
Test: adb shell cat /proc/self/mountinfo to get MAJOR:MINOR
Test: adb shell cat /sys/class/bdi/{MAJOR}:{MINOR}/read_ahead_kb
Test: created public volume, checked it's read_ahead_kb is also 256
Change-Id: Id0c149c4af1ceabf3afc33b4100563a512b38316
On devices without sdcardfs, /Android/data/com.foo and
/Android/obb/com.foo can be written by other processes (eg installers);
in those cases, file ownership may be wrong. To ensure that the original
app always has access to the files contained in this directory, add a
group to the default ACL that matches the UID of the app. Since all apps
have their own UID also as their group ID, this ensures that things keep
working correctly.
Bug: 157530951
Test: atest
android.appsecurity.cts.ExternalStorageHostTest#testExternalStorageUnsharedObb
Change-Id: I829a2a7c7b578a8328643f38681e68796adcd6b2
Change-Id: Ibbc333fb395507363830dfcf5dc6f1cfd55f008d
This can be done through binder as well as vdc, using 'vdc volume
abort_fuse'.
Bug: 153411204
Test: adb shell vdc volume abort_fuse
Change-Id: I93e46dc1cd361729cc1162c63520cf73152ea409
This allows devices that have sdcardfs enabled in the kernel to not use
it. When external_storage.sdcardfs.enabled=0, sdcardfs will not be
mounted. This is treated as default true to not affect upgrading
devices. It does not use the old ro.sys.sdcardfs as that has been
repurposed over time and no longer can be relied on to turn off
sdcardfs. This is included within emulated_storage.mk
Bug: 155222498
Test: mount|grep "type sdcardfs" should find nothing after boot complete
if external_storage.sdcardfs.enabled=0
Change-Id: I23d75fb1225aeabbcb1a035ad62fd042b6b3c7b5
The Android Emulator isn't the only virtual device the virtio-block
detection code is useful for, and those platforms might not set any
discriminating properties to indicate that they are virtual.
Rework the virtio-block major detection to use /proc/devices instead
of hardcoding the assumption that any virtual platform can have
virtio-block at any experimental major; the new code permits only the
exact experimental major assigned to virtio-block.
The new code runs everywhere, but it will only run once and could be
expanded later to detect dynamic or experimental majors.
Bug: 156286088
Change-Id: Ieae805d08fddd0124a397636f04d99194a9ef7e5
Previously every directory on external storage had project ID quota
inheritance enabled; this means that if any new file/directory is
created under such a directory, it will inherit the project ID from the
parent. We use a default project ID of 1000 for generic directories, and
application-specific project IDs for app-specific directories.
MediaProvider is responsible for updating the quota type in the generic
directories, as it scans all files there. However, there is a problem
with this approach: if you move a file to a directory with project ID
inheritance set, and the project ID of that file differs from the
project ID of the dir, that results in an EXDEV error, and requires a
copy instead. For example, if /sdcard/DCIM/test.jpg has a project ID of
1003 (for images), and you try to move it to /sdcard/Pictures/test.jpg,
that would require a copy, because the project ID of /sdcard/Pictures is
1000.
While this is not a very common scenario, it's still better to avoid it.
Luckily we can - since MediaProvider anyway scans all files, it will set
the project ID on individual files correctly - there's no need to
inherit them.
We then only need to inherit quota in application-specific directories,
since in those directories the app can create files itself, and those
need to be tagged correctly.
This change enables that, by removing quota inheritance setting from the
top-level directory, and instead doing it for app-specific directories
instead.
Bug: 151078664
Test: atest StorageHostTest
atest com.android.tests.fused.host.FuseDaemonHostTest#testRenameAndReplaceFile
Change-Id: I38a057ec61cb627e39a3ff7ac58c7218dc251bdc
- Remove bind mounting Android/ code as we want to bind mount obb dir
for each process instead.
- Set property "vold.vold.fuse_running_users" as an array of user id
for which fuse is ready to use.
- After fuse is ready for a user, fork a background process in vold
to bind mount all direct boot apps for that user so its direct boot
apps obb dir will be mounted to lower fs for imporoved performance.
Bug: 148049767
Bug: 137890172
Test: After flag is enabled, AdoptableHostTest still pass.
Change-Id: I90079fbeed1c91f9780ca71e37b0012884680b7c
This can be used to fixup application directories in case they have been
created by some other entity besides vold; the main use case for this
API right now is OBB directories, which can be created by installers
outside of vold; on devices without sdcardfs, such directories and the
files contained therein are not setup correctly. This API will make sure
everything is setup the way it needs to be setup.
Bug: 146419093
Test: inspect OBB dir after install
Change-Id: I2e35b7ac2992dbb21cc950e53651ffc07cfca907
Since installers can create directories in Android/obb, make sure those
directories end up with the correct ACL bits as well.
Bug: 146419093
Test: inspect filesystem manually
Change-Id: I211e921197560a40599938463f3171a0ff92d9aa
We want subdirectories of Android/data, Android/obb etc. to
automatically maintain their group-id.
Bug: 146419093
Test: manual inspection of /sdcard/Android
Change-Id: I36883febb01aa155dfafb0e86f8b99223cde9815
This was hard to read and understand. Instead, fall back to explicit
string operations with more comments on what we're doing and what we're
allowing.
This also fixes an issue where apps were asking us to create dirs on
their behalf that our more than 2 levels deep, eg
com.foo/files/downloads ; I thought such paths weren't allowed, but
apparently they are (and there's no good reason for us to not set them
up correctly).
Bug: 149407572
Test: launch opera
Change-Id: I7c64831032b66e90960b96e41ee42c7d616a759c
We directly pass a reference to our std::string, instead of
forcing the creation of a temporary std::string from the
result of c_str().
Test: TreeHugger
Change-Id: Ibab13f1e1ff43af076df60ae4032bf9dd111dd27
On devices without sdcardfs, application-specific directories have a
particular GID that ensure some privileged daemons (like installers) are
able to write to them. Android applications however run with a umask of 0077, which means that
any subdirectory they create within their app-specific directory has
mode 700, which in turn prevents things like DownloadManager from
working, since it can be asked to download into a subdir of the app's
private storage.
To prevent this from happening, set a default 770 ACL on the top-level
app-specific directory (eg, /data/media/0/Android/data/com.foo); the
effect of that default ACL is that all directories that are created
within these directories automatically get a 770 mask, regardless of the
umask that the process has.
Bug: 146419093
Test: atest FuseDaemonHostTest on cf_x86 (without sdcardfs)
Change-Id: I3178694e6d25ce3d04a0918ac66862f644635704
A regex allows us to be more specific in what kind of directories we
accept here, which in turn makes it easier to correctly create them.
Bug: 146419093
Test: atest FuseDaemonHostTest
Change-Id: Icb8911f6516eab81b9bbd567c7287be9f605e8b0
I3a879089422c7fc449b6a3e6f1c4b386b86687a4 enforces some gids on the
Android/ dirs but left out Android/media. We now create it
Test: atest FuseDaemonHostTest#testListFilesFromExternalMediaDirectory
Bug: 149072341
Change-Id: I260c414906cd491a6bdd83522ff45f8663e15604
This allows setting the "inherit project ID" flags on directories; in
our case, we want to set this on the root of the lower filesystem, eg
"/data/media/0".
Bug: 146419093
Test: manual invocation works
Change-Id: Ic74588fd972d464e7021bef953da0e5aaafc4286
Use PrepareAppDirsFromRoot() to setup the quota project ID on
application-specific directories correctly. App directories use
AID_EXT_GID_START + their application ID offset, whereas cache
directories use AID_CACHE_GID_START. This is consistent with the GIDs
sdcardfs used to label these directories with.
Bug: 146419093
Test: verified project IDs with lsattr -p
Change-Id: Idca8a30d185012efb0d19ceb9b346b9a4de34f18
Normally sdcardfs takes care of setting up these directories on-demand,
for example when an app requests its private data directory to be
created. On devices without sdcardfs however, we ourselves need to make
sure to setup the UID/GID of these directories correctly.
Introduce a new PrepareAndroidDirs() function which sets the dirs up
correctly. On devices without sdcardfs, that means:
Path UID GID mode
/Android media_rw media_rw 771
/Android/data media_rw ext_data_rw 771
/Android/obb media_rw ext_obb_rw 771
Bug: 146419093
Test: wipe Android/, reboot, with and without sdcardfs, verify
contents
Change-Id: I3a879089422c7fc449b6a3e6f1c4b386b86687a4
Even though /mnt/pass_through itself is 700 root root, the paths under
it are quite permissive. Now, change them from 755 to 710 root
media_rw since the FUSE daemon is the only one that should access it
and it has media_rw gid
Test: manual
Bug: 135341433
Change-Id: I743c014f2c0273c68a1cead7f4331b55a3abcb4e
This allows the FUSE daemon (with media_rw) explicitly use /mnt/user
paths for redaction.
Test: atest FuseDaemonHostTest#testVfsCacheConsistency
Change-Id: If5b5f5aa6a0ce7c8e2fd300ff6146b345b25cf04
These paths previously had 0755 permission bits
(/mnt/installer got its bits from the /mnt/user bind mount).
With such permissive bits, an unauthorized app can access a file using
the /mnt/installer path for instance even if access via /storage
would have been restricted.
In init.rc we create /mnt/user with 0755 initially, this is to keep
/sdcard working without FUSE. When mounting a FUSE filesystem, we
enusure in vold that /mnt/user is changed to 0700
Bug: 135341433
Test: adb shell ls -d /mnt/{user, installer}
Change-Id: Id387e34c5fd257858861246ad51486892653fb3a
This allows readlink(2) of /sdcard paths to work correctly
and return /storage/emulated/<userid> instead of
/mnt/user/<userid>/emulated/<userid>
Test: readlink /sdcard -> /storage/emulated/0
Bug: 135341433
Change-Id: I2cfa9cede02a93024e41d90f17c926a69ec6e052
We bind mount /mnt/user/<userid> onto /storage for normal apps and
/mnt/pass_through/<userid> for special apps like the FUSE daemon or
the old android.process.media hosting the DownloadManager. This bind
mount allows app have /storage/self/primary which is what /sdcard
symlinks to.
Before this change, we were not creating the self/primary symlink on
/mnt/pass_through/<userid> so trying to access /sdcard from the
DownloadManager would fail.
Bug: 135341433
Test: atest android.app.cts.DownloadManagerTest#testAddCompletedDownload_invalidPaths
Change-Id: I660139be3d850e6e9ea4705f86ef2b5872ddca16
In preparation of sdcardfs going away on devices launching with R,
conditionally use it.
Bug: 146419093
Test: cuttlefish with sdcardfs, cuttlefish without sdcardfs but with
FUSE
Change-Id: I2c1d4b428dcb43c3fd274dde84d5088984161993
Previously, when mounting a FUSE volume, the permission bits for
/mnt/user/<userid> were very strict, 700 which was good, however this
value was ignored because it was overriden in zygote to 755. In fact
if it wasn't ignored, apps wouldn't have had access to /sdcard becase
they would lack the directory 'execute' bit for /mnt/user/<userid>
needed while looking up /mnt/user/<userid>/emulated
Now we set it to a strict enough value, 710 that only allows apps
running under the same user id to lookup /mnt/user/<userid>.
This ensures that user 10 cannot access /mnt/user/0.
A special case is added for /mnt/user/0 for shell since it is not in
the 'everybody' group and would otherwise not be able to 'adb shell ls
/sdcard'
Bug: 135341433
Test: atest -c android.appsecurity.cts.ExternalStorageHostTest#testSecondaryUsersInaccessible
Change-Id: Ia427d1b69c7140254ae3459b98e51531d8322f1a
vold historically offerred functionality to create directories on behalf
of others. This functionality was purely used to create app-specific
data/obb/media dirs. Make this more explicit by renaming the method to
indicate this.
Additionally, in the past, we never needed to care about the UID set on
these directories, because sdcardfs would take care of that for us
automatically. But with sdcardfs going away, we need to make sure the
UID of the app-specific directories is set correctly. Allow the caller
to pass this in as an argument.
Bug: 146419093
Test: atest FuseDaemonHostTest
Change-Id: Ibeb5fdc91b40d53583bc0960ee11c4d640549c34
For apps seeing the FUSE filesystem, we want to bind-mount the Android/
directory to the lower filesystem. The main reason for this is game
performance - Android/ contains both OBBs and app-private external data,
and both are heavily accessed during game startup. This is a pretty
straightforward bind-mount on top of /mnt/user.
Bug: 137890172
Test: Running the following:
df /storge/emulated/0 ==> /dev/fuse (FUSE)
df /storage/emulated/0/Android ==> /data/media (sdcardfs)
Test: atest AdoptableHostTest
Change-Id: Ic17a5751b5a94846ee565ff935644a078044ab06
The pass-through mount is used by MediaProvider to access external
storage. Previously, it was the raw filesystem (eg ext4/f2fs); the
problem with that is that the permissions on that filesystem don't allow
MediaProvider to access all the files it needs to - in particular
directories under Android/
To solve this problem, we can have the pass-through mount sit on top of
sdcardfs instead of the raw filesystem. This means we need to mount
sdcardfs even in case we're using FUSE, but we already needed to do this
anyway for other performance reasons.
Bug: 135341433
Test: atest AdoptableHostTest
Change-Id: I893d5e5076c5096d2d55212f643c9a857242e964
symlink(2) creates a symbolic link 'linkpath' containing a
string 'target'.
linkpath was misnamed as target in MountUserFuse. This cl
s/target/linkpath/ in Utils#MountUserFuse.
Test: m
Change-Id: I274823da16b87ffc124e2e8c4563b1d16546a6e7
Up until now, the FUSE mount logic has made two assumptions:
1. The primary external volume is an emulated volume on /data/media
2. Only the primary user is running, as user zero
These assumptions are fixed by the following changes
creating an EmulatedVolume per Android user and changing the
VolumeBase id format to append the user to the id, so
s/emulated/emulated-0/. This allows us mount separate volumes per user
Some additional refactorings to re-use/clean up code.
Test: adb shell sm set-virtual-disk and partition disk operations work
even after setting up a work profile
Bug: 135341433
Change-Id: Ifabaa12368e5a591fbcdce4ee71c83ff35fdac6b
Before this, the FUSE daemon receives a setattr inode timestamp
request for every write request. This can be crippling for write
performance or read performance during writes especially random
writes where the write back cache does not effectively coagulate
requests.
We now add the MS_LAZYTIME mount flag
(http://man7.org/linux/man-pages/man2/mount.2.html) to lazily flush
the timestamp updates from memory to disk
Test: m
Bug: 135341433
Change-Id: I95a467d5682a325b4099f32634d93ed2921f815e
Also allow the state just before doMount() as a valid state for setting
fuse fd.
Test: manual
BUG:140173712
Change-Id: I012f8a83fef00e68f33010954fbc2ebc53cf8f1d
Since system_server cannot mount devices by itself,
add a binder interface to vold that system_server
can call to initiate this mount when required.
BUG: 135341433
Test: manual
Test: atest --test-mapping packages/providers/MediaProvider
Test: ExternalStorageHostTest DownloadProviderTests
Change-Id: If4fd02a1f1a8d921a3f96783d8c73e085c5b7ca1
Remove static definition of writeStringToFile, and
move it from KeyStorage to Utils
(cherry picked from commit 0bd2d11692)
Bug: 71810347
Test: Build pass and reboot stress test.
Change-Id: I38bfd27370ac2372e446dc699f518122e73c6877
Merged-In: I38bfd27370ac2372e446dc699f518122e73c6877
