Ask / Submit
6

Guide: Externalising android_storage and other directories / files to SD-card

asked 2019-04-08 00:07:25 +0300

olf gravatar image

updated 2019-04-17 00:02:06 +0300

Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on non-BTRFS devices, i.e. all but the Jolla 1 phone) with an SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.

This is an enhanced, updated and streamlined guide created from the original "[How-to] Externalising android_storage and other directories / files to SD-card". It only covers SailfishOS 2.2.0 and newer.
For SailfishOS versions before 2.2.0 and the technical background of externalising android_storage, please consult the original how-to. The section topics and numbers are mostly the same for both guides.

This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.

All this was tested for many months on Jolla 1 phones and Xperia Xs with SailfishOS versions from 2.2.0 to 3.0.2 and multiple SD-cards; if things should really turn out to look different on other devices (which is very unlikely, except for applying section 2 to the new, Android 8 based runtime environment, e.g. on an Xperia XA2), you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this guide still may have flaws. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.

1. Externalising regular files and directories in /home/nemo

For externalising android_storage, see section 2.

Mind using "hidden" directory names (starting with a ".") on the SD-card for externalised directories not to be indexed by the SailfishOS' Media Tracker service twice (there and in their original location, symlinked later in this guide).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without being symlinked somewhere under /home/nemo/) and should be indexed by the Media Tracker.

As user nemo, execute the following commands:

cd /run/media/nemo/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard 
cd
ln -s /run/media/nemo/XXXXXXXX/.nemo-sdcard SD-card

You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or anything under it, and one also shall not do that with "hidden" files or directories) onto SD-card by executing (you may want to back them up, first):

cd Foo
mkdir /run/media/nemo/XXXXXXXX/.foo
cp -av . /run/media/nemo/XXXXXXXX/.foo/  # Only continue, if copying has been successful 
cd ..
rm -r Foo
ln -s /run/media/nemo/XXXXXXXX/.foo Foo

You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):

cd Foo
mkdir /run/media/nemo/XXXXXXXX/.foo
cp -pv Bar /run/media/nemo/XXXXXXXX/.foo/  # Only continue, if copying has been successful 
rm Bar
ln -s /run/media/nemo/XXXXXXXX/.foo/Bar Bar

Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card (rsp. technically: its UUID) and "Bar" is a placeholder for a specific file.

Arbitrary examples:
~nemo/Music --> /run/media/nemo/XXXXXXXX/.music
~nemo/Videos --> /run/media/nemo/XXXXXXXX/.videos
~nemo/Documents/Maps --> /run/media/nemo/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /run/media/nemo/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin

2. Externalising /home/nemo/android_storage

  • Devices with BTRFS as their native filesystem (e.g. Jolla 1 phones) use /data/media as AlienDalvik's media directory (i.e., the "virtual SD-card", which becomes bind-mounted at /home/nemo/android_storage) for their AOSP4.1.2 based Android runtime environment.
  • "Second generation devices" employing EXT4 on LVM (e.g. Xperia X, Intex Aquafish, Jolla C, Jolla Tablet) with their AOSP4.4 based AlienDalvik all seem to use /home/nemo/android_storage directly (symlinked from /home/.android/data/media). Currently only tested with an Xperia X, please report differences on your "second generation device".
  • "Third generation devices" with their AOSP8 based Android runtime environment (e.g. the Xperia XA2 family, Cosmo and hopefully Gemini) are not analysed yet, currently lacking such a device.

Install mount-sdcard (e.g. per Storeman) to make automatic mounting on boot-up working reliably for AlienDalvik (i.e., to be race-free by a "Before=alien-service-manager.service" statement).
For automatic mounting of encrypted SD-cards, which have to be prepared as described in part 1 for externalising android_storage to them, have crypto-sdcard installed. Also running mount-sdcard is not really necessary in this case (when the content of android_storage resides on an encrypted partition), but it still provides some additional benefits.

2.1 Copying data to SD-card

Execute the following commands:

devel-su
systemctl stop alien-service-manager.service  # Stop AlienDalvik completely
cd /home/nemo  # cd /data for BTRFS-using devices
cp -av android_storage /run/media/nemo/XXXXXXXX/.android-sdcard  # Pick the right UUID!
# cp -av media /run/media/nemo/XXXXXXXX/.android-sdcard  # for BTRFS-using devices
touch /run/media/nemo/XXXXXXXX/.android-sdcard/ANDROID-SDCARD  # Set a marker
chmod a= /run/media/nemo/XXXXXXXX/.android-sdcard/ANDROID-SDCARD  # Make it persistent

Optionally check via checksums, that the data has been correctly written (needs rsync installed, per pkcon install rsync):

rsync -nc -avvhh /home/nemo/android_storage/ /run/media/nemo/XXXXXXXX/.android-sdcard/ | fgrep -v ' is uptodate'  # Should detect no files not being up-to-date on "2nd gen" devices
rsync -nc -avvhh /data/media/ /run/media/nemo/XXXXXXXX/.android-sdcard/ | fgrep -v ' is uptodate'  # Should detect no files not being up-to-date on BTRFS-using devices

2.2 Integrating an externalised android_storage when booting

Unfortunately there seems to be no proper way of integrating an externalised android_storage seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.

Alter /opt/alien/system/script/platform_envsetup.sh by executing:

devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig  # Backup

Edit this file by adapting the MEDIA_STORAGE path variable, so that a diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):

* On "2nd gen" devices:

20c20
< MEDIA_STORAGE=/home/nemo/android_storage
---
> MEDIA_STORAGE=/run/media/nemo/XXXXXXXX/.android-sdcard

* On BTRFS-using devices:

16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/run/media/nemo/XXXXXXXX/.android-sdcard

Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt them with chown and chmod, if necessary.
Finally backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched!

Mind that the altered shell script may be overwritten by SailfishOS upgrades, hence check for this then (at least when Android apps are failing to find their data after a SailfishOS upgrade) and reestablish the change (in the updated script file!), if overwritten (see timestamps with ls -l).

2.2.1 Additional step for devices with AOSP4.4 based AlienDalvik

On "2nd gen" devices (e.g. Xperia X, Jolla C, Intex Aquafish, Jolla Tablet) perform this additional step (i.e., not on BTRFS-using devices as the Jolla 1!), which can be done as user nemo (root user should also work):

cd /home/nemo
mv -v android_storage android_storage.bak
ln -s /run/media/nemo/XXXXXXXX/.android-sdcard android_storage

2.3 Finishing up

Reboot to see, if everything working as intended: Check with your Android apps, if they access their data as before, as from their perspective everything should be the same as before externalising their data.

If in doubt, check if the relevant mounts succeeded:
Issue as user nemo a mount | fgrep mmcblk1 for unencrypted SD-cards respectively mount | fgrep crypto for encrypted SD-cards (employed per part 1): You should see each partition being mounted twice when AlienDalvik is running, once regularly at /run/media/nemo/XXXXXXXX as the mounted target (i.e., in the second field after the "on") and a second time at /opt/alien/run/media/nemo/XXXXXXXX for AlienDalvik.

  • Additional check for BTRFS-using devices
    Issue a mount -t fuse: Again, you should see /dev/fuse being mounted twice when AlienDalvik is running, once regularly at /home/nemo/android_storage as the mounted target and at /opt/alien/home/nemo/android_storage for AlienDalvik.

If this does not yield the expected outcome, debug with journalctl -r as root user, while having less installed (per pkcon install less): Search with "/" for mmcblk1 rsp. crypto (if using an encrypted SD-card) (additionally on BTRFS-using devices: look for fuse), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Report your findings along with the SailfishOS version and the device you use in a comment here by describing your experience and pasting the output generated by above commands, which does not look as described.

2.3.1 Removing the old content of android_storage residing on internal eMMC

When working well, delete the old content of android_storage on internal eMMC, as this is the purpose of the whole procedure:

* On "2nd gen" devices:

devel-su
cd /home/nemo
rm -rf android_storage.bak

* On BTRFS-using devices:

devel-su
systemctl stop alien-service-manager.service  # Stop AlienDalvik completely
cd /data/media
ls -la | more  # Check: You should *not* see ANDROID-SDCARD
pwd  # Check twice that you are really in the directory you cd'ed to two lines above!
rm -rf * .[^.]*  # !!!

Done.

edit retag flag offensive close delete

Comments

note that, like said, the AOSP 8.1 Oreo layer on the XA2, works in a completely different way:

  • unlike X and J1, ~nemo/android_storage is actually empty.
  • like them, the android data is in /home/.android
  • on XA2, like on a real android, storage is mounted un /home/.android/data/media/0
  • it's fuse mounted to ~nemo/android_storage while alien dalvik is running (and before running, any data in nemo/android_storage is imported in media/0 - for backward compatibility ?)
  • i can presume that, once supported, SD in compatibility layer should appear in media/1 like on a real android ?
DrYak ( 2019-04-08 00:22:35 +0300 )edit

Oh, that is half J1 and half "2nd gen" style:

  • On a J1, ~nemo/android_storage is actually empty, too.
  • But the J1 has its android data in /data
  • /data/media is fuse mounted to ~nemo/android_storage while alien dalvik is running on a J1
  • There already are artifacts of the Android SD compatibility layer on "2nd gen" devices
olf ( 2019-04-11 00:17:31 +0300 )edit

1 Answer

Sort by » oldest newest most voted
1

answered 2019-04-08 00:21:31 +0300

M.Bln. gravatar image

First: Great work. And thanks for the guide!

But even if I would like to have more options for storage (swapping folders etc.) I am afraid to break something with the next SF-update. So I would like more to see an option in SF to do so instead of writing scripts for it.

Anyway it's nice to have the possibility and a guide to do so myself! :-)

edit flag offensive delete publish link more

Comments

@M.Bln., this is a comment, not an answer, so please convert it.

olf ( 2019-04-10 16:05:01 +0300 )edit
Login/Signup to Answer

Question tools

Follow
7 followers

Stats

Asked: 2019-04-08 00:07:25 +0300

Seen: 232 times

Last updated: Apr 17