Revision history [back]
 | 1 | initial version | posted 2018-02-12 00:26:04 +0300  |
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute: cd /media/sdcard/XXXXXXXX/ echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt mkdir .nemo-sdcard cd ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card: cd Foo mkdir /media/sdcard/XXXXXXXX/.foo cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful cd .. rm -r Foo ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees): cd Foo mkdir /media/sdcard/XXXXXXXX/.foo cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful rm Bar ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples: ~nemo/Music --> /media/sdcard/XXXXXXXX/.music ~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout ~nemo/Maps/map.navit.bin -> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data.
2.1 Copy data to SD-card
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount: devel-su cd /usr/sbin/ cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
/bin/mount --rbind "$DATADIR" "$MOUNTDIR" /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched.
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount.
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh: devel-su cd /opt/alien/system/script/ cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (insert your UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched.
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/<expression>" (e.g. for "card bind", "mmcblk1pX"), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see systemctl status data-media.mount.
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/: devel-su systemctl stop alien-service-manager.service # Stop AlienDalvik completely umount /data/media # /home/.android/data/media/ for non-BTRFS devices cd /data/media/ # Ditto! ls -la | more # Check: You should not see ANDROID-SDCARD <do delete="" large="" files,="" downloads="" etc.><="" p="">
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use: devel-su systemctl stop alien-service-manager.service # Stop AlienDalvik completely umount /data/media # /home/.android/data/media for non-BTRFS devices rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
| | 2 | No.2 Revision |
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute:
execute:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-cardSD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card:
SD-card:
cd Foo mkdir /media/sdcard/XXXXXXXX/.foo cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful cd .. rm -r Foo ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees):
trees):
cd Foo mkdir /media/sdcard/XXXXXXXX/.foo cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful rm Bar ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples: - ~nemo/Music --> /media/sdcard/XXXXXXXX/.music - ~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout - ~nemo/Maps/map.navit.bin -> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data.
2.1 Copy data to SD-card
Execute:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount:
:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.origapkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
--- ---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched.
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount.
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh:
:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.origplatform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (insert your UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched.
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/<expression>" (e.g. for "card bind", "mmcblk1pX"), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see systemctl status data-media.mount.
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
/data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should not *not* see ANDROID-SDCARD
<do delete="" large="" files,="" downloads="" etc.><="" p="">
<Do delete large files, downloads etc.>
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
3 No.3 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute:execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card:
SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo FooFoo
You can also externalise specific files (instead of whole directory trees):
trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar BarBar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
- examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
- /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
- /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin -> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data.
2.1 Copy data to SD-card
Execute:Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount: (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched.
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount.
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh: (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (insert your UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched.
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/<expression>" (e.g. for "card bind", "mmcblk1pX"), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see systemctl status data-media.mount.
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
<Do delete large files, downloads etc.>
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
4 No.4 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin -> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data.
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched.
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount.
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (insert your UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched.
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/<expression>" (e.g. for "card bind", "mmcblk1pX"), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see systemctl status data-media.mount.
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
<Do delete large files, downloads etc.>
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
5 No.5 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin -> --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (insert your UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched.
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/<expression>" (e.g. for "card bind", "mmcblk1pX"), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see systemctl "/\<expression\>" (e.g.="" for="" "card="" bind",="" "mmcblk1px"),="" use="" "n"="" to="" jump="" to="" the="" next="" hit="" and="" press="" pageup="" pagedown="" to="" look="" at="" adjacent="" messages.="" also="" see="" <code="">systemctl status data-media.mount.
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
<Do delete large files, downloads etc.>
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
6 No.6 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (insert your UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/\<expression\>" (e.g.="" for="" "card="" bind",="" "mmcblk1px"),="" use="" "n"="" to="" jump="" to="" the="" next="" hit="" and="" press="" pageup="" pagedown="" to="" look="" at="" adjacent="" messages.="" also="" see="" <code="">systemctl "/\<\expression>" (e.g. for "card bind", "mmcblk1pX"), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
<Do delete large files, downloads etc.>
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
7 No.7 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (insert your (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/\<\expression>" "/" (e.g. for "card bind", "mmcblk1pX"), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
<Do delete large files, downloads etc.>
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
8 No.8 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" firstSD-cards". first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for "card bind", "mmcblk1pX"), card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
<Do delete large files, downloads etc.>
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
9 No.9 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmc1pXmmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
<Do delete large files, downloads etc.>
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
So a cd /data/media/ && rm -rf * .[^.]* is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot. reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
overwritten.
Contributions and constructive suggestions are welcome.
10 No.10 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
<Do # Do delete large files, downloads etc.>
etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available.
available. So a cd /data/media/ && rm -rf * .[^.]*
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: P.S.:
Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
11 No.11 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.:
Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
overwritten.
Contributions and constructive suggestions are welcome.
12 No.12 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln - s -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln - s -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
13 No.13 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory ".android-sdcard" .android-sdcard is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemomount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
14 No.14 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemonemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
15 No.15 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
NB: Using Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location). location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
16 retagged
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then unmounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
17 No.17 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects the Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then unmounted) not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
18 No.18 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This generally preferable method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
19 No.19 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Do delete Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
20 No.20 Revision
[How-to] Externalising android_storage and other directories to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you have an additional backup and do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
21 No.21 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards" first.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing:executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing:executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
22 No.22 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating filesystems and partitions on SD-cards"SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards, so if things turn out to look different with other devices or SD-cards, you may have to adapt the following sections (thus some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mount containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with systemctl enable data-media.mount
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
Contributions and constructive suggestions are welcome.
23 No.23 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards, so if things turn out to look different with other devices or SD-cards, you may have to adapt the following sections (thus some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit /etc/systemd/system/data-media.mountdata-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
24 No.24 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards, so SD-cards; if things turn out to look different with other devices or SD-cards, devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
25 No.25 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin/media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
~nemo/Downloads/FollowMe --> /media/sdcard/XXXXXXXX/.comics/followme
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
26 No.26 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
/media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
~nemo/Downloads/FollowMe --> /media/sdcard/XXXXXXXX/.comics/followme
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4.2 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
27 No.27 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
~nemo/Downloads/FollowMe --> /media/sdcard/XXXXXXXX/.comics/followme
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units here (and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.4 2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.4.1 2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4.2 2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
28 No.28 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
~nemo/Downloads/FollowMe --> /media/sdcard/XXXXXXXX/.comics/followme
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount (backup first) by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig
apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compared (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other units unit files here (and (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh (backup first) by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig
platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also see output of systemctl status data-media.mount
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
P.S.: Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
29 No.29 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
~nemo/Downloads/FollowMe --> /media/sdcard/XXXXXXXX/.comics/followme
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. Also When using method A, also see output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
30 No.30 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
~nemo/Downloads/FollowMe --> /media/sdcard/XXXXXXXX/.comics/followme
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell scripts may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages. messages.
When using method A, also see check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
31 No.31 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
~nemo/Downloads/FollowMe --> /media/sdcard/XXXXXXXX/.comics/followme
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell scripts script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
32 No.32 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin
~nemo/Downloads/FollowMe --> /media/sdcard/XXXXXXXX/.comics/followme
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
33 No.33 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/.maps/osmscout/osmscout/media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/.maps/osm-maps/osm_bbox_X,Y,U,V.bin/media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, so check this functionality carefully after that and reestablish the changes (in the newly updated files!), if overwritten.
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
34 No.34 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, so hence check for this functionality carefully after that then and reestablish the changes (in the newly updated files!), if overwritten.overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
35 No.35 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copy Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: Setting By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
When done, reboot.
36 No.36 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media /media/sdcard/XXXXXXXX/.android-sdcard | grep -vF " is uptodate" # Should result in *total: matches=0 hash_hits=0 false_alarms=0 data=0*
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
rsync -av cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices.
devices; may also take a look at the option -n; switch source and destination directories for updating the other way around
When done, reboot.
37 No.37 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
- 1. Externalising regular files and directories in /home/nemo
~nemo/android_storage/home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising ~nemo/android_storage/home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media /media/sdcard/XXXXXXXX/.android-sdcard | grep -vF " is uptodate" # Should result in *total: matches=0 hash_hits=0 false_alarms=0 data=0*
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n; -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
38 No.38 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media /media/sdcard/XXXXXXXX/.android-sdcard media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should result in *total: matches=0 hash_hits=0 false_alarms=0 data=0*
detect no files to be not up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
39 No.39 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for the specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files to be not up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
40 No.40 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for SailfishOS 2.2.0 (and newer) all occurrences of */media/sdcard * must be altered to */run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /rum/media/nemo/XXXXXXXX).*
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files to be not up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
41 No.41 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for SailfishOS 2.2.0 (and newer) all occurrences of all/media/sdcard occurrences of */media/sdcard * must be altered to */run/media/nemoto /run/media/nemo (and consequently consequently /media/sdcard/XXXXXXXX /media/sdcard/XXXXXXXXto to /run/media/nemo/XXXXXXXX /rum/media/nemo/XXXXXXXX).).*
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files to be not up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
42 No.42 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files to be not up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
43 No.43 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files to be not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
44 No.44 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files to not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
45 No.45 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available.available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
46 No.46 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
If you want to utilise an encrypted SD-card, which is unlocked and mounted by crypto-sdcard, mount-sd must be replaced twice by mount-cryptosd in above bind mount unit.
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
47 No.47 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
For SailfishOS 2.2.0 and later, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
If you want to utilise an encrypted SD-card, which is unlocked and mounted by crypto-sdcard, mount-sd must be replaced twice by mount-cryptosd in above bind mount unit.
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
48 No.48 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: This guide is also applicable to SailfishOS 2.1.4, but for For SailfishOS 2.2.0 (and newer) all and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
For SailfishOS 2.2.0 and later, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
49 No.49 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 with two different SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and later, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
50 No.50 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS 2.1.3.7 versions 2.1.3 to 3.0.0 with two different various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer) all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and later, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
51 No.51 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 to 3.0.0 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 (and newer)and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and later, newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary).necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary. necessary.If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by"Before=alien-service-manager.service") for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B).
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
52 No.52 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 to 3.0.0 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per by"Before=alien-service-manager.service") by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B).B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
53 No.53 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 to 3.0.0 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for SailfishOS 2.2.0 and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2. 2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
54 No.54 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newernewer, a reworked, updated and more streamlined version of this how-to exists, adapting all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 to 3.0.0 and 2.1.4 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Also mind that for Furthermore, the instructions in this section 2 is only suitable for BTRFS-using devices under SailfishOS 2.2.0 and newer all occurrences of /media/sdcard must be altered to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ).versions below 2.2.0!
For SailfishOS versions since 2.2.0, use the reworked, updated and streamlined version of this how-to.
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
55 No.55 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer, a reworked, updated and more streamlined version of this how-to exists, adapting all occurrences of /media/sdcard to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 and 2.1.4 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Furthermore, the instructions in this section 2 is only suitable for BTRFS-using devices under SailfishOS versions below 2.2.0!
For SailfishOS versions since 2.2.0, use the reworked, updated and streamlined version of this how-to.
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
56 No.56 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer, a reworked, updated and more streamlined version of this how-to exists, adapting all occurrences of /media/sdcard to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 and 2.1.4 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Furthermore, the instructions in in thisthis section 2 is only suitable for BTRFS-using devices (i.e. Jolla 1) under SailfishOS versions below 2.2.0!
For SailfishOS versions since 2.2.0, use the reworked, updated and streamlined version of this how-to. and its newer section 2.
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
57 No.57 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer, a reworked, updated and more streamlined version of this how-to exists, adapting all occurrences of /media/sdcard to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 and 2.1.4 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Furthermore, the instructions in this section 2 is are only suitable for BTRFS-using devices (i.e. Jolla 1) under SailfishOS versions below 2.2.0!
For SailfishOS versions since 2.2.0, use the reworked, updated and streamlined version of this how-to and its newer section 2.
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
58 No.58 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer, a reworked, updated and more streamlined version of this how-to exists, adapting all occurrences of /media/sdcard to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 and 2.1.4 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Furthermore, the instructions in this section 2 are only primarily suitable for BTRFS-using devices (i.e. Jolla 1) under SailfishOS versions below before 2.2.0!
For all other devices and SailfishOS versions since 2.2.0, use the reworked, updated and streamlined version of this how-to and its newer section 2.
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
59 No.59 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer, a reworked, updated and more streamlined version of this how-to exists, adapting all occurrences of /media/sdcard to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 and 2.1.4 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Because AlienDalvik needs a filesystem providing classic UNIX access rights and ownerships, this section 2 does not work using a partition on SD-card formatted with (V)FAT.
Use "[How-to] Creating partitions on SD-card, optionally encrypted" to employ a (at least one) proper, "native" filesystem on your SD-card before continuing. Also see the shortcuts provided for this lengthy how-to.
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Furthermore, the instructions in this section 2 are primarily suitable for BTRFS-using devices (i.e. Jolla 1) under SailfishOS versions before 2.2.0!
For all other devices and SailfishOS versions since 2.2.0, use the reworked, updated and streamlined version of this how-to and its newer section 2.
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
60 No.60 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer, a reworked, updated and more streamlined version of this how-to exists, adapting all occurrences of /media/sdcard to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 and 2.1.4 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Because AlienDalvik needs a filesystem providing classic UNIX access rights and ownerships, this section 2 does not work using a partition on SD-card formatted with (V)FAT.
Use Utilise the "[How-to] Creating partitions on SD-card, optionally encrypted" to employ a (at least one) proper, "native" filesystem on your SD-card before continuing. Also see the shortcuts provided for this lengthy how-to.
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Furthermore, the instructions in this section 2 are primarily suitable for BTRFS-using devices (i.e. Jolla 1) under SailfishOS versions before 2.2.0!
For all other devices and SailfishOS versions since 2.2.0, use the reworked, updated and streamlined version of this how-to and its newer section 2.
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.
61 No.61 Revision
[How-to] Externalising android_storage and other directories / files to SD-card
Although there are a couple of guides for extending the internal mass storage (eMMC) of devices running SailfishOS (in general; this is not addressing the "tiny system partition issue" of SailfishOS on the Xperia X and Jolla C / Intex Aquafish) with a SD-card (on TMO and here on TJC), IMO none of them is completely satisfactory (technically, by functionality and usability), but they provided valuable ideas.
Note: For SailfishOS 2.2.0 and newer, a reworked, updated and more streamlined version of this how-to exists, adapting all occurrences of /media/sdcard to /run/media/nemo (and consequently /media/sdcard/XXXXXXXX to /run/media/nemo/XXXXXXXX ), as this guide was originally written for SailfishOS 2.1.x.
- 1. Externalising regular files and directories in /home/nemo
- 2. Externalising /home/nemo/android_storage
This is part 2 "Externalising android_storage and other directories / files to SD-card", you may want to read part 1 "Creating partitions on SD-card, optionally encrypted" first.
This was tested on two Jolla 1 phones under SailfishOS versions 2.1.3 and 2.1.4 with various SD-cards; if things turn out to look different with other devices, you may have to adapt the following sections (thus then some understanding of the commands issued is crucial); and last but not least this description may be faulty. Please report your experiences and adaptations in these cases, but do not blame anyone (e.g. me) for any mishaps.
Contributions and constructive suggestions are welcome.
1. Externalising regular files and directories in /home/nemo
For externalising android_storage, see section 2.
Note using "hidden" directory names (starting with a ".") on the SD-card for files not being indexed by Tracker twice (there and in their original location).
Use regular directory names for all data, which solely resides on the SD-card (i.e. without a link somewhere under /home/nemo/) and should be indexed by the media tracker.
As user nemo, execute the following commands:
cd /media/sdcard/XXXXXXXX/
echo 'Data externalised from /home/nemo/ to SD-card resides in "hidden" directories starting with a ".".' > README-data.txt
mkdir .nemo-sdcard
cd
ln -s /media/sdcard/XXXXXXXX/.nemo-sdcard SD-card
You may move existing, regular directories or subdirectories under /home/nemo/ (but not android_storage or "hidden" directories) onto SD-card by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -av . /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
cd ..
rm -r Foo
ln -s /media/sdcard/XXXXXXXX/.foo Foo
You can also externalise specific files (instead of whole directory trees) by executing (you may want to back them up, first):
cd Foo
mkdir /media/sdcard/XXXXXXXX/.foo
cp -pv Bar /media/sdcard/XXXXXXXX/.foo/ # Only continue, if copying has been successful
rm Bar
ln -s /media/sdcard/XXXXXXXX/.foo/Bar Bar
Note that ".foo" and "Foo" are just a placeholders for specific directory names, "XXXXXXXX" is a placeholder for a specific mounted partition on the SD-card and "Bar" is a placeholder for a specific file.
Arbitrary examples:
~nemo/Music --> /media/sdcard/XXXXXXXX/.music
~nemo/Videos --> /media/sdcard/XXXXXXXX/.videos
~nemo/Documents/Maps --> /media/sdcard/XXXXXXXX/maps/osmscout/osmscout
~nemo/Maps/map.navit.bin --> /media/sdcard/XXXXXXXX/maps/osm-maps/osm_bbox_X,Y,U,V.bin
2. Externalising /home/nemo/android_storage
Because AlienDalvik needs a filesystem providing classic UNIX access rights and ownerships, this section 2 does not work using a partition on SD-card formatted with (V)FAT.(v)FAT.
Utilise the "[How-to] Creating partitions on SD-card, optionally encrypted" to employ a (at least one) proper, "native" filesystem on your SD-card before continuing. Also see the shortcuts provided for this lengthy how-to.
Mind that devices with BTRFS as their native filesystem (e.g. Jolla 1 phone) use /data as AlienDalvik's data directory, while all other devices (e.g. SailfishX, Intex Aquafish, Jolla C, Jolla Tablet) use /home/.android/data
Furthermore, the instructions in this section 2 are primarily suitable for BTRFS-using devices (i.e. Jolla 1) under SailfishOS versions before 2.2.0!
For all other devices and SailfishOS versions since 2.2.0, use the reworked, updated and streamlined version of this how-to and its newer section 2.
2.1 Copying data to SD-card
Execute the following commands:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
cd /data # /home/.android/data for non-BTRFS devices
cp -av media /media/sdcard/XXXXXXXX/.android-sdcard # Pick the right UUID!
touch /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Set a marker
chmod a= /media/sdcard/XXXXXXXX/.android-sdcard/ANDROID-SDCARD # Make it persistent
In order to check via checksums, that the data has been correctly written (needs rsync installed):
rsync -nc -avvhh media/ /media/sdcard/XXXXXXXX/.android-sdcard/ | grep -vF " is uptodate" # Should detect no files not being up-to-date
2.2 Integrating externalised android_storage for automatic startup
Unfortunately there seems to be no proper way of integrating an externalised /data/media (or /home/.android/data/media) directory (which later on becomes mounted on android_storage) seamlessly into SailfishOS with AlienDalvik without altering one of its scripts.
Mind that the altered shell script (a single one for either of the two methods following) may be overwritten by SailfishOS updates, hence check for this then and reestablish the changes (in the newly updated files!), if overwritten (see timestamps with ls -l).
2.2.a Method A: With a bind mount unit
This method falls back to the original content of /data/media/ (or /home/.android/data/media), if mounting it on the SD-card fails (this SD-card not present or died, unable to unlock encrypted partition etc.). So you will still be able to use Android apps with the older data on internal eMMC in this situation, but then the data on SD-card becomes "out of sync" with what the apps last wrote, hence failing may be preferred (see method B for this).
Note: For SailfishOS 2.2.0 and newer, please use method B, as method A would need to be adapted for these SailfishOS versions.
Furthermore, while both methods have been working fine for months, method B is easier to employ.
Alter /usr/sbin/apkd-mount by executing:
devel-su
cd /usr/sbin/
cp -p apkd-mount apkd-mount.orig # Backup
Edit the second last line (insert "r") and add a new line after it (before the final "fi"), so that diff apkd-mount.orig apkd-mount outputs:
41c41,42
< /bin/mount --bind "$DATADIR" "$MOUNTDIR"
---
> /bin/mount --rbind "$DATADIR" "$MOUNTDIR"
> /bin/mount --make-rslave "$MOUNTDIR" # Without "rslave" it will not cleanly unmount
(See on using "rbind" and mount manpage.)
Backup your altered version with cp -p apkd-mount apkd-mount.patched
Check (user- and group-) ownership and access rights with ls -l apkd-mount* (compare to the backed up original) and adapt with chown and chmod, if necessary.
Create a new mount unit data-media.mount (for non-BTRFS devices: home-.android-data-media.mount) in /etc/systemd/system/, containing (you need to change =/data/media/ to =/home/.android/data/media/ twice for non-BTRFS devices):
[Unit]
Description=SD-card bind mount for AlienDalvik
ConditionPathExists=/dev/mmcblk1pX
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media/
ConditionPathIsMountPoint=/media/sdcard/XXXXXXXX/
Requires=local-fs.target systemd-user-sessions.service
Before=alien-service-manager.service
After=mount-sd@mmcblk1pX.service
BindsTo=mount-sd@mmcblk1pX.service
[Mount]
What=/media/sdcard/XXXXXXXX/.android-sdcard/
Where=/data/media/
Options=bind
[Install]
WantedBy=graphical.target aliendalvik.service
Mind to substitute all "X" with appropriate values for your configuration.
Check with ls -l that the (user- and group-) ownership and access rights for data-media.mount are as for the other unit files here (i.e. "-rw-r--r-- root root data-media.mount" on a Jolla 1 under SFOS 2.1.3.7) and adapt with chown and chmod, if necessary.
Finally activate this mount unit with: systemctl enable data-media.mount # home-.android-data-media.mount for non-BTRFS devices
Note that "Before=alien-service-manager.service" starts this mount unit quite early (possibly prolonging boot time), because alien-service-manager.service calls (among other things) /opt/alien/system/script/alien_service_manager.sh, which in turn calls the binary executable $ANDROID_ROOT/bin/servicemanager ($ANDROID_ROOT = /opt/alien/system): As this is a binary file, there is no easy way to determine, if it relies on /opt/alien/data/media to be static.
While by simple testing it appears to be O.K. to use "Before=aliendalvik.service" (which is significantly later), between "alien-service-manager.service" and "aliendalvik.service" "alien-settings.service" is started, which poses a similar issue by calling /opt/alien/system/script/alien_settings_server.sh, which in turn calls the binary executable $ANDROID_ROOT/genv/bin/alien_settings_server: Also unable to determine if it relies on /opt/alien/data/media to be static.
By reverse-engineering the somewhat convoluted call-chains from the systemd unit files to the scripts in /opt/alien/system/script/ and further, using "Before=aliendalvik.service" in above mount unit is O.K., except for the aforementioned two binaries, for which it is unknown. So I prefer to be on the safe side by starting above mount unit early.
2.2.b Method B: By setting MEDIA_STORAGE in /opt/alien/system/script/platform_envsetup.sh
This alternative method redirects Android's "data/media" directory directly to SD-card, so it will not fall back to the original content of /data/media/ (or /home/.android/data/media/), if mounting it on the SD-card fails (i.e. specific SD-card not present or died, unable to unlock encrypted partition etc.): In this case a new, empty, "hidden" directory .android-sdcard is created in the (then not mounted) mount-directory /media/sdcard/XXXXXXXX/ on internal eMMC.
This is the preferable method, if you plan to delete the original content of /data/media/ (or /home/.android/data/media/) completely (see section 2.4.1) or to configure a freshly installed device.
Alter /opt/alien/system/script/platform_envsetup.sh by executing:
devel-su
cd /opt/alien/system/script/
cp -p platform_envsetup.sh platform_envsetup.sh.orig # Backup
Edit line 16, so that diff platform_envsetup.sh.orig platform_envsetup.sh outputs (replace XXXXXXXX with the right UUID):
16c16
< MEDIA_STORAGE=$ANDROID_DATA/media
---
> MEDIA_STORAGE=/media/sdcard/XXXXXXXX/.android-sdcard
Backup your altered version with cp -p platform_envsetup.sh platform_envsetup.sh.patched
Check (user- and group-) ownership and access rights with ls -l platform_envsetup.sh* (compared to the backed up original) and adapt with chown and chmod, if necessary.
If you are using SailfishOS 2.2.0 or newer and an unencrypted SD-card, please install mount-sdcard (e.g., per Storeman) to make automatic mounting on boot-up working reliably (i.e., to be race-free by a "Before=alien-service-manager.service" statement) for method B.
For automatic mounting of encrypted SD-cards, install (additionally or alternatively) crypto-sdcard, when using SailfishOS 2.2.0 or newer. For older versions of SailfishOS one has to employ these changes manually, as described in section 4.3.5 of "[How-to] Creating partitions on SD-card, optionally encrypted" for either method (A or B) to use encrypted SD-cards.
2.3 Finishing up
Reboot to see, if everything working as intended. If in doubt, issue as user nemo mount | grep mmcblk1pX as a first check.
If needed, debug with journalctl -r as root user, while having less installed (e.g. per pkcon install less): Search with "/" (e.g. for card bind or mmcblk1pX), use "n" to jump to the next hit and press PageUp / PageDown to look at adjacent messages.
When using method A, also check output of systemctl status data-media.mount # home-.android-data-media.mount for non-BTRFS devices
2.3.1 Removing data residing on internal eMMC
When working well, you may delete large files / directories (or everything) in the original /data/media/:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media/ for non-BTRFS devices
cd /data/media/ # Ditto!
ls -la | more # Check: You should *not* see ANDROID-SDCARD
# Delete large files, downloads etc., or empty completely with: rm -rf * .[^.]*
Before emptying /data/media/ (or /home/.android/data/media/) completely, mind that this is the fallback data for method A, when the partition on the SD-card is not available. So a deleting everything there is fine, as long you do not rely on Android apps (or yourself) to have this data available without SD-card.
When done, reboot.
2.4 Updating externalised data to internal eMMC or vice versa
To update subdirectories or files in the original /data/media/ (or /home/.android/data/media/) with newer data from /media/sdcard/XXXXXXXX/.android-sdcard/ (or vice versa) you may use:
devel-su
systemctl stop alien-service-manager.service # Stop AlienDalvik completely
umount /data/media # /home/.android/data/media for non-BTRFS devices
cp -auv /media/sdcard/XXXXXXXX/.android-sdcard/osmand /data/media/osmand # Just an example; /home/.android/data/media/osmand for non-BTRFS devices; may also take a look at the option -n in cp's manpage; switch source and destination directories for updating the other way around
When done, reboot.