How to: Format your uSD-Card to share space with android
Update: For JollaC see at the bottom, things changed due to missing btrfs support and android mounting has changed a bit too Old stuff in question's history
If you are OK with becoming root and follow a step by step guide on your own account, go ahead reading this guide.
Version: >1.1.9.x
This is no beginners guide, you need to know your way around!
Understand what to do before you do!
What I did to share the space of my uSD between nemo and android without getting duplicates with bind-mounts or any other hickup, read carefully and use at your own risk:
- format the whole usdcard (mmcblk1 not mmcblk1p1) to btrfs
- mount that volume (=0)
- create two subvolumes > jolla and android
- set the default volume to jolla (the default is what gets mounted when something calls
mount /dev/mmcblk1
) - stop alien-dalvik and copy everything from /data/media to the new subvolume android (don't move the files, this way you have all basics alien-dalvik needs even after pulling the uSD)
- add a system.service to systemd that mounts and unmounts the subvolume for android as needed and honours aliendalvik.service The problems to solve are: aliendalvik fuse-mounts its /data/media to /home/nemo/android_storage so you need to make sure everything is mounted before that happens on boot, and stop aliendalvik when the sd gets pulled or the mount needs to be lifted for any other cause.
- reboot and wait for the device to settle, as soon as your jolla volume is mounted in
/media/
, you should see /dev/mmcblk1 being mounted to both /media/sdcard/#UUID and /data/media
detailed cmds use at your own risk
preparing the sdcard, should only be needed once!
first dosystemctl stop aliendalvik.service & systemctl stop aliendalvik.path
then follow this list
umount /dev/mmcblk1p1
- unmount the card might be another /dev or even more than onemkfs.btrfs -f /dev/mmcblk1
mkfs.btrfs -O ^extref -f /dev/mmcblk1
- format the card (no need for partition) - for now extref (this extends num of hardlinks available) needs to be disabled, even on the devicemount -o subvolid=0 /dev/mmcblk1 /media/sdcard
cd /media/sdcard
btrfs subvolume create jolla
- create folder called jollabtrfs subvolume create android
- create folder called androidbtrfs subvolume list .
- returns an ID for each subvolumebtrfs subvolume set-default *ID for jolla* /media/sdcard/
- sets jolla to be mounted as defaultchown nemo:nemo jolla/
- add-R
if neededchmod 775 jolla/
chmod 777 android/
cp -r -a -v /data/media/* /media/sdcard/android
cd ..
umount /dev/mmcblk1
preparing the system
- create (as in do create not a command)
/etc/systemd/system/data-media.mount
and fill with:
#
[Unit]
Description=SD card btrfs subvolume for Android
ConditionPathExists=/dev/mmcblk1
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media
Requires=local-fs.target systemd-user-sessions.service
Before=aliendalvik.service
After=mount-sd@mmcblk1.service
BindsTo=mount-sd@mmcblk1.service
[Mount]
What=/dev/mmcblk1
Where=/data/media
Options=subvol=android,compress,dirsync,noatime,users
[Install]
WantedBy=local-fs.target aliendalvik.service
Second, create /etc/systemd/system/aliendalvik.service.d/sdcard.conf file:
[Unit]
PartOf=data-media.mount
Then run systemctl enable data-media.mount
.
- the mount file gets started every time aliendalvik is started and stops aliendalvik when it gets stopped, on boot it prevents aliendalvik from starting if it is not started yet but waits for the local-fs to be settled and actually having a mountpoint
for the future we only need to load the service file to its place and enable it, if something changes I will update the code, upgrade went through without anything (see first line for updates) as the updates do not touch data-media.mount as of yet
Jolla C
This has major differences with the original solution for Jolla1 https://together.jolla.com/question/40802/how-to-format-your-usd-card-to-btrfs-and-share-space-with-android/?answer=140223#post-id-140223
- The SDcard is formatted to ext4 instead of btrfs (as there is no btrfs support on JollaC yet). I also kept a conventional partition layout (to make it more likely other systems can easily mount the card. although they need to support ext4).
- The android directory that is overlaid (bind-mount) is /home/nemo/android_storage instead of /data/media. Which is actually a good thing as now Alien is having symlinks instead of mounting in circles.
- Bind mounts are used, instead of btrfs subvolume mounts. The Android directory is hidden in an attempt to prevent tracker indexing the files twice.
New instructions for JollaC are as follows:
preparing the sdcard should only be needed once!
Note: some ext4 formatted uSD cards did not show up or rendered unmountable on JollaC, best fit is to format on device
first dosystemctl stop aliendalvik.service
then follow this list
umount /dev/mmcblk1p1
- unmount the card might be another /dev or even more than onemkfs.ext4 /dev/mmcblk1p1
- format the cardmount /dev/mmcblk1p1 /media/sdcard
cd /media/sdcard
mkdir .android
- create folder called .android - this is hidden (starts with a dot) so it does not appear in directory listingsI like to create a file on the card to remind me what I have done: so create a file called
Android.README
and fill it with:Android files are contained in a hidden directory called .android
I also like to create a file with a unique name to let me easily check that Android apps are seeing the expected SD card. Something like:
touch .android/GRC-128GB-SD.android
chown -R nemo:nemo .android/
chmod -R a=rwX .android/
cp -r -a -v /home/nemo/android_storage/* /media/sdcard/.android
cd ..
umount /dev/mmcblk1p1
preparing the system
- The next step requires knowing the Filesystem UUID for the SDcard filesystem. You can get it using
lsblk -no UUID /dev/mmcblk1p1
. The UUID is a string like 12345678-9abc-def0-1234-56789abcdef0. This string must be used instead of the string UUID-HERE in the file below (two occurrences). You can also find the UUID in the folder the uSD is mounted to as it follows that scheme (in/media/sdcard/
the folder of the currently mounted uSD). - create (as in do create not a command)
/etc/systemd/system/home-nemo-android_storage.mount
and fill with:
#
[Unit]
Description=SD card bind mount for Android
ConditionPathExists=/dev/mmcblk1p1
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/home/nemo/android_storage/
ConditionPathIsMountPoint=/media/sdcard/UUID-HERE/
Requires=local-fs.target systemd-user-sessions.service
Before=aliendalvik.service
After=mount-sd@mmcblk1p1.service
BindsTo=mount-sd@mmcblk1p1.service
[Mount]
What=/media/sdcard/UUID-HERE/.android/
Where=/home/nemo/android_storage/
Type=none
Options=bind
[Install]
WantedBy=local-fs.target aliendalvik.service
Then run systemctl enable home-nemo-android_storage.mount
.
the mount file gets started every time aliendalvik is started and stops aliendalvik when it gets stopped, on boot it prevents aliendalvik from starting if it is not started yet but waits for the local-fs to be settled and actually having a mountpoint
If you put a different SD card in, the mount file will do nothing (because the UUID does not match). The original /home/nemo/android_storage will be used. You can repeat the steps from the top, of course, for the new card.
Excellent. Many thanks:)
richardski ( 2014-04-26 02:56:05 +0300 )editworks as expected! Thanks a million. Do we need to remove the card for next sailfish-sysupdate or won't there be foreseeable issues?
mosen ( 2014-04-26 20:13:51 +0300 )edit@mosen no need to remove the card, the only things that might happen are that Jolla decides to mount all available subvolumes to /media/sdcard/$(UUID) or replace the mount-sd.sh script in another way (knowing the line by heart or having a backup of your mount-sd.sh might come handy at some point). Both cases do no harm as long as you did not delete your /data/sdcard/ folder from your device's internal storage. If there be problems in the future I will try to update the how-to accordingly.
chemist ( 2014-04-26 20:35:52 +0300 )editWhat's the best way to edit mount-sd.sh? I've done it with copying the file and editing it on computer (wordpad), but it got my phone into a bootloop... I've fixed that now and really want to try it again (1.0.5.19), but just want to be sure that what I'm doing is the right way...
RobNas ( 2014-04-27 15:16:31 +0300 )edit@RobNas I did all steps on the Jolla, for editing files I use vim. If you copy files from and to your device, you need to make sure that permissions and ownership are preserved. There was a typo, $(DEVNAME) should be ${DEVNAME} of course (updated the entries)
chemist ( 2014-04-27 15:22:13 +0300 )edit@RobNas, most convenient is ssh conection to jolla or use the native terminal app. then try one of the editors vim or nano from CLI
save to file in nano is crtl+o, exit is ctrl+x
mosen ( 2014-04-27 15:29:46 +0300 )editI've edited with vim, eventually, I think the accolades {} instead of () did the trick. It looks like it's working, when I install Sygic through Google Play, I see 55GB of space. Sweet. But now Sygic is coming with stupid errors, so I can't do anything there. Sigh...a new challenge...
Thank you really, really much for the effort :-D
EDIT: Sygic is coming up. There was still data from a pre-factoryreset install, I think. Just deleted the entire folder and installed it again through Google Play. Now downloading maps...
RobNas ( 2014-04-27 22:29:11 +0300 )edityou should see /dev/mmcblk1 mounted twice now, one to /data/sdcard and one to /media/sdcard/$(UUID), (
chemist ( 2014-04-28 13:19:52 +0300 )editdf
ormount
) - if so you are all fine, note that /data/sdcard is also exported to MTP so you can actually put data on your sdcard without any hassle, but to your android folder only (for now).I see /dev/mmcblk1 mounted twice, yes. I see that if I connect the Jolla to desktop, the reachable mount is where android is. That's not very convenient, but it's doable for the time being. It's not very clear to me how space is divided...it looks like the complete space is shared among two mounts, what's nice, but a bit confusing. Unfortunately the gallery doesn't show pictures located on the android-location.
RobNas ( 2014-04-28 20:58:44 +0300 )editThats because you are just making two datasets not two partitions. So the space is shared between all datasets on that drive.(just like on ZFS)
slaveriq ( 2014-04-28 21:27:41 +0300 )edit@eric please convert to comment
@RobNas that is exactly what the title says "share space" and a bind mount does nothing else but mount a already mounted partition's mountpoint again to another mountpoint - this is much cleaner, as subvolumes are just folders ("everything is a file") you can mount and handle like partitions without restricting the physical volume or even care about the maximum size of your FS as your whole blockdevice (or many) is available to all of them at the same time
chemist ( 2014-04-28 21:33:34 +0300 )editJust for the record - it's not technically formatting, but it's an alternative. I prefer to format my SD card to vfat and create a symlink from the folder the card is mounted in (/media/sdcard/ID) to both /home/nemo/card and /data/sdcard/card. That way the android FS stays as is and I can access the data stored on the card from both android and native apps. Thanks to it being a vfat there are no issues with permissions too. You can write a file in Android app and read it in native and vice versa.
The price to pay is that your android apps are still taking up space on the internal memory. Theoretically the sdcard could be split to contain a brtfs partition and a vfat one, both could me mounted and one could contain the android app and the second the vfat shared space. But I haven't tried that.
zlc ( 2014-04-28 23:30:51 +0300 )edit@zlc as long as you keep the card in your device, what most people (especially non-developers) do, it does not matter as it gets exported as MTP anyways. Permissions is actually something you want, not try to circumvent. And your price to pay is the main reason why I was trying to get rid of it, and bind-mounts or symlinking was not an option for me. We are not arguing about semantics are we? http://en.wikipedia.org/wiki/Disk_formatting
chemist ( 2014-04-29 10:31:12 +0300 )edit@chemist Sorry, I understand how you would think I was referring to your approach but I was actually referring to mine. I don't consider my approach to be formatting as most of the SD cards are pre-formatted to FAT32 or exFAT. I'm not arguing, really :) I was just trying to present others with an alternative that doesn't require you to move everything from the /data/ folder to the SD card and formatting it to something you might have troubles mounting on some OSes. You don't mind moving everything and you require access rights that's why you do your thing. I don't mind loosing access rights (especially because I want to be able to stuff the card into my PC and work with the data) but I don't want to move everything to the card so I do my thing. That's all there is to it, really.
zlc ( 2014-04-29 11:51:47 +0300 )editAh ok ;) I like to stuff the card in my PC as well, especially as the MTP mount has some hick-ups sometimes. But I use linux, one reason why I bought an unfinished linux mobile-phone^^ better safe than sorry!
chemist ( 2014-04-29 11:59:37 +0300 )editWell I like Linux too and that's why I bought the phone as well. And I have Win/Linux dual boot on most of my machines but I develop Windows applications for a living so I really need to have a solid Windows installation at hand and switch depending on what I'm doing. But that's OT, sorry.
zlc ( 2014-04-29 12:09:38 +0300 )editThis will break / not work as expected in a future update. The Android SD-Card will be moved to VFAT emulation via FUSE, mounted at ~/android_storage. The directory backing that (=the one you want to move to SD card) will be /data/media
Aard ( 2014-05-19 22:20:27 +0300 )edit@Aard why mount android in userhome? and why put the sdcards mountpoint onto the sdcard?
chemist ( 2014-05-20 00:38:08 +0300 )editOr better: What do we have to do in the future to still have this possibility? Probably that's a bit difficult to say at the moment. When anyone has the update, please post here if this solution is still possible...
RobNas ( 2014-05-22 19:46:37 +0300 )edit@RobNas mount the usdcard "before" alien-dalvik FUSEs to ~/android_storage and it will be fine (I am not saying that it will be easy^^)
chemist ( 2014-05-23 10:43:09 +0300 )editDoes anyone was so bold to install the 1.0.7.16 update? If so, is this solution still working?
RobNas ( 2014-06-10 07:03:28 +0300 )editYes, and no it is not working this way anymore. If anyone is faster than me, we need a way to mount the android part of the sdcard to /data/media before alien-dalvik does the fuse mount. So ideal would be a startup-script that alien-dalvik waits for anyway.
chemist ( 2014-06-10 10:53:29 +0300 )editworkaround for now is to do everything by hand...
chemist ( 2014-06-10 14:38:02 +0300 )editcan confirm, works! Thank you chemist :D
mosen ( 2014-06-10 15:44:33 +0300 )editIf I implemented the solution on top of the page, have the paarlampi version of Sailfish, update to the latest version and do this, it's good to go?
RobNas ( 2014-06-10 23:10:52 +0300 )edityeah, if alien-dalvik is stopped you can mount it that way again
chemist ( 2014-06-11 01:16:21 +0300 )editAfter updating, everything is gone.... What I did: I had the paarlampi version, 1.0.5.19, with the solution implemented. Did the update, immediately after the update did #!/bin/hash etc... through terminal. Wanted to start Sygic. Nothing. Started to install data after a reboot. Checked sd-card: everything gone. Thanks, I can start again.
IMPORTANT: EVERYTHING ON SD-CARD WILL BE REMOVED AFTER UPDATE TO SAPUNKI IF YOU'VE IMPLEMENTED THE SOLUTION IN THE FIRST POST!
RobNas ( 2014-06-11 11:45:14 +0300 )editehrm...NO! I have no idea what you have done but if the subvolume is not mounted the way I told, you wont have the data available for android and you see the default folder fused to /home/nemo/android_storage/ and there wont be any data in /data/sdcard that you need... I will update the how-to in a few minutes to a working with current upgrade... FYI your subvolume on your microsd is not altered...
chemist ( 2014-06-11 14:42:53 +0300 )editupdated, if followed correctly this survives reboots and even pulling the sdcard...
chemist ( 2014-06-11 15:32:28 +0300 )editI modified it to be a .mount unit:
Vuubi ( 2014-06-11 20:22:11 +0300 )edithmm... ah I have to creat those^^ testing... but well, we had two files with that, with the service we have only one, but good to know what the config needs to look like, can you please edit it to reflect that it is a file you need to generate?
chemist ( 2014-06-11 20:30:57 +0300 )editNo, somehow I don't see the link to edit the comment
Vuubi ( 2014-06-11 20:38:07 +0300 )edit@eric can you fix the post above (separate code lines) please
chemist ( 2014-06-11 21:02:51 +0300 )edit@chemist: done.
eric ( 2014-06-12 11:15:43 +0300 )edit@eric thanks.
chemist ( 2014-06-12 13:16:44 +0300 )editSo that's why you set android subvolume to chmod 777, I thought it was strange. Original permissions are 770.
Vuubi ( 2014-06-12 19:20:54 +0300 )editthat is only for the folder so nothing comes to the idea to not being able to write it... this problem does only occure if you update this from the previous version and already have all your files on your sdcard as they changed user and group for the storage
chemist ( 2014-06-12 19:28:25 +0300 )edit@Vuubi can you put your comment about how to make it a .mount into an answer so we can edit it?
chemist ( 2014-06-12 19:30:15 +0300 )editThanks ! This saved my day !.I inversed the principle slightly to mount android storage by default and in the service description I created the necessary mount points in nemo's home directory. For some reason, I had to make a symbolic link to android SD card part as /data/sdcard, otherwise some Android applications did not like to start. I made my Evernote page on this public if somebody is interested: http://bit.ly/1qea9rB
canne ( 2014-06-14 13:28:43 +0300 )editSry but you are doing it wrong :/ what you do is taking everything upside-down, the default volume gets mounted by mount-sd.sh anyways so why bother, that is fine with the system and that is where you have your traditional sdcard mount - android on the other hand is now a bit more of a trick as the userdata gets fuse mounted to $HOME - even if... you had made the upside down work for you, you symlink the wrong directory to it and all the heavy userdata from android is still on device and not on sdcard, my solution does not care if some
chemist ( 2014-06-14 13:54:55 +0300 )editinstall
of an android app is huge, sure we could change that as well but the userdata like maptiles (grow easily to 12GB if you want detailed hiking maps) are stored in ~/android_storage and that is actually /data/media which we need to mount before it gets fused to $HOME@chemist Your method is certainly the one I would advice, thank you once again for sharing your insight.
canne ( 2014-06-14 14:57:05 +0300 )editOnly the following command is correctly: chown -R media_rw:media_rw /data/media/ as chmod changes the read write rights.
holgern ( 2014-06-16 16:14:31 +0300 )edit@holgern thanks for commenting, changed it.
chemist ( 2014-06-16 20:49:38 +0300 )editYou should also remove the stuff in the "old" /data/media after copying to the sdcard (to free up space)... ;)
You could change 12. to:
cp -r -a -v /data/media/* /media/sdcard/android && rm -rf /data/media
The '&&' means only run second command ('rm -rf /data/media') if the first command ('cp -r -a -v /data/media/* /media/sdcard/android') is successfully executed... :)
I freed up 3.1 GB on the internal storage... ;)
DrWilken ( 2014-07-12 02:44:53 +0300 )edit@DrWilken to free up space yes, to have "some" things available without SD, no... - in general it might be just moved as it is "user" data anyways, so nothing important anyways
chemist ( 2014-07-14 01:38:37 +0300 )editVery small change to the instructions: You have to 'cd ..' out of the /media/sdcard before unmounting or the system claims that the device is busy.
Manatus ( 2014-07-14 17:56:40 +0300 )editIs there any change/progress/update regarding to the 1.0.8.19 (Tahkalampi)? Or in other words: what do I need to do before/during/after the update to keep this working, or is this solution not working anymore?
RobNas ( 2014-07-20 13:59:23 +0300 )edit@RobNas, it's working, no changes
Vuubi ( 2014-07-20 14:05:27 +0300 )editSo I apply the update, no manual action needed and the space from SD-card is still shared?
RobNas ( 2014-07-20 14:07:59 +0300 )edit@RobNas, yes
Vuubi ( 2014-07-20 14:10:36 +0300 )editHi,
I think I got this done successfully, but I cannot access the card over USB in Windows 7. I can see it under Sailfish - SD Card, but I cannot copy or see anything in there.
Did I fail at something, has anyone gotten it to work over normal USB connection "PC Connection" -mode?
EDIT: After loading some music to the uSD Card, I can now see and access the Card over
Juuba ( 2014-08-21 10:29:50 +0300 )editMTP
in Windows 7. I don't know why or how, but now it seems to work. Will test at home with Win 8.1.Thinking about doing this, I'm a carefult guy. Would anyone consider this causing problems if Jolla comes with an update? I means, the only real change is the addition of this service file to mount the card. But still, is there anything with that that would cause any problems when Jolla comes with an update?
Larswad ( 2014-09-09 10:42:00 +0300 )edit@Larswad no need to be scared, at least only the btrfs formatting part is something to break, as long as they do not change anything, I mean, change path of android directories, there won't be any issue (in software you never can be sure but confident) - what can happen is that the service fails but iirc aliendalvik.service starts anyways just without the sdcard mounted to /data/media
chemist ( 2014-09-09 11:42:47 +0300 )edit@chemist: Thanks! I'll give it a shot, this way I can move my subsonic music cache and the DCIM folder for the android camera apps that I have (won't MOVE anything else except those, rest of the files I'll make a copy, like stated in theguid).
Sorry, but a side question pops up: Is there a benefit using the Vuubi guy's guide below to convert your service unit into a mount unit? Is it just "more correct", but have no practical difference?
Larswad ( 2014-09-09 12:15:18 +0300 )editTechnically it is right but I think it prevents aliendalvik from starting if it fails to mount (not sure) - it was what I wanted to do in the first place but ended up doing it as service and stick to it. Comment on his answer if you run into any problems, please.
chemist ( 2014-09-09 17:23:29 +0300 )editbetween step 12 and 13 add:
qbot ( 2014-10-15 22:40:06 +0300 )editUitukka, anyone?
RobNas ( 2014-10-28 17:43:27 +0300 )edit@RobNas I updated the question, everything is fine. If you upgrade to update9 with this setup everything is fine too, no user action needed.
chemist ( 2014-10-29 12:18:57 +0300 )editThanks :-)
RobNas ( 2014-10-29 12:35:05 +0300 )editIn the androidsdcard.service file it says
After=local-fs.target mount-sd@mmcblk1.serviceBindsTo=mount-sd@mmcblk1.service
This "mount-sd@mmcblk1.service", is it the one located at /etc/systemd/system?
If it is, the one in my Jolla is called mount-sd@.service and not mount-sd@mmcblk1.service. What should I do?
Aashish ( 2014-11-07 23:17:06 +0300 )editI think it has worked fine. Its evident now that some android apps are using SD card. However, I have seen that the 'used' phone memory usage increases by the same amount that the sd card usage increases. Any ideas why that is?
Aashish ( 2014-11-08 22:25:06 +0300 )editthe service created is mount-sd@mmcblk1.service if you followed the re-formatting part above, if you format your card with a partition the service will be called mount-sd@mmcblk1p1.service as the sd-card mounting service creates them, note the @ in the service file?! It is a bit difficult to explain though, services with an @ in their name are created from a meta service file that has nothing after the @ which then gets replaced with the called name, in the case above, mount-sd calls the service for mmcblk1, therefor the service is called mount-sd@mmcblk1.service. For the FS usage, I have more GB used on the SD than I have space on the internal storage, so NO this does not sound right! Are you sure it is setup correctly?
chemist ( 2014-11-10 02:18:52 +0300 )editSettings
->System
->About product
shows 5.3GB used for me, and I have like 8GB maps downloaded to the android part of the sd-card.i have tired this on 1.0.8.21 and it does not work when i connect the phone to my pc i am unable to copy files to it say the write permissions is not allowed this is in windows
also when doing the umount /dev/mmcblk1p1 it tells me there is no mount point for this
so when doing this umount /dev/mmcblk1 my sd card is unmounted and can be formatted to btrfs
is there a new way of doing this or have i done something wrong
ReflexDarky ( 2014-11-25 22:23:50 +0300 )editAnd I thought I wrote enough disclaimer... if your sdcard is indeed /dev/mmcblk1 you need to unmount that, /dev/mmcblk1p1 is what "most" stock sdcards show up as. If you just changed
chemist ( 2014-11-25 22:35:11 +0300 )edit1.
and followed all steps afterwards it should work fine. To check, look on your phone, not in an MTP mount by Windows. Is everything mounted where it belongs? If you need further help, please join the jolla IRC channel on freenodeokie well did it again and checked to see if they were mounted on the phone it now shows this
/dev/mmcblk1 is already mounted or /data/media busy
/dev/mmcblk1 is already mounted on /media/sdcard/53b4d0b9-f5a9-4d79-b6f9-707d839dcd46
/dev/mmcblk1 is already mounted on /data/media
but still can not copy data to the sd and nothing shows up i also checked to see if androidsdcard.service and it is active
ReflexDarky ( 2014-11-25 23:03:56 +0300 )editEhrm you don't look if something is properly mounted by trying to umnount it! Just type
mount
in terminal and you get a listing of all currently mounted mount points!One user (don't recall where it was posted might be above^^) had similar issues, I think he copied files to the sdcard by ssh and after that it showed properly in windows and was write enabled. Just so you know I often get warnings about not being able to write MTP devices (not just Jolla) but it does write, sometimes at least.
chemist ( 2014-11-25 23:12:51 +0300 )editokies I was typing just mount /dev/mmcblk1 which was shown in my last post
I have now just typed mount and it shows the sd card as mounted I did the whole guild again and formatted my sd card in i did this a root and all worked no problem
I then rebooted the phone and checked to see if the sd is mount and its only being mounted once
/dev/mmcblk1 mounted on /data/media
ReflexDarky ( 2014-11-26 01:18:22 +0300 )editare you sure you set the "ID" for default right (step 8)? Not sure what causes this as if it is able to mount to /data/media it is also able to mount to /media/sdcard/$UUID , do you see a /dev/fuse on /home/nemo/android_storage ? that would mean the sdcard mount script does not work for some reason but the android storage does
chemist ( 2014-11-26 10:14:14 +0300 )editokies well i tried again and still getting the same result on step 8 i am getting this output
ID 257 gen 12 top level 5 path jolla ID 258 gen 12 top level 5 path android
so i type in btrfs subvolume set-default 257 /media/sdcard/
for step 8
the two mounts i can see are
/dev/mmcblk1 /data/media /dev/fuse /home/nemo/android_storage
its almost like its not mounting the card to the /media/sdcard
ReflexDarky ( 2014-11-28 00:15:04 +0300 )editand that is strange, if you are familiar with systemd and journalctl you may have a look what is going wrong, please join IRC and ask for help there, it is a bit to much to discuss in "comments" (this section is now very long to scroll^^)
chemist ( 2014-11-28 10:11:37 +0300 )editokies i checked with journalctl and i can see this ........ cannot add dependency job for unit android.sdcard.service, ignoring:unit mount-sd@mmcblk1.service failed to load: No such file or directory which i guess means its not working i checked in the systemd folder and there is no mount-sd file there where to go next which channel to go on the irc
ReflexDarky ( 2014-12-06 00:22:08 +0300 )edit#jollamobile on IRC freenode, strange or did you format the card as mmcblk1p1?
chemist ( 2014-12-06 12:52:49 +0300 )editThe file is /etc/systemd/system/mount-sd@.service btw. command to find things is
chemist ( 2014-12-06 12:56:43 +0300 )editfind / -type f -iname mount-sd*
sorry i mean the /etc/systemd/system folder in my previous post, I done the find command it displays find: /proc/11980 (and goes all the way to) 13019: No Such File or directory. The only file it finds is /usr/sbin/mount-sd.sh
ReflexDarky ( 2014-12-06 13:54:48 +0300 )editcommands that want to search the whole devices are sent as root so you don't get all the "permission denied" msgs. Your .service file went missing? You better join up on IRC!
chemist ( 2014-12-07 13:30:00 +0300 )editQuick question - what are the benefits of using "compress" ? on the android mount command? Wouldn't it slow things down with no great benefit on large media files?
icebox ( 2014-12-09 12:22:00 +0300 )editMost files (close to all) in there are program related mini files cache etc, compression on btrfs increases high IO single thread performance massively as btrfs supports transparent single file compression. Performance is about the majority of files used on a device, so for me that is maptiles and other cache - on the far end you have large files, just a few. Thanks for pointing me to re-read up on these things, we probably need to specify
chemist ( 2014-12-09 14:11:04 +0300 )editcompress=lzo
and not justcompress
, read https://wiki.archlinux.org/index.php/Btrfs#Compression there is also a howto for compressing single directories or files only without having it mounted that wayHi all, Not vsited this page for a while. Implemented trickery here last spring/summer. Been working fine. Now I discover copying backups to sdcard fails. Probably some permission issue? Experience anyone?
JayBeRayBearGun ( 2014-12-17 02:35:11 +0300 )editIt might be that you got what I got - my card first became read only and in a few days failed completely despite my efforts to fix it. Could it be that btrfs is to hard on sd cards?
icebox ( 2014-12-17 08:18:03 +0300 )edit@icebox, There might be an issue with misaligned partition since there is no partitions used in this tutorial. After that it is up to the sdcard controller. For instance SanDisk speaks against of removing the existing partition boundaries on their cards. Some controllers also require extra space left unpartitioned for wear leveling.
EDIT: Typed previous in the bus... More about this, or how I have understood the issue:
Sdcard.org's SD formatter leaves out extra space after and often before partitions, depending on the make of the card. The excess space is for wear leveling and/or to align partitions to the internal logic of the sdcard's controller.
On flash technology to write into a cell/block (terms are often used wildly) on sdcard requires complete rewrite of the cell, which may also contain information from blocks of several other files.
The problem is that when the whole card or its partitions are not aligned, the block written in the OS filesystem hits the area of twofold amount of memory cells on the sdcard, causing a lot of additional reads and writes by the controller. This alone is bad.
If the card model requires extra space for rearranging data, it gets even worse.
Manatus ( 2014-12-17 10:05:58 +0300 )editAs I understand flash tech and controllers there is no need to reserve actual space - the controller reserves space that it's never exposed to the OS for wear leveling and bad blocks - how much depends on the quality of the card.
Even if missaligned 1 year life for a sd card in a phone is very short - especially a 64Gb card that was usually one third empty. I have cards that lived years in phone and now moved to raspberry pi's that I never bothered to configure for minimum writes and still work fine. If it was only my card I would write it off to a dud card - it was a Kingston so not really the best out there. But I distinctly remember multiple people reporting the same type of card error developing inside the Jolla.
That's why I suspect btrfs might be at fault here - but based on intuition - no solid proof.
I also dislike that btrfs inside the Jolla is so old - but that's another story.
icebox ( 2014-12-17 11:09:52 +0300 )edit@icebox, I tried to find the article where SanDisk (or Kingston) mentioned that partition boundaries are set to stone but could not find it, at least yet, so I cannot prove this. (And I'm not really an expert about this, I might add.)
With similar but more robust SSD technology, while it is common that users generally do not have to worry about the space reserved for wear leveling, in my experience at least one model of Violin SSD drive require that user reserves the extra space for wear leveling themselves. This was mentioned by Violin themselves in their partitioning tool. I would expect that wear leveling somewhat works without it too, as there are several types involved (garbage collection algorithms and also actual trim), but this makes it a lot easier for drive to handle it, and prolongs the drive's longevity.
Manatus ( 2014-12-17 11:28:58 +0300 )editYou are right - and I wouldn't be surprised that cheap or knock-off brands do away with hidden reserved space and leave everything up to the user.
icebox ( 2014-12-17 11:39:21 +0300 )editBut I also remember distinctly reading that - at least on some make/models - the controller will reserve space in the flash chip for future bad sectors - due to wear, etc. (probably bad sectors already exist and are mapped - most of the time the silicon is not perfect and so the same die can be used for multiple size cards depending on number of defects)
Bunnie has a couple of interesting articles on the subject: http://www.bunniestudios.com/blog/?p=918http://www.bunniestudios.com/blog/?p=3554
Thanks for the links icebox. :)
Also found this: http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Components/General/SDSpec.pdf
"If the host uses partial blocks whose accumulated length is not block aligned and block misalignment is not allowed (CSD parameter WRITE_BLK_MISALIGN is not set), the card shall detect the block misalignment error and abort programming before the beginning of the first misaligned block. The card shall set the ADDRESS_ERROR error bit in the status register, and while ignoring all further data transfer, wait in the Receive-data-State for a stop command."
No time to read it though, but at least SanDisk detects misaligned blocks and does something else. Might not have to worry about misaligned partitions at least.
Manatus ( 2014-12-17 11:52:59 +0300 )edit@JayBeRayBearGun I ran into this with the first systemd script, are the partitions mounted in the right order? Are the permissions right? - thevault (the jolla backup tool) tries to copy the file to the "first-1st" mounted device that is mmcblk1* so if you have an old service file that might cause android to be mounted first and that must not happen. For Sdcard controllers, you wont be able to touch or misalign the $userspace of your sdcard, you may fry the controller by unplugging it while it writes but that is about it (that needs a UART write tool of the manufacturer to be solved). I format most of my pendrives and sdcard without partitions for years now and only ran into issues when pulling the devices from a windows machine without "unmounting" it first. If your sdcard is read-only, there are several ways to get it back to rw service but that is something else.
chemist ( 2014-12-17 12:49:49 +0300 )edit@chemist, Ok, thanks for sorting that out. So we just have to be on the lookout for bad batches or not genuine sdcards. :)
Manatus ( 2014-12-17 14:18:58 +0300 )editWe recovered a transcend 64GB pendrive's controller just last week, it was pulled from a laptop while running a debian installer from it, so I think some controllers are just madly bad controllers and their developers should burn in hell - good there are controller writing tools available for close to any controller. So far I managed to fry some WD USB3 disk controllers only, fry as in I had to send them in as not even the controller flashing software was able to read it (fried two in a row and bought a toshiba instead). All sdcards I ever owned worked fine with ext2/3/4 btrfs xfs and iirc all of them where formatted directly as device or even with LUKS without any real partitions
chemist ( 2014-12-17 17:33:16 +0300 )editCan anyone confirm the solution still works after updating to vaarainjärvi?
RobNas ( 2014-12-19 10:05:49 +0300 )editI can confirm that it is working OK after upgrading on my Jolla.
richardski ( 2014-12-19 11:15:16 +0300 )editI just did the solution on update 10 (after receiving a replacement uSD :)) and it worked as far as I can tell, the only note I have is that there seems to be a mismatch between the kernel driver of btrfs and mkfs.btrfs and when formatting you need to add "-O ^extref" to the mkfs command.
As mentioned here: https://together.jolla.com/question/39007/mounting-sd-card-fails-btrfs-extref-option-of-recent-kernels/
Keeper-of-the-Keys ( 2014-12-26 01:32:33 +0300 )editGot myself a Samsung 32GB microSD card, placed in my Jolla directly from package and it worked ok with vfat fs usable in /media/sdcard.
Then decided to follow this guide in order to make it a btrfs card,, unmounted /dev/mmcblk1p1, run mkfs.btrfs -f command and then when trying to "mount -o subvolid=0 /dev/mmcblk1 /media/sdcard" it fails with syslog complaoning abt insupported mount optipnd. Mount command faiils even without -o or with -t btrfs.
What's wtong and how to mount this card now that had btrfs in it.
I'm at 1.1.1.27 and don't have AlienDalvik installed at all.
foss4ever ( 2014-12-26 19:57:53 +0300 )editAs @Keeper-of-the-Keys pointed out the step 2 in the guide should read "mkfs.btrfs -O ^extref -f /dev/mmcblk1" so as to make it posssible to mount the btrfs volume and continue with the setup. By doing that I was able to successfully enable my SDcard with a Jolla subvolume to which I could copy my backups from the device. So, @chenist might want to comsider updating the guide accordingly so as to make ot work for users with current 1.1.1.27 #SailfishOS version. Thanks for the info regardless, cause now I have +32GB storage in my Jolla ;)
foss4ever ( 2014-12-26 23:43:12 +0300 )editBecause the OP has not (as of yet) corrected this guide to take in consideration the mismatch of mkfs and mount cmds in the current 1.1.1.27 #SailfishOS version of our Jolla's in which the step 2 and step 3. i.e the mkfs.btrfs and the following mount commands (of the prepratory phase) just fail because of unsupported mount options (seen in the syslog when trying to mount incorrectly created fs) I'm happy to inform that I was able to make this work if the second step in the guide is replaced with a command that disables that extension of the btrfs filesystem. Then atleast the following steps can be completed and btrfs sdcard is operational (even if the /media/sdcard folder does not show the created subvolumes but generates blkid-type of ids for those "partitions").
So, the command in step 2 of this guide should read: "mkfs.btrfs -O ^extref -f /dev/mmcblk1", so as the disable the extref option that the Jolla mount command (the kernel btrfs support) does not understand.
HTH ;)
foss4ever ( 2014-12-27 06:06:16 +0300 )editJust tried on another phone, mine is just working though - updated the guide
chemist ( 2014-12-27 12:19:40 +0300 )editThe guide claimed that the mkfs.btrfs command without -O ^extref should work when formatting and mounting the card in device with 1.1.1.27 but it just fails. That's why I provided a correction to the guide as an Answer to this Q:. Good that the guide is now fixed, just wondering why my correct and relevant answer was changed to comment? A simple thanks would have been ok and/or letting a correct answer to stay as an Answer .. ;)
foss4ever ( 2014-12-28 03:28:49 +0300 )edit@chemist: What do you think about these 2 solutions? http://talk.maemo.org/showpost.php?p=1408865&postcount=29 and http://talk.maemo.org/showthread.php?t=92449&page=5 (J4ZZ on 2014-02-01) I try to understand the differences between your solution and theirs..
Also, is there a way to have the android applications on the microSD card and some Jolla files like saving the phone backups on the micro SD or moving those files take takes a lot of space (pictures, movies, docs or even application files - some games or some utilities) - and all the above "accessible"?
aki ( 2015-01-03 19:37:32 +0300 )edit@aki they are using the sdcard as $home partition which is in my opinion not that a good idea, I'd rather put some larger stuff like maps, music and pictures on the sdcard instead of everything (reads system-irrelevant-things). Also I'd not use bind-mounts or mounting scripts when there is btrfs and systemd services. This howto leaves everything on your sdcard accessible, jolla backup eg TheVault is saving everything to a git repository in your home which you may then export to the sdcard from within the backup tool, afterwards you may just delete the backups in your home (from within the backup-tool NOT by hand in the filesystem). I did expect Jolla to make something like this or other stuff in regards to sdcard usage available to normal users by now. If you have specific folders you want to put on your sdcard you may extend this howto with additional subvolumes and mounts as it suits you.
chemist ( 2015-01-04 14:26:40 +0300 )edit@chemist: First, THX a lot for all the work. Still I don't understand why Jolly permit Android-App to use the sdcard. I'm using brandnew installed 1.1.1.27, all Android stuff is now stored on my sdcard, yeeah. But I have no more access via USB/MTP Connection to the sdcard. E.g. I can't see any more the backup.tar. Using ssh and devel-su, everything looks fine:
On my desktop via MTP I can see 2 Icons, 'Phone Memory' and 'SD Card'. With Phone Memory everything works as before (can read/write files from desktop), 'SD Card' is empty, cannot read or write anything from desktop (is a arch linux pc).
Any advise for me?
woody ( 2015-01-06 20:32:12 +0300 )edit@woody for MTP it is just the same for me, we need to wait for jolla to do something about that as it seems to be an export issue of some kind
chemist ( 2015-01-06 20:56:11 +0300 )edit@chemist: I now experience serious lagging in music under android after moving to the external SD card. I use the subsonic media streaming client, and it keeps its cache in the android sdcard. Even if the file is cached it now stutters often and quite badly. So I suspect latency problems from the SD card. Yes, it IS a slow card I have put in there, but I didn't expect it to give problems by just reading an mp3 (I had it in a tablet before and that one could stream movies without problems). Can it be so, that what you wrote about compress=lzo could be the better alternative here? What do you think? If so, is it harmless to recompress the partitions with:
btrfs filesystem defragment -r -v -clzo /data/media and
btrfs filesystem defragment -r -v -clzo /media/sdcard/sdcardpath/
Or do I have to unmount them first and do it another way?
Larswad ( 2015-02-02 18:52:53 +0300 )edityou can just mount it with that flag, no need to re-compress (everything written in the future will be comressed), issues with slow sdcards arise when read and write happens at the same time, most slower and cheap cards write 2Mbit/s and read 5Mbit/s no problem, but when it comes to simultaneous RW some of those cards do not even get >50Kbit/s - I don't think you need to test the card for its speed as you already know it is not working out.
chemist ( 2015-02-03 12:46:05 +0300 )editdamned if that wasn't IT! Thanks again chemist, you're very helpful. Seems even that existant files are faster too, at least they don't seem to skip...for some reason. So for a deadslow sd card I really recommend the compression to be turned on. You could change the guide to that safely I think! Now I'm considering that alternative with making it a mount unit. But I'm not sure what the gain is. Maybe its more 'correct'?
Larswad ( 2015-02-07 12:20:24 +0300 )editmount option in the .service file is
chemist ( 2015-02-07 15:27:26 +0300 )editcompress
already if someone wants something else than default they should do that on their own (compress=xxx), difference between .service and .mount is only that .mount is what it should be but it was easier for me to get .service to work (I did all of this on my Jolla, not even remote) +plus less lines in the guide and honestly... there is no gain fmpovany news ? want my offline maps back for Here Maps ...
elastic ( 2015-02-20 18:50:56 +0300 )editAnyone tried the latest update (Yliaavanlampi, 1.1.2.15) ?
RobNas ( 2015-02-20 20:01:58 +0300 )edityes, doesn't work anymore, but chemist stated this already in his thread.
Fellfrosch ( 2015-02-20 20:13:46 +0300 )editwhat are you guys saying? That it doesnt work with the data on externa sd after the update? In that case I don't agree. They work for me, and I'm sure I deleted the internal cards data after the move, so it can't be using that. What gives?
Larswad ( 2015-02-20 20:33:08 +0300 )editAnd what did you do to make it work?
Fellfrosch ( 2015-02-20 21:26:24 +0300 )editdevel-su
systemctl stop aliendalvik.service
systemctl stop aliendalvik.path
mount -o subvol=android,compress,dirsync ,noatime,users /dev/mmcblk1 /data/media
lechuck ( 2015-02-20 21:58:35 +0300 )editkind of right, in 1.1.2.15 you need to wait for aliendalvik to settle for the first time, it optifies (chroot) all alien files and then fuse-mounts to /home/nemo/android_storage - then you do what @lechuck just said above !!-> if you do not have your androidsdcard.service running or your .mount - if you have the .mount or .service active you need to stop that and do not need to mount by hand as it will start on its own once alien is started again - I do not have a proper idea how to do that in an automatic way for $user for now.
chemist ( 2015-02-21 03:19:47 +0300 )editOk thanx for your explanation. Got it to work. I have the androidsdcard.service active, so all I do is stop alien dalvik, stop androidsdcard.service and start an android app afterwards.
Fellfrosch ( 2015-02-21 08:54:25 +0300 )editNew path for android is
So you just need to change mount path
alexxy ( 2015-02-21 11:53:04 +0300 )edit