We have moved to a new Sailfish OS Forum. Please start new discussions there.
116

How to: Format your uSD-Card to share space with android

asked 2014-04-25 14:46:10 +0300

chemist gravatar image

updated 2016-08-03 14:24:49 +0300

Update: For JollaC see at the bottom, things changed due to missing btrfs support and android mounting has changed a bit too Old stuff in question's history

If you are OK with becoming root and follow a step by step guide on your own account, go ahead reading this guide.

Version: >1.1.9.x


This is no beginners guide, you need to know your way around!

Understand what to do before you do!

What I did to share the space of my uSD between nemo and android without getting duplicates with bind-mounts or any other hickup, read carefully and use at your own risk:

  1. format the whole usdcard (mmcblk1 not mmcblk1p1) to btrfs
  2. mount that volume (=0)
  3. create two subvolumes > jolla and android
  4. set the default volume to jolla (the default is what gets mounted when something calls mount /dev/mmcblk1)
  5. stop alien-dalvik and copy everything from /data/media to the new subvolume android (don't move the files, this way you have all basics alien-dalvik needs even after pulling the uSD)
  6. add a system.service to systemd that mounts and unmounts the subvolume for android as needed and honours aliendalvik.service The problems to solve are: aliendalvik fuse-mounts its /data/media to /home/nemo/android_storage so you need to make sure everything is mounted before that happens on boot, and stop aliendalvik when the sd gets pulled or the mount needs to be lifted for any other cause.
  7. reboot and wait for the device to settle, as soon as your jolla volume is mounted in /media/, you should see /dev/mmcblk1 being mounted to both /media/sdcard/#UUID and /data/media

detailed cmds use at your own risk

preparing the sdcard, should only be needed once!

first dosystemctl stop aliendalvik.service & systemctl stop aliendalvik.path then follow this list

  1. umount /dev/mmcblk1p1 - unmount the card might be another /dev or even more than one
  2. mkfs.btrfs -f /dev/mmcblk1 mkfs.btrfs -O ^extref -f /dev/mmcblk1 - format the card (no need for partition) - for now extref (this extends num of hardlinks available) needs to be disabled, even on the device
  3. mount -o subvolid=0 /dev/mmcblk1 /media/sdcard
  4. cd /media/sdcard
  5. btrfs subvolume create jolla - create folder called jolla
  6. btrfs subvolume create android - create folder called android
  7. btrfs subvolume list . - returns an ID for each subvolume
  8. btrfs subvolume set-default *ID for jolla* /media/sdcard/ - sets jolla to be mounted as default
  9. chown nemo:nemo jolla/ - add -R if needed
  10. chmod 775 jolla/
  11. chmod 777 android/
  12. cp -r -a -v /data/media/* /media/sdcard/android
  13. cd ..
  14. umount /dev/mmcblk1

preparing the system

  • create (as in do create not a command) /etc/systemd/system/data-media.mount and fill with:

#

[Unit]
Description=SD card btrfs subvolume for Android
ConditionPathExists=/dev/mmcblk1
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/data/media
Requires=local-fs.target systemd-user-sessions.service
Before=aliendalvik.service
After=mount-sd@mmcblk1.service
BindsTo=mount-sd@mmcblk1.service

[Mount]
What=/dev/mmcblk1
Where=/data/media
Options=subvol=android,compress,dirsync,noatime,users

[Install]
WantedBy=local-fs.target aliendalvik.service

Second, create /etc/systemd/system/aliendalvik.service.d/sdcard.conf file:

[Unit] 
PartOf=data-media.mount

Then run systemctl enable data-media.mount.

  • the mount file gets started every time aliendalvik is started and stops aliendalvik when it gets stopped, on boot it prevents aliendalvik from starting if it is not started yet but waits for the local-fs to be settled and actually having a mountpoint

for the future we only need to load the service file to its place and enable it, if something changes I will update the code, upgrade went through without anything (see first line for updates) as the updates do not touch data-media.mount as of yet


Jolla C

This has major differences with the original solution for Jolla1 https://together.jolla.com/question/40802/how-to-format-your-usd-card-to-btrfs-and-share-space-with-android/?answer=140223#post-id-140223

  1. The SDcard is formatted to ext4 instead of btrfs (as there is no btrfs support on JollaC yet). I also kept a conventional partition layout (to make it more likely other systems can easily mount the card. although they need to support ext4).
  2. The android directory that is overlaid (bind-mount) is /home/nemo/android_storage instead of /data/media. Which is actually a good thing as now Alien is having symlinks instead of mounting in circles.
  3. Bind mounts are used, instead of btrfs subvolume mounts. The Android directory is hidden in an attempt to prevent tracker indexing the files twice.

New instructions for JollaC are as follows:

preparing the sdcard should only be needed once!

Note: some ext4 formatted uSD cards did not show up or rendered unmountable on JollaC, best fit is to format on device

first dosystemctl stop aliendalvik.service then follow this list

  1. umount /dev/mmcblk1p1 - unmount the card might be another /dev or even more than one
  2. mkfs.ext4 /dev/mmcblk1p1 - format the card
  3. mount /dev/mmcblk1p1 /media/sdcard
  4. cd /media/sdcard
  5. mkdir .android - create folder called .android - this is hidden (starts with a dot) so it does not appear in directory listings
  6. I like to create a file on the card to remind me what I have done: so create a file called Android.README and fill it with:

    Android files are contained in a hidden directory called .android

    See https://together.jolla.com/question/40802/how-to-format-your-usd-card-to-btrfs-and-share-space-with-android/

  7. I also like to create a file with a unique name to let me easily check that Android apps are seeing the expected SD card. Something like: touch .android/GRC-128GB-SD.android

  8. chown -R nemo:nemo .android/
  9. chmod -R a=rwX .android/
  10. cp -r -a -v /home/nemo/android_storage/* /media/sdcard/.android
  11. cd ..
  12. umount /dev/mmcblk1p1

preparing the system

  • The next step requires knowing the Filesystem UUID for the SDcard filesystem. You can get it using lsblk -no UUID /dev/mmcblk1p1. The UUID is a string like 12345678-9abc-def0-1234-56789abcdef0. This string must be used instead of the string UUID-HERE in the file below (two occurrences). You can also find the UUID in the folder the uSD is mounted to as it follows that scheme (in /media/sdcard/ the folder of the currently mounted uSD).
  • create (as in do create not a command) /etc/systemd/system/home-nemo-android_storage.mount and fill with:

#

[Unit]
Description=SD card bind mount for Android
ConditionPathExists=/dev/mmcblk1p1
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/home/nemo/android_storage/
ConditionPathIsMountPoint=/media/sdcard/UUID-HERE/
Requires=local-fs.target systemd-user-sessions.service
Before=aliendalvik.service
After=mount-sd@mmcblk1p1.service
BindsTo=mount-sd@mmcblk1p1.service

[Mount]
What=/media/sdcard/UUID-HERE/.android/
Where=/home/nemo/android_storage/
Type=none
Options=bind

[Install]
WantedBy=local-fs.target aliendalvik.service

Then run systemctl enable home-nemo-android_storage.mount.

  • the mount file gets started every time aliendalvik is started and stops aliendalvik when it gets stopped, on boot it prevents aliendalvik from starting if it is not started yet but waits for the local-fs to be settled and actually having a mountpoint

  • If you put a different SD card in, the mount file will do nothing (because the UUID does not match). The original /home/nemo/android_storage will be used. You can repeat the steps from the top, of course, for the new card.

edit retag flag offensive close delete

Comments

Excellent. Many thanks:)

richardski ( 2014-04-26 02:56:05 +0300 )edit

works as expected! Thanks a million. Do we need to remove the card for next sailfish-sysupdate or won't there be foreseeable issues?

mosen ( 2014-04-26 20:13:51 +0300 )edit
4

@mosen no need to remove the card, the only things that might happen are that Jolla decides to mount all available subvolumes to /media/sdcard/$(UUID) or replace the mount-sd.sh script in another way (knowing the line by heart or having a backup of your mount-sd.sh might come handy at some point). Both cases do no harm as long as you did not delete your /data/sdcard/ folder from your device's internal storage. If there be problems in the future I will try to update the how-to accordingly.

chemist ( 2014-04-26 20:35:52 +0300 )edit

What's the best way to edit mount-sd.sh? I've done it with copying the file and editing it on computer (wordpad), but it got my phone into a bootloop... I've fixed that now and really want to try it again (1.0.5.19), but just want to be sure that what I'm doing is the right way...

RobNas ( 2014-04-27 15:16:31 +0300 )edit
1

@RobNas I did all steps on the Jolla, for editing files I use vim. If you copy files from and to your device, you need to make sure that permissions and ownership are preserved. There was a typo, $(DEVNAME) should be ${DEVNAME} of course (updated the entries)

chemist ( 2014-04-27 15:22:13 +0300 )edit
1

@RobNas, most convenient is ssh conection to jolla or use the native terminal app. then try one of the editors vim or nano from CLI

nano /usr/sbin/mount-sd.sh

save to file in nano is crtl+o, exit is ctrl+x

mosen ( 2014-04-27 15:29:46 +0300 )edit

I've edited with vim, eventually, I think the accolades {} instead of () did the trick. It looks like it's working, when I install Sygic through Google Play, I see 55GB of space. Sweet. But now Sygic is coming with stupid errors, so I can't do anything there. Sigh...a new challenge...

Thank you really, really much for the effort :-D

EDIT: Sygic is coming up. There was still data from a pre-factoryreset install, I think. Just deleted the entire folder and installed it again through Google Play. Now downloading maps...

RobNas ( 2014-04-27 22:29:11 +0300 )edit
1

you should see /dev/mmcblk1 mounted twice now, one to /data/sdcard and one to /media/sdcard/$(UUID), (df or mount) - if so you are all fine, note that /data/sdcard is also exported to MTP so you can actually put data on your sdcard without any hassle, but to your android folder only (for now).

chemist ( 2014-04-28 13:19:52 +0300 )edit

I see /dev/mmcblk1 mounted twice, yes. I see that if I connect the Jolla to desktop, the reachable mount is where android is. That's not very convenient, but it's doable for the time being. It's not very clear to me how space is divided...it looks like the complete space is shared among two mounts, what's nice, but a bit confusing. Unfortunately the gallery doesn't show pictures located on the android-location.

RobNas ( 2014-04-28 20:58:44 +0300 )edit

Thats because you are just making two datasets not two partitions. So the space is shared between all datasets on that drive.(just like on ZFS)

slaveriq ( 2014-04-28 21:27:41 +0300 )edit

@eric please convert to comment

@RobNas that is exactly what the title says "share space" and a bind mount does nothing else but mount a already mounted partition's mountpoint again to another mountpoint - this is much cleaner, as subvolumes are just folders ("everything is a file") you can mount and handle like partitions without restricting the physical volume or even care about the maximum size of your FS as your whole blockdevice (or many) is available to all of them at the same time

chemist ( 2014-04-28 21:33:34 +0300 )edit

Just for the record - it's not technically formatting, but it's an alternative. I prefer to format my SD card to vfat and create a symlink from the folder the card is mounted in (/media/sdcard/ID) to both /home/nemo/card and /data/sdcard/card. That way the android FS stays as is and I can access the data stored on the card from both android and native apps. Thanks to it being a vfat there are no issues with permissions too. You can write a file in Android app and read it in native and vice versa.

The price to pay is that your android apps are still taking up space on the internal memory. Theoretically the sdcard could be split to contain a brtfs partition and a vfat one, both could me mounted and one could contain the android app and the second the vfat shared space. But I haven't tried that.

zlc ( 2014-04-28 23:30:51 +0300 )edit

@zlc as long as you keep the card in your device, what most people (especially non-developers) do, it does not matter as it gets exported as MTP anyways. Permissions is actually something you want, not try to circumvent. And your price to pay is the main reason why I was trying to get rid of it, and bind-mounts or symlinking was not an option for me. We are not arguing about semantics are we? http://en.wikipedia.org/wiki/Disk_formatting

chemist ( 2014-04-29 10:31:12 +0300 )edit

@chemist Sorry, I understand how you would think I was referring to your approach but I was actually referring to mine. I don't consider my approach to be formatting as most of the SD cards are pre-formatted to FAT32 or exFAT. I'm not arguing, really :) I was just trying to present others with an alternative that doesn't require you to move everything from the /data/ folder to the SD card and formatting it to something you might have troubles mounting on some OSes. You don't mind moving everything and you require access rights that's why you do your thing. I don't mind loosing access rights (especially because I want to be able to stuff the card into my PC and work with the data) but I don't want to move everything to the card so I do my thing. That's all there is to it, really.

zlc ( 2014-04-29 11:51:47 +0300 )edit
1

Ah ok ;) I like to stuff the card in my PC as well, especially as the MTP mount has some hick-ups sometimes. But I use linux, one reason why I bought an unfinished linux mobile-phone^^ better safe than sorry!

chemist ( 2014-04-29 11:59:37 +0300 )edit

Well I like Linux too and that's why I bought the phone as well. And I have Win/Linux dual boot on most of my machines but I develop Windows applications for a living so I really need to have a solid Windows installation at hand and switch depending on what I'm doing. But that's OT, sorry.

zlc ( 2014-04-29 12:09:38 +0300 )edit

This will break / not work as expected in a future update. The Android SD-Card will be moved to VFAT emulation via FUSE, mounted at ~/android_storage. The directory backing that (=the one you want to move to SD card) will be /data/media

Aard ( 2014-05-19 22:20:27 +0300 )edit

@Aard why mount android in userhome? and why put the sdcards mountpoint onto the sdcard?

chemist ( 2014-05-20 00:38:08 +0300 )edit

Or better: What do we have to do in the future to still have this possibility? Probably that's a bit difficult to say at the moment. When anyone has the update, please post here if this solution is still possible...

RobNas ( 2014-05-22 19:46:37 +0300 )edit

@RobNas mount the usdcard "before" alien-dalvik FUSEs to ~/android_storage and it will be fine (I am not saying that it will be easy^^)

chemist ( 2014-05-23 10:43:09 +0300 )edit

Does anyone was so bold to install the 1.0.7.16 update? If so, is this solution still working?

RobNas ( 2014-06-10 07:03:28 +0300 )edit

Yes, and no it is not working this way anymore. If anyone is faster than me, we need a way to mount the android part of the sdcard to /data/media before alien-dalvik does the fuse mount. So ideal would be a startup-script that alien-dalvik waits for anyway.

chemist ( 2014-06-10 10:53:29 +0300 )edit

workaround for now is to do everything by hand...

#!/bin/bash
systemctl stop aliendalvik.service
sleep 2
mount -o subvol=android,compress,dirsync,noatime,users /dev/mmcblk1 /data/media
chemist ( 2014-06-10 14:38:02 +0300 )edit

can confirm, works! Thank you chemist :D

mosen ( 2014-06-10 15:44:33 +0300 )edit

If I implemented the solution on top of the page, have the paarlampi version of Sailfish, update to the latest version and do this, it's good to go?

RobNas ( 2014-06-10 23:10:52 +0300 )edit

yeah, if alien-dalvik is stopped you can mount it that way again

chemist ( 2014-06-11 01:16:21 +0300 )edit

After updating, everything is gone.... What I did: I had the paarlampi version, 1.0.5.19, with the solution implemented. Did the update, immediately after the update did #!/bin/hash etc... through terminal. Wanted to start Sygic. Nothing. Started to install data after a reboot. Checked sd-card: everything gone. Thanks, I can start again.

IMPORTANT: EVERYTHING ON SD-CARD WILL BE REMOVED AFTER UPDATE TO SAPUNKI IF YOU'VE IMPLEMENTED THE SOLUTION IN THE FIRST POST!

RobNas ( 2014-06-11 11:45:14 +0300 )edit

ehrm...NO! I have no idea what you have done but if the subvolume is not mounted the way I told, you wont have the data available for android and you see the default folder fused to /home/nemo/android_storage/ and there wont be any data in /data/sdcard that you need... I will update the how-to in a few minutes to a working with current upgrade... FYI your subvolume on your microsd is not altered...

chemist ( 2014-06-11 14:42:53 +0300 )edit

updated, if followed correctly this survives reboots and even pulling the sdcard...

chemist ( 2014-06-11 15:32:28 +0300 )edit

I modified it to be a .mount unit:

    # cat /etc/systemd/system/data-media.mount 
    [Unit]
    Description=SD card btrfs subvolume for Android
    ConditionPathExists=/dev/mmcblk1
    ConditionPathIsDirectory=/data/media
    Requires=local-fs.target
    Before=aliendalvik.service
    After=local-fs.target
    BindsTo=mount-sd@mmcblk1.service

    [Mount]
    What=/dev/mmcblk1
    Where=/data/media
    Options=subvol=android,compress,dirsync,noatime,users

    [Install]
    WantedBy=local-fs.target aliendalvik.service


    # cat /etc/systemd/system/aliendalvik.service.d/sdcard.conf 
    [Unit]
    PartOf=data-media.mount
Vuubi ( 2014-06-11 20:22:11 +0300 )edit

hmm... ah I have to creat those^^ testing... but well, we had two files with that, with the service we have only one, but good to know what the config needs to look like, can you please edit it to reflect that it is a file you need to generate?

chemist ( 2014-06-11 20:30:57 +0300 )edit

No, somehow I don't see the link to edit the comment

Vuubi ( 2014-06-11 20:38:07 +0300 )edit

@eric can you fix the post above (separate code lines) please

chemist ( 2014-06-11 21:02:51 +0300 )edit

@chemist: done.

eric ( 2014-06-12 11:15:43 +0300 )edit

@eric thanks.

chemist ( 2014-06-12 13:16:44 +0300 )edit

For upgrading from previous version: the group and owner of the files have changed

So that's why you set android subvolume to chmod 777, I thought it was strange. Original permissions are 770.

Vuubi ( 2014-06-12 19:20:54 +0300 )edit

that is only for the folder so nothing comes to the idea to not being able to write it... this problem does only occure if you update this from the previous version and already have all your files on your sdcard as they changed user and group for the storage

chemist ( 2014-06-12 19:28:25 +0300 )edit

@Vuubi can you put your comment about how to make it a .mount into an answer so we can edit it?

chemist ( 2014-06-12 19:30:15 +0300 )edit

Thanks ! This saved my day !.I inversed the principle slightly to mount android storage by default and in the service description I created the necessary mount points in nemo's home directory. For some reason, I had to make a symbolic link to android SD card part as /data/sdcard, otherwise some Android applications did not like to start. I made my Evernote page on this public if somebody is interested: http://bit.ly/1qea9rB

canne ( 2014-06-14 13:28:43 +0300 )edit

Sry but you are doing it wrong :/ what you do is taking everything upside-down, the default volume gets mounted by mount-sd.sh anyways so why bother, that is fine with the system and that is where you have your traditional sdcard mount - android on the other hand is now a bit more of a trick as the userdata gets fuse mounted to $HOME - even if... you had made the upside down work for you, you symlink the wrong directory to it and all the heavy userdata from android is still on device and not on sdcard, my solution does not care if some install of an android app is huge, sure we could change that as well but the userdata like maptiles (grow easily to 12GB if you want detailed hiking maps) are stored in ~/android_storage and that is actually /data/media which we need to mount before it gets fused to $HOME

chemist ( 2014-06-14 13:54:55 +0300 )edit

@chemist Your method is certainly the one I would advice, thank you once again for sharing your insight.

canne ( 2014-06-14 14:57:05 +0300 )edit

Only the following command is correctly: chown -R media_rw:media_rw /data/media/ as chmod changes the read write rights.

holgern ( 2014-06-16 16:14:31 +0300 )edit

@holgern thanks for commenting, changed it.

chemist ( 2014-06-16 20:49:38 +0300 )edit

You should also remove the stuff in the "old" /data/media after copying to the sdcard (to free up space)... ;)

You could change 12. to:

cp -r -a -v /data/media/* /media/sdcard/android && rm -rf /data/media

The '&&' means only run second command ('rm -rf /data/media') if the first command ('cp -r -a -v /data/media/* /media/sdcard/android') is successfully executed... :)

I freed up 3.1 GB on the internal storage... ;)

DrWilken ( 2014-07-12 02:44:53 +0300 )edit

@DrWilken to free up space yes, to have "some" things available without SD, no... - in general it might be just moved as it is "user" data anyways, so nothing important anyways

chemist ( 2014-07-14 01:38:37 +0300 )edit

Very small change to the instructions: You have to 'cd ..' out of the /media/sdcard before unmounting or the system claims that the device is busy.

Manatus ( 2014-07-14 17:56:40 +0300 )edit

Is there any change/progress/update regarding to the 1.0.8.19 (Tahkalampi)? Or in other words: what do I need to do before/during/after the update to keep this working, or is this solution not working anymore?

RobNas ( 2014-07-20 13:59:23 +0300 )edit

@RobNas, it's working, no changes

Vuubi ( 2014-07-20 14:05:27 +0300 )edit

So I apply the update, no manual action needed and the space from SD-card is still shared?

RobNas ( 2014-07-20 14:07:59 +0300 )edit

@RobNas, yes

Vuubi ( 2014-07-20 14:10:36 +0300 )edit

Hi,

I think I got this done successfully, but I cannot access the card over USB in Windows 7. I can see it under Sailfish - SD Card, but I cannot copy or see anything in there.

Did I fail at something, has anyone gotten it to work over normal USB connection "PC Connection" -mode?

EDIT: After loading some music to the uSD Card, I can now see and access the Card over MTP in Windows 7. I don't know why or how, but now it seems to work. Will test at home with Win 8.1.

Juuba ( 2014-08-21 10:29:50 +0300 )edit

Thinking about doing this, I'm a carefult guy. Would anyone consider this causing problems if Jolla comes with an update? I means, the only real change is the addition of this service file to mount the card. But still, is there anything with that that would cause any problems when Jolla comes with an update?

Larswad ( 2014-09-09 10:42:00 +0300 )edit
1

@Larswad no need to be scared, at least only the btrfs formatting part is something to break, as long as they do not change anything, I mean, change path of android directories, there won't be any issue (in software you never can be sure but confident) - what can happen is that the service fails but iirc aliendalvik.service starts anyways just without the sdcard mounted to /data/media

chemist ( 2014-09-09 11:42:47 +0300 )edit

@chemist: Thanks! I'll give it a shot, this way I can move my subsonic music cache and the DCIM folder for the android camera apps that I have (won't MOVE anything else except those, rest of the files I'll make a copy, like stated in theguid).

Sorry, but a side question pops up: Is there a benefit using the Vuubi guy's guide below to convert your service unit into a mount unit? Is it just "more correct", but have no practical difference?

Larswad ( 2014-09-09 12:15:18 +0300 )edit
1

Technically it is right but I think it prevents aliendalvik from starting if it fails to mount (not sure) - it was what I wanted to do in the first place but ended up doing it as service and stick to it. Comment on his answer if you run into any problems, please.

chemist ( 2014-09-09 17:23:29 +0300 )edit

between step 12 and 13 add:

cd ..
qbot ( 2014-10-15 22:40:06 +0300 )edit

Uitukka, anyone?

RobNas ( 2014-10-28 17:43:27 +0300 )edit

@RobNas I updated the question, everything is fine. If you upgrade to update9 with this setup everything is fine too, no user action needed.

chemist ( 2014-10-29 12:18:57 +0300 )edit

Thanks :-)

RobNas ( 2014-10-29 12:35:05 +0300 )edit

In the androidsdcard.service file it says

After=local-fs.target mount-sd@mmcblk1.serviceBindsTo=mount-sd@mmcblk1.service

This "mount-sd@mmcblk1.service", is it the one located at /etc/systemd/system?

If it is, the one in my Jolla is called mount-sd@.service and not mount-sd@mmcblk1.service. What should I do?

Aashish ( 2014-11-07 23:17:06 +0300 )edit

I think it has worked fine. Its evident now that some android apps are using SD card. However, I have seen that the 'used' phone memory usage increases by the same amount that the sd card usage increases. Any ideas why that is?

Aashish ( 2014-11-08 22:25:06 +0300 )edit

the service created is mount-sd@mmcblk1.service if you followed the re-formatting part above, if you format your card with a partition the service will be called mount-sd@mmcblk1p1.service as the sd-card mounting service creates them, note the @ in the service file?! It is a bit difficult to explain though, services with an @ in their name are created from a meta service file that has nothing after the @ which then gets replaced with the called name, in the case above, mount-sd calls the service for mmcblk1, therefor the service is called mount-sd@mmcblk1.service. For the FS usage, I have more GB used on the SD than I have space on the internal storage, so NO this does not sound right! Are you sure it is setup correctly? Settings -> System -> About product shows 5.3GB used for me, and I have like 8GB maps downloaded to the android part of the sd-card.

chemist ( 2014-11-10 02:18:52 +0300 )edit

i have tired this on 1.0.8.21 and it does not work when i connect the phone to my pc i am unable to copy files to it say the write permissions is not allowed this is in windows

also when doing the umount /dev/mmcblk1p1 it tells me there is no mount point for this

so when doing this umount /dev/mmcblk1 my sd card is unmounted and can be formatted to btrfs

is there a new way of doing this or have i done something wrong

ReflexDarky ( 2014-11-25 22:23:50 +0300 )edit

And I thought I wrote enough disclaimer... if your sdcard is indeed /dev/mmcblk1 you need to unmount that, /dev/mmcblk1p1 is what "most" stock sdcards show up as. If you just changed 1. and followed all steps afterwards it should work fine. To check, look on your phone, not in an MTP mount by Windows. Is everything mounted where it belongs? If you need further help, please join the jolla IRC channel on freenode

chemist ( 2014-11-25 22:35:11 +0300 )edit

okie well did it again and checked to see if they were mounted on the phone it now shows this

/dev/mmcblk1 is already mounted or /data/media busy

/dev/mmcblk1 is already mounted on /media/sdcard/53b4d0b9-f5a9-4d79-b6f9-707d839dcd46

/dev/mmcblk1 is already mounted on /data/media

but still can not copy data to the sd and nothing shows up i also checked to see if androidsdcard.service and it is active

ReflexDarky ( 2014-11-25 23:03:56 +0300 )edit

Ehrm you don't look if something is properly mounted by trying to umnount it! Just type mount in terminal and you get a listing of all currently mounted mount points!

One user (don't recall where it was posted might be above^^) had similar issues, I think he copied files to the sdcard by ssh and after that it showed properly in windows and was write enabled. Just so you know I often get warnings about not being able to write MTP devices (not just Jolla) but it does write, sometimes at least.

chemist ( 2014-11-25 23:12:51 +0300 )edit

okies I was typing just mount /dev/mmcblk1 which was shown in my last post

I have now just typed mount and it shows the sd card as mounted I did the whole guild again and formatted my sd card in i did this a root and all worked no problem

I then rebooted the phone and checked to see if the sd is mount and its only being mounted once

/dev/mmcblk1 mounted on /data/media

ReflexDarky ( 2014-11-26 01:18:22 +0300 )edit

are you sure you set the "ID" for default right (step 8)? Not sure what causes this as if it is able to mount to /data/media it is also able to mount to /media/sdcard/$UUID , do you see a /dev/fuse on /home/nemo/android_storage ? that would mean the sdcard mount script does not work for some reason but the android storage does

chemist ( 2014-11-26 10:14:14 +0300 )edit

okies well i tried again and still getting the same result on step 8 i am getting this output

ID 257 gen 12 top level 5 path jolla ID 258 gen 12 top level 5 path android

so i type in btrfs subvolume set-default 257 /media/sdcard/

for step 8

the two mounts i can see are

/dev/mmcblk1 /data/media /dev/fuse /home/nemo/android_storage

its almost like its not mounting the card to the /media/sdcard

ReflexDarky ( 2014-11-28 00:15:04 +0300 )edit

and that is strange, if you are familiar with systemd and journalctl you may have a look what is going wrong, please join IRC and ask for help there, it is a bit to much to discuss in "comments" (this section is now very long to scroll^^)

chemist ( 2014-11-28 10:11:37 +0300 )edit

okies i checked with journalctl and i can see this ........ cannot add dependency job for unit android.sdcard.service, ignoring:unit mount-sd@mmcblk1.service failed to load: No such file or directory which i guess means its not working i checked in the systemd folder and there is no mount-sd file there where to go next which channel to go on the irc

ReflexDarky ( 2014-12-06 00:22:08 +0300 )edit

#jollamobile on IRC freenode, strange or did you format the card as mmcblk1p1?

chemist ( 2014-12-06 12:52:49 +0300 )edit

The file is /etc/systemd/system/mount-sd@.service btw. command to find things is find / -type f -iname mount-sd*

chemist ( 2014-12-06 12:56:43 +0300 )edit

sorry i mean the /etc/systemd/system folder in my previous post, I done the find command it displays find: /proc/11980 (and goes all the way to) 13019: No Such File or directory. The only file it finds is /usr/sbin/mount-sd.sh

ReflexDarky ( 2014-12-06 13:54:48 +0300 )edit

commands that want to search the whole devices are sent as root so you don't get all the "permission denied" msgs. Your .service file went missing? You better join up on IRC!

chemist ( 2014-12-07 13:30:00 +0300 )edit

Quick question - what are the benefits of using "compress" ? on the android mount command? Wouldn't it slow things down with no great benefit on large media files?

icebox ( 2014-12-09 12:22:00 +0300 )edit

Most files (close to all) in there are program related mini files cache etc, compression on btrfs increases high IO single thread performance massively as btrfs supports transparent single file compression. Performance is about the majority of files used on a device, so for me that is maptiles and other cache - on the far end you have large files, just a few. Thanks for pointing me to re-read up on these things, we probably need to specify compress=lzo and not just compress, read https://wiki.archlinux.org/index.php/Btrfs#Compression there is also a howto for compressing single directories or files only without having it mounted that way

chemist ( 2014-12-09 14:11:04 +0300 )edit

Hi all, Not vsited this page for a while. Implemented trickery here last spring/summer. Been working fine. Now I discover copying backups to sdcard fails. Probably some permission issue? Experience anyone?

JayBeRayBearGun ( 2014-12-17 02:35:11 +0300 )edit

It might be that you got what I got - my card first became read only and in a few days failed completely despite my efforts to fix it. Could it be that btrfs is to hard on sd cards?

icebox ( 2014-12-17 08:18:03 +0300 )edit

@icebox, There might be an issue with misaligned partition since there is no partitions used in this tutorial. After that it is up to the sdcard controller. For instance SanDisk speaks against of removing the existing partition boundaries on their cards. Some controllers also require extra space left unpartitioned for wear leveling.

EDIT: Typed previous in the bus... More about this, or how I have understood the issue:

Sdcard.org's SD formatter leaves out extra space after and often before partitions, depending on the make of the card. The excess space is for wear leveling and/or to align partitions to the internal logic of the sdcard's controller.

On flash technology to write into a cell/block (terms are often used wildly) on sdcard requires complete rewrite of the cell, which may also contain information from blocks of several other files.

The problem is that when the whole card or its partitions are not aligned, the block written in the OS filesystem hits the area of twofold amount of memory cells on the sdcard, causing a lot of additional reads and writes by the controller. This alone is bad.

If the card model requires extra space for rearranging data, it gets even worse.

Manatus ( 2014-12-17 10:05:58 +0300 )edit

As I understand flash tech and controllers there is no need to reserve actual space - the controller reserves space that it's never exposed to the OS for wear leveling and bad blocks - how much depends on the quality of the card.

Even if missaligned 1 year life for a sd card in a phone is very short - especially a 64Gb card that was usually one third empty. I have cards that lived years in phone and now moved to raspberry pi's that I never bothered to configure for minimum writes and still work fine. If it was only my card I would write it off to a dud card - it was a Kingston so not really the best out there. But I distinctly remember multiple people reporting the same type of card error developing inside the Jolla.

That's why I suspect btrfs might be at fault here - but based on intuition - no solid proof.

I also dislike that btrfs inside the Jolla is so old - but that's another story.

icebox ( 2014-12-17 11:09:52 +0300 )edit

@icebox, I tried to find the article where SanDisk (or Kingston) mentioned that partition boundaries are set to stone but could not find it, at least yet, so I cannot prove this. (And I'm not really an expert about this, I might add.)

With similar but more robust SSD technology, while it is common that users generally do not have to worry about the space reserved for wear leveling, in my experience at least one model of Violin SSD drive require that user reserves the extra space for wear leveling themselves. This was mentioned by Violin themselves in their partitioning tool. I would expect that wear leveling somewhat works without it too, as there are several types involved (garbage collection algorithms and also actual trim), but this makes it a lot easier for drive to handle it, and prolongs the drive's longevity.

Manatus ( 2014-12-17 11:28:58 +0300 )edit
2

You are right - and I wouldn't be surprised that cheap or knock-off brands do away with hidden reserved space and leave everything up to the user.
But I also remember distinctly reading that - at least on some make/models - the controller will reserve space in the flash chip for future bad sectors - due to wear, etc. (probably bad sectors already exist and are mapped - most of the time the silicon is not perfect and so the same die can be used for multiple size cards depending on number of defects)
Bunnie has a couple of interesting articles on the subject: http://www.bunniestudios.com/blog/?p=918http://www.bunniestudios.com/blog/?p=3554

icebox ( 2014-12-17 11:39:21 +0300 )edit

Thanks for the links icebox. :)

Also found this: http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Components/General/SDSpec.pdf

"If the host uses partial blocks whose accumulated length is not block aligned and block misalignment is not allowed (CSD parameter WRITE_BLK_MISALIGN is not set), the card shall detect the block misalignment error and abort programming before the beginning of the first misaligned block. The card shall set the ADDRESS_ERROR error bit in the status register, and while ignoring all further data transfer, wait in the Receive-data-State for a stop command."

No time to read it though, but at least SanDisk detects misaligned blocks and does something else. Might not have to worry about misaligned partitions at least.

Manatus ( 2014-12-17 11:52:59 +0300 )edit

@JayBeRayBearGun I ran into this with the first systemd script, are the partitions mounted in the right order? Are the permissions right? - thevault (the jolla backup tool) tries to copy the file to the "first-1st" mounted device that is mmcblk1* so if you have an old service file that might cause android to be mounted first and that must not happen. For Sdcard controllers, you wont be able to touch or misalign the $userspace of your sdcard, you may fry the controller by unplugging it while it writes but that is about it (that needs a UART write tool of the manufacturer to be solved). I format most of my pendrives and sdcard without partitions for years now and only ran into issues when pulling the devices from a windows machine without "unmounting" it first. If your sdcard is read-only, there are several ways to get it back to rw service but that is something else.

chemist ( 2014-12-17 12:49:49 +0300 )edit

@chemist, Ok, thanks for sorting that out. So we just have to be on the lookout for bad batches or not genuine sdcards. :)

Manatus ( 2014-12-17 14:18:58 +0300 )edit
1

We recovered a transcend 64GB pendrive's controller just last week, it was pulled from a laptop while running a debian installer from it, so I think some controllers are just madly bad controllers and their developers should burn in hell - good there are controller writing tools available for close to any controller. So far I managed to fry some WD USB3 disk controllers only, fry as in I had to send them in as not even the controller flashing software was able to read it (fried two in a row and bought a toshiba instead). All sdcards I ever owned worked fine with ext2/3/4 btrfs xfs and iirc all of them where formatted directly as device or even with LUKS without any real partitions

chemist ( 2014-12-17 17:33:16 +0300 )edit

Can anyone confirm the solution still works after updating to vaarainjärvi?

RobNas ( 2014-12-19 10:05:49 +0300 )edit

I can confirm that it is working OK after upgrading on my Jolla.

richardski ( 2014-12-19 11:15:16 +0300 )edit

I just did the solution on update 10 (after receiving a replacement uSD :)) and it worked as far as I can tell, the only note I have is that there seems to be a mismatch between the kernel driver of btrfs and mkfs.btrfs and when formatting you need to add "-O ^extref" to the mkfs command.

As mentioned here: https://together.jolla.com/question/39007/mounting-sd-card-fails-btrfs-extref-option-of-recent-kernels/

Keeper-of-the-Keys ( 2014-12-26 01:32:33 +0300 )edit

Got myself a Samsung 32GB microSD card, placed in my Jolla directly from package and it worked ok with vfat fs usable in /media/sdcard.

Then decided to follow this guide in order to make it a btrfs card,, unmounted /dev/mmcblk1p1, run mkfs.btrfs -f command and then when trying to "mount -o subvolid=0 /dev/mmcblk1 /media/sdcard" it fails with syslog complaoning abt insupported mount optipnd. Mount command faiils even without -o or with -t btrfs.

What's wtong and how to mount this card now that had btrfs in it.

I'm at 1.1.1.27 and don't have AlienDalvik installed at all.

foss4ever ( 2014-12-26 19:57:53 +0300 )edit

As @Keeper-of-the-Keys pointed out the step 2 in the guide should read "mkfs.btrfs -O ^extref -f /dev/mmcblk1" so as to make it posssible to mount the btrfs volume and continue with the setup. By doing that I was able to successfully enable my SDcard with a Jolla subvolume to which I could copy my backups from the device. So, @chenist might want to comsider updating the guide accordingly so as to make ot work for users with current 1.1.1.27 #SailfishOS version. Thanks for the info regardless, cause now I have +32GB storage in my Jolla ;)

foss4ever ( 2014-12-26 23:43:12 +0300 )edit

Because the OP has not (as of yet) corrected this guide to take in consideration the mismatch of mkfs and mount cmds in the current 1.1.1.27 #SailfishOS version of our Jolla's in which the step 2 and step 3. i.e the mkfs.btrfs and the following mount commands (of the prepratory phase) just fail because of unsupported mount options (seen in the syslog when trying to mount incorrectly created fs) I'm happy to inform that I was able to make this work if the second step in the guide is replaced with a command that disables that extension of the btrfs filesystem. Then atleast the following steps can be completed and btrfs sdcard is operational (even if the /media/sdcard folder does not show the created subvolumes but generates blkid-type of ids for those "partitions").

So, the command in step 2 of this guide should read: "mkfs.btrfs -O ^extref -f /dev/mmcblk1", so as the disable the extref option that the Jolla mount command (the kernel btrfs support) does not understand.

HTH ;)

foss4ever ( 2014-12-27 06:06:16 +0300 )edit

Just tried on another phone, mine is just working though - updated the guide

chemist ( 2014-12-27 12:19:40 +0300 )edit

The guide claimed that the mkfs.btrfs command without -O ^extref should work when formatting and mounting the card in device with 1.1.1.27 but it just fails. That's why I provided a correction to the guide as an Answer to this Q:. Good that the guide is now fixed, just wondering why my correct and relevant answer was changed to comment? A simple thanks would have been ok and/or letting a correct answer to stay as an Answer .. ;)

foss4ever ( 2014-12-28 03:28:49 +0300 )edit

@chemist: What do you think about these 2 solutions? http://talk.maemo.org/showpost.php?p=1408865&postcount=29 and http://talk.maemo.org/showthread.php?t=92449&page=5 (J4ZZ on 2014-02-01) I try to understand the differences between your solution and theirs..

Also, is there a way to have the android applications on the microSD card and some Jolla files like saving the phone backups on the micro SD or moving those files take takes a lot of space (pictures, movies, docs or even application files - some games or some utilities) - and all the above "accessible"?

aki ( 2015-01-03 19:37:32 +0300 )edit

@aki they are using the sdcard as $home partition which is in my opinion not that a good idea, I'd rather put some larger stuff like maps, music and pictures on the sdcard instead of everything (reads system-irrelevant-things). Also I'd not use bind-mounts or mounting scripts when there is btrfs and systemd services. This howto leaves everything on your sdcard accessible, jolla backup eg TheVault is saving everything to a git repository in your home which you may then export to the sdcard from within the backup tool, afterwards you may just delete the backups in your home (from within the backup-tool NOT by hand in the filesystem). I did expect Jolla to make something like this or other stuff in regards to sdcard usage available to normal users by now. If you have specific folders you want to put on your sdcard you may extend this howto with additional subvolumes and mounts as it suits you.

chemist ( 2015-01-04 14:26:40 +0300 )edit

@chemist: First, THX a lot for all the work. Still I don't understand why Jolly permit Android-App to use the sdcard. I'm using brandnew installed 1.1.1.27, all Android stuff is now stored on my sdcard, yeeah. But I have no more access via USB/MTP Connection to the sdcard. E.g. I can't see any more the backup.tar. Using ssh and devel-su, everything looks fine:

[root@Jolla /]# mount | grep mmcblk1
/dev/mmcblk1 on /media/sdcard/62db2af4-c188-4422-8e26-b4cf2358e65a type btrfs (rw,nosuid,nodev,noexec,noatime,dirsync,thread_pool=4,ssd,noacl,space_cache)
/dev/mmcblk1 on /data/media type btrfs (rw,nosuid,nodev,noexec,noatime,dirsync,thread_pool=4,ssd,noacl,space_cache)
[root@Jolla /]# mount | grep mmcblk1
/dev/mmcblk1 on /media/sdcard/62db2af4-c188-4422-8e26-b4cf2358e65a type btrfs (rw,nosuid,nodev,noexec,noatime,dirsync,thread_pool=4,ssd,noacl,space_cache)
/dev/mmcblk1 on /data/media type btrfs (rw,nosuid,nodev,noexec,noatime,dirsync,thread_pool=4,ssd,noacl,space_cache)
[root@Jolla /]# ls /data/media/   
Alarms/        DCIM/          Movies/        Notifications/ Podcasts/      WhatsApp/      
Android/       Download/      Music/         Pictures/      Ringtones/     osmand/        
[root@Jolla /]# ls /media/sdcard/62db2af4-c188-4422-8e26-b4cf2358e65a/
Backup.tar

On my desktop via MTP I can see 2 Icons, 'Phone Memory' and 'SD Card'. With Phone Memory everything works as before (can read/write files from desktop), 'SD Card' is empty, cannot read or write anything from desktop (is a arch linux pc).

Any advise for me?

woody ( 2015-01-06 20:32:12 +0300 )edit

@woody for MTP it is just the same for me, we need to wait for jolla to do something about that as it seems to be an export issue of some kind

chemist ( 2015-01-06 20:56:11 +0300 )edit
1

@chemist: I now experience serious lagging in music under android after moving to the external SD card. I use the subsonic media streaming client, and it keeps its cache in the android sdcard. Even if the file is cached it now stutters often and quite badly. So I suspect latency problems from the SD card. Yes, it IS a slow card I have put in there, but I didn't expect it to give problems by just reading an mp3 (I had it in a tablet before and that one could stream movies without problems). Can it be so, that what you wrote about compress=lzo could be the better alternative here? What do you think? If so, is it harmless to recompress the partitions with:

btrfs filesystem defragment -r -v -clzo /data/media and

btrfs filesystem defragment -r -v -clzo /media/sdcard/sdcardpath/

Or do I have to unmount them first and do it another way?

Larswad ( 2015-02-02 18:52:53 +0300 )edit
1

you can just mount it with that flag, no need to re-compress (everything written in the future will be comressed), issues with slow sdcards arise when read and write happens at the same time, most slower and cheap cards write 2Mbit/s and read 5Mbit/s no problem, but when it comes to simultaneous RW some of those cards do not even get >50Kbit/s - I don't think you need to test the card for its speed as you already know it is not working out.

chemist ( 2015-02-03 12:46:05 +0300 )edit

damned if that wasn't IT! Thanks again chemist, you're very helpful. Seems even that existant files are faster too, at least they don't seem to skip...for some reason. So for a deadslow sd card I really recommend the compression to be turned on. You could change the guide to that safely I think! Now I'm considering that alternative with making it a mount unit. But I'm not sure what the gain is. Maybe its more 'correct'?

Larswad ( 2015-02-07 12:20:24 +0300 )edit
1

mount option in the .service file is compress already if someone wants something else than default they should do that on their own (compress=xxx), difference between .service and .mount is only that .mount is what it should be but it was easier for me to get .service to work (I did all of this on my Jolla, not even remote) +plus less lines in the guide and honestly... there is no gain fmpov

chemist ( 2015-02-07 15:27:26 +0300 )edit

any news ? want my offline maps back for Here Maps ...

elastic ( 2015-02-20 18:50:56 +0300 )edit

Anyone tried the latest update (Yliaavanlampi, 1.1.2.15) ?

RobNas ( 2015-02-20 20:01:58 +0300 )edit

yes, doesn't work anymore, but chemist stated this already in his thread.

Fellfrosch ( 2015-02-20 20:13:46 +0300 )edit

what are you guys saying? That it doesnt work with the data on externa sd after the update? In that case I don't agree. They work for me, and I'm sure I deleted the internal cards data after the move, so it can't be using that. What gives?

Larswad ( 2015-02-20 20:33:08 +0300 )edit

And what did you do to make it work?

Fellfrosch ( 2015-02-20 21:26:24 +0300 )edit
2

devel-su

systemctl stop aliendalvik.service

systemctl stop aliendalvik.path

mount -o subvol=android,compress,dirsync ,noatime,users /dev/mmcblk1 /data/media

lechuck ( 2015-02-20 21:58:35 +0300 )edit
4

kind of right, in 1.1.2.15 you need to wait for aliendalvik to settle for the first time, it optifies (chroot) all alien files and then fuse-mounts to /home/nemo/android_storage - then you do what @lechuck just said above !!-> if you do not have your androidsdcard.service running or your .mount - if you have the .mount or .service active you need to stop that and do not need to mount by hand as it will start on its own once alien is started again - I do not have a proper idea how to do that in an automatic way for $user for now.

chemist ( 2015-02-21 03:19:47 +0300 )edit

Ok thanx for your explanation. Got it to work. I have the androidsdcard.service active, so all I do is stop alien dalvik, stop androidsdcard.service and start an android app afterwards.

Fellfrosch ( 2015-02-21 08:54:25 +0300 )edit
2

New path for android is

/opt/alien/data/media/

So you just need to change mount path

alexxy ( 2015-02-21 11:53:04 +0300 )edit

haha, even more incentive to start an "uptime race" on our jollas :P as is understand as of now we have to manually stop services and mount after jolla settled after (every) reboot? correct?

mosen ( 2015-02-21 13:34:52 +0300 )edit
1

my uptime is only cut by upgrades... and you should probably stop androidsdcard before you stop aliendalvik as aliendalvik.path might start aliendalvik again before you stoped androidsdcard or it unmounted and it will blow the fuse and you'd need to reboot (happened to me)

chemist ( 2015-02-21 15:40:51 +0300 )edit

nope that is a bindmount of /data and it does not take submounts into accoutn when mounting (for some reason)

chemist ( 2015-02-21 15:45:36 +0300 )edit

I just want to explain my misunderstanding before. I assumed that since all android apps appeared to work for me the external card was still used. But so wasn't the case. For some apps like.my banking id it still worked so that was why I thought thimgs were still ok. But I suppose not all apps store their things in the data location but where they are installed. So it seems jolla.rebuilt the data for all apps into a clean state. Going to try the stopping and remounting.stuff mentioned here then.

Larswad ( 2015-02-21 17:27:24 +0300 )edit

@Larswad, could it be that you did not do this btrfs hack on a fresh setup but on an already populated android folder (already containing eg. your banking id)? This would explain you would again see an oder state of that folder with your historic data before that was copied to sd and then used/modified with further use?

mosen ( 2015-02-21 17:41:15 +0300 )edit

@Larswad, @mosen, The situation looks to me that after the update the internal mass storage is still there and in use, but the external storage (sdcard) points at wrong path. In most android phones the sdcard is emulated these days, so one does not have to think about difference of these two. Apps use them as they like.

So any applications that normally use internal memory in Android still work as they did before. But any apps that actively used emulated sdcard, while still launch, miss all their previous in use generated data.

For instance I lost my HERE application offline maps but I haven't lost my Zookeeper scores as they are saved in internal storage area. That hasn't stopped working in update.

Manatus ( 2015-02-21 18:33:50 +0300 )edit

/dev/fuse also has a second mount point in /opt/alien/home/nemo/android_storage

Does anybody knows what these additional /opt/alien mounts are good for?

J4ZZ ( 2015-02-21 19:24:07 +0300 )edit

Maybe a helper-script would be a solution like here: link text

elastic ( 2015-02-21 19:38:30 +0300 )edit
1

The way I did this btrfs hack was to first follow it through every point. And.when I saw things worked fine when mounted on the external card I disabled the services and then deleted EVERYTHING recursively under the /data/media folder so that the internal card was freed up. I know I did that right since the same amount of space was regained on the internal card after this. Then restarted the dalvik and all was fine (even if very slow like i discussed with chemist earlier). Anyhow, my guess.is the following is what have happened at some.point: The aliendalviksdcard service fails to mount for some reason or gets skipped. The internal.one kicks into use again and at that point it probably rebuild the data folders for the.apps i happened to use during that startup until the next time where the phone was rebooted.

Today I could see that only a few songs from one single.album had been cached by subsonic.on the internal storage. I think i at some time saw that it recached songs and wondered why. So, after the update to update11 it started to use that half-assed.internal /data/media again since the.aliendalviksdcard service is not started anymore. Guess maybe chemist who seems to be knowledgeable about this may find a new way to have the.service started on boottime. Its a hell writing with this virtual keyboard by the way, spelling errors everywhere, dots when missing the space all the time. I hate virtual keyboards with a passion.

Larswad ( 2015-02-21 19:44:14 +0300 )edit
1

@Larswad you don't have to type all the commands new every time. just use the up and down arrows. And you can switch between the last entered commands. It doesn't matter if you closed the terminal in between.

Fellfrosch ( 2015-02-21 21:21:57 +0300 )edit

@Fellfrosch: Thanks. But that part is ok, I'm a software developer by profession. I meant writing in forums and mail and alike. I just loathe the vkb. Its a trend for cellphones that should have died off instead of escalated.

Larswad ( 2015-02-21 21:45:17 +0300 )edit

Interesting to read about your interpretation how things work. To put it simple: You had your uSD mounted to android's /data/media in <1.1.2 - what happens on (probably any) update is that /data/media gets populated with the default setup so what happens in the case you had your stuff on your uSD but that does not get mounted -> nothing simpler than that! Everything works like expected as /data/media is android's userdata folder - everything works fine as it gets rebuilt upon app's start so you only recognize missing userdata like downloaded maps, podcasts etc nothing else. You wont miss anything until you try to access said userdata, which is the bloating balloon of your app-data filling your storage.

chemist ( 2015-02-21 21:57:01 +0300 )edit

Yes, exactly chemist. That's my take on it, or at least the best I can come up with.

Some sailor from Jolla maybe could help us out to find the best way to do this again. I don't want android to swallow my poor 16gb.

Larswad ( 2015-02-21 22:49:29 +0300 )edit

Without altering alien files? The only non-intrusive way is to run a script after aliendalvik was started that stops the androidsdcard.service (or .mount) and stops aliendalvik so it will mount accordingly the next time it is started (each boot).

chemist ( 2015-02-22 00:22:21 +0300 )edit

something like this?

 #!/bin/bash
service=aliendalvik.service

if (( $(ps -ef | grep -v grep | grep $service | wc -l) > 0 ))
then 
systemctl stop aliendalvik.service
systemctl stop aliendalvik.path
fi
elastic ( 2015-02-22 00:43:30 +0300 )edit

no, there is no process for aliendalvik.service, the process if any is alien.sh but even then how do you know it has finished starting? and don't forget that you have your androidsdcard.service to stop first, then you might even drop the stopping of aliendalvik.path.

chemist ( 2015-02-22 01:18:06 +0300 )edit

got the point - so we could try if it works with

  • alien_bridge_server
  • alien_gps_server
  • aliensensorserver
  • alien_audio_server

they are always loaded when Aliendalvik is running ...

elastic ( 2015-02-22 01:47:07 +0300 )edit

yes, for example, but how about to look if the fuse mount is there?

chemist ( 2015-02-22 03:21:09 +0300 )edit

isn't systemd capable of surveying such thing ? We could also test a specific directory that should always be there with the fuse mount. PS: regarding an event-based solution, I think the mount unit is more flexible than a service. But maybe I'm wrong.

Tofe ( 2015-02-22 11:49:22 +0300 )edit
1

I misread the alien.sh as I was probably drunk - coderus an myself were at it and it might be a bit easier, I will probably release a patch once I am sure I do not break anything (move data from bind to rbind section should just work). If you want to keep working on a non-intrusive solution please be my guest! Once done I will update the top post and will change that over to .mount as that is what I am using atm anyway.

chemist ( 2015-02-22 14:56:04 +0300 )edit

opt/alien is used to chroot all alien the fuse mount is to have /data/media available to nemo as $home/android_storage

chemist ( 2015-02-22 21:09:47 +0300 )edit
1

just to get it right and don't mess up my system ;-)

"move data form the bind section of alien.sh to its rbind section

means in /opt/alien/system/script/alien.sh

move the value [data] from section

for d in bin sbin lib usr var etc tmp home vendor **data**; do
if [ "" == "$(grep $ROOT/$d /proc/mounts)" ]; then
    echo "mount $ROOT/$d"
    mount --bind /$d $ROOT/$d

to

for d in dev sys run **data**; do 
if [ "" == "$(grep $ROOT/$d /proc/mounts)" ]; then
    echo "mount $ROOT/$d"
    mount --rbind /$d $ROOT/$d
elastic ( 2015-02-23 22:02:34 +0300 )edit
2

yes that should be it :) - with androidsdcard.service still enabled of course (or the .mount) - reboot and check that everything is mounted as expected after a minute or two (alien takes ages to start for me)

chemist ( 2015-02-23 22:15:27 +0300 )edit

works perfect thx - btw: my alien is available directly after startup ...

elastic ( 2015-02-23 22:42:49 +0300 )edit

Mayday, I could need some small assistance here. I accidentally deleted the folder android_storage from /home/nemo/ while using (harbour) file browser (it was empty because no mounts were active at this time). The thing is, that now after recreating the folder /dev/fuse doesn't mount to this folder anymore. Only /opt/alien/home/nemo/android_storage gets filled with content. /home/nemo/android_storage stays empty. Even though mount | grep fuse list both directories as successfully mounted. As a consequence of this the sdcard softlinks under / and /opt/alien stay empty as they both point to /home/nemo/android_storage and therefore android apps don't get any sdcard content and complain about it.

Does someone possibly has an idea how to get this fixed... Since it seems some kind of a bug for me.

Thanks in advance

J4ZZ ( 2015-02-24 01:46:20 +0300 )edit

permissions maybe? android_storage is owned root:nemo and is 775 permission wise - if that does not help maybe uninstall -> reinstall dalvik?

chemist ( 2015-02-24 10:15:41 +0300 )edit
1

I couldn't clearly figure out a summary from the long conversation here:

  1. What changes in the OS / Alien Dalvik with update 1.1.2? Files from /data/media get moved to a different location once and chroot is executed in runtime?

  2. What is the recommended upgrade path?

  3. How alien.sh is connected to our little hack?

I don't have the update yet, just preparing.

Vuubi ( 2015-02-24 14:14:12 +0300 )edit
  1. dalvik gets chrooted and the optification does not mount /data recursively so the previously mounted /data/media is not recognized
  2. upgrade and follow instructions for 1.1.2 in top post
  3. alien.sh is optifying dalvik to have it then chrooted
chemist ( 2015-02-24 15:55:41 +0300 )edit

@chemist The permissions are fine here, still it won't work. As I actually don't want to reinstall dalvik in the first place, is there any mount command I can use to remount both fuses and see if that helps? Edit: Plus could you please check the ownership of /home/nemo for me, too?

J4ZZ ( 2015-02-24 16:02:12 +0300 )edit

you defuse by stopping aliendalvik.service and .path (that restarts it faster than you type mount sometimes), if that does not defuse something is corrupted and you probably need to reboot... oh that permission stuff we had before... /home/nemo is 750 and nemo:nemo

chemist ( 2015-02-24 16:10:30 +0300 )edit

I am puzzled by this whole discussion, as far as I can tell though alien may be mounting data differently now all my old data is still accessible and it's not preventing me from adding new data....

What are the symptoms I 'should' be seeing?

(And just to be clear I am using 1.1.2)

Keeper-of-the-Keys ( 2015-02-24 16:42:49 +0300 )edit

@Keeper-of-the-Keys /home/nemo/android_storage == /opt/alien/data/media != /data/media if you are using the androidsdcard.service without changing anything... dalvik is now chrooted to /opt/alien/ so the fuse-mount is mounted from /opt/alien/data/media which is not (!=) /dev/mmcblk1 mounted at /data/media as the optification for the chroot does bind-mount /data but not /data/media as it is not "r"bind-mounting -> that has to be fixed in alien.sh

chemist ( 2015-02-24 17:27:27 +0300 )edit

But shouldn't my saved games, installed programs etc. be gone in that case or does this only affect new stuff I install?

Keeper-of-the-Keys ( 2015-02-24 18:05:24 +0300 )edit

They are not part of /data/media - /data/media is the pure userspace - so downloaded maps, music-caches etc...and that is why it is fused to /home/nemo/android_storage! Installed apps are /data/app for example (that is the folder aliendalvik.path is checking if empty and starting dalvik when it is not empty) - /data/media is the part that can grow huge very fast, all the rest is minor compared to it. You can also read it as "the data not really needed to run anything".

chemist ( 2015-02-24 19:00:18 +0300 )edit

A ".path" unit is what I talked about: couldn't we monitor the /opt/alien/data/media/something, and when this unit is activated, we activate our mount ? I feel that with a path unit, we can achieve what we would like to do...

Tofe ( 2015-02-24 19:07:52 +0300 )edit
1

@chemist... Ok, permissions are fine here... I also changed alien.sh to rbind data. mount | grep fuse shows both folders

/dev/fuse on /opt/alien/home/nemo/android_storage type fuse (rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other)
/dev/fuse on /home/nemo/android_storage type fuse (rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other)

but /home/nemo/android_storage is still empty...

What else could it be?

J4ZZ ( 2015-02-24 19:17:22 +0300 )edit

@J4ZZ what else... how about you haven't anything mounted to it? and/or deleted the folder for real, eg ther is actually nothing?

chemist ( 2015-02-24 20:23:45 +0300 )edit

Hehe, nope.... As I wrote, I can see all files in /opt/alien/home/nemo/android_storage and since it's just some kind of mirror of /home/nemo/android_storage I'm sure my files are intact ...

And that's exactly what I'm not getting... Why there are two successful mounts but only one shows the content...

J4ZZ ( 2015-02-24 20:42:39 +0300 )edit

It is not some kind of mirror, both are separate fuses of /opt/alien/data/media, so your fusemount fails... the unmounted /home/nemo/android_storage is drwxrwsr-x - note the s for group, have you checked the directory as nemo or root?

chemist ( 2015-02-24 21:11:21 +0300 )edit

I checked with both but the result is the same. /home/nemo/android_storage stays empty... Permissions are fine and I additionally changed the creation date with 'touch' to match the folder from /opt/alien/ but no success either.

Are those fusemount folders somehow special, so you can't simply recreate them?

This is my current grep fuse ouput:

fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
statefs on /run/state type fuse.statefs (rw,nosuid,nodev,relatime,user_id=0,group_id=100,default_permissions,allow_other)
statefs on /run/user/100000/state type fuse.statefs (rw,nosuid,nodev,relatime,user_id=100000,group_id=100000,default_permissions,allow_other)
fusectl on /opt/alien/sys/fs/fuse/connections type fusectl (rw,relatime)
statefs on /opt/alien/run/state type fuse.statefs (rw,nosuid,nodev,relatime,user_id=0,group_id=100,default_permissions,allow_other)
statefs on /opt/alien/run/user/100000/state type fuse.statefs (rw,nosuid,nodev,relatime,user_id=100000,group_id=100000,default_permissions,allow_other)
/dev/fuse on /opt/alien/home/nemo/android_storage type fuse (rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other)
/dev/fuse on /home/nemo/android_storage type fuse (rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other)
J4ZZ ( 2015-02-24 22:35:53 +0300 )edit

no idea... would you mind to use pastebin or something instead of posting it in here... and how about pasting (to pastebin) journalctl -b output and mount please, and probably ls -la /home/nemo/android_storage in both states fused and not fused.

chemist ( 2015-02-24 23:53:38 +0300 )edit

Thanks for helping me mate. Here is my journal mount is here and ls of /home/nemo/android_storage is always

total 0
drwxrwsr-x 1 root nemo    0 Feb 11 01:50 .
drwxr-x--- 1 nemo nemo 1022 Feb 25 01:13 ..

no matter if fused or unfused

I hope you find something

J4ZZ ( 2015-02-25 02:43:49 +0300 )edit

You need to find out why it does show as fused but is not fused, maybe ask some jolla people on IRC for help and tell that you accidentally deleted the folder...

chemist ( 2015-02-25 10:38:50 +0300 )edit

@J4ZZ@chemist I have the same issue, tried many combinations of permission, no mirror at /home/nemo/android_storage (and all dependent links).

jolladiho ( 2015-02-25 22:44:34 +0300 )edit

I just did the "data" move, and everything worked A-OK!

Thanks Chemist!

Juuba ( 2015-02-25 22:51:06 +0300 )edit

@jolladiho same for you, ask for help somewhere with Jolla staff (IRC for example), as I won't be able to help you despite suggesting to start over (reinstall aliendalvik for example)

chemist ( 2015-02-25 22:55:24 +0300 )edit

I'm still at 1.1.1.27 update (and have the shared space implemented), I want to go to 1.1.2.x, but what do I need to do: the part after "prepare the system" or after "1.1.2.x"?

RobNas ( 2015-02-26 12:58:39 +0300 )edit

@RobNas

For me, I didn't need to do anything before hand. After update I just moved the "data" word from bind to rbind section in alien.sh.

Just like the original post describes (it has been edited numerous times, but it is current!)

Juuba ( 2015-02-26 13:48:53 +0300 )edit

@RobNas the starting post is only to be done from scratch if you start from scratch, if your state is 1.1.1 then of course you do not need to do that again. As @Juuba pointed out, just like described. (sometimes I wonder if people actually read what I write or if they just filter cmd prompts while crossreading)

chemist ( 2015-02-26 14:07:49 +0300 )edit
2

@J4ZZ I found a solution for you (and me) ;-). Am I right, that you have (like me) your "nemo" folder on a btrfs sdcard and mount it into /home/nemo at startup?

All sailors who did this have to do one more change in /opt/alien/system/script/alien.sh. The home entry has to be moved to the "rbind" section. It should look like this:

for d in bin sbin lib usr var etc tmp vendor; do
    if [ "" == "$(grep $ROOT/$d /proc/mounts)" ]; then
        echo "mount $ROOT/$d"
        mount --bind /$d $ROOT/$d
    fi
done

for d in dev sys run data home; do
    if [ "" == "$(grep $ROOT/$d /proc/mounts)" ]; then
        echo "mount $ROOT/$d"
        mount --rbind /$d $ROOT/$d
    fi
done

Without this fix, the alien script mounts the original "/home" from the mmcblk0p28 partition, and also the /home/nemo from the mmcblk0p28. With this fix you will get your /home/nemo. As I could investigate, there is a bug in 1.1.2.15 and 16, that makes this change necessary. BUG: I think the fuse mount to /home/nemo/android_storage is never done.

MountService: Duplicate state transition (mounted -> mounted) for /home/nemo/android_storage

if you make the change to alien.sh, you will use the same folder in /opt/alien/home/nemo and /home/nemo.

jolladiho ( 2015-02-26 17:58:48 +0300 )edit
1

Brilliant.... That did it!!! Huge thanks!

J4ZZ ( 2015-02-27 03:05:05 +0300 )edit

@ Juuba: thanks for the reply. I assume alien.sh will be made during the update (because I can't find it now)? So what's on the shared part now will still be there after the update and modification of alien.sh? @chemist: I think people are reading, but don't understand everything (counts for me). In that case I'd rather ask the obvious question instead of a chance of losing all data...

RobNas ( 2015-03-01 19:08:19 +0300 )edit

@RobNas sure, better safe than sorry ;)

chemist ( 2015-03-01 22:35:28 +0300 )edit

Chemist, my sincere apologies for being such a newbie... I've seen my androidsdcard.service has 'subvolid' instead of 'subvol'. I didn't notice any issues, shall I change it anyway?

RobNas ( 2015-03-01 23:29:09 +0300 )edit

@RobNas you can mount by ID or by path, if you mount subvolid=android that is wrong and should be subvol=android, if you mount by subvolid="$ID_of_android_subvolume" for example subvolid=123 that is fine. If it works the other way round that is probably a glitch in the mounting options and could vanish over time (at least I do not know that this should be possible intentionally).

chemist ( 2015-03-02 01:23:18 +0300 )edit

Hmm, something changed???

Now my sdcard is mounted "read-only"... for android at least..

mount | grep 'mmcblk1\|android' output: https://gist.github.com/juuba/4924e4d0882c3c078de2

What do?

Juuba ( 2015-03-04 09:08:43 +0300 )edit

Seems there might be something wrong, since the OS turns the mount to "read-only" after some write errors.. It worked 2 days ago, but not anymore...

Juuba ( 2015-03-04 10:22:52 +0300 )edit

Tried to make a backup and store it on pc, but unfortunately the copy to sd card function doesn't work. Manually copy...doesn't work. Now what?

RobNas ( 2015-03-04 13:58:38 +0300 )edit

You mean copy backup to sdcard from settings backup? Is the jolla volume mounted before any other mount of the sdcard? Manually copy does not work either? Is it even mounted? Cannot help you without pastebin of /proc/mounts and maybe journalctl -b

chemist ( 2015-03-04 14:56:18 +0300 )edit

I can confirm, you have to do the same changes to alien.sh in 1.1.4 as in 1.1.2, like chemist wrote. It would be fine if Jolla could add the change to next version - it should work with a standard config too - or am I wrong with this?

jolladiho ( 2015-04-24 17:59:06 +0300 )edit

you missed please be aware that copying backup to uSD will stop working while aliendalvik is running and therefor the mount is active (you have to stop all of it to be able to restore from or save to uSD - TheVault has mismanagement, also MTP stopped working for me while the mount is active! there is more to it than just having the right bindmounts

chemist ( 2015-04-24 21:24:27 +0300 )edit

Just some sidething here, I am a little baffled that the android media partition is mounted on so many places. It's mounted or symlinked at all of these places, it's hysterical:

/mnt/sdcard/ (symlink)

/data/sdcard/ (symlink)

/data/media/ (mount)

/home/nemo/android_storage (mount)

/opt/alien/sdcard/ (symlink)

/opt/alien/data/sdcard (symlink)

/opt/alien/data/media (mount)

What's the reason, is it because of the android nature?

And it makes it really hard to know what is taking up space on the internal card because of those mounts that are everywhere that reflects a size but it is actually mounted on the external.

And: Have I understood it right: If I want to either make a Backup.tar using the builtin backup tool, do I first stop all the alien services like described.

Then umount the mmcblk1, then issuing the mount -o subvolid=0 /dev/mmcblk1 /media/sdcard

and finally doing the backup to or from, and then finally umount the mmcblk1 one again,

and then restart the services?

Larswad ( 2015-05-12 18:01:45 +0300 )edit

Just a little comment to the update "MTP stopped working completely, tested Debian & Win7.": For me MTP stopped working only on the SD card. I can access my Phone Memory just fine, only folder "SD Card" is empty. Tried on Windows 7.

casanunda ( 2015-05-12 18:45:10 +0300 )edit

@Larswad you are getting confused by the fact that mmcblk1 is btrfs and has several subvolumes, they all show as "mmcblk1 is mounted to X" in /proc/mounts - there are no symlinks, this is all bindmounts (the same just for mounts, works a tad different than symlinks) - the /opt/alien* are bind-mounts for the aliendalvik chroot - df -h should give you an impression of what is where and how much space is left. <> for backups to work you stop the .path/.service/.mount only (systemctl stop ...) like described and nothing else! > then you open settings and go to backup (if you are in backup already you need to leave it as it is a program that initializes the proper path on startup). No need to start anything again either. If you want to use android stuff again just start it, it will take a couple of seconds but everything will be fine again.

@casanunda for me both work when stopping everything all together, for you too?

chemist ( 2015-05-12 20:25:36 +0300 )edit

Thanks. Yes, that part I get about btrfs, its like sub-partitions to the created volume. But nevertheless, all of these mountpoints (well some of those I listed DO show up as if they actually were symlinks, not bind mounts, because using ls -alh those are pointed with an arrow to the actual location). Anyhow, these are shown up on all these different places for the very same file system. I don't get the idea for each of these places? I CAN understand lets say /mnt/sdcard and /data/media, but the rest.. Why on the /opt locations as well (or vice versa, but not both)? The 'alien' hierarchy is duplicated under /opt, /data, /home/nemo/ /mnt/. The chroot should only need one of those places or?

Indeed the df -h gives me the space for the internal. But trying to find out if there are some places where data is taking up most space (maybe old junk or so) is difficult using du -hs since these alien mounts are all over the place, counting it into the result (Same goes for the Space Inspector app, it gets confused on its results).

But maybe, if I simply stop the service it will dismount them all... Then 'du -hs' won't count those parts.

Larswad ( 2015-05-13 15:47:25 +0300 )edit

@chemist: I see now heh, du -x is said to exclude other mounts, and by default it should ignore symlinks. That's good.

Larswad ( 2015-05-13 16:02:47 +0300 )edit

Maybe try a GUI app like space inspector if you aren't getting your way with du the /opt/alien duplication will not vanish, that is the chroot and gets bind-mounted only once on first start - the duplication is needed for said chroot. I have no idea what you are talking about symlink wise, there are symlinks for sdcard and stuff but they have nothing to do with those bind-mounts. Can it be that you maybe left lots of/all stuff in the internal /data/media? /data/media is what takes most of the space, there is all kinds of caches and userdata. Some handy things, du -hsx /data will show only the size of data without the from somewhere else mounted /data/media, while du -hsx /data/* will show all folders of /data with the sum of those subfolders also ignoring bindmounts below, both do not follow symlinks. So du -hsxc /* returns only the internal memory with a "total" at the end - no need to unmount or stop any services... for occupation of the android part of your uSD you have to look at du -hsxc /data/media/*

chemist ( 2015-05-13 16:14:42 +0300 )edit

@chemist: hmm, you may be right. This may be really old stuff, these symlinks that I see. may be some old attempts or something to get the android environment to 'see' the external sdcard (or something). I have no memory of particularly creating those symlinks, but I may be wrong.

Anyhow, there must be some other "duplication" going on as well, because the /data/data (or /opt/data/data which is the same) mounted on internal has the application data (900mb).

BUT, under /data/media/Android/data/ (or /opt/alien/data/media/Android/data) mounted on external there is ALSO application data (709mb). Which one is the right one? This is confusing. ANDROID is confusing. So, maybe according to this: http://android.stackexchange.com/questions/46926/android-folder-hierarchy The same app may have data on several location if its a big one?

Larswad ( 2015-05-13 18:40:41 +0300 )edit

Android apps have data all over the filesystem, it is close to impossible to properly track what is where as for example HERE mapfiles are on a completely different location than MTVmusic soundfiles (both below /data/media/ though), both are considerably the same type of data, user-downloaded content not useful for anything else but yet they are not located below the same structure... one app says it is a cache even if it is "offline" data, one says it is app-data and so on, there is no "single" place where you find all of it but in general most or all "downloaded by an app" content will be in /data/media (bound to /opt/alien/data/media/).

chemist ( 2015-05-13 21:35:49 +0300 )edit

I'm following the steps indicated but im having a problem getting the jolla sdcard and android mount on the sdcard

The problem is that i cant have at the same time the jolla and android subvolume mounted. What i found is that the script /usr/sbin/mound_sd.sh can not mount a device which has an entry on /proc/mounts.

/usr/sbin/mount-sd.sh

"DIR=$(grep -w ${DEVNAME} /proc/mounts | cut -d \ -f 2) if [ -n "$DIR" ]; then systemd-cat -t mount-sd /bin/echo "${DEVNAME} already mounted on ${DIR}, ignoring" exit 0 fi"

This should be bypassed with the entries on the file /etc/systemd/system/data-media.mount After=mount-sd@mmcblk1.service and BindsTo=mount-sd@mmcblk1.service. But they dont seem to work.

Any help?

corsarium ( 2015-06-25 16:22:03 +0300 )edit

Hard to debug, go through the how-to again and check for typos, is it really mmcblk1 or did you format mmcblk1p1? The file content I posted is copy&paste of my on-device stuff...

chemist ( 2015-06-25 18:57:22 +0300 )edit

[Unit]

Description=SD card btrfs subvolume for Android

ConditionPathExists=/dev/mmcblk1

ConditionPathExists=!/tmp/os-update-running

ConditionPathIsDirectory=/data/media

Requires=local-fs.target systemd-user-sessions.service

Before=aliendalvik.service

After=local-fs.target mount-sd@mmcblk1.service

BindsTo=mount-sd@mmcblk1.service

[Mount]

What=/dev/mmcblk1

Where=/data/media

Options=subvol=android,compress,dirsync,noatime,users

[Install]

WantedBy=local-fs.target aliendalvik.service

I have modified:

lechuck ( 2015-06-25 21:18:57 +0300 )edit

good find with the sessions! requires is like after+bindsTo iirc so local-fs.target is not needed in After but well that should work nevertheless

chemist ( 2015-06-25 22:34:42 +0300 )edit

Hi,

what about the upcoming upgrade to 1.1.7.24?

https://together.jolla.com/question/98953/release-notes-117-bjorntrasket/

Do we have to revert the android settings?

Regards

gfwp ( 2015-07-15 15:13:05 +0300 )edit

It still says >1.1.4 right? ;) You might want to disable the .mount systemctl disable data-media.mount for the upgrade and enable it again afterwards, can be done after upgrade (is already in the starting question!)

chemist ( 2015-07-15 15:32:52 +0300 )edit

6 Answers

Sort by » oldest newest most voted
13

answered 2016-07-31 18:04:53 +0300

gcobb gravatar image

I have created a variant of this for use on the Jolla C, with SailfishOS 2.0.2.43 (Aurajoki). This has the following main differences with the original solution @chemist created:

  1. The SDcard is formatted to ext4 instead of btrfs (as there is no btrfs support in this Sailfish version). I also kept a conventional partition layout (to make it more likely other systems can easily mount the card. although they need to support ext4).
  2. The android directory that is overlaid is /home/nemo/android_storage instead of /data/media.
  3. Bind mounts are used, instead of btrfs subvolume mounts. The Android directory is hidden in an attempt to prevent tracker indexing the files twice.

Here is my replacement for @chemist's instructions:

preparing the sdcard, should only be needed once!

first dosystemctl stop aliendalvik.service then follow this list

  1. umount /dev/mmcblk1p1 - unmount the card might be another /dev or even more than one
  2. mkfs.ext4 /dev/mmcblk1p1 - format the card
  3. mount /dev/mmcblk1p1 /media/sdcard
  4. cd /media/sdcard
  5. mkdir .android - create folder called .android - this is hidden (starts with a dot) so it does not appear in directory listings
  6. I like to create a file on the card to remind me what I have done: so create a file called Android.README and fill it with:

    Android files are contained in a hidden directory called .android

    See https://together.jolla.com/question/40802/how-to-format-your-usd-card-to-btrfs-and-share-space-with-android/

  7. I also like to create a file with a unique name to let me easily check that Android apps are seeing the expected SD card. Something like: touch .android/GRC-128GB-SD.android

  8. chown -R nemo:nemo .android/
  9. chmod -R a=rwX .android/
  10. cp -r -a -v /home/nemo/android_storage/* /media/sdcard/.android
  11. cd ..
  12. umount /dev/mmcblk1p1

preparing the system

  • The next step requires knowing the Filesystem UUID for the SDcard filesystem. You can get it using lsblk -no UUID /dev/mmcblk1p1. The UUID is a string like 12345678-9abc-def0-1234-56789abcdef0. This string must be used instead of the string UUID-HERE in the file below (two occurrences).
  • create (as in do create not a command) /etc/systemd/system/home-nemo-android_storage.mount and fill with:

#

[Unit]
Description=SD card bind mount for Android
ConditionPathExists=/dev/mmcblk1p1
ConditionPathExists=!/tmp/os-update-running
ConditionPathIsDirectory=/home/nemo/android_storage/
ConditionPathIsMountPoint=/media/sdcard/UUID-HERE/
Requires=local-fs.target systemd-user-sessions.service
Before=aliendalvik.service
After=mount-sd@mmcblk1p1.service
BindsTo=mount-sd@mmcblk1p1.service

[Mount]
What=/media/sdcard/UUID-HERE/.android/
Where=/home/nemo/android_storage/
Options=bind

[Install]
WantedBy=local-fs.target aliendalvik.service

Then run systemctl enable home-nemo-android_storage.mount.

  • the mount file gets started every time aliendalvik is started and stops aliendalvik when it gets stopped, on boot it prevents aliendalvik from starting if it is not started yet but waits for the local-fs to be settled and actually having a mountpoint

  • If you put a different SD card in, the mount file will do nothing (because the UUID does not match). The original /home/nemo/android_storage will be used. You can repeat the steps from the top, of course, for the new card.

edit flag offensive delete publish link more

Comments

Thanks mate, I tried that myself a couple of days ago and was already in progress of updating the main article when something went wrong, I guess it was because I used putty out of windows with a wrong lineending or something, as I was unable to start the .mount. I will copy this in the OP if that is ok with you.

chemist ( 2016-08-02 00:17:46 +0300 )edit

Feel free (particularly if you or someone else has tested the instructions!)

gcobb ( 2016-08-02 00:56:04 +0300 )edit

I wrote it myself but in general just the same but one line of yours, I will copy it upstairs then with a link to your answer ;)

chemist ( 2016-08-03 14:11:11 +0300 )edit

Hey guys,

After 2 weeks mutiple trys I still have the same problem that my jolla phone recognizes my sdcard, but it does'nt save any android data on it and android app don't show the sd card space. I worked with gcobb's solution but it still doesn't work.

I'm using Aurajoki 2.0.2.51 and first generation Jolla Phone 1.0.0.13 and I don't know which Android Version is simulated by the Jolla Android Support.

Strangley after serval times of trying gcobb's solutions I have now 2 folders on my SD card both with a different UUID!

I need some new input and help, please!

Quermitschi ( 2016-10-19 18:23:40 +0300 )edit

@Quermitschi: Android apps and data have two main locations on Sailfish OS (release 2.0.5). One is /home/nemo/android_storage and the other /home/.android.
Bind mapping proposed by chemist and gcobb here concern directory /home/nemo/android_storage. With that approach some big junks of data like map tiles (of Here WeGo, etc.) can be moved and added to the SD card and used from there. Very good. However, Android apps themselves and their data reside under /home/.android. For instance, the game of Clash of Clans has nothing under /home/nemo/android_storage (afaics). No, both the apk and the game data are under the tree of /home/.android.
I wonder if a similar bind mapping of /home/.android (or of an appropriate subdir, e.g. /home/.android/data/data) would make it possible to collect the app data to the memory card. It would be interesting to know. Making this work fluently and reliably would probably need more insight to Alien Dalvik.

jovirkku ( 2016-11-22 17:06:07 +0300 )edit
4

answered 2014-06-12 19:39:58 +0300

Vuubi gravatar image

This is how you can make chemist's version of systemd service unit (androidsdcard.service) into systemd mount unit.

First, create /etc/systemd/system/data-media.mount file:

[Unit]
Description=SD card btrfs subvolume for Android
ConditionPathExists=/dev/mmcblk1
ConditionPathIsDirectory=/data/media
Requires=local-fs.target
Before=aliendalvik.service
After=local-fs.target mount-sd@mmcblk1.service
BindsTo=mount-sd@mmcblk1.service

[Mount]
What=/dev/mmcblk1
Where=/data/media
Options=subvol=android,compress,dirsync,noatime,users

[Install]
WantedBy=local-fs.target aliendalvik.service

Second, create /etc/systemd/system/aliendalvik.service.d/sdcard.conf file:

[Unit] 
PartOf=data-media.mount

Then run systemctl enable data-media.mount (androidsdcard.service must be disabled before that).

edit flag offensive delete publish link more

Comments

there should be a button to convert to answer at your comment... that is why I asked @eric can you clean up again please (sry)

chemist ( 2014-06-12 20:18:32 +0300 )edit

No, I don't see any buttons there except vote arrow.

Vuubi ( 2014-06-12 20:27:31 +0300 )edit
2

Running a fresh install of 1.1.0.39, with Android support installed and Yandex Store working, but I don't seem to have the folder "aliendalvik.service.d/" in etc/systemd/system/. Should I create it and then create /etc/systemd/system/aliendalvik.service.d/sdcard.conf file or is there something else I need to do?

oenone ( 2014-11-26 00:30:47 +0300 )edit
2

@oenone

i just created a folder called aliendalvik.service.d and inside this folder i made the file sdcard.conf

aki ( 2014-12-14 17:09:49 +0300 )edit

I missed something crucial here. Why does Android apps complain "not enough space"? My lucky guess is that alien-dalvik cannot access android subvolume or somewhat. I only used wuubi's mount script.

DarkWhite ( 2015-01-06 16:05:31 +0300 )edit
2

answered 2014-06-11 12:47:51 +0300

alci gravatar image

If you want to use the sdcard as an extended memory and put your /home on it, you need to make sure it is mounted at the right moment and available for the ambiance setting, etc...

I tried the solution mentioned by the OP, but it implies modifying a system script, which is not upgrade-proof. Moreover, the script is not called early enough to let ambiance and some settings work...

I have found that a much solution is this one: Maemo thread to mount sdcard as /home I am using this for 2 month and a half now, 2 upgrades, and it has worked reliably for now. Of course, YMMV :-)

edit flag offensive delete publish link more

Comments

3

changed it to a systemd service, and this is not about using your sdcard as /home...

chemist ( 2014-06-11 15:09:38 +0300 )edit
1

answered 2015-02-22 18:36:23 +0300

jolladiho gravatar image

updated 2015-02-24 21:24:04 +0300

This is a workaround for 1.1.2. Someone changed it from a comment to an answer, thanks for this. I wrote it as comment, before chemist published the fix for 1.1.2 in the top thread. I recommend to use the fix from chemist.

As difference to the fix: No changes to delivered system files are necessary. You need the script and a change in the androidsdcard.service.

After reading comments from top thread, I wrote a little script, for all, who use a .service for the android-sd mount, like described in top thread. It puts some messages in the journalctl if you use it with 'mydebug=1'. Change the 'mymountoptions' var to your needs (the subvol entry at minimum!). The script waits for all alien processes, before it stops the aliendalvik.service and .path. Then it mounts the subvol to /data/media and starts the aliendalvik.path again. Use at your own risk.

If you make some mistakes, you have to know, how to start the recovery shell, mount the root subvol and remove a systemd service in that environment.

The script: /usr/local/bin/tmpfix-alien-mount-1.1.2.15.sh

#!/bin/bash
# dirty but handy
# stop android after startup, do mount and restart aliendalvik.path
# set the vars to your needs (user variables)
mymountoptions="subvol=.android,compress,dirsync,noatime,users,autodefrag,rw"
mydebug=1 # set to 1 for logging (jounalctl | grep tmpfix-alien-mount)
# no changes below needed
myservices="alien_audio_server aliensensorserver alien_bridge_server alien_gps_server alien_keyboard_server"
[ $mydebug == 1 ] && systemd-cat -t tmpfix-alien-mount /bin/echo "started with: PID=$$"
for i in $myservices
do
  mycount=$(/bin/ps -ef | /bin/grep -v grep | /bin/grep $i | wc -l)
  while [ $mycount -lt 1 ]
  do
    /bin/sleep 1
    mycount=$(/bin/ps -ef | /bin/grep -v grep | /bin/grep $i | wc -l)
  done
  [ $mydebug == 1 ] && systemd-cat -t tmpfix-alien-mount /bin/echo "$i found, counts: $mycount"
done
[ $mydebug == 1 ] && systemd-cat -t mount-sd.custom /bin/echo "all aliendalvik processes found."
/bin/sleep 1
systemctl stop aliendalvik.service
/bin/sleep 1
systemctl stop aliendalvik.path
[ $mydebug == 1 ] && systemd-cat -t tmpfix-alien-mount /bin/echo "aliendalvik stopped."
/bin/sleep 1
/bin/mount -o $mymountoptions /dev/mmcblk1 /data/media
[ $mydebug == 1 ] && systemd-cat -t tmpfix-alien-mount /bin/echo "mount with $mymountoptions done."
/bin/sleep 1
systemctl start aliendalvik.path
[ $mydebug == 1 ] && systemd-cat -t tmpfix-alien-mount /bin/echo "aliendalvik.path restarted, all done, exit."

I put it in /usr/local/bin and make it executable for user/group root.

In my androidsdcard.service I replaced the 'ExecStart' and 'ExecStop' part with.

ExecStart=/usr/local/bin/tmpfix-alien-mount-1.1.2.15.sh
ExecStop=/bin/umount /data/media

My Jolla did not shutdown correctly as I tried to add a 'systemctl stop aliendalvik.service' and .path to the ExecStop.

edit flag offensive delete publish link more

Comments

Please delete

elastic ( 2015-08-04 00:16:24 +0300 )edit
1

answered 2015-12-30 18:14:22 +0300

tyler gravatar image

I meet a wired problem: if I was downloading to card through an android app, software cannot write/read sd card after some time.

$ cd android_storage/    # another android app is downloading in background
$ while true; do
> date
> date > write_test
> sleep 3
> done
Wed Dec 30 23:46:46 CST 2015
Wed Dec 30 23:46:49 CST 2015
Wed Dec 30 23:46:52 CST 2015
Wed Dec 30 23:46:55 CST 2015
Wed Dec 30 23:46:58 CST 2015
Wed Dec 30 23:47:01 CST 2015
Wed Dec 30 23:47:04 CST 2015
Wed Dec 30 23:47:07 CST 2015
Wed Dec 30 23:47:10 CST 2015
Wed Dec 30 23:47:13 CST 2015    # frozen forever, android app downloading frozen
^C
$ ls    # frozen here forever
edit flag offensive delete publish link more

Comments

iirc that is android hooking the device and stalling any write, like stuck maps download (happened quiet often to me) - no idea if it is the btrfs driver or alien though. Stop alien and try again from scratch. Does it happen always or random? Would you mind checking if that happens without the sdcard mounted (internal storage), aka reproduce.

chemist ( 2015-12-31 13:48:05 +0300 )edit
0

answered 2014-06-11 12:32:11 +0300

RobNas gravatar image

updated 2014-06-13 14:21:14 +0300

This answer is obsolete now. I've left the part of editing with VIM, for anyone who's interested. To create a file, just type in terminal: touch /to/path/filename.extension Where "/to/path/" is the location of where the file needs to be located, and "filename.extension" is, surprisingly, the filename and extension. I've edited it with VIM. How to:
pkcon install zypper - if you haven't installed vim yet
zypper in vim - if you haven't installed vim yet
vim /usr/sbin/mount-sd.sh
To edit in vim, type "i"
Now you can type your text. After editing, get out of edit-mode, by pressing ESC twice. Then enter shift twice (capslocking it) and tap z twice ("ZZ"is the command for vim to save and exit).

edit flag offensive delete publish link more

Comments

1

I updated the top post, you need to be more patient with me please... and stop scaring people! you do not loose no data from your sdcard, it is just not mounted! my workaround answer is a bash-script, if you are unable to see that you probably should stay away from this how-to!

chemist ( 2014-06-11 15:11:39 +0300 )edit

You're right, I've screwed up. To be honest, I don't understand half of the code what is be done there and I'm just trying to use common sense (and a good bit of good luck). I've used part of the rewritten code for 1.0.7.16 now, and it's working again.

RobNas ( 2014-06-13 14:17:31 +0300 )edit

@RobNas join me on IRC if you need help or anything, if you updated to 1.0.7.16/18 the only thing you need to do is create the androidsdcard.service and enable it (or the .mount Vuubi describes in his answer), if you had the early version without the "after mount-sd@..." and update the service file, you need to stop and disable it before you do, systemd needs to be reloaded to reflect the changes then you enable it again and everything should work as expected!

chemist ( 2014-06-13 15:27:08 +0300 )edit

@chemist from step 1 ( umount /dev/mmcblk1p1 - unmount the card might be another /dev or even more than one) I am met with the message "umount: /media/sdcard/E42F-2BD2: umount failed: Operation not permitted"

please can you advise

ozzyo1212 ( 2015-01-11 12:14:18 +0300 )edit

@ozzyo1212 you probably need to be root to do that, if that did not help you need to be privileged (devel-su -p or -P iirc)

chemist ( 2015-01-11 23:36:30 +0300 )edit
Login/Signup to Answer

Question tools

Follow
95 followers

Stats

Asked: 2014-04-25 14:46:10 +0300

Seen: 25,382 times

Last updated: Aug 03 '16