btrfs problem / home dir gone

Question

1

btrfs problem / home dir gone

asked 2018-06-07 11:37:14 +0200

nas
491 ●13 ●26 ●32

Hello,

few days ago my Jolla1 show me that memory was almost full. Yesterday I empty a lot of space (~3Gb) and yesterday noon I started btrfs-balancer balance . Today I saw it stuck at 10% and the phone was unresponsive. I took out the battery (which as I read after wasn't very good idea), and now the phone when it starts, it shows the Jolla logo, then it is gone and the white light remains and stops there.

I went to recovery mode, where I tried:

/ # mount  -o  subvol=@home  /dev/mmcblk0p28 /myhome
mount: mounting /dev/mmcblk0p28 on /myhome failed: No space left on device

I mounted /dev/mmcblk0p28 (mount /dev/mmcblk0p28 /myhome) and I saw that the /home directory is totally empty :( .

When I do a check, I get these errors:

/ # btrfs check --repair /dev/mmcblk0p28
enabling repair mode
Checking filesystem on /dev/mmcblk0p28
UUID: 808158a7-ac77-4713-e316-7a3191b759cf
checking extents
checking free space cache
cache and super generation don''t match, space cache will be invalidated
checking fs roots
root 257 inode 26327 errors 400, nbytes wrong
root 258 inode 4859363 errors 400, nbytes wrong
root 258 inode 7984606 errors 400, nbytes wrong
root 258 inode 10125125 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125126 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125127 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125128 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125129 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125130 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125131 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125132 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125133 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125134 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125135 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125136 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125137 errors 2400, nbytes wrong, link count wrong
root 258 inode 10125138 errors 2400, nbytes wrong, link count wrong
found 4667207882 bytes used err is 1
total csum bytes: 10819840
total tree bytes: 236953600
total fs tree bytes: 205643776
total extent tree bytes: 16134144
btree space waste bytes: 70494524
file data blocks allocated: 15631282176
 referenced 10313031680
Btrfs v3.16
/ #

Did I lost my entire home directory? Is there ANY chance I get it back, or at least I can save some parts of it? Mostly I am interested of the phonebook, smses, bookmarks and notes.

Any suggestions welcome..

edit retag flag offensive close delete

Comments

Factory reset and then copy back your data from your backups.

olf ( 2018-06-07 12:50:22 +0200 )edit

I made a dd image from the partition once upon a time, mounted it on a moderer linux distro with newer btrfs, and fixed it (just enough) there.

Direc ( 2018-06-13 00:17:50 +0200 )edit

1

Direc your other post (Dec 2017) on tjc was huge source of inspiration, thanks ;) . But unfortunately didn't work for me. As the #btrfs guys told me:

This may get tricky. There was a feature introduced some time ago to improve the performance of receive. That's the UUID tree. If a more recent kernel sees a filesystem without one, it'll try to build one. However, if you've hit ENOSPC, it can't create a UUID tree, and you can't do anything to fix the ENOSPC because you can't mount the FS... I'm also not 100% certain what happens when you try to mount an FS with a UUID tree on an older kernel that doesn't know abou tit.

The next step would be to try to use older/ different linux operating systems, hopefully didn't need to.

nas ( 2018-06-19 11:13:44 +0200 )edit

Factory reset and then copy back your data from your backups.
olf ( 2018-06-07 12:50:22 +0200 )edit
I made a dd image from the partition once upon a time, mounted it on a moderer linux distro with newer btrfs, and fixed it (just enough) there.
Direc ( 2018-06-13 00:17:50 +0200 )edit
Direc your other post (Dec 2017) on tjc was huge source of inspiration, thanks ;) . But unfortunately didn't work for me. As the #btrfs guys told me:
This may get tricky. There was a feature introduced some time ago to improve the performance of receive. That's the UUID tree. If a more recent kernel sees a filesystem without one, it'll try to build one. However, if you've hit ENOSPC, it can't create a UUID tree, and you can't do anything to fix the ENOSPC because you can't mount the FS... I'm also not 100% certain what happens when you try to mount an FS with a UUID tree on an older kernel that doesn't know abou tit.
The next step would be to try to use older/ different linux operating systems, hopefully didn't need to.
nas ( 2018-06-19 11:13:44 +0200 )edit

Accepted Answer

3

answered 2018-06-12 20:50:28 +0200

nas
491 ●13 ●26 ●32

Well, after 5 days I managed to make my phone work again! It was quite tricky, and since none of the methods I read worked for me as there were, I write here my own experience, hoping it will help someone else.

As I said, my phone became unresponsive. From what I saw in Rescue I managed to mount the / , but I couldn't see /home. When I tried to make a directory so I can mount /home, it showed an error about no space left on device.

So I did:

mkdir myjolla
mount /dev/mmcblk0p28 /myjolla/
mount /proc/ /myjolla/proc/
mount /dev/ /myjolla/dev/

chroot /myjolla/
mount /dev/mmcblk1p1 /media/sdcard/UUID

from there I managed to mount home!!! I copied my data on the SD.

Any try to make a balance failed. Also, I have to say, btrfs-balancer balance, couldn't find battery status and it couldn't start the balance!

I "dd" the filesystem to the SD, I tried to balance it on my pc, which didn't work, and as guys from #btrfs@freenode told, the errors I got had to do with different ways kernels of my mobile and my pc were trying to handle btrfs trees.

So I tried to add a file 500 Mb I created on my SD as loop device using losetup, and didn't work.

Next thing I did was to format an entire SD as btrfs, and then I managed to add it on the btrfs in rescue shell !!!

btrfs device add /dev/mmcblk1p1 /myjolla/

I started balance, and I had a strange surprise: On Jolla phone, it appeared a message like " Entering rescue file system, please do not power of the phone " and SailfishOS logo in the center of the screen.

After about 1 min, phone rebooted itself, and it started normaly !!!

It was very slow, and I saw it connected to wifi, so I ssh it and saw in DF that the phone memory was 22 Gb!!! ~14 of its own, plus 8 Gb from SD.

I deleted some more data from /home/nemo dir, and then I tried to delete the device from btrfs. It didn't work since it appeared that BTRFS saw it as Raid0 AND Raid1:

bash-3.2# btrfs filesystem df /
Data, RAID0: total=11.70GiB, used=6.92GiB
Data, single: total=1.00GiB, used=999.38MiB
System, RAID1: total=64.00MiB, used=4.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, RAID1: total=512.00MiB, used=220.09MiB

So I made a balance that way:

 btrfs balance start -f -sconvert=single -mconvert=single -dconvert=single /myjolla/

and after 3 - 4 mins I had again the " Entering rescue file system .." thing!

I closed the phone, I entered Rescue menu-> Shell, and without chroot this time, I made the balance. I waited only 1 - 2 hours, and then I did deleted the device:

 btrfs device delete /dev/mmcblk1p1 /myjolla

And it gone! btrfs filesystem show didn't show me the second device (the SD) but showed me an error message like it was missing. I made one balance more, and btrfs filesystem show was as it should. I restarted the phone, and everything went like all this never happened.

I noticed two things: a) Date/Time was set at the time my mobile phone stoped responding 5 days ago. But last week the time was becoming wrong, and I am not sure yet why/ didn't have time to check it. b) tracker took a couple of hours running my phone's cpu at 100%. Now everything seems fine.

Anyone wants to ask anything, please do! I wrote this as I remembered, and since I did a lot of things try and error, I didn't logged exactly the whole thing.

Thanks to nephros for the reference links, and the idea to contact directly the btrfs guys, even I did it on irc and not in mailing list ;) And a big thanks to #btrfs@freenode channel :D