Last call for help: Restore btrfs or bricked?
asked 2017-05-15 23:04:32 +0300
Hello community, I already contacted the care and support, but they need my Jolla 1 with an insecure outcome which means: not an option.
History of what happened:
Undefined behavioral on 2.1.0.11, sth. like file system full, but wasn't the case, I always use my SD card. You'll see that below:
- After reboot apps didn't start anymore,
- 3 times hooray for my backups, tried factory reset over preference app, got stuck.
- Saw, read and applied the recovery mode article
- Boot loop
Recovery mode console: Mount, analyse and repair:
/ # mount -t btrfs -o skip_balance /dev/mmcblk0p28 /mnt/ / # btrfs fi show ERROR: unable get label Inappropriate ioctl for device Label: 'sailfish' uuid: … Total devices 1 FS bytes used 2.73GiB devid 1 size 13.75GiB used 5.75GiB path /dev/mmcblk0p28 Btrfs v3.16 ----- Reboot ---- / # btrfs rescue chunk-recover /dev/mmcblk0p28 Check chunks successfully with no orphans Recover the chunk tree successfully. / # btrfs check --repair /dev/mmcblk0p28 enabling repair mode Checking filesystem on /dev/mmcblk0p28 UUID: … checking extents bad item end 3999 wanted 3995 […] Incorrect local backref count on 4032372736 root 269 owner 4957 offset 655360 found 0 wanted 1 back 0x2229818 Backref disk bytenr does not match extent record, bytenr=4032372736, ref bytenr=0 backpointer mismatch on [4032372736 528384] owner ref check failed [4032372736 528384] ref mismatch on [4032921600 720896] extent item 1, found 0 attempting to repair backref discrepency for bytenr 4032921600 Failed to find [158896128, 168, 4096] btrfs unable to find ref byte nr 158896128 parent 0 root 269 owner 2 offset 0 repaired damaged extent references Failed to find [31232000, 168, 4096] btrfs unable to find ref byte nr 31232000 parent 0 root 1 owner 1 offset 0 btrfs: extent-tree.c:1752: write_one_cache_group: Assertion `!(ret)' failed. Aborted / # btrfs fi show Label: 'sailfish' uuid: … Total devices 1 FS bytes used 2.73GiB devid 1 size 13.75GiB used 5.75GiB path /dev/mmcblk0p28 Btrfs v3.16Nothing changed, first support mail. Their solution: an over-full file system --> balancing. I was wondering, because my home is empty and the file system only uses 5GB, but why not:
/mnt/rootfs # ls -la … drwxr-xr-x 1 0 0 2164 Aug 19 2016 etc -rw-r----- 1 0 2000 980 Oct 2 2015 fstab.qcom drwxr-xr-x 1 0 0 0 Aug 19 2016 home …
fresh restart and mount everything:
----------------------------- Jolla Recovery v0.3.1 ----------------------------- Welcome to the recovery tool! The available options are: 1) Reset device to factory state 2) Reboot device 3) Bootloader unlock [Current state: locked] 4) Shell 5) Try btrfs recovery if your device is in bootloop 6) Exit Type the number of the desired action and press [Enter]: 4 If you continue, this may void your warranty. Are you really SURE? [y/N] y [OK] No lockcode has been set. Proceeding... / # mount /dev/mmcblk0p28 /mnt / # mount rootfs on / type rootfs (rw) none on /dev type devtmpfs (rw,relatime,size=412500k,nr_inodes=103125,mode=755) proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) none on /tmp type tmpfs (rw,relatime,size=10240k) /proc/sys/debug on /proc/sys/debug type debugfs (rw,relatime) none on /var/run type tmpfs (rw,nosuid,noatime,size=256k,mode=755) devpts on /dev/pts type devpts (rw,relatime,mode=600) /dev/mmcblk0p28 on /mnt type btrfs (rw,relatime,ssd,noacl,space_cache) / # btrfs balance start -v -dusage=50 /mnt Dumping filters: flags 0x1, state 0x0, force is off DATA (flags 0x2): balancing, usage=50 ERROR: error during balancing '/mnt' - Read-only file system There may be more info in syslog - try dmesg | tail / # dmesg | tail [ 29.293087] [<c038f13c>] (relocate_block_group+0x528/0x5a0) from [<c038f2fc>] (btrfs_relocate_block_group+0x148/0x2a4) [ 29.293117] [<c038f2fc>] (btrfs_relocate_block_group+0x148/0x2a4) from [<c036eba4>] (btrfs_relocate_chunk.isra.12+0x3c/0x688) [ 29.293148] [<c036eba4>] (btrfs_relocate_chunk.isra.12+0x3c/0x688) from [<c0372aa8>] (btrfs_balance+0xbbc/0xd74) [ 29.293148] [<c0372aa8>] (btrfs_balance+0xbbc/0xd74) from [<c0372cac>] (balance_kthread+0x4c/0x6c) [ 29.293178] [<c0372cac>] (balance_kthread+0x4c/0x6c) from [<c0191ecc>] (kthread+0x84/0x90) [ 29.293209] [<c0191ecc>] (kthread+0x84/0x90) from [<c0106758>] (kernel_thread_exit+0x0/0x8) [ 29.293239] Code: e3a03001 ebfeabe0 e3500000 aa000000 (e7f001f2) [ 29.293270] ---[ end trace da227214a82491ba ]--- [ 29.293270] kevent_trigger: queue event work event id is 0 [ 29.293331] kevent_work: send kobject uevent, EVENT=Oops / # mount rootfs on / type rootfs (rw) none on /dev type devtmpfs (rw,relatime,size=412500k,nr_inodes=103125,mode=755) proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) none on /tmp type tmpfs (rw,relatime,size=10240k) /proc/sys/debug on /proc/sys/debug type debugfs (rw,relatime) none on /var/run type tmpfs (rw,nosuid,noatime,size=256k,mode=755) devpts on /dev/pts type devpts (rw,relatime,mode=600) /dev/mmcblk0p28 on /mnt type btrfs (ro,relatime,ssd,noacl,space_cache) / # btrfs fi show ERROR: unable get label Inappropriate ioctl for device Label: 'sailfish' uuid: … Total devices 1 FS bytes used 2.73GiB devid 1 size 13.75GiB used 5.75GiB path /dev/mmcblk0p28 Btrfs v3.16 / #The file system turns suddenly into a read only system
Found this post: https://together.jolla.com/question/160081/phone-bricked-during-last-update/?answer=160350#post-id-160350 used dd in this readme' http://www.htheb.com/list/jolla/factoryimage/Readme.txt
- Boot loop [end of ideas]
TL;DR: Brick?
Can you enable persistent logging while in recovery mode and post the result of journalctl here?
louisbob ( 2017-05-16 00:04:10 +0300 )editBtrfs is odd, but don't you do a
chemist ( 2017-05-16 15:37:58 +0300 )editcheckon an unmounted volume?When I've had btrfs issues in the past (non-Jolla) I've had success using newer versions of the btrfs tools (eg, you might be able to get a newer version off OBS, or from a different distro and then copy them on?) -- or maybe try exposing the partition via the telnet interface (eg, I'm pretty sure I've used sshfs (to the device) when in rescue mode) to allow interacting with the partition directly (eg to allow dd-ing it to make an image to hack at more aggressively)
bruce_one ( 2017-05-17 12:32:15 +0300 )edit@luisbob: which recovery do you mean? btrfs balance, btrfs rescue chunk-recover, btrfs rescue super-recover, factory reset or the dd?
@chemist: Nope, I did a reboot in between
@bruce_one: can you give me hint which version you used? I don't want to run into unresolved dependencies or sth.
Ah and sorry for my delay, you know, work and stuff to do, but i appreciate your help, thanks a lot!
NGC-Ollie ( 2017-05-21 16:44:37 +0300 )edit1.) Go into recovery mode: https://jolla.zendesk.com/hc/en-us/articles/204709607 (watch out, for jolla C you need to use the UP volume key, not the down one)
2.) In the menu, select Shell
3.) When the shell prompt appears, and edit the following file: /rootfs/etc/systemd/journald.conf
4.) Change the line "Storage=volatile" to "Storage=persistent"
5.) Reboot, wait a bit so your log will be populated with error hopefully.
6.) Go back to the recovery mode -> Shell, and use journalctl command to display the log. Note that you will probably need to do a chroot before (command is "chroot /rootfs/")
Resources: https://together.jolla.com/question/4842/how-to-enable-more-detailed-and-persistent-logs-on-jolla-device/
louisbob ( 2017-05-21 21:27:28 +0300 )edit