btrfs allocation reached 100%. Balance does not seem to end and I can not remove files, what do you suggest me? [answered]

asked 2016-11-05 00:43:19 +0200

antoniovr gravatar image

updated 2016-11-06 02:56:46 +0200

[Solved]

My phone reached 100% of allocation (I have around 11G used, 76%). Yesterday evening I run btrfs balance start -d=5 /, and it is still running. I have tried to cancel it from another term with btrfs balance cancel / around 4h ago, but it has not stopped it nor cancel it (it is also still running).

At the same time, as I guess is so slow because it has no space for moving data while "sorting" the data I have tried to make some space (I guess I should have done it first...). But now I can not remove any file it tells "No space left on device". I have tried to replace file contents before removing them as suggested in https://together.jolla.com/question/30822/root-and-home-disks-full-and-causing-various-problems/ but as it can be seen:

echo > video.mkv -bash: echo: write error: No space left on device

it also fails.

I have checked the status of the balance and it's running: btrfs balance status / Balance on '/' is running, cancel requested 2 out of about 5 chunks balanced (18 considered), 60% left

I don't know what to do, at the moment I am able to make or receive calls, but I can not check mail, receive sms or open nothing related with android (due to the lack of space), I am afraid to "kill the balance" but it seems it won't reach an end and I am afraid I can run out of battery at any time. What should I do?

Here, I paste the last messages in dmesg:

[91271.392481] ------------[ cut here ]------------
[91271.392481] WARNING: at fs/btrfs/extent-tree.c:6274 btrfs_alloc_free_block+0xd8/0x35c()
[91271.392511] Modules linked in: wlan(PO) cfg80211
[91271.392542] [<c010b71c>] (unwind_backtrace+0x0/0x118) from [<c017365c>] (warn_slowpath_common+0x4c/0x64)
[91271.392572] [<c017365c>] (warn_slowpath_common+0x4c/0x64) from [<c0173704>] (warn_slowpath_null+0x18/0x20)
[91271.392603] [<c0173704>] (warn_slowpath_null+0x18/0x20) from [<c03398dc>] (btrfs_alloc_free_block+0xd8/0x35c)
[91271.392603] [<c03398dc>] (btrfs_alloc_free_block+0xd8/0x35c) from [<c03280f4>] (__btrfs_cow_block+0x120/0x508)
[91271.392633] [<c03280f4>] (__btrfs_cow_block+0x120/0x508) from [<c0328658>] (btrfs_cow_block+0x17c/0x230)
[91271.392664] [<c0328658>] (btrfs_cow_block+0x17c/0x230) from [<c038c0b0>] (do_relocation+0x2ac/0x480)
[91271.392694] [<c038c0b0>] (do_relocation+0x2ac/0x480) from [<c038e034>] (relocate_tree_blocks+0x334/0x500)
[91271.392725] [<c038e034>] (relocate_tree_blocks+0x334/0x500) from [<c038eecc>] (relocate_block_group+0x2b8/0x5a0)
[91271.392755] [<c038eecc>] (relocate_block_group+0x2b8/0x5a0) from [<c038f2fc>] (btrfs_relocate_block_group+0x148/0x2a4)
[91271.392755] [<c038f2fc>] (btrfs_relocate_block_group+0x148/0x2a4) from [<c036eba4>] (btrfs_relocate_chunk.isra.12+0x3c/0x688)
[91271.392786] [<c036eba4>] (btrfs_relocate_chunk.isra.12+0x3c/0x688) from [<c0372aa8>] (btrfs_balance+0xbbc/0xd74)
[91271.392817] [<c0372aa8>] (btrfs_balance+0xbbc/0xd74) from [<c0372cac>] (balance_kthread+0x4c/0x6c)
[91271.392847] [<c0372cac>] (balance_kthread+0x4c/0x6c) from [<c0191ecc>] (kthread+0x84/0x90)
[91271.392878] [<c0191ecc>] (kthread+0x84/0x90) from [<c0106758>] (kernel_thread_exit+0x0/0x8)
[91271.392878] ---[ end trace da227214a8249def ]---
[91568.986285] [BAT]## SOC= 94( 94, 94),4157(4178),A=0174,T=305,On=0,0,(0),Unknown 000(100),vd=4370(0,0),ib=1000,0,F03,R08,W00
[91688.454986] [BAT] 93( 93, 93.9, 91.6( 91.6) 0.0)v4180(4212)( 145, 153)t304,c -69658, 10(17650)r218,L4350,s10
[91688.457489] [BAT]## SOC= 93( 93, 93),4176(4180),A=0118,T=304,On=0,0,(0),Unknown 000(100),vd=4370(0,0),ib=1000,0,F03,R08,W10
[91728.178625] btrfs: block rsv returned -28
[91728.178625] ------------[ cut here ]------------
[91728.178686] WARNING: at fs/btrfs/extent-tree.c:6274 btrfs_alloc_free_block+0xd8/0x35c()
[91728.178686] Modules linked in: wlan(PO) cfg80211
[91728.178747] [<c010b71c>] (unwind_backtrace+0x0/0x118) from [<c017365c>] (warn_slowpath_common+0x4c/0x64)
[91728.178777] [<c017365c>] (warn_slowpath_common+0x4c/0x64) from [<c0173704>] (warn_slowpath_null+0x18/0x20)
[91728.178808] [<c0173704>] (warn_slowpath_null+0x18/0x20) from [<c03398dc>] (btrfs_alloc_free_block+0xd8/0x35c)
[91728.178869] [<c03398dc>] (btrfs_alloc_free_block+0xd8/0x35c) from [<c03280f4>] (__btrfs_cow_block+0x120/0x508)
[91728.178899] [<c03280f4>] (__btrfs_cow_block+0x120/0x508) from [<c0328658>] (btrfs_cow_block+0x17c/0x230)
[91728.178930] [<c0328658>] (btrfs_cow_block+0x17c/0x230) from [<c032bce4>] (btrfs_search_slot+0x770/0x79c)
[91728.178960] [<c032bce4>] (btrfs_search_slot+0x770/0x79c) from [<c038bf38>] (do_relocation+0x134/0x480)
[91728.178991] [<c038bf38>] (do_relocation+0x134/0x480) from [<c038e034>] (relocate_tree_blocks+0x334/0x500)
[91728.179052] [<c038e034>] (relocate_tree_blocks+0x334/0x500) from [<c038eecc>] (relocate_block_group+0x2b8/0x5a0)
[91728.179082] [<c038eecc>] (relocate_block_group+0x2b8/0x5a0) from [<c038f2fc>] (btrfs_relocate_block_group+0x148/0x2a4)
[91728.179113] [<c038f2fc>] (btrfs_relocate_block_group+0x148/0x2a4) from [<c036eba4>] (btrfs_relocate_chunk.isra.12+0x3c/0x688)
[91728.179143] [<c036eba4>] (btrfs_relocate_chunk.isra.12+0x3c/0x688) from [<c0372aa8>] (btrfs_balance+0xbbc/0xd74)
[91728.179174] [<c0372aa8>] (btrfs_balance+0xbbc/0xd74) from [<c0372cac>] (balance_kthread+0x4c/0x6c)
[91728.179235] [<c0372cac>] (balance_kthread+0x4c/0x6c) from [<c0191ecc>] (kthread+0x84/0x90)
[91728.179265] [<c0191ecc>] (kthread+0x84/0x90) from [<c0106758>] (kernel_thread_exit+0x0/0x8)
[91728.179296] ---[ end trace da227214a8249df0 ]---
[91728.179784] btrfs: block rsv returned -28
[91728.179784] ------------[ cut here ]------------
[91728.179815] WARNING: at fs/btrfs/extent-tree.c:6274 btrfs_alloc_free_block+0xd8/0x35c()
[91728.179845] Modules linked in: wlan(PO) cfg80211
[91728.179876] [<c010b71c>] (unwind_backtrace+0x0/0x118) from [<c017365c>] (warn_slowpath_common+0x4c/0x64)
[91728.179937] [<c017365c>] (warn_slowpath_common+0x4c/0x64) from [<c0173704>] (warn_slowpath_null+0x18/0x20)
[91728.179967] [<c0173704>] (warn_slowpath_null+0x18/0x20) from [<c03398dc>] (btrfs_alloc_free_block+0xd8/0x35c)
[91728.179998] [<c03398dc>] (btrfs_alloc_free_block+0xd8/0x35c) from [<c03280f4>] (__btrfs_cow_block+0x120/0x508)
[91728.180028] [<c03280f4>] (__btrfs_cow_block+0x120/0x508) from [<c0328658>] (btrfs_cow_block+0x17c/0x230)
[91728.180059] [<c0328658>] (btrfs_cow_block+0x17c/0x230) from [<c038c0b0>] (do_relocation+0x2ac/0x480)
[91728.180089] [<c038c0b0>] (do_relocation+0x2ac/0x480) from [<c038e034>] (relocate_tree_blocks+0x334/0x500)
[91728.180151] [<c038e034>] (relocate_tree_blocks+0x334/0x500) from [<c038eecc>] (relocate_block_group+0x2b8/0x5a0)
[91728.180181] [<c038eecc>] (relocate_block_group+0x2b8/0x5a0) from [<c038f2fc>] (btrfs_relocate_block_group+0x148/0x2a4)
[91728.180212] [<c038f2fc>] (btrfs_relocate_block_group+0x148/0x2a4) from [<c036eba4>] (btrfs_relocate_chunk.isra.12+0x3c/0x688)
[91728.180242] [<c036eba4>] (btrfs_relocate_chunk.isra.12+0x3c/0x688) from [<c0372aa8>] (btrfs_balance+0xbbc/0xd74)
[91728.180273] [<c0372aa8>] (btrfs_balance+0xbbc/0xd74) from [<c0372cac>] (balance_kthread+0x4c/0x6c)
[91728.180334] [<c0372cac>] (balance_kthread+0x4c/0x6c) from [<c0191ecc>] (kthread+0x84/0x90)
[91728.180364] [<c0191ecc>] (kthread+0x84/0x90) from [<c0106758>] (kernel_thread_exit+0x0/0x8)
[91728.180395] ---[ end trace da227214a8249df1 ]---

P.D.: I have read this https://together.jolla.com/question/30822/root-and-home-disks-full-and-causing-various-problems/ . I write down the link here in case someone is having the same problem and has not started trying to solve it, it can read it before doing nothing.

SOLVED: So, solution is not to wait the 100% before acting, Anyway, balancing metadata first, but If somebody faces the same problem, recovery-mode could be a solution I just got into the shell by pressing 4 (https://jolla.zendesk.com/hc/en-us/articles/204709607-Miten-k%C3%A4yt%C3%A4n-Recovery-Modea- ) and then mount the home partition: mount -o skip_balance -o subvol=@home /dev/mmcblk0p28 /mnt/ Remove a lot of heavy files, umount /mnt and follow the rules to get back to normal state. The boot took me more time than usual (to scary me a little bit ehehe) and everything is working again.

edit retag flag offensive reopen delete

The question has been closed for the following reason "the question is answered, an answer was accepted" by antoniovr
close date 2016-11-06 02:55:36.828859

Comments

2

You may try to delete files in recovery mode. Cancel action on a running btrfs balance does proceed after the current block allocation only. So this might never happen for you. Be aware that it might not be that a good idea to reboot or shutdown (eg my recovery mode suggestion) either. While on a normal balance run it can just suspend balancing and restart after boot, this might not be working out here. What does echo > video.mkv do good here? If you just want to create an empty file, I do not understand why you even try to write more to the full disk?! With a filled up device in an unwriteable state you might need to forcefully do it from recovery mode - do you know your way around that?

chemist ( 2016-11-05 01:07:18 +0200 )edit

What does echo > video.mkv do good here?

It was an attempt to empty the file for removing it as rm is not working.

About recovery mode I have read this https://jolla.zendesk.com/hc/en-us/articles/204709607-Miten-k%C3%A4yt%C3%A4n-Recovery-Modea- I guess I can manage... even though shutting down scares me a bit. I am thinking to wait until tomorrow night (and I will try to carry the charger with me during the day :)) to see if it gets cancel, and otherwise I try anyway the recovery mode... what do you think? I guess the risk of running out of battery in middle of the process should be equally risky to shutting down, shouldn’t it?

antoniovr ( 2016-11-05 01:15:06 +0200 )edit
2

No, running out of battery during balancing will probably shred the FS. Either it is able to hibernate the balance on shutdown or it doesn't. I'd go for a regular shutdown, then recovery-mode. Mount the internal storages. If that does not go RO already you may be able to delete some files. And just balance metadata at first - always!

chemist ( 2016-11-05 13:20:14 +0200 )edit

Thanks!

As it didn't change in the following 24h I decided to try to get into the recovery-mode.

I am trying to mount it (in order to remove some files), but it seems the mount tool has started balancing before mounting (at least that seems when looking into top information and also because of the time)

267     2 0        DW       0  0.0   0  1.0 [btrfs-balance]
  261     2 0        DW       0  0.0   1  1.0 [btrfs-transacti]
  110     2 0        DW       0  0.0   0  0.6 [mmcqd/0]
  264     2 0        SW       0  0.0   1  0.5 [btrfs-worker-3]
  249     2 0        SW       0  0.0   0  0.3 [btrfs-submit-1]
  518   517 0        R     1728  0.2   1  0.0 top
   15     2 0        SW       0  0.0   0  0.0 [kworker/0:1]
    3     2 0        SW       0  0.0   0  0.0 [ksoftirqd/0]
  496     2 0        SW       0  0.0   0  0.0 [kworker/u:1]
  135     1 0        S     1728  0.2   0  0.0 telnetd -l/usr/bin/recovery-menu
  235   230 0        S     1728  0.2   0  0.0 sh
  326   320 0        S     1728  0.2   1  0.0 sh
  337   326 0        D     1728  0.2   0  0.0 mount -o subvol=@home /dev/mmcblk0p28 /mntdisco/

[...]

Halt is also not working so I guess I can only wait or remove the battery and try to factory reset (getting somehow an sd card first eje).

antoniovr ( 2016-11-06 00:51:44 +0200 )edit

now I've seen I should have used "skip_balance" option while mounting :(

antoniovr ( 2016-11-06 01:53:00 +0200 )edit