answered
2019-09-30 20:47:37 +0300
First, a a shameless plug for ny own
howto to switch metadata from DUP to Single.
Now the details: Btrfs is closer to other modern filesystems like ZFS or BcacheFS - they aren't pure filesystems, they are also block device managers (like LVM and Mdadm).
Btrfs works by allocating chunks for either metadata or data.
It might needs to allocated e.g. a new chunk to write metadata, but there isn't unallocated space left on the device anymore. So even if there is still space left in the current data chunks (== what df will show), you get out of space errors (enosp) because it's impossible to allocate a much needed new metadata chunk.
This is made worse by the fact that btrfs never overwrites data in-place. It's a CoW (copy on write) system, it will always append first a new copy of the change, and only then it will remove the old copy if not needed anymore (not snapshotted). This makes btrfs extremely resilient to some type of problems, it makes snapshots "basically for free", and fsck is redundant (pro-tip, only run fsck on a CoW system as a last resort, usually you can recover the data without). BUT it means that btrfs is constantly writing new stuff and thus might often need a new chunk for something.
This is made worse that on Sailfish the default behaviour is to allocate metadata twice. That would be useful on a HDD (if one copy is bust, you can read from the other), but is hopeless on flash (the flash controler will group writes together, and thus both copies will go bust together). It's even worse as on Jolla 1, once you factor the gazillion of extra partitions that you need for the Android drivers, you're left with a ridiculously small amount of flash: allocating twice the chunks is terribky wasteful.
that's why rebalancing (euther manually with dusage or with jolla's tool) between upgrades is important: it avoids the situation where e.g. you have tons of half empty data chunk and no space allocatable for a new metadata chunk.
