We have moved to a new Sailfish OS Forum. Please start new discussions there.
27

complete flashable image of the os as backup

asked 2018-01-01 15:58:33 +0200

toba gravatar image

updated 2018-01-06 15:35:16 +0200

ralooyar gravatar image

hello everybody,

i'm really happy with my xperia, but i would love to have a feature i had on the n900 before: backing up my whole system how it is to use it as a imagefile i could flash my new device from. was called backupmenu or similar and worked flawlessly...

is ther a way to do this? a noobproove tutorial?

anyone who would like to have this?

sincerly

toba

edit retag flag offensive close delete

Comments

12

dd last mmcblk0pX

coderus ( 2018-01-02 01:41:53 +0200 )edit

would love to have that, too.

Moo-Crumpus ( 2018-01-02 10:53:47 +0200 )edit
5

i'm really honored that the mighty coderus himself is replying to my question, but as i said: im too noobie to execute this wihitout further more precise and noobproof instruction.

please enlight me, coderus. спасибо

toba ( 2018-01-02 22:58:24 +0200 )edit

3 Answers

Sort by » oldest newest most voted
36

answered 2018-01-11 00:27:53 +0200

ralooyar gravatar image

updated 2018-07-08 17:12:03 +0200

Table of Contents:

This answer contains of the following chapters:

  • Disclaimer
  • HowTo backup
  • HowTo restore the backup
  • Backup recovery test
  • Inspecting the backup file
  • About SailfishX partition layout

Disclaimer

(!) These commands worked for my SailfishX device F5121 with SailfishOS 2.1.3.7. They are not guaranteed to work for all SailfishX devices! (!)

(!) Please be carefull, using these commands! I guess, you can do heavy damage to your SailfishX device or your data if something goes wrong. And I can not guarantee that these commands are correct in all use cases. Use this information at own risk. (!)

HowTo backup

Step 1: Insert SD card

Insert an SD card into your SailfishX device, large enough to hold your backup data. In my case, my "system data" partition is filled with 1.5 GB of data and my "user data" is filled with 3.8 GB of data. In my test scenario, I ended up with a compressed backup image of 6.0 GB.

In these instructions, I use a 16 GB SD card with one single partition, formatted with a ext4 filesystem.

(!) Don't forget to format your SD card with a filesystem that is capable of holding single files large than 4 GB. Using FAT32 as filesystem will let the dd ... | gzip ... process fail with "gzip: short write". (!)

Step 2: Enter recovery mode according to my other HowTo at Answer 177584 on Question 169302 "XperiaX recovery mode"

Step 3: Mount SD card

In the recovery shell type:

mkdir /mnt/sdcard
mount /dev/mmcblk1p1 /mnt/sdcard/

mmcblk1p1 means "first partition on the SD card. mmcblk1p2 would be second partition....

OPTIONAL: Typing ls -lh /mnt/sdcard/ should now display the contents of your SD card.

OPTIONAL: You can check with df -h if there is enough space free on your mounted SD card partition.

Step 4: Backup your partition to SD card

Type the command:

dd if=/dev/mmcblk0p51 | gzip > /mnt/sdcard/backup.img.gz

I personally don't use backup.img.gz as filename, but sailfishx-backup-mmcblk0p51-yyyy-mm-dd.img.gz and replace yyyy-mm-dd with the current date to document the date the backup has been created.

This command will read every single bit from partition mmcblk0p51 (see later for information about partitions), pipe those bits to the compression program gzip and finally write those compressed bytes to the backup file backup.img.gz on your SD card.

This process takes about 59 minutes in my case, but heavily depends on the amount of data to write and writing speed of your SD card. The output should look like:

43409408+0 records in
43409408+0 records out
22225616896 bytes (20.7GB) copied, 3518.607151 seconds, 6.0MB/s

Please check, that your number of records and bytes are identical to mine to ensure everything went fine. Time and writing speed will vary.

If you get gzip: short write, your backup failed. See section "Insert SD card" above.

Just to be sure, type the command

sync

to flush the filesystem buffer and make sure everything has been written to your SD card.

Step 5: Unmount your SD card

Type the following commands to unmount your SD card again:

cd /
umount /mnt/sdcard/

Step 6: Exiting recovery mode

Now you can exit recovery mode, as described in Answer 177584 on Question 169302 "XperiaX recovery mode"

Step 7: Remove your SD card

As last step, I would recommend to shut down the SailfishX device once more and remove the backup SD card. Otherwise this backup won't help in cases like losing your SailfishX device. ;)

Then you are done! Congratulations! :)

HowTo restore the backup

Step 1, 2 and 3: Preparations

Insert your SD card with your backup file in your SailfishX device and do steps 2 and 3 from chapter "HowTo backup".

Step 4: Restore from SD card

In step 4 instead of dd ... | gzip ... do this:

gzip -d -c backup.img.gz | dd of=/dev/mmcblk0p51

Replace backup.img.gz with the name of your backup file.

The device identifier /dev/mmcblk0p51 is very crutial: If you use a wrong one, this command might destroy a partition of your SailfishX device, which contains import data like firmware, drivers, etc.. You might be able to "brick" your device with this command!

This process will read your backup file backup.img.gz, decompress the bits and pipe them to dd, which will write them to the internal partition mmcblk0p51 of your SailfishX device.

The restore process should output the same number of records and bytes as the backup process (time and writing speed will vary):

43409408+0 records in
43409408+0 records out
22225616896 bytes (20.7GB) copied, 1058.962191 seconds, 20.0MB/s

Surprisingly the restore process is about 3 times faster than the backup process. (I guess compressing is the bottleneck.)

Now lets do a

sync

again, to make sure all data has been flushed to the internal storage of your SailfishX device.

Steps 5, 6 and 7: Back to normal

Proceed with steps 5, 6 and 7 as described in chapter "HowTo backup".

Backup recovery test

Since a backup method is useless, unless you have tested it, I performed the following process:

  1. create a backup according to the instructions above
  2. changed some data on my SailfishX device
  3. restore the backup (of point 1) according to the instructions above
  4. check whether the changes (of point 2) have been reverted

Data I have changed and checked:

  • modify address book
  • delete existing and add new notes
  • delete existing calendar events and add new ones
  • settings:
    • change developer password
    • turn off mobile data
    • disable display flip cover
    • change usb mode from "charge" to "always ask"
  • delete existing and add new pictures
  • jolla apps:
    • uninstall installed jolla app
    • update installed jolla app
  • uninstall installed android app
  • revert the unlock sound hack
  • jolla app settings:
    • add new alarm
    • disable "request SMS delive report"
  • android app settings:
    • change Threema color theme
    • disable Threema start at boot time
    • add new favorite POV to OsmAnd
  • delete music file A and renamed music file B

=> All changes have been perfectly reverted by restoring the backup. :)

Inspecting the backup file

If you want to know, which data is contained in your backup file (created in backup step 4), or want to extract some file directly from the backup file, you can do the following on a linux PC (I use Debian, but it surely will work with other distros, too):

Step 1: install necessary programs:

apt-get install lvm2

Step 2: unzip your backup file:

gzip -d -k backup.img.gz

Step 3: setup loop device

If loop0 is already in use on your PC, use the next free device name like loop1, loop2, .... You can list used device names with cat /proc/mounts | grep /dev/loop.

losetup /dev/loop0 backup.img

Step 4: start logical volume manager

Update: This step seems to be obsolete in Debian Stretch and does not work. The service is no longer available in Debian Stretch.

systemctl start lvm2

Step 5: check logical volume manager

Now you can list your logical volume groups with:

vgdisplay

You should see something like:

--- Volume group ---
VG Name               sailfish
System ID             
Format                lvm2
...

Step 6: mount the partitions of the backup

mkdir /mnt/sailfish
mkdir /mnt/sailfish/root
mkdir /mnt/sailfish/home

mount /dev/sailfish/home /mnt/sailfish/home/
mount /dev/sailfish/root /mnt/sailfish/root/

Now you should be able to access your data at /mnt/sailfish/home/ and /mnt/sailfish/root/.

E.g.:

ls -lh /mnt/sailfish/home/

Step 7: umount the partitions again

I you are done with inspecting and/or extracting, you can unmount the partitions again with:

umount /mnt/sailfish/home/
umount /mnt/sailfish/root/

I you like, you can remove the directories, too:

rmdir /mnt/sailfish/home
rmdir /mnt/sailfish/root
rmdir /mnt/sailfish

Step 8: shutdown logical volume manager

To be honest, I have no clue, whether if have to cleanly shutdown the logical volume manager in some way. Nor do I know, how to do it. But I guess a clean system shutdown will handle this correctly.

Hints are welcome

Thanks to:

About SailfishX partition layout

A few words about SailfishX partition layout:

fdisk -l does not work very well in the recovery mode used in the steps above. So please don't use it.

A better way to get partition information in recovery mode is:

cat /proc/partitions

This will list a partition layout which is quite identical to using the following command in SailfishX normal operation mode (use option -O to get a lot of output):

lsblk

Output of lsblk ... is something like:

NAME                KNAME       FSTYPE      SIZE    MOUNTPOINT       LABEL
...
├─mmcblk0p51        mmcblk0p51  LVM2_member 20.7G       
│ ├─sailfish-root   dm-0        ext4         2.5G   /                root
│ └─sailfish-home   dm-1        ext4        18.3G   /opt/alien/home  home
└─mmcblk0p52        mmcblk0p52  ext4         7.3G   /fimage          fimage

@DrYak wrote a few word about those partitions. Please see his comment to this answer.

edit flag offensive delete publish link more

Comments

6

NOTE:

/dev/mmcblk0p51 - is actually not a self contained filesystem, but further sub-partitionned using LVM.

your dd method should work, but you're actually packing together both LVM sub-partitions (/dev/sailfish/root and /dev/sailfish/home), so for some very specific use-cases (e.g.: want to mount the data on a Linux PC to look for a single file) it would require additional steps.

On the other hand, this packing makes restoring by doing a dd or even using plain fastboot (just like the official Sailfish X installation) just a piece of cacke.

/dev/mmcblk0p52 - you can safely ignore. That's just some data used by recovery. It's gzip image files of the 2 LVM subpartition. Doing a factory reset of the device on the device-it self from the recovery telnet menu basically overwrite the sub-partition with these. It's not necessary if you do the reset from a laptop using your dd or the installer's fastboot.

Some users actually even resize and remove part of it to give more free space to mmcblk0p51 (popular mod on F5122 dual SIM).

DrYak ( 2018-01-15 13:25:39 +0200 )edit

@DrYak Thanks a lot for your information! :)

ralooyar ( 2018-01-15 23:19:23 +0200 )edit

thank you guys. i have to buy me a 5122 and gonna try to flash my old image onto it. i'm very thankfull for the fast and detailed replies..

sincerly

toba

toba ( 2018-01-16 09:52:51 +0200 )edit
2

First class, @ralooyar@DrYak !

lakutalo ( 2018-01-17 23:19:09 +0200 )edit
1

Very nice tutorial indeed.

I myself don't usually backup whole partitions like that since it is pretty wasteful IMHO, I just rsync /home/nemo daily; that's enough for most cases.

juiceme ( 2018-07-09 08:29:43 +0200 )edit
3

answered 2018-09-18 14:38:23 +0200

Levone1 gravatar image

updated 2018-09-21 05:24:38 +0200

The other answer here is obviously well thought out and knowledgeable, and no doubt will work. I just wondered about coderus' first comment seemingly implying that a simple dd dump would suffice, and I decided to test. So my thought was, if you consider that flashing the rom initially to the phone happens through 4 fastboot commands:

  1. fastboot flash boot hybris-boot.img
  2. fastboot flash system fimage.img001
  3. fastboot flash userdata sailfish.img001
  4. fastboot flash oem (name of Sony oem file),

and 2 of those files don't change based on usage, (boot and oem), then, could you just dump your current state of the other 2 partitions, with:

  1. dd if=/dev/mmcblk0p52 of=/...(wherever you want)/system.img
  2. dd if=/dev/mmcblk0p51 of=/...(wherever you want)/userdata.img,

using the terminal on the phone, and saving files.

So, here's what I did:

  • In terminal on phone, dd if=/dev/mmcblk0p51 of=/(your choice)/userdata.imgThis was 24gb for me, (so you'll probably need a 32gb sdcard), and took about 25 minutes. dd if=/dev/mmcblk0p52 of=/(your choice)/system.img This was about 6gb for me, and took 5 minutes.
  • Copy files to computer, in the same directory you have your Sailfish image download. This could have some obstacles, with the files being so large, if you're using Windows, or something. What I did was upload to file storage from the phone, and then download to computer. What also works on Windows is to use Linux Reader app, which can read ext4 fs on Windows, (like from sd card), and export files. If you know how to copy direct to your computer in recovery via ssh or something, more power to you.
  • Do whatever you want to do with your phone until you're ready to restore your backup.
  • Flash Sailfish to your phone, just like you did at first, but using your backup files as 'system' and 'userdata'. For me, I just manual fastbooted the files, (which is what I did with the original official image, also). I know Jolla includes an auto-flash script in the download, so If you used that, you could either: 1) copy your original system and userdata files somewhere else, and then rename your new, backup files to match those names, or 2) edit the flash script to flash your new files, instead of the original ones.
    • Note - fastboot will initially give an error about mismatch sparse header, or something, but just wait, and it will flash ok. Also, flashing the 24gb userdata file happened in 38 chunks, and took about 20 minutes.

After flashing, reboot phone, and it will be just like it was before.

edit flag offensive delete publish link more

Comments

1

Ok, so in attempting to do my idea, I discovered:

  • filesystem in recovery isn't the same. There's /rootfs, which is the main filesystem, but there's nothing in /rootfs/home, (where /nemo and others are when booted). So, I used directions in ralooyar's answer to mount sd card to /mnt. So updated dd command is: dd if=/dev/mmcblk0p52 of=/mnt/sdcard/system.img, etc. So then, that part works, but...
  • as also stated by ralooyar, must make sure card is formatted properly, or else error: 'file too large... 4gb copied...' Took about 4 minutes.

Going to format card and test again.

Update - formatted sd, and dd working fine, but, system img is about 6.5gb, and userdata img is about 9.5, so ran out of space on my 16gb card, (so close...). I did the dd commands on the phone. Anyway, not sure when I'll get around to testing all the way, but seems like should work... Any input appreciated. Going to edit answer.

Levone1 ( 2018-09-18 16:24:20 +0200 )edit

If you read the accepted answer thoroughly you'll see that it actually uses dd. And that you safely can skip /dev/mmcblk0p52. Your answer is redundant.

Mohjive ( 2018-09-18 18:34:32 +0200 )edit

My suggestion is that you actually delete your answer, as it only adds confusion.

Mohjive ( 2018-09-18 18:36:43 +0200 )edit

Ok, sorry. I saw that it uses dd, but my thought was that is seems much simpler to just copy your imgs, and then fastboot it back when you want to, rather than go through all that. I'm fine with accepting that I lack understanding, but it definitely doesn't seem to be redundant - it's a totally different method. Also, I have been only putting it out there as a question, looking for input. I'm assuming it would work, but I can't say for sure at this point. Just hoping people who know would confirm/deny. I apologize if my posts came across as spammy... As far as /mmcblk0p52, are you saying that the /system files are included in mmcblk0p51? I thought that was only userdata, and you still needed to flash something to system. I saw what was written in the other answer about it, but it says you can safely ignore it, given that you either do a factory reset on the device, or fastboot the official image. In my case, my /system is modified from original, so using the official image wouldn't restore it, and factory reset from device is only applicable if you're currently using a booted device. My idea was to restore everything back to a device totally from nothing. Again, I'm just asking. I appreciate the input.

Levone1 ( 2018-09-18 18:45:52 +0200 )edit

I'm sorry, I shouldn't have expressed myself so confident about things I'm not completely sure of. All I know is what I read in the first comment of the accepted answer. It states that mmcblk0p52 just has data related to recovery and mmcblk0p51 has both home and root partitions.

The same comment also states that the images created in the answer is also usable by fastboot flashing. The only info missing is how to use fastboot to flash the device, as a complement to using recovery mode.

You should read that comment thoroughly as well.

Mohjive ( 2018-09-19 13:26:20 +0200 )edit
1

answered 2018-12-16 11:31:23 +0200

toba gravatar image

again, i have to thank a lot for jans tutorial. but i think it would be nice to have all this in a noobfriendly programm like it we had it on the n900: http://talk.maemo.org/showthread.php?t=63975

would also be a feature i think everybody would like on his os in general...

gonna try the instructions over xmas when i have time to flash an xperia xa2

edit flag offensive delete publish link more

Comments

1

would be pretty easy to make a script for my answer and put it in a one-click app, like Shellex or something. The whoke method is just 2 commands on the phone, then copy files to computer for restoring later. If anyone's a noob, I am ...

Levone1 ( 2018-12-16 20:31:59 +0200 )edit
Login/Signup to Answer

Question tools

Follow
31 followers

Stats

Asked: 2018-01-01 15:58:33 +0200

Seen: 4,328 times

Last updated: Dec 16 '18