Ask / Submit
18

HowTo: Synchronize backups with remote servers using git

asked 2015-01-29 12:20:15 +0200

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

updated 2015-04-30 21:51:28 +0200

rdmo gravatar image

NOTE: Vault has an extra directory in its .git directory that must be synchronized separately. See Dez's answer for details.

Sailfish backups are stored in a git repository in /home/nemo/.vault. Git is designed to make it easy to keep a local data store syncronized with another server, and this is possible in Sailfish.

On your ssh enabled server with git installed, initialize a bare git repository like this:

git init --bare <path to your repository>

Add your server as a remote like this:

cd /home/nemo/.vault
git remote add <remote name> <your remote git url>

e.g.

git init --bare /var/git/jolla-backup.git (on server)
git remote add origin ssh://me@homeserver/var/git/jolla-backup.git (on phone)

then push the master branch to your remote from the phone:

git  push --set-upstream origin master

which will do the initial push then when you create a new backup you can copy that over:

git push origin

Git manages changes rather than the files themselves, so this is very efficient.

If your phone gets wiped/lost, then you should be able to restore your backup vault from the remote using git clone, though I haven't tested this:

git clone <your remote git url> /home/nemo/.vault

My question is, is restoring from a cloned remote vault supposed to work, and is this likely to be a supported way of keeping your phone backups safe in the long term? Pushing git commits is very efficient and versatile for those who can host their own serve, though it's not suitable at all for public services such as github unless you want all your private data copied to the US and possibly published on the web. This could easily get wrapped up in a small app or added to the backup UI with options such as auto-pushing new backups as well as cloning/merging/rebasing remote backups.

edit retag flag offensive close delete

Comments

Beautiful idea!

I already back up /home/nemo via nightly rsync, but using git is another neat thing. I suppose it could end up being more efficient since rsync treats the repository as black-box binary.

juiceme ( 2015-01-29 13:05:54 +0200 )edit
1

I think the beautiful idea is serializing user data to text files and versioning them in git! The implementation is a little odd though. There are symlinks directly into the .git repo...

Andy Branson ( 2015-01-29 16:45:33 +0200 )edit

@rdmo stop messing up the code segments

kimmoli ( 2015-04-30 21:09:23 +0200 )edit

@kimmoli: Sorry. Manually reverted. Should the '(on phone)' parts be '# (on phone)' ?

rdmo ( 2015-04-30 21:55:05 +0200 )edit

1 Answer

Sort by » oldest newest most voted
4

answered 2015-02-19 15:28:42 +0200

dez gravatar image

updated 2015-05-20 09:29:11 +0200

You can't just clone it: some binaries are stored inside .git in the blobs directory because git can't handle large binaries efficiently. So, you should add rsync step to synchronize .git/blobs subdir.

Also, I am preparing simple script (to be released soon, maybe it will even reside in each .vault storage) to finish .vault cloning in the right way. Also some kind of README file will appear in each storage.

edit flag offensive delete publish link more

Comments

1

That's a shame. Thanks for the info.

Andy Branson ( 2015-02-19 15:40:25 +0200 )edit
1

@andy-branson git is handy to handle metadata and files with intermediate size (especially if it is possible to export user data into known format), so one more step needed for binaries (like videos/pictures etc.) does not overweight git advantages. .vault should just contain README telling about it, or even better - it can just contain (shell) script to finish full cloning.

dez ( 2015-02-19 16:37:34 +0200 )edit

Does this blob directory have any relevance to vanilla git? Do the filenames refer to commits? Is it just the gallery that will break if it's not present, or is it any binary files over a certain size produced by a backup source?

Andy Branson ( 2015-02-19 20:44:35 +0200 )edit
1

The basic rule to choose files to be stored in this directory is: if file is really a blob (binary large object, sometimes with some specific and non-portable content) it is stored there, all structured data is stored in the git storage. Files in this blobs storage has sha-1-based name as files stored in the git object storage, while these "blob" files are stored unmodified: e.g. no header is added to the file. These files are referenced from the git tree by symlinks. See https://github.com/nemomobile/vault

dez ( 2015-02-19 23:20:19 +0200 )edit

It does look like supporting standard git is planned though, it's mentioned at the bottom of that readme. Thanks again for the additional info.

Andy Branson ( 2015-02-20 12:27:59 +0200 )edit
Login/Signup to Answer

Question tools

Follow
14 followers

Stats

Asked: 2015-01-29 12:20:15 +0200

Seen: 1,426 times

Last updated: May 20 '15