Home Directory Snapshots

25 Oct 2009 16:12

Your home directory is where the most important data is stored.

But from time to time you just simply rm -Rf ~ and your all precious data is totally out of luck. Backups you say, don't you?

So let's think how do you do backups. cp /home/quake /my/distant/location ? On my 24 GB home directory? It would take hours. cp /home/quake /home/backups/quake/date ? Better, this will take a few minutes, but wait, I have like 120 GB of disk space, which means I can have no more than 5 backups.

What to do with this? There are two posibilities. Either you minimize data to backup to only backup important data (but figuring out what data is important may take some time and be inappropriate) or move to smarter solution, like incremental backups. Or snapshots.

Having brtfs as the filesystem for my home directory, I chose to make a snapshot of it each hour. It takes between 0 and 1 second to complete and uses almost no disk space. Why? Btrfs is copy-on-write filesystems, which means cloning a filesystems is instant as it only makes it available under two locations. And then modifying one of the two makes a real copy of the modified fragment of it and changes that copy.

OK. How to do it.

First create a btrfs filesystem (you'll need a recent kernel and btrfs-utils):

# mkfs.btrfs /dev/sda7

(sda7 is partition for my /home directory)

Then mount it somewhere else than /home, let's use /vol as an example:

# mount /dev/sda7 /vol

Create some volumes on that filesystem: home, quake, snapshots:

# btrfsctl -S home /vol
# btrfsctl -S quake /vol
# btrfsctl -S snapshots /vol

The volumes are accessible as the subdirectories of /vol:

# ls -la /vol
drwx------  1 root  root     36 1970-01-01 01:00 .
drwxr-xr-x 24 root  root   4096 2009-10-25 15:04 ..
drwx------  1 root  root     20 2009-10-25 15:51 home
drwx------  1 root  root  11488 2009-10-25 16:17 quake
drwx------  1 root  root     76 2009-10-25 15:40 snapshots

But you can mount then separately:

# mount /dev/sda7 /home -o subvolume=home
# mkdir /home/quake
# mount /dev/sda7 /home/quake -o subvolume=quake

Fix permissions:

# chown quake:quake /home/quake /vol/quake /vol/snapshots
# chmod 0755 /home/ /home/quake

Now you're ready to do snapshots. Now populate the /home/quake directory:

$ mkdir /home/quake/abcd
$ mkdir /home/quake/dddd
$ mkdir /home/quake/abcd/eeee
$ echo testtest > /home/quake/testfile

Aaaaaand, make snapshots!

$ btrfs -s /vol/snapshots/quake-`date +%Y%m%d-%H%M` /vol/quake

I figured out, that it's quite important to point to /vol/quake and not /home/quake. At first it seams that it's totally the same, but on /home/quake there can be some other filesystems mounted (like .gvfs for GNOME virtual file systems) and /vol/quake contains "pure data". When doing snapshots of /home/quake with filesystems mounted under it, the filesystems freezes for me (btrfs is still experimental, they say). So as noted above, it's better to snapshot pure data directory.

Now, the /vol/snapshots/quake-20091025-1653 (or whatever your date is) and /vol/quake should list the same files and the operation of "cloning" should be just instant no matter how much data you have. But now modifying the contents of /vol/quake should not change anything in /vol/snapshots/quake-20091025-1653 (but of course should in /home/quake).

Also the snapshot doesn't really take any disk space as long as you keep the /vol/quake directory unchanged. Once you change some file from /vol/quake, it needs to really keep two copies of it, so this is when additional space is allocated.

To sum up let's have a table listing possibilities to have the same contents in two directories:

More notes on btrfs and snapshotting:

• What can be done with btrfs-bcp on directories level can be done with snapshots on volumes level (as described in this post)
• Snapshots in btrfs are not removable yet. You can clear them and reclaim the space taken by saved difference from the starting point. Still a few bytes is taken by having the directory that is not removable. Deleting snapshots is to be implemented in stable version of btrfs.

Comments: 1

Ubuntu And Intel AGN Wireless Card

19 May 2009 15:20

I finally managed to get a version of kernel/modules that work nice on my wireless card:

06:00.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN [Kedron] Network Connection (rev 61)

I had problems since late Hardy or Intrepid, using different kernel from that time with no luck. The worst case was connecting to WPA2 Enterprise (user/password) secured network. In worst subcase, I had the connection for 5-20 minutes and after that the network card or driver hanged and then only computer reboot used to help.

I was used to see iwlist scan errors like "Resource temporarily unavailable" or "Busy". dmesg showed different things on different kernels.

My frustration was great, when the problem started (after some upgrades that meant to improve things) to appear even on unsecured network. The connection was broken and I had to notoriously reconnect with my NetworkManager.

Finally I found a version of kernel that plays well.

It's from jaunty-proposed repository (enable it in your Synaptic or other package manager). Don't use jaunty-backports (this one was only broken). The package that solves things is: linux-backports-modules-jaunty version 2.6.28.12.16. The install should also update the linux-image to 2.6.18-12 (update your grub.conf as usual to include the changes).

After rebooting, run uname, to verify the kernel version:

# uname -a
Linux vaio 2.6.28-12-generic #43-Ubuntu SMP Fri May 1 19:31:32 UTC 2009 x86_64 GNU/Linux

This is crucial here: 2.6.28-12-generic. The original Ubuntu kernel is 2.6.28-11 not -12.

Hope this helps someone (with VAIO SZ or any other Intel AGN-enabled notebook with Ubuntu).

Comments: 0

Snapshots Of Filesystems Under Linux

07 Mar 2008 21:47

Today I've learnt about making snapshots of regular filesystems in Linux.

First of all, this is a link to the article I've found and seems to be quite OK: http://howtoforge.com/linux_lvm_snapshots_p2

The idea is simple. We have to:

1. Have LVM partition
2. Set up some (I believe this is not limited to standard ext3) partitions on it
3. Prepare some place for backups (over network or on a separate disk)
4. Then at any time, we can just create a snapshot - this does not really consume MUCH resources — but consumes SOME.
5. This takes not more than 1 second and creates a device/file/something that is an image of the filesystem in the exact moment of creating the snapshot.
6. Having the image (snapshot) we can do anything with it (like with a block device) — mount (and backup the files), create a raw-copy, export to another machine, clone, whatever.

Notes

• The author believes one can safely restore a backup without even restarting services.
  • Will we need to rely on this? 60 seconds of down-time is acceptable and guaranties that nothing bad happens.

Comments: 1