Lustre discuss - Mar 2009 - MDT hardlink snapshots?

Hi,

I was wondering if it is possible to mount an MDT device with ldiskfs, hardlink
a dir tree and then remount (lustre) without Lustre clients getting confused?
Are hardlinks in Lustre a "bad idea" for any reason (i.e. linking
files into a different dir tree). I was toying with the idea (madcap Fridays) of
doing hardlink style file based snapshots by doing something like this:

  [root at mds ~] umount -f /dev/lustre/mdt1
  [root at mds ~] mount -t ldiskfs /dev/lustre/mdt1 /mnt/lustre/mdt1
  [root at mds ~] cp -fal /mnt/lustre/mdt1/ROOT/blah /mnt/lustre/mdt1/ROOT/blah0
  [root at mds ~] mount -t lustre /dev/lustre/mdt1 /mnt/lustre/mdt1

Something akin to "rsync -H --link-dest" backups (dirvish). A quick
benchmark suggests that you can link around 100 million files in 6 hours. It is
debatable how viable it would be to take a filesystem down for that long - maybe
a a weekend operation. I was just curious if there was anything in Lustre that
makes this impossible/bad. This would probably be impossible in Lustre 2.0 (CMD
and changelogs).

Hard linking over the network is obviously an order of magnitude slower although
I have not tested yet with lots of clients. Eagerly awaiting unlimited LVM
shared exception store snapshots....

  http://people.redhat.com/mpatocka/patches/kernel/new-snapshots/devel/

Daire

On Mar 20, 2009  11:46 +0000, Daire Byrne wrote:
> I was wondering if it is possible to mount an MDT device with ldiskfs,
> hardlink a dir tree and then remount (lustre) without Lustre clients
> getting confused? Are hardlinks in Lustre a "bad idea" for any
reason
> (i.e. linking files into a different dir tree). I was toying with the
> idea (madcap Fridays) of doing hardlink style file based snapshots by
> doing something like this:
>
>   [root at mds ~] umount -f /dev/lustre/mdt1
>   [root at mds ~] mount -t ldiskfs /dev/lustre/mdt1 /mnt/lustre/mdt1
>   [root at mds ~] cp -fal /mnt/lustre/mdt1/ROOT/blah
/mnt/lustre/mdt1/ROOT/blah0
>   [root at mds ~] mount -t lustre /dev/lustre/mdt1 /mnt/lustre/mdt1
> 
> Something akin to "rsync -H --link-dest" backups (dirvish). A
quick
> benchmark suggests that you can link around 100 million files in 6
> hours. It is debatable how viable it would be to take a filesystem down
> for that long - maybe a a weekend operation. I was just curious if there
> was anything in Lustre that makes this impossible/bad. This would probably
> be impossible in Lustre 2.0 (CMD and changelogs).
There shouldn''t be any real problem, since clients don''t track
the link
counts themselves.  This will protect from file deletion, but whether
it handles file modification depends on if/how your applications modify
files.

Note also that having many hard links on a file will slow down e2fsck,
there were some problems like this in the past with some backup tools
that did exactly what you are proposing.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.