Hi All: We have been looking at implementing deduplication on a backup server.>From what I have been able to find the available documentation ispretty thin. I ended up trying to install LessFS on this CentOS 5.7 box but we have now encountered problems with fuse version. Has anyone out there been able to get LessFS running on CentOS 5.7 and can provide some pointers? If not LessFS can you suggest an alternate deduplication software? TIA Regards, Hugh -- Hugh E Cruickshank, Forward Software, www.forward-software.com
> We have been looking at implementing deduplication on a backup server. > > If not LessFS can you suggest an alternate deduplication software? >http://openindiana.org/ Solaris 11 Express http://www.freebsd.org/releases/9.0R/announce.html (ZFS pool version >= 28)
On Mon, Jan 16, 2012 at 5:50 PM, Hugh E Cruickshank <hugh at forsoft.com> wrote:> > We have been looking at implementing deduplication on a backup server. > >From what I have been able to find the available documentation is > pretty thin. I ended up trying to install LessFS on this CentOS 5.7 > box but we have now encountered problems with fuse version. > > Has anyone out there been able to get LessFS running on CentOS 5.7 and > can provide some pointers? > > If not LessFS can you suggest an alternate deduplication software?Backuppc dedups (and compresses) at the file level using hardlinks. Not quite as effective as a block level if you have frequent small changes in large files, but still very good with no unusual filesystem requirements other than keeping the whole archive on one filesystem. It will link all identical content, whether from the same or different systems and it's rsync implementation can work with local compressed copies while chatting with a stock remote version. -- Les Mikesell lesmikesell at gmail.com
Dne 17.1.2012 0:50, Hugh E Cruickshank napsal(a):> Hi All: > > We have been looking at implementing deduplication on a backup server. > >From what I have been able to find the available documentation is > pretty thin. I ended up trying to install LessFS on this CentOS 5.7 > box but we have now encountered problems with fuse version. > > Has anyone out there been able to get LessFS running on CentOS 5.7 and > can provide some pointers? > > If not LessFS can you suggest an alternate deduplication software? > > TIA > > Regards, Hugh >Hi Hugh, I've got something in my repo http://fs12.vsb.cz/hrb33/el5/hrb/stable/i386/repoview/fuse-lessfs.html. Might be somewhat outdated. You can try it and we can build new versions. As to alternatives I'm happy with rdiff-backup. DH
On 01/16/2012 03:50 PM, Hugh E Cruickshank wrote:> Hi All: > > We have been looking at implementing deduplication on a backup server. > >From what I have been able to find the available documentation is > pretty thin. I ended up trying to install LessFS on this CentOS 5.7 > box but we have now encountered problems with fuse version. > > Has anyone out there been able to get LessFS running on CentOS 5.7 and > can provide some pointers? > > If not LessFS can you suggest an alternate deduplication software? > > TIA > > Regards, Hugh >The ZFSonlinux project from LLNL looks promising (native mode kernel implementation, pool version 28), although the version that supports mountable filesystems is still in the RC stage. I would want some solid testing before deploying in a backup system. http://zfsonlinux.org/ Nataraj
Hugh E Cruickshank writes:> Hi All: > > We have been looking at implementing deduplication on a backup server. > From what I have been able to find the available documentation is > pretty thin. I ended up trying to install LessFS on this CentOS 5.7 > box but we have now encountered problems with fuse version.Maybe try CentOS6. We've had numerous fuse issues with other software on CentOS5 and one recommendation was to use a newer kernel, which essentially means a newer distro.
On Mon, 2012-01-16 at 15:50 -0800, Hugh E Cruickshank wrote:> We have been looking at implementing deduplication on a backup server. > >From what I have been able to find the available documentation is > pretty thin. I ended up trying to install LessFS on this CentOS 5.7 > box but we have now encountered problems with fuse version > Has anyone out there been able to get LessFS running on CentOS 5.7 and > can provide some pointers > If not LessFS can you suggest an alternate deduplication software?SFDS / openDEDUPE <http://wmmi.net/documents/OpenDedup.pdf> <http://www.opendedup.org/> -- Adam Tauno Williams <http://www.whitemiceconsulting.com> System Administrator, OpenGroupware Developer, LPI / CNA Fingerprint 8C08 209A FBE3 C41A DD2F A270 2D17 8FA4 D95E D383 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: <http://lists.centos.org/pipermail/centos/attachments/20120122/e83b1dad/attachment-0005.sig>
This thread has been beat to death, so perhaps my $0.02 isn't so meaningful, but I wrote a set of rsync scripts in php that I've used for years to manage terabytes of backups going back years of time. It's called TINBackupBuddy and you can get it at http://www.effortlessis.com/thisisnotbackupbuddy/ - it is a set of scripts that allow you to manage and back up numerous hosts, called via cron on a regular, graceful failure basis, via rsync. It de-duplicates files that have not changed between backup sets, so depending on the churn on your servers, you can get an astonishing number of backups onto a single drive... I've managed backups for a rather large cluster (now over 200 schools and school districts) of data automatically, on a 24 hour basis using these scripts, for years, so they really do work. And for our development team, we recover from these backups in order to replicate reported issues, so these backups are verified numerous times per day. Get a computer with some big disks in it. (We have about 20 TB of disk space on our backups server right now) Set up TinBackupBuddy and point to the big disks, use symlinks where it makes sense. Set a few options, call bbbackup.php via cron, and you're golden. Been doing it for close to 10 years now.... Good luck! On 01/16/2012 03:50 PM, Hugh E Cruickshank wrote:> Hi All: > > We have been looking at implementing deduplication on a backup server. > > From what I have been able to find the available documentation is > pretty thin. I ended up trying to install LessFS on this CentOS 5.7 > box but we have now encountered problems with fuse version. > > Has anyone out there been able to get LessFS running on CentOS 5.7 and > can provide some pointers? > > If not LessFS can you suggest an alternate deduplication software? > > TIA > > Regards, Hugh >-- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.