rsync - Apr 2008 - rsync checksums

Hi all !

I am currently trying to find a smart way to long-term-backup my data.
I currently do link-dest-updates to an external hdd. However, I am concerned 
about data corruption in the backup or the original, i.e. corruption of the 
data inside the files.

Thus i am wondering if there is a way to check the integrity of the backups on 
the external disk. i.e. to avoid data degradation for example when the hdd 
starts dying.

What i think are two things:
rsync calculates a md4 checksum of every file transferred. 
Would it be possible to store this checksum for future use, i.e. to recheck 
the files of each rsync snapshot at any later time ?

If not, could you suggest a tool to do that, keeping in mind that most files 
will be hard-links which I dont want to check in each backup (i.e. 
find+md5sum wont work).

And a question about the --checksum option:
What happens if --checksum finds CRC differences in a file which has the same 
size and modification time as the copy in the archive ? 
Would that not be a highly unusual thing to happen, showing some kind of data 
error ? Could this be used to alert the user about possible data corruption 
in the original or the backup ?

Thanks for your time,
Cheers,
Karl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url :
http://lists.samba.org/archive/rsync/attachments/20080427/f39cfef2/attachment.bin

On Sun, Apr 27, 2008 at 11:06:00PM +0200, Karl Kashofer
wrote:
> rsync calculates a md4 checksum of every file transferred.  Would it
> be possible to store this checksum for future use, i.e. to recheck the
> files of each rsync snapshot at any later time ?
The "db.diff" patch in the latest source has the start of some
checksum-
caching.  It stores checksums in a DB (currently either MySQL or SQLite)
and provides support for storing/fetching the checksums during the rsync
transfer and/or updating and checking of the checksums via a provided
perl script.  The data is kept by inode, so it is only computed once per
hard-linked file (though there is not currently any optimizing of the
checking to avoid testing the same hard-linked file more than once when
using the perl script to verify the checksums).
> What happens if --checksum finds CRC differences in a file which has
> the same size and modification time as the copy in the archive ? 
Rsync just updates the file.  This happens regularly with certain kinds
of files, such as MP3s where an editing program tweaked some tags but
reset the mtime, or the __db.* files inside the /var/lib/rpm dir (for
reasons unknown).  If you itemize the transfer (-i) with checksums (-c),
you can see any files where the checksum differed but the time did not
by having a program analyze the output.  The case that would be really
worrisome would be a file whose ctime had not changed, but whose
checksum was no longer correct.  Regular rsync can't help with that,
but the db patch has enough data to be able to check for such an error.

..wayne..