Hello, Suppose that every day cron runs this: rsync -a --times --delete $HOME /my/backups/dir/latest In general, rsync will only update a file if it has been modified. Now, imagine that one of the files becomes corrupted in the backup directory, but the timestamp hasn't changed. Will rsync detect this? I am using rsync in a backup solution which could be summarized to: cd /my/external/disk/backups rsync -a --times --delete $HOME latest cp -al latest $TODAYS_DATE So, if a file never changes, I just end up with a lot of hard links pointing to the same block of data. That's meant to make the backup space efficient. But if that block of data gets corrupted, all my "backups" for that file are gone. And I'm concerned that if that happens, rsync will just merrily go along ignoring this file because the timestamp says that the file hasn't been modified so it doesn't need to be updated. Thanks for the help. Daniel.
On Fri 22 May 2009, Daniel Carrera wrote:> > In general, rsync will only update a file if it has been modified. Now, > imagine that one of the files becomes corrupted in the backup directory, > but the timestamp hasn't changed. Will rsync detect this?Not in the usual case. You may want to consider adding -c (--checksum) to the options periodically, as this tells rsync to checksum each file and use that as a basis to determine whether to update a file or not, instead of modtime and size. Note that this will probably slow rsync down considerably, as it now needs to read each file on both ends of the transfer. Paul
Ok, thanks. Do you have any idea if a corruption that leaves the file
size intact is common or rare? What I could do is add the --checksum
option only once a week:
if [ `date +%a` = "Sat" ]; then
OPT='-a --numeric-ids --delete --times --checksum'
else
OPT='-a --numeric-ids --delete --times'
fi
...
rsync $OPT $HOME latest
Thanks again.
Daniel.
Giorgos Gaganis wrote:> Hello Daniel
>
> The default check is the time of last modification and size so if your
> corruption also leaves the file size the same the file will not be
> included for update.
>
> You can use the --checksum option that also checks the file contents but
> this will significantly increase your disk I/O.
>
> Giorgos
Hello Daniel The default check is the time of last modification and size so if your corruption also leaves the file size the same the file will not be included for update. You can use the --checksum option that also checks the file contents but this will significantly increase your disk I/O. Giorgos Daniel Carrera wrote:> Hello, > > Suppose that every day cron runs this: > > rsync -a --times --delete $HOME /my/backups/dir/latest > > > In general, rsync will only update a file if it has been modified. > Now, imagine that one of the files becomes corrupted in the backup > directory, but the timestamp hasn't changed. Will rsync detect this? > > I am using rsync in a backup solution which could be summarized to: > > cd /my/external/disk/backups > rsync -a --times --delete $HOME latest > cp -al latest $TODAYS_DATE > > > So, if a file never changes, I just end up with a lot of hard links > pointing to the same block of data. That's meant to make the backup > space efficient. But if that block of data gets corrupted, all my > "backups" for that file are gone. And I'm concerned that if that > happens, rsync will just merrily go along ignoring this file because > the timestamp says that the file hasn't been modified so it doesn't > need to be updated. > > Thanks for the help. > > Daniel.
On 22.05.2009 13:43, Daniel Carrera wrote:> Hello, > > Suppose that every day cron runs this: > > rsync -a --times --delete $HOME /my/backups/dir/latest > > > In general, rsync will only update a file if it has been modified. Now, > imagine that one of the files becomes corrupted in the backup directory, > but the timestamp hasn't changed. Will rsync detect this? > > I am using rsync in a backup solution which could be summarized to: > > cd /my/external/disk/backups > rsync -a --times --delete $HOME latest > cp -al latest $TODAYS_DATE > > > So, if a file never changes, I just end up with a lot of hard links > pointing to the same block of data. That's meant to make the backup > space efficient. But if that block of data gets corrupted, all my > "backups" for that file are gone. And I'm concerned that if that > happens, rsync will just merrily go along ignoring this file because the > timestamp says that the file hasn't been modified so it doesn't need to > be updated.Exactly. But you can (periodically) add "-c", then rsync while checksum the whole content of all files. But IF you have (or suspect) such type of corrution, you have have an even greater problem: Your hardware is crap. Bis denn -- Real Programmers consider "what you see is what you get" to be just as bad a concept in Text Editors as it is in women. No, the Real Programmer wants a "you asked for it, you got it" text editor -- complicated, cryptic, powerful, unforgiving, dangerous.