Norbert Hoeller
2010-Jun-19 22:48 UTC
rsync 3.0.5 Appears to be Transferring Unchanged Files
I have been using backuppc (rsync protocol) to backup up a Linux web server onto a local server for a number of years. I recall that data volumes were quite low (about 40MB), regardless of whether I was doing full or incremental backups. I recently switched to both a new web server and backuppc server. Early morning traffic volumes when backuppc runs are much higher than expected - I have seen volumes as high as 450MB. Full backups appear to drive particularly high volumes, although an incremental backup after cloning a bunch of MediaWiki installations also transferred large amounts of data. As far as I can tell, backuppc itself seems to be working fine and reports that the most files are identical to already backed up files. Many things have changed. I am backing up a virtual private server with more files. The backup server is a 'plug computer' with an ARM processor running Ubuntu 9.04, backuppc 3.1.0 and rsync 3.0.5. The web server is running CentOS 5 with rsync 2.6.9. Unfortunately, the old web and backup servers are no longer available. I enabled verbose logging and itemized-changes and ran two full backups. Both logs showed a large number of entries similar to the following. If I read the line correctly, these files were transferred, the type is a file, and the remaining attributes are unknown (possible rsync mismatch). 2010/06/19 16:12:44 [7321] <f????????? vault_scripts/phplist/scripts/schema.sql 2010/06/19 16:12:44 [7321] <f????????? vault_scripts/phplist/scripts/upgrade-app.php I upgraded the web server to rsync 3.0.5 but that does not appear to have reduced data volumes and the log file entries are unchanged A 'du' on the web server indicates that there are about 32MB of files. None of these files have changed recently. Statistics at the end of the log: 2010/06/19 16:13:04 [7321] total: matches=8154 hash_hits=19066 false_alarms=26 data=8005565 2010/06/19 16:13:04 [7321] sent 8248288 bytes received 143322 bytes total size 22385893 I am using default rsync options on backuppc: rsync --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . /var/<symlink>/ Any suggestions on how I can debug this problem? Thanks, Norbert -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20100619/d481e97d/attachment.html>
Norbert Hoeller
2010-Jun-30 18:01 UTC
rsync 3.0.5 Appears to be Transferring Unchanged Files
Further testing has determined that the rsync command is working as advertised. backuppc uses a Perl implementation of File::RsyncP that communicates with rsync. The Ubuntu port for ARM processors appears to label a significant number of blocks as changed even through the source and target files are identical. The problem has been reported to Craig Barratt and the ubuntu-devel-discuss mailing list.> I have been using backuppc (rsync protocol) to backup up a Linux web > server onto a local server for a number of years. I recall that data > volumes were quite low (about 40MB), regardless of whether I was doing > full or incremental backups. I recently switched to both a new webserver> and backuppc server. Early morning traffic volumes when backuppc runsare> much higher than expected - I have seen volumes as high as 450MB.-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20100630/f3c33a40/attachment.html>