Hi List
I've experienced some problems with rsync. I'm backuping a complete
machine's rootdirectory. After completion I see in the log of the
output, that rsync links some files which are surely *not* the same
on the source System. Or well, it says, that it makes links, using
the filename1 => filename2 notation. On the source System this files
are not the same and some files are missing on the target System
after the sync. Well, the log says, "partial transfer". But why the
link-sign or why does rsync think, that this files are the same??
Below you will find all the information about the systems and rsync
that I'm using as well as the invocation, tail of the log and stats
of the wrong-linked files.
Source and target hosts both
rsync 2.5.5-0.1
from Debian Woody
source host linux 2.4.20, target host 2.4.22 (both unpatched
kernel.org kernels), both running reiserfs.
rsync --version:
rsync version 2.5.6cvs protocol version 26
Copyright (C) 1996-2002 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks,
batchfiles,
IPv6, 64-bit system inums, 64-bit internal inums
invocation:
<<<<<<<<<<<<<<<<<<<<<<<<<<<<
rsync | -v | -aHx | --numeric-ids | --delete |
--exclude-from=Synchronisationsausnahmen | / | 192.168.1.1:/
<<<<<<<<<<<<<<<<<<<<<<<<<<<<
tail of the logfile:
<<<<<<<<<<<<<<<<<<<<<<<<<<<<
var/cache/httpd-accel/a/f4/4d0d0d7f891a691ad5c739448f37f =>
etc/sv/qmail-smtp/log/supervise/status
var/cache/httpd-accel/e/b8/fc770c62d513c4032b02da7b526c2 =>
etc/sv/qmail/log/supervise/status
var/www/ethlife/articles/news/.xmlstyle_cache/a5/ba/f217e3bdffeccb7d16c0e277fabf.type
=> var/cache/httpd-accel/0/0f/1ecf467cbcf662efc18681c7f6fe7
wrote 586980781 bytes read 4511014 bytes 577346.80 bytes/sec
total size is 7661625252 speedup is 12.95
rsync error: partial transfer (code 23) at main.c(578)
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
stats of the files with "=>" on the source host
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
ls -li var/cache/httpd-accel/a/f4/4d0d0d7f891a691ad5c739448f37f
898149 -rw-r----- 1 www-data www-data 4094 17. Sep 14:37
var/cache/httpd-accel/a/f4/4d0d0d7f891a691ad5c739448f37f
ls -li etc/sv/qmail-smtp/log/supervise/status
890631 -rw-r--r-- 1 root root 18 17. Sep 14:26
etc/sv/qmail-smtp/log/supervise/status
ls -li var/cache/httpd-accel/e/b8/fc770c62d513c4032b02da7b526c2
681656 -rw-r----- 1 www-data www-data 6068 17. Sep 15:41
var/cache/httpd-accel/e/b8/fc770c62d513c4032b02da7b526c2
ls -li etc/sv/qmail/log/supervise/status
888784 -rw-r--r-- 1 root root 18 17. Sep 15:33
etc/sv/qmail/log/supervise/status
ls -li
var/www/ethlife/articles/news/.xmlstyle_cache/a5/ba/f217e3bdffeccb7d16c0e277fabf.type
886182 -rw-rw-r-- 1 elcms_ht elcms 29 17. Sep 15:40
var/www/ethlife/articles/news/.xmlstyle_cache/a5/ba/f217e3bdffeccb7d16c0e277fabf.type
ls -li var/cache/httpd-accel/0/0f/1ecf467cbcf662efc18681c7f6fe7
701948 -rw-r----- 1 www-data www-data 18649 17. Sep 15:40
var/cache/httpd-accel/0/0f/1ecf467cbcf662efc18681c7f6fe7
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Thanks in hope for answers, cheers, Chris
--
Chris T.
tarnutzer@ethlife.ethz.ch
-------------------------
Human beings often make some mistakes.
But for real trouble, you need Windows...
On Thu, Sep 18, 2003 at 03:17:22PM +0200, Chris Tarnutzer wrote:> Hi List > > I've experienced some problems with rsync. I'm backuping a complete > machine's rootdirectory. After completion I see in the log of the > output, that rsync links some files which are surely *not* the same > on the source System. Or well, it says, that it makes links, using > the filename1 => filename2 notation. On the source System this files > are not the same and some files are missing on the target System > after the sync. Well, the log says, "partial transfer". But why the > link-sign or why does rsync think, that this files are the same?? > > Below you will find all the information about the systems and rsync > that I'm using as well as the invocation, tail of the log and stats > of the wrong-linked files. > > Source and target hosts both > rsync 2.5.5-0.1 > from Debian Woody > > source host linux 2.4.20, target host 2.4.22 (both unpatched > kernel.org kernels), both running reiserfs. > > rsync --version: > rsync version 2.5.6cvs protocol version 26 > Copyright (C) 1996-2002 by Andrew Tridgell and others > <http://rsync.samba.org/> > Capabilities: 64-bit files, socketpairs, hard links, symlinks, > batchfiles, > IPv6, 64-bit system inums, 64-bit internal inums > > invocation: > <<<<<<<<<<<<<<<<<<<<<<<<<<<< > rsync | -v | -aHx | --numeric-ids | --delete | > --exclude-from=Synchronisationsausnahmen | / | 192.168.1.1:/ > <<<<<<<<<<<<<<<<<<<<<<<<<<<< > > tail of the logfile: > <<<<<<<<<<<<<<<<<<<<<<<<<<<< > var/cache/httpd-accel/a/f4/4d0d0d7f891a691ad5c739448f37f => > etc/sv/qmail-smtp/log/supervise/status > var/cache/httpd-accel/e/b8/fc770c62d513c4032b02da7b526c2 => > etc/sv/qmail/log/supervise/status > var/www/ethlife/articles/news/.xmlstyle_cache/a5/ba/f217e3bdffeccb7d16c0e277fabf.type > => var/cache/httpd-accel/0/0f/1ecf467cbcf662efc18681c7f6fe7 > wrote 586980781 bytes read 4511014 bytes 577346.80 bytes/sec > total size is 7661625252 speedup is 12.95 > rsync error: partial transfer (code 23) at main.c(578) > <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > stats of the files with "=>" on the source host > <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< > ls -li var/cache/httpd-accel/a/f4/4d0d0d7f891a691ad5c739448f37f > 898149 -rw-r----- 1 www-data www-data 4094 17. Sep 14:37 > var/cache/httpd-accel/a/f4/4d0d0d7f891a691ad5c739448f37f > ls -li etc/sv/qmail-smtp/log/supervise/status > 890631 -rw-r--r-- 1 root root 18 17. Sep 14:26 > etc/sv/qmail-smtp/log/supervise/status > > ls -li var/cache/httpd-accel/e/b8/fc770c62d513c4032b02da7b526c2 > 681656 -rw-r----- 1 www-data www-data 6068 17. Sep 15:41 > var/cache/httpd-accel/e/b8/fc770c62d513c4032b02da7b526c2 > ls -li etc/sv/qmail/log/supervise/status > 888784 -rw-r--r-- 1 root root 18 17. Sep 15:33 > etc/sv/qmail/log/supervise/status > > ls -li > var/www/ethlife/articles/news/.xmlstyle_cache/a5/ba/f217e3bdffeccb7d16c0e277fabf.type > 886182 -rw-rw-r-- 1 elcms_ht elcms 29 17. Sep 15:40 > var/www/ethlife/articles/news/.xmlstyle_cache/a5/ba/f217e3bdffeccb7d16c0e277fabf.type > ls -li var/cache/httpd-accel/0/0f/1ecf467cbcf662efc18681c7f6fe7 > 701948 -rw-r----- 1 www-data www-data 18649 17. Sep 15:40 > var/cache/httpd-accel/0/0f/1ecf467cbcf662efc18681c7f6fe7 > <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > Thanks in hope for answers, cheers, ChrisGiven that the files don't seem be be transient i'd say reiserfs might be playing fast and loose with inode numbers. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
On Thu, 18 Sep 2003, Chris Tarnutzer <tarnutzer@ethlife.ethz.ch>?wrote:>?Hi List >? >?I've experienced some problems with rsync. I'm backuping a complete >?machine's rootdirectory. After completion I see in the log of the >?output, that rsync links some files which are surely *not* the same >?on the source System. Or well, it says, that it makes links, using >?the filename1 =>?filename2 notation. On the source System this files >?are not the same and some files are missing on the target System >?after the sync. Well, the log says, "partial transfer". But why the >?link-sign or why does rsync think, that this files are the same??...>?stats of the files with "=>" on the source host >?<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< >?ls -li var/cache/httpd-accel/a/f4/4d0d0d7f891a691ad5c739448f37f >? 898149 -rw-r----- ? ?1 www-data www-data ? ? 4094 17. Sep 14:37 >?var/cache/httpd-accel/a/f4/4d0d0d7f891a691ad5c739448f37f >?ls -li etc/sv/qmail-smtp/log/supervise/status >? 890631 -rw-r--r-- ? ?1 root ? ? root ? ? ? ? ? 18 17. Sep 14:26 >?etc/sv/qmail-smtp/log/supervise/status >? >?ls -li var/cache/httpd-accel/e/b8/fc770c62d513c4032b02da7b526c2 >? 681656 -rw-r----- ? ?1 www-data www-data ? ? 6068 17. Sep 15:41 >?var/cache/httpd-accel/e/b8/fc770c62d513c4032b02da7b526c2 >?ls -li etc/sv/qmail/log/supervise/status >? 888784 -rw-r--r-- ? ?1 root ? ? root ? ? ? ? ? 18 17. Sep 15:33 >?etc/sv/qmail/log/supervise/status >? >?ls -li var/www/ethlife/articles/news/.xmlstyle_cache/a5/ba/f217e3bdffeccb7d16c0e277fabf.type >? 886182 -rw-rw-r-- ? ?1 elcms_ht elcms ? ? ? ? ?29 17. Sep 15:40 > var/www/ethlife/articles/news/.xmlstyle_cache/a5/ba/f217e3bdffeccb7d16c0e277fabf.type >?ls -li var/cache/httpd-accel/0/0f/1ecf467cbcf662efc18681c7f6fe7 >? 701948 -rw-r----- ? ?1 www-data www-data ? ?18649 17. Sep 15:40 >?var/cache/httpd-accel/0/0f/1ecf467cbcf662efc18681c7f6fe7 >?<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<These pairs of timestamps are recent and pretty close together, which makes me think this is happening for a given pair of files - FileA and FileB: - rsync begins building the filelist on the source and adds FileA - FileA is a dynamic file which gets deleted (and possibly replaced later) and so the inode goes in the free inode pool. - FileB is created, using that recycled inode. - rsync slogs along, encounters FileB, and adds it to its filelist. Now FileA and FileB have the same inode in rsync's filelist. FileA may even no longer exist (that's a risk you take of rsyncing live systems). Since the inode number are what's used to figure out hardlinks, FileA and FileB are hardlinked together on the target system. I have no suggestion for avoiding this pitfall and still use hard links. -- John Van Essen Univ of MN Alumnus <vanes002@umn.edu> 3DGamers Systems Software Support <jve@3dgamers.com>