rsync - Jan 2011 - rsync -rcv printing out filenames when content identical

Hi,

I'm heavly relying on rsync -nvrc during my day-to-day work in UNIX-like
enviroment. I'm using rsync for few years now and today I've found a
situation which I do not understand.

I have two directories with few maildirs in it. Each maildir containg
some emails. I'm testing scripts which I wrote to manipulate those
emails.

Each time I want to revert changes in one set of maildirs I've rsync'ing
them from backup copy:

$ rsync -rcv --delete backup/ mail/

Nothing special about that. However my scripts are modyfing only
mail/INBOX maildir, nothing else. To my surprize when I ran rsync
command to revert my changes here is what I got (I will use -n in below
examples for now):

$ rsync -v -n -rc --delete backup/ mail/
sending incremental file list
deleting
INBOX/new/1247266872_0.9989.myhost,U=759,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
deleting
INBOX/new/1247249243_0.9989.myhost,U=758,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
deleting
INBOX/new/1247085243_0.9989.myhost,U=685,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
deleting
INBOX/new/1247083982_0.9989.myhost,U=684,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
deleting
INBOX/new/1247070709_0.9989.myhost,U=655,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
deleting
INBOX/new/1246460627_0.9989.myhost,U=472,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
deleting
INBOX/new/1246372564_0.20677.myhost,U=407,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
deleting
INBOX/new/1246370187_1.20677.myhost,U=40,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
deleting
INBOX/new/1246370187_0.20677.myhost,U=39,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
INBOX/new/1246370187_0.20677.myhost,U=39,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
INBOX/new/1246370187_1.20677.myhost,U=40,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
INBOX/new/1246372564_0.20677.myhost,U=407,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
INBOX/new/1246460627_0.9989.myhost,U=472,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
INBOX/new/1247070709_0.9989.myhost,U=655,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
INBOX/new/1247083982_0.9989.myhost,U=684,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
INBOX/new/1247085243_0.9989.myhost,U=685,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
INBOX/new/1247249243_0.9989.myhost,U=758,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
INBOX/new/1247266872_0.9989.myhost,U=759,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
Jira/cur/1246370207_0.20677.myhost,U=1,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370208_0.20677.myhost,U=2,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370208_1.20677.myhost,U=3,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370208_2.20677.myhost,U=4,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370208_3.20677.myhost,U=5,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370209_0.20677.myhost,U=6,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370209_1.20677.myhost,U=7,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370209_2.20677.myhost,U=8,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370212_3.20677.myhost,U=19,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370213_0.20677.myhost,U=20,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370213_1.20677.myhost,U=21,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370213_2.20677.myhost,U=22,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370213_3.20677.myhost,U=23,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370214_0.20677.myhost,U=24,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
Jira/cur/1246370214_1.20677.myhost,U=25,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
...

so even when I'm modyfing INBOX rsync shows that it's syncing files from
Jira as well. Now if I run above command (rsync -v -n -rc --delete
backup/ mail/) again and again and again, I will randomly get sometimes
sync of INBOX only or sometimes INBOX and Jira. Ofcourse INBOX sync is
correct, as files where moved around, buy why Jira? Nothing changed
there.

If I run diff -ruN backup/Jira mail/Jira there are no differences
reported. If I cmp(1) all files from both Jira folders there are no
differences.

How is it possible that rsync(1) sometimes reports (via -v option) that
some files from Jira were synced and sometimes it doesn't?

If I rsync -v -n -rc --delete backup/Jira mail/Jira I never have report
that something was copied.

If someone could shed some light on this for me I would really
appreciate it.

Cheers.


I can reproduce that on RHEL6beta1 rsync-3.0.6-4.el6.x86_64, OpenBSD
4.9-beta rsync-3.0.7, OpenBSD 4.6 rsync-3.0.6

-- 
best regards
q#

I'm suing -n on purposes to see what will happen. Not sure did I express
myself clearly. When using -n and reaparing dry run for my
direcotories sometimes rsync shows that only INBOX has some differences
and sometimes INBOX and Jira. Not sure was that clear in my previous
email.

-- 
best regards
q#

Anyone?

On Sat, Jan 22, 2011 at 06:21:13PM +0000, Mikolaj Kucharski
wrote:
> Hi,
> 
> I'm heavly relying on rsync -nvrc during my day-to-day work in
UNIX-like
> enviroment. I'm using rsync for few years now and today I've found
a
> situation which I do not understand.
> 
> I have two directories with few maildirs in it. Each maildir containg
> some emails. I'm testing scripts which I wrote to manipulate those
> emails.
> 
> Each time I want to revert changes in one set of maildirs I've
rsync'ing
> them from backup copy:
> 
> $ rsync -rcv --delete backup/ mail/
> 
> Nothing special about that. However my scripts are modyfing only
> mail/INBOX maildir, nothing else. To my surprize when I ran rsync
> command to revert my changes here is what I got (I will use -n in below
> examples for now):
> 
> $ rsync -v -n -rc --delete backup/ mail/
> sending incremental file list
> deleting
INBOX/new/1247266872_0.9989.myhost,U=759,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
> deleting
INBOX/new/1247249243_0.9989.myhost,U=758,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
> deleting
INBOX/new/1247085243_0.9989.myhost,U=685,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
> deleting
INBOX/new/1247083982_0.9989.myhost,U=684,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
> deleting
INBOX/new/1247070709_0.9989.myhost,U=655,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
> deleting
INBOX/new/1246460627_0.9989.myhost,U=472,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
> deleting
INBOX/new/1246372564_0.20677.myhost,U=407,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
> deleting
INBOX/new/1246370187_1.20677.myhost,U=40,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
> deleting
INBOX/new/1246370187_0.20677.myhost,U=39,FMD5=7e33429f656f1e6e9d79b29c3f82c57e
>
INBOX/new/1246370187_0.20677.myhost,U=39,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
INBOX/new/1246370187_1.20677.myhost,U=40,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
INBOX/new/1246372564_0.20677.myhost,U=407,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
INBOX/new/1246460627_0.9989.myhost,U=472,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
INBOX/new/1247070709_0.9989.myhost,U=655,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
INBOX/new/1247083982_0.9989.myhost,U=684,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
INBOX/new/1247085243_0.9989.myhost,U=685,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
INBOX/new/1247249243_0.9989.myhost,U=758,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
INBOX/new/1247266872_0.9989.myhost,U=759,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,
>
Jira/cur/1246370207_0.20677.myhost,U=1,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370208_0.20677.myhost,U=2,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370208_1.20677.myhost,U=3,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370208_2.20677.myhost,U=4,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370208_3.20677.myhost,U=5,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370209_0.20677.myhost,U=6,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370209_1.20677.myhost,U=7,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370209_2.20677.myhost,U=8,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370212_3.20677.myhost,U=19,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370213_0.20677.myhost,U=20,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370213_1.20677.myhost,U=21,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370213_2.20677.myhost,U=22,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370213_3.20677.myhost,U=23,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370214_0.20677.myhost,U=24,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
>
Jira/cur/1246370214_1.20677.myhost,U=25,FMD5=89f846f264788bfaf58c4efbf05de4eb:2,S
> ...
> 
> so even when I'm modyfing INBOX rsync shows that it's syncing files
from
> Jira as well. Now if I run above command (rsync -v -n -rc --delete
> backup/ mail/) again and again and again, I will randomly get sometimes
> sync of INBOX only or sometimes INBOX and Jira. Ofcourse INBOX sync is
> correct, as files where moved around, buy why Jira? Nothing changed
> there.
> 
> If I run diff -ruN backup/Jira mail/Jira there are no differences
> reported. If I cmp(1) all files from both Jira folders there are no
> differences.
> 
> How is it possible that rsync(1) sometimes reports (via -v option) that
> some files from Jira were synced and sometimes it doesn't?
> 
> If I rsync -v -n -rc --delete backup/Jira mail/Jira I never have report
> that something was copied.
> 
> If someone could shed some light on this for me I would really
> appreciate it.
> 
> Cheers.
> 
> 
> I can reproduce that on RHEL6beta1 rsync-3.0.6-4.el6.x86_64, OpenBSD
> 4.9-beta rsync-3.0.7, OpenBSD 4.6 rsync-3.0.6
-- 
best regards
q#

On Sat, Jan 22, 2011 at 10:21 AM, Mikolaj Kucharski
<mikolaj at kucharski.name>wrote:
> $ rsync -rcv --delete backup/ mail/

Unless the "backup" dir only contains files that came from inside the
mail
dir, you need to specify what source inside the backup you want to copy.
 Like one of these:

rsync -rcv --del backup/mail/ mail/
rsync -rcv --del backup/mail .

In the former, the contents of the mail dir are put into the mail dir.  In
the latter, the mail dir is copied, along with its contents into the current
dir (for "mail") and its subdirs.

..wayne..
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20110125/32c71384/attachment.html>

On Wed, Jan 26, 2011 at 12:48:42PM +0100, Paul Slootman
wrote:
> On Wed 26 Jan 2011, Mikolaj Kucharski wrote:
> > 
> > both folders. I run always rsync -n -rcv --delete backup/ mail/ and
> > rsync randomly repors that Jira should be synced or not if I run this
> > command five times in a row in 1 second interval.
> 
> Use -i to show what attributes rsync thinks are different, leading to
> the sync.
Cool. Thanks for that Paul.

	rsync -v -n -rc --delete -i backup/ mail/

	>fc.T......

	> means that a file is being transferred to the local host (received)
	f for a file
	c means either that a regular file has a different checksum
	T means that the modification time will be set to the transfer time,
	  which happens when a file/symlink/device is updated without --times

Is it possible that rsync(1) can compute different checksum once in a
while for a file?

Can I see somehow checksum for each file computed by rsync(1)?

-- 
best regards
q#