rsync - Sep 2010 - DO NOT REPLY [Bug 7683] New: New rsync deletes destination files it shouldn't

https://bugzilla.samba.org/show_bug.cgi?id=7683

           Summary: New rsync deletes destination files it shouldn't
           Product: rsync
           Version: 3.0.7
          Platform: x86
        OS/Version: Linux
            Status: NEW
          Severity: critical
          Priority: P3
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: vaskorbmail-bugzilla at yahoo.co.uk
         QAContact: rsync-qa at samba.org
                CC: vaskorbmail-bugzilla at yahoo.co.uk


I have just upgraded to Ubuntu 10.04.1 (from 8.04, I think). I previously used
the rsync program regularly to backup files incrementally between locally
mounted Windows type filesystems (hard disks and memory pen etc.) without
problems. However, when I do a trial run using the -n parameter on the upgraded
system, the computer deletes a large number of files on the destination
filesystem (and does not evidently recopy these across).

I have copied and pasted some output below to demonstrate the issue. The first
command shows the execution of rsync on a small sample of the original fileset
and the output shows the files being deleted (in -n trial mode). The
'ls'
command next shows the contents of the "New recovery" source folder to
show
that the roa.doc file does indeed exist and shouldn't have been deleted on
the
destination without being recopied. Finally, I have just run the rsync command
on its own to show the version number.

Interestingly, I have copied an old version on the rsync program from a old
Linux CD and it exhibits the same problem running under the new Ubuntu, so it
may be an issue relating to the way rsync reads a VFAT filesystem.

vb at celery:~$ sudo rsync --modify-window=2 -r -u -t -n --delete-during
--progress /media/sda1/My\ Documents/Vaskor/Tasks /media/4BEF-DCDA/Vaskor
sending incremental file list
Tasks/
deleting Tasks/cleanvir.bat
deleting Tasks/CPD Technical Updates 2011 - Topics nomination form.doc
Tasks/Moringa-Brochure-Anstiss.pdf
Tasks/Orange free SIM with ?5 credit.htm
Tasks/Tasks.txt
Tasks/Time.txt
Tasks/attachments_18_07_2010.zip
Tasks/benefits.txt
Tasks/benefits.txt~
Tasks/check address.TXT
Tasks/chris-howard-bwb.pdf
Tasks/phone_usa.jsp.html
Tasks/ppp36.pdf
Tasks/preaching peace.pdf
Tasks/spam2.txt
Tasks/statistics.txt
Tasks/tci-guide-to-investments.pdf
Tasks/tiscali.ins
Tasks/trendexplosion.pdf
Tasks/tripletrend34 - bad.zip
Tasks/New recovery/
deleting Tasks/New recovery/roa.doc
deleting Tasks/New recovery/3-s098~1.jpg
deleting Tasks/New recovery/2/annual~1.doc
deleting Tasks/Taken on disk/Permanent/friends.txt

sent 20578 bytes  received 134 bytes  13808.00 bytes/sec
total size is 124405234  speedup is 6006.43 (DRY RUN)
vb at celery:~$ ls /media/sda1/My\ Documents/Vaskor/Tasks/New\ recovery
1  Careers.zip  desktop.ini  New Folder  Thumbs.db
2  CV.doc       linux        ROA.DOC     Vaskor.zip
vb at celery:~$ rsync
rsync  version 3.0.7  protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7683





------- Comment #1 from paul at debian.org  2010-09-16 10:21 CST
-------
> ... The 'ls'
> command next shows the contents of the "New recovery" source
folder to show
> that the roa.doc file does indeed exist ...
roa.doc != ROA.DOC


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7683





------- Comment #2 from vaskorbmail-bugzilla at yahoo.co.uk  2010-09-16 10:37
CST -------
I agree, but shouldn't rsync attempt to copy across ROA.DOC after deleting
roa.doc in that case?

(In reply to comment #1)
> > ... The 'ls'
> > command next shows the contents of the "New recovery" source
folder to show
> > that the roa.doc file does indeed exist ...
> roa.doc != ROA.DOC

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7683





------- Comment #3 from paul at debian.org  2010-09-16 10:43 CST -------
I expect that rsync has also checked for "DOA.DOC" and has determined
that it
exists and is identical to the source. After that a check for extraneous files
(by reading the directory listing) shows a "doa.doc", so that gets
deleted.

Just make sure you use a case-preserving filesystem.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7683


matt at mattmccutchen.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID




------- Comment #4 from matt at mattmccutchen.net  2010-09-16 16:08 CST -------
(In reply to comment #3)
> I expect that rsync has also checked for "DOA.DOC" and has
determined that it
> exists and is identical to the source. After that a check for extraneous
files
> (by reading the directory listing) shows a "doa.doc", so that
gets deleted.
That analysis is almost correct.  The delete mode is --delete-during, so rsync
considers "doa.doc" for deletion before it considers
"DOA.DOC" for transfer.
The reason why a transfer of "DOA.DOC" is not itemized is that on a
dry run,
"doa.doc" is not actually deleted, and rsync has no way to anticipate
that
deleting it would cause "DOA.DOC" to no longer exist on the
destination.  On a
real run, rsync should transfer "DOA.DOC".

The main version of rsync currently does not claim to support case-insensitive
filesystems (there is a maintained patch "ignore-case.diff" that adds
the
functionality), so this report is invalid.

Vaskor, the options I see for you are to make a one-time change to harmonize
the filenames (with rsync or otherwise), use the ignore-case patch, or fiddle
with the "shortname" mount option of the source and/or destination
filesystem
to try to make the discrepancy in the names go away.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7683





------- Comment #5 from vaskorbmail-bugzilla at yahoo.co.uk  2010-09-18 08:51
CST -------
(In reply to comment #4)

I actually prefer the ext3 filesystem, but the filesystems in question needed
to be readable by Windows computers.

I assumed by DOA.DOC you were referring to ROA.DOC (or perhaps you were
suggesting that the file was "Dead On Arrival"!)

Many thanks for the suggestions, which has resolved my issue - I wish
commercial software was as well supported!


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.