rsync - Feb 2010 - DO NOT REPLY [Bug 7129] New: rsync 3.0.7 - unlink failed (21) - unable to delete directory

https://bugzilla.samba.org/show_bug.cgi?id=7129

           Summary: rsync 3.0.7 - unlink failed (21) - unable to delete
                    directory
           Product: rsync
           Version: 3.0.7
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P3
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: p.herz at allied-internet.ag
         QAContact: rsync-qa at samba.org


Hi,

from a nightly cronjob we're running the following rsync command:

rsync -axpHgotvr --bwlimit=2500 --delete --ignore-errors --force
--numeric-ids --progress --stats / /backup/

After having updated to rsync-3.0.7 (debian /lenny) we constantly get the
followning error message on every run:

rsync: unlink "/backup/usr/lib/gcc/x86_64-linux-gnu/4.1.3" failed: Is
a
directory (21)

We would appreciate any hints, thoughs and comments on how to solve the isssue
that rsync is no longer able to delete the content of e.g. that namend folder
above. 

Kind regards - philipp


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7129





------- Comment #1 from matt at mattmccutchen.net  2010-03-17 02:32 CST -------
That's weird, rsync shouldn't be trying to unlink a directory.  Please
provide
strace output for the generator so we can see what is going on.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7129


wayned at samba.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WORKSFORME




------- Comment #2 from wayned at samba.org  2011-02-22 09:15 CST -------
Looks like you are missing an --exclude=/backup/ option.  Without that, rsync
will copy /backup into /backup/backup.

As for the unlink error, I've never seen it and with no further comments
from
the user, I'm closing this.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7129

--- Comment #3 from Roel van Meer <rolek at bokxing.nl> 2011-07-25
10:43:17 UTC ---
Hi,

I've got the same problem, and I have a script to reproduce it easily. This
is
also with 3.0.7. I'll also test it with a later version and report the
results.

Regards,

Roel

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7129

--- Comment #4 from Roel van Meer <rolek at bokxing.nl> 2011-07-25
10:44:45 UTC ---
Created attachment 6713
  --> https://bugzilla.samba.org/attachment.cgi?id=6713
Script that reproduces bug 7129

Output of the script:

rolek at combineer:/tmp$ sh /tmp/rsync-7129-reproducer.sh 
receiving incremental file list
aa
bb -> aa
dd -> aa
cc => dd

sent 46 bytes  received 147 bytes  386.00 bytes/sec
total size is 6  speedup is 0.03
receiving incremental file list
rsync: unlink "/tmp/to/dd" failed: Is a directory (21)

sent 11 bytes  received 95 bytes  212.00 bytes/sec
total size is 6  speedup is 0.06
rsync error: some files/attrs were not transferred (see previous errors) (code
23) at main.c(1508) [generator=3.0.7]

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7129

--- Comment #5 from Roel van Meer <rolek at bokxing.nl> 2011-07-25
11:18:08 UTC ---
The bug also exists in 3.0.9pre1, but not with the latest git version (which is
3.1.0dev).

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7129

--- Comment #6 from Roel van Meer <rolek at bokxing.nl> 2011-07-25
11:19:57 UTC ---
If you need more info please let me know. I'll be on holiday soon, so I
might
not answer before mid august, though.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7129

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|3.0.7                       |3.1.0

--- Comment #7 from Wayne Davison <wayned at samba.org> 2011-07-26
03:20:57 UTC ---
(In reply to comment #5)
> The bug also exists in 3.0.9pre1, but not with the latest git version
(which is
> 3.1.0dev).
Good -- that is what I expected.  3.1.0 has much-improved code for maintaining
the backup hierarchy.  The 3.0.x code can get confused by a directory that
needs to be replaced by a file, and other fun (or not so fun) path conflicts. 
With the 3.0.x code, you have to manually clean up your backup dir when rsync
outputs an error about it.  Beginning with 3.1.0, rsync will do a much better
job of keeping the backup directory hierarchy in good shape.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7129

--- Comment #8 from Chris H. <qrstuv+sambabugzilla at gmail.com>
2013-04-17 15:04:58 UTC ---
I'm also affected by this bug, albeit intermittently.

I'm using version 3.0.6 on CentOS 6.4 x86_64.

The attached script does not reproduce the issue with me, but I had a large
backup job that failed last night due to this error:

2013/04/16 21:36:28 [19677] rsync: unlink
"/mnt/storage1/backups/daily.1/beefy/home/.ecryptfs/user/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbAq6pvIi4anETlUvMqFdHuA2E9ZDJ1XuQ2PHT6BcS.3jDHuHoFDj4MTk--"
failed: Is a directory (21)
2013/04/16 21:36:28 [19677] rsync error: errors with program diagnostics (code
13) at log.c(235) [sender=3.0.6]

Command run:

/usr/bin/rsync -aHvv --delete --delete-before --delete-excluded --progress
--numeric-ids --sparse --exclude-from=/usr/local/bin/backup_excludes.txt
/mnt/storage1/special_backups/
root at server.domain.local:/mnt/storage1/special_backups/

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=7129

--- Comment #9 from basti <bastian-samba-rsync-bugzilla at t6l.de>
2013-09-19 10:05:33 UTC ---
Just for Info

I have hit this bug on debian 7.1, when creating a backup of my rsnapshot
backup.
I need to preserve hard links for this.
"""
% rsync --version
rsync  version 3.0.9  protocol version 30
% rsync -aH $SRC $DST
"""

Error message is:
"""
rsync: unlink "/mnt/backup/rsnapshot/root/daily.1/host/home/user/.psi"
failed:
Is a directory
"""

In my case this is triggered, when a directory became a link or vice versa:
"""
% cd /mnt/backup/rsnapshot/root
% stat */host/home/user/.psi | grep -E '(File|Size)'
  File: `daily.0/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `daily.1/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `daily.2/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `daily.3/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `daily.4/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `daily.5/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `daily.6/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `monthly.0/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `monthly.1/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `monthly.2/host/home/user/.psi' -> `fuse/mnt/private/rc/psi'
  Size: 23            Blocks: 0          IO Block: 4096   symbolic link
  File: `monthly.3/host/home/user/.psi' -> `fuse/mnt/private/rc/psi'
  Size: 23            Blocks: 0          IO Block: 4096   symbolic link
  File: `monthly.4/host/home/user/.psi' -> `fuse/mnt/private/rc/psi'
  Size: 23            Blocks: 0          IO Block: 4096   symbolic link
  File: `monthly.5/host/home/user/.psi' -> `fuse/mnt/private/rc/psi'
  Size: 23            Blocks: 0          IO Block: 4096   symbolic link
  File: `weekly.0/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `weekly.1/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `weekly.2/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory
  File: `weekly.3/host/home/user/.psi'
  Size: 4096          Blocks: 8          IO Block: 4096   directory

"""


Solution was, as descibed above, to run latest rsync code. I cloned from git
and compiled v3.0.3-568-g60cc5d4
"""
% /opt/rsync/local/bin/rsync --version
rsync  version 3.1.0pre1  protocol version 31.PR14
"""


Now, these unlink problems dont exist anymore.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.