rsync - Oct 2004 - [Bug 1959] writefd_unbuffered failed to write 4092 bytes phase send_file_entry broken pipe

https://bugzilla.samba.org/show_bug.cgi?id=1959


wayned@samba.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|major                       |normal
             Status|NEW                         |ASSIGNED




------- Additional Comments From wayned@samba.org  2004-10-21 15:19 -------
Please see the issues/debugging page for instructions on how to figure out what
is going on:

http://rsync.samba.org/issues.html

Note the recommendation to upgrade to 2.6.3 for its improved error reporting.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=1959


gaurav_verma22@yahoo.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |NEW




------- Additional Comments From gaurav_verma22@yahoo.com  2004-10-22 07:20
-------
(In reply to comment #1)
> Please see the issues/debugging page for instructions on how to figure out
what
> is going on:
> 
> http://rsync.samba.org/issues.html
> 
> Note the recommendation to upgrade to 2.6.3 for its improved error
reporting.

Hi.. we did extensive reserach on this and tried various alternatives:

One interesting thing that we noticed is that if the size of "any one
particular" directory is more than 152M, the rsync fails and encounters a
hang
state. 

Again, this is only the case with with A->C transfer. 
A->B Transfer is working fine with the same set of directories.

A has rsync 2.6.2
B has rsync 2.5.7 : A->B works
C has rsync 2.6.2 : A->C doesnt work for directories > 152m

We have checked that this is also not due to some processes running and rsync
trying to overwrite that file.

This error is encountered for admin/log 

Are there any specific kernal parameters which need to be set for enabling
transfer of huge (read directories with size > some threshold) directories
using
rsync. It pretty much looks as if some OS Limit is being crossed here.

e.g. the ls command needs some maxsize kernal parameter to be able to show file
listing if the # of files > some threshold number.

any pointers on this will be appreciated. We would really like to use rsync for
syncing up code trees for our erp product which we are implementing for our
customer.

thanks a lot.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=1959





------- Additional Comments From jlowery@zertis.net  2004-11-19 05:48 -------
Created an attachment (id=798)
 --> (https://bugzilla.samba.org/attachment.cgi?id=798&action=view)
rsync hanging strace output


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=1959





------- Additional Comments From jlowery@zertis.net  2004-11-19 05:47 -------
Created an attachment (id=797)
 --> (https://bugzilla.samba.org/attachment.cgi?id=797&action=view)
Command line session of rsync hanging


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=1959





------- Additional Comments From jlowery@zertis.net  2004-11-19 05:50 -------
I am experiencing similar problems on debian woody (linux 2.4.26 and after
upgrading to linux 2.4.28). This problem started suddenly after over 100 days of
clean daily operation.

Rsyncing any two folders of larger size causes rsync to hang and the process
waits until it is manually killed. I have reproduced this with rsyncing a remote
directory to a local directory and rsyncing two local directories. I have tried
rsyncing two local directories on different hard drives with the same result. To
test if it was some disk IO error, I've ran md5deep on the directory to be
rsynced and it finishes properly. This error occurs with the debian woody
version of rsync (2.5.5) and also with the latest rsync downloaded from source
(rsync-2.6.3). 

Rsync always hangs during the first stage when calculating what files to sync.
I have attached a sample shell session and strace output.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=1959





------- Additional Comments From wayned@samba.org  2004-11-19 10:13 -------
The cited strace shows that rsync is hanging because of all the verbose messages
coming from the receiver aren't getting read by the sender.  So, just reduce
the
verbosity and it should run fine (2.6.3, that is -- I assume that 2.5.5 was
hanging for a different reason).

I'll check into this to see about resolving the problem, but it may take a
while.

Finally, a hang bug is quite different from write-failed bug, so your bug,
Jeremy, is not related to this bug report's original purpose.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.