rsync - Jan 2003 - rsync hanging with openssh-2.9.9p2 as the transport

Hi,

This is perhaps a stupid question.  I apologize in advance if it's already
been covered, but I'm stumped...

I'm using rsync to backup some file systems to a remote host.
The transport is openssh-2.9.9p2.

The problem does not occur when the transport is rsh.

I'm invoking rsync as follows:

  rsync -RlHptgoD --numeric-ids -e "ssh" -rzx --stats .
remotehost:/nfs/mirror

But it hangs when there are large numbers of files in the "."
filesystem.
If I run with -vvv, I see lots of make_file entries as send_file_list
starts sending the list of files to the remote host, but then it freezes
after around 4500 to 6500 make_file messages.

I see the same problem with versions 2.4.6, 2.5.5, and 2.5.6pre1.
And I see it on linux 2.4.18 (red hat 8.0) and solaris 8/x86.

I can get it to work by breaking it up into chunks of 1000 files or
so.  It might work with more, I haven't tested exhaustively.

I have tried using the --blocking-io and --no-blocking-io options, but
neither one solves the problem.

I could provide more info about where it hangs, but I thought somebody
might know the answer, since this is clearly related to the
interaction with openssh (since there's no problem with rsh).

Is there a trick to make rsync work nicely with openssh?  I searched the
archives and haven't found anything...  Does upgrading to openssh 3 solve
the problem?

Thanks,
Andy

On Tue, Jan 14, 2003 at 05:29:06PM -0500, Andrew J. Schorr
wrote:
> Hi,
> 
> This is perhaps a stupid question.  I apologize in advance if it's
already
> been covered, but I'm stumped...
> 
> I'm using rsync to backup some file systems to a remote host.
> The transport is openssh-2.9.9p2.
> 
> The problem does not occur when the transport is rsh.
> 
> I'm invoking rsync as follows:
> 
>   rsync -RlHptgoD --numeric-ids -e "ssh" -rzx --stats .
remotehost:/nfs/mirror
> 
> But it hangs when there are large numbers of files in the "."
filesystem.
> If I run with -vvv, I see lots of make_file entries as send_file_list
> starts sending the list of files to the remote host, but then it freezes
> after around 4500 to 6500 make_file messages.
> 
> I see the same problem with versions 2.4.6, 2.5.5, and 2.5.6pre1.
> And I see it on linux 2.4.18 (red hat 8.0) and solaris 8/x86.
> 
> I can get it to work by breaking it up into chunks of 1000 files or
> so.  It might work with more, I haven't tested exhaustively.
> 
> I have tried using the --blocking-io and --no-blocking-io options, but
> neither one solves the problem.
> 
> I could provide more info about where it hangs, but I thought somebody
> might know the answer, since this is clearly related to the
> interaction with openssh (since there's no problem with rsh).
> 
> Is there a trick to make rsync work nicely with openssh?  I searched the
> archives and haven't found anything...  Does upgrading to openssh 3
solve
> the problem?
Unfortunately I don't think anybody is going to be able to tell you.
I've not heard of anybody lately posting a similar problem.  In the
past hanging problems have been traced to many different sources.
Rsync stresses network (and filesystem) implementations greatly, and
combining it with ssh stresses things that much more.  I think it's
worth a try to use openssh 3.1 or 3.2 (I've not been happy with versions
after 3.2).  What's the network between the local and remote machines?
Does the name /nfs/mirror imply that the files are not directly mounted
on "remote" but are instead on an NFS server?  That has been the cause
of many problems in the past.

- Dave