Hello, I am using the latest rsync version that is available as of today, which is 2.6.5, that I compiled from the src downloaded from rsync website (my understanding is that the latest version includes all the patches that were required for cygwin). Both the source and target systems are Windows 2000 server, and I am trying to sync over ssh (through inetd). I am frequently facing problem with rsync as it hangs with no messages. If I kill the client, it remains running on the server side, so I have to kill that too. I have searched for cygwin hanging problems and found a few posts regarding some old versions, but I expected them to be fixed by now, and I didn't find any solutions/resolutions. The last time it happened, I initiated the sync from the source m/c instead of the target m/c and it went through fine. After that, I switched back to running it on the target m/c, and it ran fine for about a week or so, and now it is hanging again (all through the weekend). I looked at the issues page on rsync website, and tried to gather netstat output, but netstat on windows doesn't seem to print the queue states. I should be able to get the strace output, though I accidentally killed the rsync command while killing the strace. I am currently trying to rerun the sync to see if it hangs again and post with the strace output, but meanwhile, I would like to know how I can increase the reliability of rsync on cygwin. I am trying to create a hot backup server for our production perforce installation with about 20gigs of depot files, but the problem is not probably due to the size of data, as I am sure there are several other users syncing even larger data sets, but also because I had the same problem earlier in the test environment syncing a smaller perforce installation of having about only 50mb of depot files. The problem went away in the test environment when I restarted rsync, probably because it already copied a few files before hanging, and so the state was not exactly the same. The hanging problem is really bothering me as I would like to make this completely automated in a loop. Can anyone help me resolve this issue? Will initiating rsync from the source be more reliable? Will using a different protocol, such as rsh or rsync instead of ssh be any better? Currently rsh usage is broken on cygwin, but I found a workaround to use a wrapper on the list archive. Thanks a lot, Hari
On Mon, Jul 25, 2005 at 04:44:00PM -0700, Hari Krishna Dara wrote:> I am frequently facing problem with rsync as it hangs with no > messages.It has been my experience that this is caused by bugs in cygwin's pipe handling -- if you rsync directly to a daemon process, the data gets sent over a socket and rsync does not hang. If you use a remote-shell connection, rsync has to talk over a pipe to the remote-shell, and the cygwin code mangles the data, resulting in a hang. If this is still the case (which I believe it is), this hang is out of rsync's control. ..wayne..
On 7/26/05, Wayne Davison <wayned@samba.org> wrote:> On Mon, Jul 25, 2005 at 04:44:00PM -0700, Hari Krishna Dara wrote: > > I am frequently facing problem with rsync as it hangs with no > > messages. > > It has been my experience that this is caused by bugs in cygwin's pipe > handling -- if you rsync directly to a daemon process, the data gets > sent over a socket and rsync does not hang. If you use a remote-shell > connection, rsync has to talk over a pipe to the remote-shell, and the > cygwin code mangles the data, resulting in a hang. If this is still the > case (which I believe it is), this hang is out of rsync's control. > > ..wayne.. >In daemon mode, I coun't reproduce the problem, which is very good, so I have decided to change my scripts to use this method. Now to avoid adding another manual step to my setup process (such as manually configuring inetd.conf and rsyncd.conf), I was looking at the options to start rsync via remote shell, and have a really hard time to get it working. After trying out various options, the documentation seems to be missing something or I am misunderstaning something (I hope it is the later). I thought all that I need to do is to pass "--daemon" option along with the --rsh that I am already passing, and use rsync://<host>/<module>/<path> syntax instead of the <host>:/<path> syntax, something like this: rsync --daemon --rsh=ssh -av rsync://host/module/path_from_module /local_path But this seems to work just like the below command: rsync --daemon as it just starts up a local daemon and does no sync. The man page on rsyncd.conf also says (in "LAUNCHING THE RSYNC DAEMON" section), that the --daemon option need not be passed explicitly while using "::" or "rsync://" in remote server path, so I even tried that: rsync --rsh=ssh -av rsync://host/module/path_from_module /local_path but this assumes that rsync server is already running on the default port on the remote host. I tried other minor modifications such as passing --server (though the manpage says it is meant for using with the ssh keys), but no luck. Can someone please help me? I searched the archives, and found a number of references to using the daemon mode from inet or with the ssh keys, but not the above simple case. I appreciate any help. Thanks a lot, Hari
On 7/27/05, Wayne Davison <wayned@samba.org> wrote:> On Wed, Jul 27, 2005 at 12:04:22PM -0700, Hari Krishna Dara wrote: > > Now to avoid adding another manual step to my setup process (such as > > manually configuring inetd.conf and rsyncd.conf), I was looking at the > > options to start rsync via remote shell > > The reason the daemon mode avoids hanging is that it is providing a > socket service that rsync can connect to. Trying to get rsync to talk > the daemon protocol via a remote shell will take you back to a failure > mode since rsync will be sending its data via a pipe again (and you > still need an rsyncd.conf file on the remote system to start up a > daemon). What you can do is to use normal remote-shell commands to > setup and take-down a remote rsync daemon: > > rsync -a rsync.conf host:/somedir # or use scp > ssh host rsync --daemon --config=/somedir/rsync.conf > rsync -av /src/ host::module # do the actual transfer > ssh host killall rsync # or perhaps do something safer >I was under the impression that "--daemon --rsh" functionality is meant to simplify the above, ie., rsync will take care of starting up the daemon over remote shell, and will in turn connect to it (and preferably even shut it down when done, though the document didn't say anything). If all that it is supposed to do is startup rsync server over a pipe, then how is it different from the remote shell access? When I explicitly specify the remote path as rsync://, is not supposed to connect to the default rsync port directly? Why would it use a pipe? Starting up rsync daemon separately using ssh is ok, though it would be nice to to have a way to shutdown the daemon from an rsync client. I might just take this route, however, could someone tell me why the "--daemon --rsh" method in my case behaved just like "--daemon"? Thank you, Hari