I'm just looking for some advice on a project I am currently working on. I have two OS X servers and I am looking to use rsync to mirror data between the two. I just love the fact that rsync only moves the data incrementally. It is very efficient and since I'm more of a Linux user and I feel comfortable with rsync. I'm just worried that the way the Apple File Sharing service handles files from the OS 9 machines is causing rsync to hang while rsync is trying to do it's diff on the files. Server A has data being placed on the server constantly, from OS 9 clients, and then those files are to be moved to server B. The script is running on server B and is not being run from cron. It kicks off rsync and then a script to change permissions of the files so clients have access to them. After both have run the script sleeps for 120 seconds. Plus the script is run as root. Unfortunately it seems rsync catches files being copied from the OS 9 clients to the OS X server and hangs. I'm wondering if rsync doesn't see any file locking by the Apple Fileserver? Has anyone else run into this? Any Suggestions? Do know of a more efficient solution? I'm all ears! Thanks in Advance, Adam
On Wed, Jul 30, 2003 at 11:22:58AM -0500, Adam Behn wrote:> I'm just looking for some advice on a project I am currently working on. > > I have two OS X servers and I am looking to use rsync to mirror data between > the two. I just love the fact that rsync only moves the data incrementally. > It is very efficient and since I'm more of a Linux user and I feel > comfortable with rsync. I'm just worried that the way the Apple File Sharing > service handles files from the OS 9 machines is causing rsync to hang while > rsync is trying to do it's diff on the files. > Server A has data being placed on the server constantly, from OS 9 clients, > and then those files are to be moved to server B. The script is running on > server B and is not being run from cron. It kicks off rsync and then a > script to change permissions of the files so clients have access to them. > After both have run the script sleeps for 120 seconds. Plus the script is > run as root. > Unfortunately it seems rsync catches files being copied from the OS 9 > clients to the OS X server and hangs. I'm wondering if rsync doesn't see any > file locking by the Apple Fileserver? > > Has anyone else run into this? Any Suggestions? Do know of a more efficient > solution?No one else has reported it. Rsync has no concept of file locking. In *ix file locking is advisory so this isn't a problem. The only time a process notices a file lock is when it tries to lock the file itself; something rsync never does. Windows has implicit mandatory locks but all they do is cause open to fail so that rsync skips (with errors) open files. If OSX has implemented mandatory locks that cause blocking on open() or read() rsync would indeed hang until the lock is released. In such a case there should be a flag that can be set on the open() call to either ignore the locks or fail. Either would be an improvement on blocking. I'd suggest you search the OSX resources regarding this. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
What filesystem are you using? If you are using HFS or HFS+ I suggest you give UFS a try because it all the filesystem semantics are what unix tools expect. While I haven't seen your specific problem, I am sure there are quite a few situations where rsync and other unix tools will be confused on HFS. Additionally, HFS and HFS+ store resource forks that rsync (and scp, amanda, tar, etc) won't see -- therefore your files are likely to be incomplete after being transferred with rsync or any other unix-level utility. If you are using UFS, then the resource forks will be transferred ok, because MacOSX will store them as hidden files instead of resource forks (or data streams, as they are known in the Win32 world). regards, martin Adam Behn wrote:>I'm just looking for some advice on a project I am currently working on. > >I have two OS X servers and I am looking to use rsync to mirror data between >the two. I just love the fact that rsync only moves the data incrementally. >
I am using HFS but the files are being zipped up using stuff-it(MAC winzip/tar.gz) So forks aren't a problem since I don't need the metadata from the forked file when it is from a stuffed/zipped file. Unfortunatly the server in question is already online. It works most the time, but we do have to kill rsync an awful lot. I will see if I can find a test server to try UFS on. Maybe UFS will play better for some reason. Thanks for all the good suggestions so far, any other suggestions are welcome since it will be a couple of days before I can scrounge this testbed together. And please don't say use linux... I've tried to tell the owner that :) hehe Adam -----Original Message----- From: Martin Langhoff Sent: Wed 7/30/2003 6:59 PM To: Adam Behn Cc: rsync@lists.samba.org Subject: Re: Mirroring OS X folders with Rsync What filesystem are you using? If you are using HFS or HFS+ I suggest you give UFS a try because it all the filesystem semantics are what unix tools expect. While I haven't seen your specific problem, I am sure there are quite a few situations where rsync and other unix tools will be confused on HFS. Additionally, HFS and HFS+ store resource forks that rsync (and scp, amanda, tar, etc) won't see -- therefore your files are likely to be incomplete after being transferred with rsync or any other unix-level utility. If you are using UFS, then the resource forks will be transferred ok, because MacOSX will store them as hidden files instead of resource forks (or data streams, as they are known in the Win32 world). regards, martin Adam Behn wrote:>I'm just looking for some advice on a project I am currently workingon.> >I have two OS X servers and I am looking to use rsync to mirror databetween>the two. I just love the fact that rsync only moves the dataincrementally.>-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 3382 bytes Desc: not available Url : http://lists.samba.org/archive/rsync/attachments/20030730/ddf0dff2/attachment.bin