Hi all, Allow me to quickly sketch our backup situation: we have about 40 sites. On each site there's a Rsync client on an NT machine (Cygwin). In the main site we have a huge Win2K fileserver which acts as a Rsync server. I'm doing tests now with syncing the volumes of several Netware servers on the sites (to which the Rsync NT clients have a mapping) to the Win2K server. Initially we wanted to put a Rsync server on each site, but for some unknown reason the Rsync daemon does not want to use mappings to a Netware volume in its modules. Right now I'm doing about 10 syncs at the same time. What I'm seeing now, is that for every Rsync session that's initiated towards the Rsync service on the Win2K server, a seperate Rsync.exe is spawned. Each Rsync uses up to 20% CPU util. Obviously, the machine is constantly running at 100% CPU. Not desireable, since I want to add yet another 30 sites! Do you people think that if we would use a Linux server instead of a Win2K server, the CPU would be less burdoned? Thx already! Bart Coninckx Network Administrator CNE, ASE ************************************* Watco ICT Services Lilsedijk 19 B-2340 Beerse e-mail: bart.coninckx@watco.be Tel: + 32 (0) 14 60 99 42 Fax: + 32 (0) 14 62 41 47 ************************************* ========================== Disclaimer =================================The information in this email is confidential, and is intended solely for the addressee(s). If you are not the intended recipient of this email please let us know by reply and then delete it from your system; you should not copy this message or disclose its contents to anyone, not even by forwarding it. Due to the integrity risk of sending emails over the Internet, Watco ICT will accept no liability for any comments and/or attachments contained within this email. ========================== Disclaimer ==================================
On Fri, Aug 23, 2002 at 08:31:35PM +0200, bart.coninckx@watco.be wrote:> Hi all, > > Allow me to quickly sketch our backup situation: we have about 40 sites. On > each site there's a Rsync client on an NT machine (Cygwin). In the main > site we have a huge Win2K fileserver which acts as a Rsync server. I'm > doing tests now with syncing the volumes of several Netware servers on the > sites (to which the Rsync NT clients have a mapping) to the Win2K server. > Initially we wanted to put a Rsync server on each site, but for some > unknown reason the Rsync daemon does not want to use mappings to a Netware > volume in its modules. > > Right now I'm doing about 10 syncs at the same time. What I'm seeing now, > is that for every Rsync session that's initiated towards the Rsync service > on the Win2K server, a seperate Rsync.exe is spawned. Each Rsync uses up to > 20% CPU util. Obviously, the machine is constantly running at 100% CPU. Not > desireable, since I want to add yet another 30 sites! > > Do you people think that if we would use a Linux server instead of a Win2K > server, the CPU would be less burdoned?There are so many factors i couldn't even start. It is possible changing OS for the server would help as linux has historically been more efficient at disk I/O in the general case but as i haven't done comparisons and have no NT or W2K servers to even try. I recall there being a patch to preserve ACLs in NT-NT rsync so there may be a functional benefit to sticking with an NT server (yuck!). Some things that would help in evaluation would be the rsync command-line and the disk configuration (are you running RAID, what level and HW vs. SW, etc), approximate file counts and size, and it also might help if we knew if the overloaded server were low powered. As a start i would check to be sure that you aren't using the --checksum option unnecessarily. That was apparently the problem the last time someone reported excessive load. Also, don't run software RAID-5. The other thing that would be worth doing is to manage the load better by reducing the number of simultaneous connections.> ========================== Disclaimer =================================> The information in this email is confidential, and is intended solely for > the addressee(s). If you are not the intended recipient of this email > please let us know by reply and then delete it from your system; > you should not copy this message or disclose its contents to anyone, > not even by forwarding it. > Due to the integrity risk of sending emails over the Internet, > Watco ICT will accept no liability for any comments and/or attachments > contained within this email. > ========================== Disclaimer =================================Leave off the disclaimer. When you post to the list there is no confidentiality and it wastes electrons :) -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
>There are so many factors i couldn't even start. It is >possible changing OS for the server would help as linux has >historically been more efficient at disk I/O in the general >case but as i haven't done comparisons and have no NT or >W2K servers to even try. I recall there being a patch to >preserve ACLs in NT-NT rsync so there may be a functional >benefit to sticking with an NT server (yuck!).ACLs are not important to us. Mind you, we sync Netware data and permissions there aren't synced anyway. BTW: my first choice would be Linux as well, but I'm the only one able to support it, so there's get to be a real functional motive to "upgrade" ;-)>Some things that would help in evaluation would be the >rsync command-line and the disk configuration (are you running >RAID, what level and HW vs. SW, etc), approximate file counts >and size, and it also might help if we knew if the overloaded >server were low powered.this is the command line of the server: rsync --daemon --config=rsyncd.conf client: rsync -rtv --delete --modify-window=2 --stats /cygdrive/r/ bee2bs01::d/backup/servername/data/ We're using hardware RAID 5.>As a start i would check to be sure that you aren't using >the --checksum option unnecessarily.Nope, we use the default...>That was apparently the >problem the last time someone reported excessive load. >Also, don't run software RAID-5.No, we indeed prefer hardware RAID. We use a SCSI to ATA RAID configuration (a device with a SCSI connector, but with IDE disks in it), but this should not influence CPU performance, since the machine sees natice SCSI.>The other thing that would be worth doing is to manage the >load better by reducing the number of simultaneous >connections.True, but we have a limited timeframe (about 10 hours), to sync everything. It could well be that Rsync manages to do an avarage site in less than 2 hours, but we don't know yet. There are a lot of variables there: what is changed in data on a particular day on a particular site, how fast is the WAN link, ... Since the clients are scattered, it's also hard to orchestrate a steady scheme. This would have been a lot easier in a pull configuration, but as I mentionned, for some reason the Rsync service (daemon) won't work with Netware mappings.>Leave off the disclaimer. When you post to the list there >is no confidentiality and it wastes electrons :)Sorry, auto signature, you know how it goes ... Thx for the reply! Bart