Hello, I started monitoring lan traffic with RRDTool on a linux box the other day that runs rsync, and I've found what I would consider a strange traffic pattern. This linux box rsync about 2Gb of data to a local samba share that is connected to a Windows 2003 server. Based on the literal data stat, roughly 20% is changed nightly and uploaded. However, the strange thing is that the traffic monitoring shows that there is 3mb/s saturation both ways, upload and download (somehow the server only had a 10mb ethernet adapter). This behavior could not possibly be correct, could it? Or is this normal. If needed I can post some stats and command line options. Thanks, Max
On Thu, Oct 16, 2003 at 01:44:46PM -0500, Max Kipness wrote: [reformatted, please use shorter lines next time]> Hello, > > I started monitoring lan traffic with RRDTool on a linux > box the other day that runs rsync, and I've found what I > would consider a strange traffic pattern. This linux box > rsync about 2Gb of data to a local samba share that is > connected to a Windows 2003 server. Based on the literal > data stat, roughly 20% is changed nightly and uploaded. > However, the strange thing is that the traffic monitoring > shows that there is 3mb/s saturation both ways, upload and > download (somehow the server only had a 10mb ethernet > adapter). > > This behavior could not possibly be correct, could it? Or > is this normal. If needed I can post some stats and > command line options.Why couldn't it? Fast disks and CPU will easily overrun a 10Mb connection for network file service. Don't forget 3Mb/s is only 300KB/s compared to 5-30MB/s for disks. Your description leaves a couple of important things out: version of rsync and how rsync views the transfer with regard to networking. Make sure that rsync is doing a --whole-file transfer. Otherwise it will really thrash the destination. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
> > I started monitoring lan traffic with RRDTool on a linux > > box the other day that runs rsync, and I've found what I would > > consider a strange traffic pattern. This linux box rsync > about 2Gb of > > data to a local samba share that is connected to a Windows 2003 > > server. Based on the literal data stat, roughly 20% is > changed nightly > > and uploaded. However, the strange thing is that the traffic > > monitoring shows that there is 3mb/s saturation both ways, > upload and > > download (somehow the server only had a 10mb ethernet > > adapter). > > > > This behavior could not possibly be correct, could it? Or > > is this normal. If needed I can post some stats and > > command line options. > > Why couldn't it? Fast disks and CPU will easily overrun a > 10Mb connection for network file service. Don't forget > 3Mb/s is only 300KB/s compared to 5-30MB/s for disks.Sorry, I meant to emphasize the point that saturation occurred both ways, inbound and outbound. I can understand the saturation on the upload as it is transfering changed files up to the destination, but what is it downloading that is so large?> Your description leaves a couple of important things out: > version of rsync and how rsync views the transfer with regard > to networking.Rsync version 2.5.6. Both servers are on the same local segment.> Make sure that rsync is doing a --whole-file transfer. > Otherwise it will really thrash the destination.I am not doing --whole-file transfer, but will try that for the session tonight. Thanks, Max
> On Thu, Oct 16, 2003 at 09:38:32PM -0500, Max Kipness wrote: > > > > I started monitoring lan traffic with RRDTool on a > linux box the > > > > other day that runs rsync, and I've found what I would > consider a > > > > strange traffic pattern. This linux box rsync > > > about 2Gb of > > > > data to a local samba share that is connected to a Windows 2003 > > > > server. Based on the literal data stat, roughly 20% is > > > changed nightly > > > > and uploaded. However, the strange thing is that the traffic > > > > monitoring shows that there is 3mb/s saturation both ways, > > > upload and > > > > download (somehow the server only had a 10mb ethernet adapter). > > > > > > > > This behavior could not possibly be correct, could it? > Or is this > > > > normal. If needed I can post some stats and command > line options. > > > > > > Why couldn't it? Fast disks and CPU will easily overrun a 10Mb > > > connection for network file service. Don't forget 3Mb/s is only > > > 300KB/s compared to 5-30MB/s for disks. > > > > Sorry, I meant to emphasize the point that saturation occurred both > > ways, inbound and outbound. I can understand the saturation on the > > upload as it is transfering changed files up to the > destination, but > > what is it downloading that is so large? > > I assume by upload you mean write and download, read. > > stat(), readdir, and if --no-whole-file reading so it can do > checksums followed by reading in order to merge matched > blocks with new. > > > > Your description leaves a couple of important things out: > > > version of rsync and how rsync views the transfer with regard > > > to networking. > > > > Rsync version 2.5.6. Both servers are on the same local segment. > > As rsync views the transfer, not the physical layout. > A local smbfs mount does not look to rsync like a network tranfer. > > > > > > Make sure that rsync is doing a --whole-file transfer. > > > Otherwise it will really thrash the destination. > > > > I am not doing --whole-file transfer, but will try that for the > > session tonight.Just to let you know, including the --whole-file made a huge difference. The upload (write) speed was still at the 3mb level, but the read (download) was now at less than 100k, instead of matching the upload (write) speed. Thanks for the help. Max