Hi all I hope no one minds but I was asked to post my timetrial findings back to the list. Hope it helps someone else as well, if you have any suggestions please mention them as I need all the speed that I can get The scenaro is as follows I have to switch mail servers and I need to copy all my mbox files over to the new machine. As you may well know time if off the utmost importance so I need to do things as quickly as possible, we are an ISP so we cannot have people offline for to long. I have from 12:00 to 5:00 in the morning I conducted some tests to see what will be the most effecient to copy the data over with and tried to simulate the enviroment as closely as possible but on a smaller scale. Here are my results Smaller test(not whole directory) All test were done with the same set of files that I put in a special directory for the test there are 4593 files in the dir with a total size of 3.3G All tests were done from the same two machines in the same direction,I just changed the options It was done on two Gigabit nic's with a crosover cable so network usage and speed were effectively not a factor. I also tested tar but that took to long scp failed misserably I didn't test ftp at all as I cannot see that it will be faster time rsync -pogrve 'ssh -c arcfour' /var/spool/mqueue/testwind root@10.0.0.9:/var/spool/mail wrote 3429040714 bytes read 73492 bytes 5287762.85 bytes/sec total size is 3428390423 speedup is 1.00 real 10m48.081s user 3m1.530s sys 1m56.540s ------------------------------------------------------------------------- time rsync -pogrve ssh /var/spool/mqueue/testwind root@10.0.0.9:/var/spool/mail wrote 3429040714 bytes read 73492 bytes 3820740.06 bytes/sec total size is 3428390423 speedup is 1.00 real 14m56.454s user 4m58.250s sys 2m5.180s [root@ais-mail01 spool]# ------------------------------------------------------------------------- [root@ais-mail01 spool]# scp /var/spool/mqueue/testwind/* 10.0.0.9:/var/spool/mail bash: /usr/bin/scp: Argument list too long You have new mail in /var/spool/mail/stefaans [root@ais-mail01 spool]# ------------------------------------------------------------------------- time tar -cvzf sendmail.tgz * real 34m13.435s user 14m17.790s sys 2m15.140s ------------------------------------------------------------------------- time rsync -arve ssh /var/spool/mqueue/testwind root@10.0.0.9:/var/spool/mail wrote 3429040714 bytes read 73492 bytes 4410436.28 bytes/sec total size is 3428390423 speedup is 1.00 real 12m57.454s user 4m56.800s sys 1m56.860s ------------------------------------------------------------------------- time rsync -arve 'ssh -c arcfour' /var/spool/mqueue/testwind root@10.0.0.9:/var/spool/mail real 9m51.771s user 2m55.240s sys 1m52.290s [root@ais-mail01 spool]# Mozzi -- _ ( ) Ascii ribbon campaign against html email X / \
Today, Mozzi wrote:> All tests were done from the same two machines in the same direction,I > just changed the options > It was done on two Gigabit nic's with a crosover cable so network usage > and speed were effectively > not a factor.This is not true ! A Gigabit card itself never uses jumbo packets if you don't tell them to use it. With jumbo packets it should be possible to have almost 50-70% more throughput !> scp failed misserablyNo, it was the shell that failed, you used globbing with the *. Try it with . and you can see, that it works. Even here it's recommended to user -c arcfour (as proven in the rest of your results that it's the fastest encryption at this time). Mermgfurt, Udo -- Udo Wolter | /"\ email: uwp@dicke-aersche.de | \ / ASCII RIBBON CAMPAIGN www: www.dicke-aersche.de | X AGAINST HTML MAIL dark: heaven@lutz-ziffer.de | / \
On Wed, Nov 13, 2002 at 01:27:46PM +0200, Mozzi wrote:> Hi all > > I hope no one minds but I was asked to post my timetrial findings back > to the list. > Hope it helps someone else as well, if you have any suggestions please > mention them as I need all the speed that I can get > > The scenaro is as follows > I have to switch mail servers and I need to copy all my mbox files over > to the new machine. > As you may well know time if off the utmost importance so I need to do > things as quickly as possible, we are an ISP so we cannot have people > offline for to long. > I have from 12:00 to 5:00 in the morning > I conducted some tests to see what will be the most effecient to copy > the data over with and tried to simulate the enviroment as closely as > possible but on a smaller scale. > Here are my results > > Smaller test(not whole directory) > All test were done with the same set of files that I put in a special > directory for the test > there are 4593 files in the dir with a total size of 3.3G > All tests were done from the same two machines in the same direction,I > just changed the options > It was done on two Gigabit nic's with a crosover cable so network usage > and speed were effectively > not a factor. > I also tested tar but that took to long > scp failed misserably > I didn't test ftp at all as I cannot see that it will be fasterStop using * Aside from the time the shell will be spending sorting the list, argument processing is generally inefficient compared to directory walking. Also, runing tar and writing to the same disk, controller or even bus will bear no relationship to the network. Better to do tar cf - .|rsh dest 'cd destdir; tar xf -' The verbose option will kill performance, particularly if it goes to a CRT.> > > time rsync -pogrve 'ssh -c arcfour' /var/spool/mqueue/testwind > root@10.0.0.9:/var/spool/mail > > wrote 3429040714 bytes read 73492 bytes 5287762.85 bytes/sec > total size is 3428390423 speedup is 1.00 > > real 10m48.081s > user 3m1.530s > sys 1m56.540s > > ------------------------------------------------------------------------- > > time rsync -pogrve ssh /var/spool/mqueue/testwind > root@10.0.0.9:/var/spool/mail > > wrote 3429040714 bytes read 73492 bytes 3820740.06 bytes/sec > total size is 3428390423 speedup is 1.00 > > real 14m56.454s > user 4m58.250s > sys 2m5.180s > [root@ais-mail01 spool]# > ------------------------------------------------------------------------- > > [root@ais-mail01 spool]# scp /var/spool/mqueue/testwind/* > 10.0.0.9:/var/spool/mail > bash: /usr/bin/scp: Argument list too long > You have new mail in /var/spool/mail/stefaans > [root@ais-mail01 spool]# > > ------------------------------------------------------------------------- > > time tar -cvzf sendmail.tgz * > > real 34m13.435s > user 14m17.790s > sys 2m15.140s > > ------------------------------------------------------------------------- > time rsync -arve ssh /var/spool/mqueue/testwind > root@10.0.0.9:/var/spool/mail > > wrote 3429040714 bytes read 73492 bytes 4410436.28 bytes/sec > total size is 3428390423 speedup is 1.00 > > real 12m57.454s > user 4m56.800s > sys 1m56.860s > > ------------------------------------------------------------------------- > > time rsync -arve 'ssh -c arcfour' /var/spool/mqueue/testwind > root@10.0.0.9:/var/spool/mail > > real 9m51.771s > user 2m55.240s > sys 1m52.290s > [root@ais-mail01 spool]# > > Mozzi > > -- > _ > ( ) Ascii ribbon campaign against html email > X > / \ > > -- > To unsubscribe or change options: > http://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html-- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
On Wed, Nov 13, 2002 at 01:27:46PM +0200, Mozzi wrote:> Hi all > > I hope no one minds but I was asked to post my timetrial findings back > to the list. > Hope it helps someone else as well, if you have any suggestions please > mention them as I need all the speed that I can get > > The scenaro is as follows > I have to switch mail servers and I need to copy all my mbox files over > to the new machine. > As you may well know time if off the utmost importance so I need to do > things as quickly as possible, we are an ISP so we cannot have people > offline for to long. > I have from 12:00 to 5:00 in the morning > I conducted some tests to see what will be the most effecient to copy > the data over with and tried to simulate the enviroment as closely as > possible but on a smaller scale. > Here are my results[...] In all the tests you are doing a "full copy", so all you are excercising is the various methods directory walking and data transfer. I'd be interested to see if actualy using the rsync delta for this would improve things. Try the following; 1) rsync the data over from the "live system" 2) start the stopwatch... now beginneth the downtime 3) rsync the data over again 4) stop the stopwatch... now endeth the down time :-) Idea being you minimize the transfer by getting a rough copy over first without bringing the live system down, then just "update" it during the downtime. I suspect with gigabit ethernet the "sacrifice CPU for bandwidth" philosopy of rsync will mean this is actually worse than the raw copies, but certainly for anything slower it should make an improvement. -- ---------------------------------------------------------------------- ABO: finger abo@minkirri.apana.org.au for more info, including pgp key ----------------------------------------------------------------------
> Hi all > > I hope no one minds but I was asked to post my timetrial findings back > to the list. > Hope it helps someone else as well, if you have any suggestions please > mention them as I need all the speed that I can get > > The scenaro is as follows > I have to switch mail servers and I need to copy all my mbox files over > to the new machine. > As you may well know time if off the utmost importance so I need to do > things as quickly as possible, we are an ISP so we cannot have people > offline for to long. > I have from 12:00 to 5:00 in the morning > I conducted some tests to see what will be the most effecient to copy > the data over with and tried to simulate the enviroment as closely as > possible but on a smaller scale. > Here are my resultsWe recently moved things to a new mailserver as well. It was at the other end of an 8Meg wireless link with 6Megs already in use, so we couldn't zap the mailspools (5GB) over in 10-20 minutes like you. I redirected incomning mail to the new server, shut down POP3 on the old one, forwarded pop3 request with simpleproxy from the old pop server to the new one. Then transferred the files with rsync (with the z option in my case) to the new server into a temporary directory. Since we changed from /var/spool/mail/username to /home/username/Mailbox, I wrote a little script that appended with cat the username named mailbox to /home/username/Mailbox. Then when the cutover time came, I switched hostnames around and shut down sendmail and pop3 on the old server. So during this transition (which I did during the day and evening!), people could not access old delivered email, but could stil send and receive new email messages.> > Smaller test(not whole directory) > All test were done with the same set of files that I put in a special > directory for the test > there are 4593 files in the dir with a total size of 3.3G > All tests were done from the same two machines in the same direction,I > just changed the options > It was done on two Gigabit nic's with a crosover cable so network usage > and speed were effectively > not a factor. > I also tested tar but that took to long > scp failed misserably > I didn't test ftp at all as I cannot see that it will be faster > > > time rsync -pogrve 'ssh -c arcfour' /var/spool/mqueue/testwind > root@10.0.0.9:/var/spool/mail > > wrote 3429040714 bytes read 73492 bytes 5287762.85 bytes/sec > total size is 3428390423 speedup is 1.00 > > real 10m48.081s > user 3m1.530s > sys 1m56.540s > > ------------------------------------------------------------------------- > > time rsync -pogrve ssh /var/spool/mqueue/testwind > root@10.0.0.9:/var/spool/mail > > wrote 3429040714 bytes read 73492 bytes 3820740.06 bytes/sec > total size is 3428390423 speedup is 1.00 > > real 14m56.454s > user 4m58.250s > sys 2m5.180s > [root@ais-mail01 spool]# > ------------------------------------------------------------------------- > > [root@ais-mail01 spool]# scp /var/spool/mqueue/testwind/* > 10.0.0.9:/var/spool/mail > bash: /usr/bin/scp: Argument list too long > You have new mail in /var/spool/mail/stefaans > [root@ais-mail01 spool]# > > ------------------------------------------------------------------------- > > time tar -cvzf sendmail.tgz * > > real 34m13.435s > user 14m17.790s > sys 2m15.140s > > ------------------------------------------------------------------------- > time rsync -arve ssh /var/spool/mqueue/testwind > root@10.0.0.9:/var/spool/mail > > wrote 3429040714 bytes read 73492 bytes 4410436.28 bytes/sec > total size is 3428390423 speedup is 1.00 > > real 12m57.454s > user 4m56.800s > sys 1m56.860s > > ------------------------------------------------------------------------- > > time rsync -arve 'ssh -c arcfour' /var/spool/mqueue/testwind > root@10.0.0.9:/var/spool/mail > > real 9m51.771s > user 2m55.240s > sys 1m52.290s > [root@ais-mail01 spool]# > > Mozzi > > -- > _ > ( ) Ascii ribbon campaign against html email > X > / \ > > -- > To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html >