Hi All, I have a production server that's running our corporate mail (Cyrus-IMAP). I used to use cpio to archive the mail store, but that's just getting silly now that there's about 40GB that it needs to run through. I tried running rsync in order to backup the data, but as soon as I started running the script the machine came to a grinding halt. I left it running for about 30 minutes while rsync said it was analyzing the file system but the server was un-usable and I was forced to kill it. Is there some way to slow rsync down? I don?t mind if the initial sync takes a day to run, I just dont want it to kill the machine. I was given a suggestion to: Use the -W flag to stop it comparing files byte for byte, don't use checksums and add --progress so you can see what's happening. Which seemed to help, but the machine is still pretty useless when rsync runs. Does anyone have any other ideas? Thanks Warrick
On Mon, Sep 26, 2005 at 02:17:40PM -0400, Warrick FitzGerald wrote:> Is there some way to slow rsync down?It all depends on what resource rsync is consuming on your system. If it does not have a lot of CPU, you should be able to use "nice" to make rsync use only left-over CPU cycles. Also, if you're using ssh as the remote shell, compressing the transfer takes a bunch of CPU too, so you could switch cyphers to a less-CPU intensive one (such as arcfour or blowfish) and/or nice ssh (or even switch over to using rsh or a daemon connection). If your system does not have a lot of disk I/O, then you might try using the --bwlimit option to make rsync use less network I/O, and thus make it consume disk I/O more slowly (as long as you don't use --checksum). ..wayne..
On Mon 26 Sep 2005, Warrick FitzGerald wrote:> > I used to use cpio to archive the mail store, but that's just getting > silly now that there's about 40GB that it needs to run through. > > I tried running rsync in order to backup the data, but as soon as I > started running the script the machine came to a grinding halt. I left > it running for about 30 minutes while rsync said it was analyzing the > file system but the server was un-usable and I was forced to kill it.How many files are involved? If the number is very high, the memory usage of rsync will cause the machine to run out of memory. 40GB of mail would suggest a lot of little files... In that case, see if it's possible to run rsync on subdirectories of the tree (hardlinks outside such a subdir is a showstopper for that approach, assuming you're using -H). Paul Slootman