search for: encodek

...The backup server has 48GB RAM, and during the rsyncs about 30GB get used at any given time, but not more than that. The server has 16 3GHz cores and doesn't even break a sweat. All my rsyncs are called via simple bash scripts in the format: #!/bin/bash /usr/bin/rsync -a --delete rsync://encodek-0-4/data/genomes/[q-u]* /hive/data/genomes/ /usr/bin/rsync -a --delete rsync://encodek-0-4/data/genomes/[A-Z]* /hive/data/genomes/ etc... And while many of them work, as time moves on I see some fail with this: rsync error: timeout in data send/receive (code 30) at io.c(200) [sender=3.0.5] rsy...

...are the same on both sides, but there are just so many files (160 million). I don't seem to be out of memory, but I don't know why rsync would go to 100% CPU and just stall. I am rsyncing from an rsync server to my local server, with commands similar to this: rsync -a --delete rsync://encodek-0-4/data/genomes/ /hive/data/genomes/ Again, both sides at version 3.0.5. Nothing fancy or special. I have confirmed that it does count the files incrementally by running a few manually, it does report "getting incremental file list...". Any ideas why the processes go to 100% CPU an...