We are using rsync to mirror around 10 of our websites from dedicated server A to dedicated server B runing RHEL 3.0 and rysnc v2.6.3pre1 in a daemon mode. I am using the following crontab script command on dedicated server A: /usr/local/bin/rsync -avz -e "ssh -i /home/users/admin/publicKey-for-server-B" /home/serverA/site3 serverB::home/mirror Dedicated server B (where the mirroring of websites from server A is done) is sometimes crashing with the following error message from the linux kernel: Aug 29 22:47:02 ns2 kernel: eth0: Memory squeeze,deferring packet. Aug 29 22:47:23 ns2 kernel: eth0: NULL pointer encountered in Rx ring, skipping The following is the error message from rsync log in dedicated server B: 2004/08/29 22:45:02 [18367] rsync allowed access on module home from ns.serverA.co.uk 2004/08/29 22:45:02 [18367] rsync to home/mirror from ns.serverA.co.uk 2004/08/29 22:45:51 [18367] site3/logs/ 2004/08/30 00:58:15 [18367] rsync: connection unexpectedly closed (355347 bytes received so far) [receiver] 2004/08/30 00:58:15 [18367] rsync: writefd_unbuffered failed to write 84 bytes: phase "unknown" [generator]: Broken pipe (32) 2004/08/30 00:58:15 [18367] rsync error: error in rsync protocol data stream (code 12) at io.c(909) I've 10 separate crontab rsync scripts for mirroring the 10 websites. I've allowed enough time intervalls so that the each crontab rsync process finishes before the next one starts. Occasionally, however (about once a week) rsync crashes dedicated server B during any one of these 10 crontab processes. I would be very grateful for your help. regards, Daniel. Daniel Berhane bmj.com web administrator The BMJ Publishing Group Limited BMA House, Tavistock Square, London WC1H 9JR http://bmj.com Tel: +44 (0)20 7383 6362 Fax: +44 (0)20 7383 6997
On Tue 31 Aug 2004, Daniel Berhane wrote:> We are using rsync to mirror around 10 of our websites from dedicated > server A to dedicated server B runing RHEL 3.0 and rysnc v2.6.3pre1 in a > daemon mode. I am using the following crontab script command on dedicated > server A: > > /usr/local/bin/rsync -avz -e "ssh -i > /home/users/admin/publicKey-for-server-B" /home/serverA/site3 > serverB::home/mirror > > > Dedicated server B (where the mirroring of websites from server A is done) > is sometimes crashing with the following error message from the linux > kernel: > > Aug 29 22:47:02 ns2 kernel: eth0: Memory squeeze,deferring packet. > Aug 29 22:47:23 ns2 kernel: eth0: NULL pointer encountered in Rx ring, > skippingSounds like you're running out of memory (and swap as well). I'd suggest adding memory (adding more swap doesn't make it faster...) or rsyncing smaller bits at a time if possible (if there are many hardlinks across directory trees it may not be feasible to do parts separately as those hardlinks will be lost, for example). Paul Slootman
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1> I would be very grateful for your help. > > regards, > Daniel.Hi Daniel, This isn't a direct answer to your problem, but I have just done a basic perl script for rdiff-backup which could be changed for rsync (I will be doing this soon anyway), which writes a log file and sends you an e-mail on fail or complete. If you would like it, it is here: http://www.perl.me.uk/downloads. I would easily changes bits for you, as I want to get the ssh passphrase goig in it anyway. Gavin. - -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1467 624141 M +44 (0) 7930 323266 F +44 (0) 1224 742001 E ghenry@suretecsystems.com Open Source. Open Solutions. http://www.suretecsystems.com/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBNGgyeWseh9tzvqgRAkmRAJ0dmXpwqX1640Ez0ugLnT1ZQw1hcQCfenjc Wo0QAI4HPO2RJT1GL56b1Zs=C7aG -----END PGP SIGNATURE-----
Daniel Berhane/BMJ wrote on 31/08/2004 12:49:34:> Currently, the dedicated server has 1GBytes of RAM. I have done a > quick test and found out that it is using only 20% of the RAM while > the rsync crontab scripts are running. > > Thank you, > Daniel. > > rsync-bounces+dberhane=bmj.com@lists.samba.org wrote on 31/08/200412:43:17:> > > On Tue 31 Aug 2004, Daniel Berhane wrote: > > > > > We are using rsync to mirror around 10 of our websites fromdedicated> > > server A to dedicated server B runing RHEL 3.0 and rysnc v2.6.3pre1in a> > > daemon mode. I am using the following crontab script command ondedicated> > > server A: > > > > > > /usr/local/bin/rsync -avz -e "ssh -i > > > /home/users/admin/publicKey-for-server-B" /home/serverA/site3 > > > serverB::home/mirror > > > > > > > > > Dedicated server B (where the mirroring of websites from server > A is done) > > > is sometimes crashing with the following error message from thelinux> > > kernel: > > > > > > Aug 29 22:47:02 ns2 kernel: eth0: Memory squeeze,deferring packet. > > > Aug 29 22:47:23 ns2 kernel: eth0: NULL pointer encountered in Rxring,> > > skipping > > > > Sounds like you're running out of memory (and swap as well). > > I'd suggest adding memory (adding more swap doesn't make it faster...) > > or rsyncing smaller bits at a time if possible (if there are many > > hardlinks across directory trees it may not be feasible to do parts > > separately as those hardlinks will be lost, for example). > > > > > > Paul Slootman > > -- > > To unsubscribe or change options: http://lists.samba. > > org/mailman/listinfo/rsync > > Before posting, read:http://www.catb.org/~esr/faqs/smart-questions.html> > ForwardSourceID:NT00023BA6 > ForwardSourceID:NT00023BAA
Congratulations, you've found a weakness in the ethernet driver. Check up2date, as there may be a kernel update for that. You could also maybe plug in a different type of ethernet card. The kernel is explaining that there's a problem at that level, and your rsync log shows what happens when that problem comes up. Don't worry about adding memory (or swap), as you're likely to be using only a few hundred MB when this happens. Tim Conway Unix System Administration Contractor - IBM Global Services desk:3032734776 conway@us.ibm.com We are using rsync to mirror around 10 of our websites from dedicated server A to dedicated server B runing RHEL 3.0 and rysnc v2.6.3pre1 in a daemon mode. I am using the following crontab script command on dedicated server A: /usr/local/bin/rsync -avz -e "ssh -i /home/users/admin/publicKey-for-server-B" /home/serverA/site3 serverB::home/mirror Dedicated server B (where the mirroring of websites from server A is done) is sometimes crashing with the following error message from the linux kernel: Aug 29 22:47:02 ns2 kernel: eth0: Memory squeeze,deferring packet. Aug 29 22:47:23 ns2 kernel: eth0: NULL pointer encountered in Rx ring, skipping The following is the error message from rsync log in dedicated server B: 2004/08/29 22:45:02 [18367] rsync allowed access on module home from ns.serverA.co.uk 2004/08/29 22:45:02 [18367] rsync to home/mirror from ns.serverA.co.uk 2004/08/29 22:45:51 [18367] site3/logs/ 2004/08/30 00:58:15 [18367] rsync: connection unexpectedly closed (355347 bytes received so far) [receiver] 2004/08/30 00:58:15 [18367] rsync: writefd_unbuffered failed to write 84 bytes: phase "unknown" [generator]: Broken pipe (32) 2004/08/30 00:58:15 [18367] rsync error: error in rsync protocol data stream (code 12) at io.c(909) I've 10 separate crontab rsync scripts for mirroring the 10 websites. I've allowed enough time intervalls so that the each crontab rsync process finishes before the next one starts. Occasionally, however (about once a week) rsync crashes dedicated server B during any one of these 10 crontab processes. I would be very grateful for your help. regards, Daniel. Daniel Berhane bmj.com web administrator The BMJ Publishing Group Limited BMA House, Tavistock Square, London WC1H 9JR http://bmj.com Tel: +44 (0)20 7383 6362 Fax: +44 (0)20 7383 6997 -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html