I have been using the built in Xen Paravirt NIC drivers for paravirt networking and had bad performance until I disabled all offload setting on the bridge and paravirt nic (From Dom0). There are 2 related questions I have, where should I be tweaking the offload settings, the Dom0 Physical NIC, The Bridge or the DomU NIC? Which settings in particular tend to cause trouble? (I just disabled all of them because this was a server that is in use that I converted to paravirt dirvers) Does anyone have any Ethtool -K recommendations for a Xen, Redhat/Centos, Intel PCI-Express and Intel Legacy PCI drivers? Thanks, Rob The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ SAQ is the trading name of SEMTEC Limited. Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. ISPA Member Find us in http://www.thebestof.co.uk/petersfield _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Apr 24, 2009 at 5:41 PM, Robert Dunkley <Robert@saq.co.uk> wrote:> I have been using the built in Xen Paravirt NIC drivers for paravirt > networking and had bad performance until I disabled all offload setting > on the bridge and paravirt nic (From Dom0).Are they PV guests, or HVM guests with PV drivers?> > There are 2 related questions I have, where should I be tweaking the > offload settings, the Dom0 Physical NIC, The Bridge or the DomU NIC? > Which settings in particular tend to cause trouble? (I just disabled all > of them because this was a server that is in use that I converted to > paravirt dirvers)I''d start with tx and LSO on domU.> > Does anyone have any Ethtool -K recommendations for a Xen, > Redhat/Centos, Intel PCI-Express and Intel Legacy PCI drivers? >It should be "None". If you''re using RHEL and you still have to disable some offload, then you should file a bug to Redhat :) That being said, I''m using RHEL5.3 with Broadcom (tg3 and bnx2) NICs mostly, and they''re working correctly without any need for offload setting change. If you''re still using old version, updating with yum might help. Regards, Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Apr 24, 2009 at 8:13 PM, Robert Dunkley <Robert@saq.co.uk> wrote:> Hi Fajar, > > These are HVM guests with PV Drivers: > DOM0 is Centos 5.2 with Xen 3.31 > DOMU is Centos 5.3 using integrated PV DriversAh, I see. Is there a particular reason to use HVM? Unless you''re using some propietary binary-only kernel module, you''ll have much better results using PV guests. Converting HVM to PV guest is pretty straightforward. I can share the steps if you''re interested.> > The problem seems to be gone with all offload disabled on the bridge and > DomU but rx, tx and scatter enabled on the physical NIC. > > The problem is very noticeable, about 5-10 seconds after turning on an > offload it does not like the SMTP and DNS service on the DOMU begin to > slow. I wonder if this is the Intel driver included with Centos 5.2 that > causes the problem: > filename: > /lib/modules/2.6.18-92.1.6.el5xen/kernel/drivers/net/e1000e/e1000e.ko > version: 0.2.0 > license: GPL > description: Intel(R) PRO/1000 Network Driver > author: Intel Corporation, <linux.nics@intel.com> > > This version looks very old (Latest is 0.5.18.3) but Centos 5.3 seems to > use the same version, I''m not sure how I''m meant to update this driver, > some sources say it requires a kernel rebuild but others like the link > below show it as a simple install: > http://downloadcenter.intel.com/filter_results.aspx?strTypes=all&Product > ID=2248&OSFullName=Linux*&lang=eng&strOSs=39&submit=Go%21I suggest you try it if possible. It''s usually pretty simple (and MUCH easier if you can find a dkms version). On early RHEL 5.0 series I had to recompile tg3 modules manually since RHEL''s driver was having problems. The result was well worthed the effort.> > Can you think of any reason why the included Redhat driver is so old?I don''t know. Possible licensing issue, or that the new version is not tested enough. Good luck. Regards, Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I have the same exact questions and problems. F.Alves -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Robert Dunkley Sent: Friday, April 24, 2009 6:42 AM To: xen-users@lists.xensource.com Subject: [Xen-users] NIC Offloading Confusion I have been using the built in Xen Paravirt NIC drivers for paravirt networking and had bad performance until I disabled all offload setting on the bridge and paravirt nic (From Dom0). There are 2 related questions I have, where should I be tweaking the offload settings, the Dom0 Physical NIC, The Bridge or the DomU NIC? Which settings in particular tend to cause trouble? (I just disabled all of them because this was a server that is in use that I converted to paravirt dirvers) Does anyone have any Ethtool -K recommendations for a Xen, Redhat/Centos, Intel PCI-Express and Intel Legacy PCI drivers? Thanks, Rob The SAQ Group Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ SAQ is the trading name of SEMTEC Limited. Registered in England & Wales Company Number: 06481952 http://www.saqnet.co.uk AS29219 SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. ISPA Member Find us in http://www.thebestof.co.uk/petersfield _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Vie 24 Abr 2009 09:26:37 Fajar A. Nugraha wrote:> On Fri, Apr 24, 2009 at 8:13 PM, Robert Dunkley <Robert@saq.co.uk> wrote: > > Hi Fajar, > > > > These are HVM guests with PV Drivers: > > DOM0 is Centos 5.2 with Xen 3.31 > > DOMU is Centos 5.3 using integrated PV Drivers > > Ah, I see. > Is there a particular reason to use HVM? > Unless you''re using some propietary binary-only kernel module, you''ll > have much better results using PV guests. Converting HVM to PV guest > is pretty straightforward. I can share the steps if you''re interested.I''m interested in the steps to convert a HVM to PV guest. Thanks> > > The problem seems to be gone with all offload disabled on the bridge and > > DomU but rx, tx and scatter enabled on the physical NIC. > > > > The problem is very noticeable, about 5-10 seconds after turning on an > > offload it does not like the SMTP and DNS service on the DOMU begin to > > slow. I wonder if this is the Intel driver included with Centos 5.2 that > > causes the problem: > > filename: > > /lib/modules/2.6.18-92.1.6.el5xen/kernel/drivers/net/e1000e/e1000e.ko > > version: 0.2.0 > > license: GPL > > description: Intel(R) PRO/1000 Network Driver > > author: Intel Corporation, <linux.nics@intel.com> > > > > This version looks very old (Latest is 0.5.18.3) but Centos 5.3 seems to > > use the same version, I''m not sure how I''m meant to update this driver, > > some sources say it requires a kernel rebuild but others like the link > > below show it as a simple install: > > http://downloadcenter.intel.com/filter_results.aspx?strTypes=all&Product > > ID=2248&OSFullName=Linux*&lang=eng&strOSs=39&submit=Go%21 > > I suggest you try it if possible. It''s usually pretty simple (and MUCH > easier if you can find a dkms version). > > On early RHEL 5.0 series I had to recompile tg3 modules manually since > RHEL''s driver was having problems. The result was well worthed the > effort. > > > Can you think of any reason why the included Redhat driver is so old? > > I don''t know. Possible licensing issue, or that the new version is not > tested enough. > > Good luck. > > Regards, > > Fajar > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Jorge Armando Medina Computación Gráfica de México Web: www.e-compugraf.com Tel: 55 51 40 72 email: jmedina@e-compugraf.com GPG Key: 1024D/28E40632 2007-07-26 GPG Fingerprint: 59E2 0C7C F128 B550 B3A6 D3AF C574 8422 28E4 0632 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sat, Apr 25, 2009 at 1:34 AM, Jorge Armando Medina <jmedina@e-compugraf.com> wrote:> On Vie 24 Abr 2009 09:26:37 Fajar A. Nugraha wrote: >> Is there a particular reason to use HVM? >> Unless you''re using some propietary binary-only kernel module, you''ll >> have much better results using PV guests. Converting HVM to PV guest >> is pretty straightforward. I can share the steps if you''re interested. > > I''m interested in the steps to convert a HVM to PV guest. > > ThanksFirst of all, since apparently a fresh-install of centos HVM domU loads PV drivers but does NOT actually use them, you should read this first : Part 1. Creating a Centos HVM domU with working PV drivers : http://pastebin.com/fb6fe631 Then you can read Part 2, converting HVM guest to PV guest : http://pastebin.com/f6a5022bf If you follow both parts correctly you should have a working PV domU. If anything goes wrong during conversion process, you should still be able to boot the previous HVM domU config if you select the non-xen kernel (second entry) from grub menu.list. Regards, Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
My machine has Suse 11 and 16 VM''s with Red hat 5.3.under heavy load the dom0 freezes, and the VM''s of course are inaccessible. How do I troubleshoot this? I enabled kdump, but don''t know how to trigger a dump, since PRTSCR only triggers a screen capture. Is there anything that I can do to find out what is failing? Novell found that I has packet loss, but that does not make a system freeze. How do I capture a memory dump if the machine is frozen? F.Alves _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Maybe you can use netconsole... http://www.cyberciti.biz/tips/linux-netconsole-log-management-tutorial.html Just my 2 cents! 2009/4/24 Venefax <venefax@gmail.com>> My machine has Suse 11 and 16 VM''s with Red hat 5.3.under heavy load the > dom0 freezes, and the VM''s of course are inaccessible. How do I > troubleshoot > this? I enabled kdump, but don''t know how to trigger a dump, since PRTSCR > only triggers a screen capture. Is there anything that I can do to find out > what is failing? Novell found that I has packet loss, but that does not > make > a system freeze. How do I capture a memory dump if the machine is frozen? > F.Alves > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
"Venefax" <venefax@gmail.com> writes:> My machine has Suse 11 and 16 VM''s with Red hat 5.3.under heavy load the > dom0 freezes, and the VM''s of course are inaccessible. How do I troubleshoot > this? I enabled kdump, but don''t know how to trigger a dump, since PRTSCR > only triggers a screen capture. Is there anything that I can do to find out > what is failing? Novell found that I has packet loss, but that does not make > a system freeze. How do I capture a memory dump if the machine is frozen?does it unfreze? or does it stay frozen? if it unfrezes, try this command in the dom0 (I put it in /etc/rc.local in the dom0) xm sched-credit -d 0 60000 (you can do the same thing by dedicating a core to the dom0) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I had two cores out of 16 dedicated to dom0, and 6GB of RAM out of 128. It froze solid. I had to recycle the box. It died about 5 mins after the heavy load started. Is there a simple to setup load simulator that can make me reproduce it? I cannot call my clients again to send calls (this is a telephony switch). I would like to capture the console over the network, but from a PC, since I don''t have another linux box available. -----Original Message----- From: Luke S Crawford [mailto:lsc@prgmr.com] Sent: Friday, April 24, 2009 7:59 PM To: Venefax Cc: ''Fajar A. Nugraha''; ''Xen List'' Subject: Re: [Xen-users] Frozen Dom0 "Venefax" <venefax@gmail.com> writes:> My machine has Suse 11 and 16 VM''s with Red hat 5.3.under heavy load the > dom0 freezes, and the VM''s of course are inaccessible. How do Itroubleshoot> this? I enabled kdump, but don''t know how to trigger a dump, since PRTSCR > only triggers a screen capture. Is there anything that I can do to findout> what is failing? Novell found that I has packet loss, but that does notmake> a system freeze. How do I capture a memory dump if the machine is frozen?does it unfreze? or does it stay frozen? if it unfrezes, try this command in the dom0 (I put it in /etc/rc.local in the dom0) xm sched-credit -d 0 60000 (you can do the same thing by dedicating a core to the dom0) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
"Venefax" <venefax@gmail.com> writes:> I had two cores out of 16 dedicated to dom0, and 6GB of RAM out of 128. It > froze solid. I had to recycle the box. It died about 5 mins after the heavy > load started. Is there a simple to setup load simulator that can make me > reproduce it? I cannot call my clients again to send calls (this is a > telephony switch). I would like to capture the console over the network, but > from a PC, since I don''t have another linux box available.do you have a windows box you could hook a null modem cable to? a serial console would really be best for figuring out what is going on. Someone else suggested netconsole, which might also work (though I have not personally used it with xen) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Apr 24, 2009 at 11:41:46AM +0100, Robert Dunkley wrote:> I have been using the built in Xen Paravirt NIC drivers for paravirt > networking and had bad performance until I disabled all offload setting > on the bridge and paravirt nic (From Dom0). > > There are 2 related questions I have, where should I be tweaking the > offload settings, the Dom0 Physical NIC, The Bridge or the DomU NIC? > Which settings in particular tend to cause trouble? (I just disabled all > of them because this was a server that is in use that I converted to > paravirt dirvers) > > Does anyone have any Ethtool -K recommendations for a Xen, > Redhat/Centos, Intel PCI-Express and Intel Legacy PCI drivers? >I can''t remember having to disable any offloading settings for ages.. Running RHEL5.x or CentOS5.x on both dom0 and (PV) domU. -- Pasi _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Vie 24 Abr 2009 18:40:29 Fajar A. Nugraha wrote:> On Sat, Apr 25, 2009 at 1:34 AM, Jorge Armando Medina > > <jmedina@e-compugraf.com> wrote: > > On Vie 24 Abr 2009 09:26:37 Fajar A. Nugraha wrote: > >> Is there a particular reason to use HVM? > >> Unless you''re using some propietary binary-only kernel module, you''ll > >> have much better results using PV guests. Converting HVM to PV guest > >> is pretty straightforward. I can share the steps if you''re interested. > > > > I''m interested in the steps to convert a HVM to PV guest. > > > > Thanks > > First of all, since apparently a fresh-install of centos HVM domU > loads PV drivers but does NOT actually use them, you should read this > first : >Thank you I will try the instructions.> Part 1. Creating a Centos HVM domU with working PV drivers : > http://pastebin.com/fb6fe631 > > Then you can read > > Part 2, converting HVM guest to PV guest : http://pastebin.com/f6a5022bf > > If you follow both parts correctly you should have a working PV domU. > If anything goes wrong during conversion process, you should still be > able to boot the previous HVM domU config if you select the non-xen > kernel (second entry) from grub menu.list. > > Regards, > > Fajar > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Jorge Armando Medina Computación Gráfica de México Web: www.e-compugraf.com Tel: 55 51 40 72 email: jmedina@e-compugraf.com GPG Key: 1024D/28E40632 2007-07-26 GPG Fingerprint: 59E2 0C7C F128 B550 B3A6 D3AF C574 8422 28E4 0632 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users