Simons IT-Service
2007-Feb-14 16:20 UTC
[Xen-users] Poor performance with Fibre-channel in domU and dom0
Hello, I have a server with 4GB of Memory, two Intel QuadCore Xeon CPUs and a LSI FibreChannel 4GBit HBA Card connected to a SAN-Box. When I install this with a normal Linux (CentOS 4.4) with kernel 2.6.9-42.0.3.ELsmp and the mptfc driver, which is deliverd with this kernel, everything works fine. I can copy a 600 MB file from one directory to an other on the same partition in 6 seconds. When I install on the same server a Xen-kernel (2.6.16.29-xen) with the mptfc driver, which is deliverd with this kernel, the I/O performance to the SAN is very,very,very slow. When I copy in a domU a 600 MB file from one directory to an other on the same partition it take 90 seconds. The config file for the disk in my domU is this: disk = [ ''phy:/dev/OracleVG/OraAdmin10gR2,/dev/sda1,w'', ''phy:/dev/OracleVG/OraDumps10gR2,/dev/sda2,w'', ''phy:/dev/OracleVG/OraProduct10gR2,/dev/sda3,w'', ''phy:/dev/OracleVG/OraSoftware10gR2,/dev/sda4,w'', ''phy:/dev/OraDataVG/pdevkh1,/dev/sdb1,w'', ''phy:/dev/OraDataVG/pskh,/dev/sdb2,w'', ''phy:/dev/OraDataVG/temp,/dev/sdb3,w'', ''phy:/dev/sda13,/dev/sdc1,w'' ] The filesystem is ext3. You can see that we would run Oracle 10g R2 in a Xen DomU. We work with datafiles greater then 6GB. Copy from a partiton to an other is ok. Can anybody help us? greetz from Krefeld, Germany Dietmar Simons -- ______________________________________________ Simons IT-Service Dietmar Simons Matthiasstr. 18a 41468 Neuss Germany Phone: 02131 - 153498 Fax: 02131 - 153499 Mobil: 0172 - 253 0523 email: info@simons-it-service.de http://www.simons-it-service.de ______________________________________________ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Tim Post
2007-Feb-15 04:26 UTC
Re: [Xen-users] Poor performance with Fibre-channel in domU and dom0
On Wed, 2007-02-14 at 17:20 +0100, Simons IT-Service wrote:> Hello, > > I have a server with 4GB of Memory, two Intel QuadCore Xeon CPUs and a LSI FibreChannel 4GBit HBA Card connected to a SAN-Box. > > When I install this with a normal Linux (CentOS 4.4) with kernel 2.6.9-42.0.3.ELsmp and the mptfc driver, which is deliverd with this kernel, everything works fine. > > I can copy a 600 MB file from one directory to an other on the same partition in 6 seconds. > > When I install on the same server a Xen-kernel (2.6.16.29-xen) with the mptfc driver, which is deliverd with this kernel, the I/O performance to the SAN is very,very,very slow. > > When I copy in a domU a 600 MB file from one directory to an other on the same partition it take 90 seconds. > > The config file for the disk in my domU is this: > > disk = [ ''phy:/dev/OracleVG/OraAdmin10gR2,/dev/sda1,w'', ''phy:/dev/OracleVG/OraDumps10gR2,/dev/sda2,w'', ''phy:/dev/OracleVG/OraProduct10gR2,/dev/sda3,w'', ''phy:/dev/OracleVG/OraSoftware10gR2,/dev/sda4,w'', ''phy:/dev/OraDataVG/pdevkh1,/dev/sdb1,w'', ''phy:/dev/OraDataVG/pskh,/dev/sdb2,w'', ''phy:/dev/OraDataVG/temp,/dev/sdb3,w'', ''phy:/dev/sda13,/dev/sdc1,w'' ] >I''m not 100% sure its going to *significantly* improve things, but giving the dom-u some swap space may help, unless this one : ''phy:/dev/sda13,/dev/sdc1,w'' is handling swap locally, but it looks like it may be the boot vbd, unless sda1 also contains the root filesystem for the guest? Best, --Tim _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Simons IT-Service
2007-Feb-15 07:02 UTC
Re: [Xen-users] Poor performance with Fibre-channel in domU and dom0
Am Donnerstag, den 15.02.2007, 12:26 +0800 schrieb Tim Post:> On Wed, 2007-02-14 at 17:20 +0100, Simons IT-Service wrote: > > Hello, > > > > I have a server with 4GB of Memory, two Intel QuadCore Xeon CPUs and a LSI FibreChannel 4GBit HBA Card connected to a SAN-Box. > > > > When I install this with a normal Linux (CentOS 4.4) with kernel 2.6.9-42.0.3.ELsmp and the mptfc driver, which is deliverd with this kernel, everything works fine. > > > > I can copy a 600 MB file from one directory to an other on the same partition in 6 seconds. > > > > When I install on the same server a Xen-kernel (2.6.16.29-xen) with the mptfc driver, which is deliverd with this kernel, the I/O performance to the SAN is very,very,very slow. > > > > When I copy in a domU a 600 MB file from one directory to an other on the same partition it take 90 seconds. > > > > The config file for the disk in my domU is this: > > > > disk = [ ''phy:/dev/OracleVG/OraAdmin10gR2,/dev/sda1,w'', ''phy:/dev/OracleVG/OraDumps10gR2,/dev/sda2,w'', ''phy:/dev/OracleVG/OraProduct10gR2,/dev/sda3,w'', ''phy:/dev/OracleVG/OraSoftware10gR2,/dev/sda4,w'', ''phy:/dev/OraDataVG/pdevkh1,/dev/sdb1,w'', ''phy:/dev/OraDataVG/pskh,/dev/sdb2,w'', ''phy:/dev/OraDataVG/temp,/dev/sdb3,w'', ''phy:/dev/sda13,/dev/sdc1,w'' ] > > > > I''m not 100% sure its going to *significantly* improve things, but > giving the dom-u some swap space may help, unless this one : > > ''phy:/dev/sda13,/dev/sdc1,w'' > > is handling swap locally, but it looks like it may be the boot vbd, > unless sda1 also contains the root filesystem for the guest? > > Best, > --Tim >Hello Tim, thanks for your answer. Every DomU in my solution have a 2 GB Swap (''phy:/dev/sda13,/dev/sdc1,w'') for him self. And every Dom0 and DomU boot over NFS, I use openQRM 3.1.2 from www.openqrm.org Any idea? Best Dietmar Simons (Neuss, Germany) =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Scanned with Copfilter Version 0.83beta3a (ProxSMTP 1.4) by Markus Madlener @ http://www.copfilter.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jordi Segues
2007-Feb-15 10:32 UTC
Re: [Xen-users] Poor performance with Fibre-channel in domU and dom0
Hello, Could you specify the exact model of your quad-core xeon cpu please? I''m looking for VT processors well recognized. Is it a Xeon E5300, or Xeon X5300 family? Thanks for your answer, and sorry not to help with your issue. Jordi Segues On 2/15/07, Simons IT-Service <info@simons-it-service.de> wrote:> Am Donnerstag, den 15.02.2007, 12:26 +0800 schrieb Tim Post: > > On Wed, 2007-02-14 at 17:20 +0100, Simons IT-Service wrote: > > > Hello, > > > > > > I have a server with 4GB of Memory, two Intel QuadCore Xeon CPUs and a LSI FibreChannel 4GBit HBA Card connected to a SAN-Box. > > > > > > When I install this with a normal Linux (CentOS 4.4) with kernel 2.6.9-42.0.3.ELsmp and the mptfc driver, which is deliverd with this kernel, everything works fine. > > > > > > I can copy a 600 MB file from one directory to an other on the same partition in 6 seconds. > > > > > > When I install on the same server a Xen-kernel (2.6.16.29-xen) with the mptfc driver, which is deliverd with this kernel, the I/O performance to the SAN is very,very,very slow. > > > > > > When I copy in a domU a 600 MB file from one directory to an other on the same partition it take 90 seconds. > > > > > > The config file for the disk in my domU is this: > > > > > > disk = [ ''phy:/dev/OracleVG/OraAdmin10gR2,/dev/sda1,w'', ''phy:/dev/OracleVG/OraDumps10gR2,/dev/sda2,w'', ''phy:/dev/OracleVG/OraProduct10gR2,/dev/sda3,w'', ''phy:/dev/OracleVG/OraSoftware10gR2,/dev/sda4,w'', ''phy:/dev/OraDataVG/pdevkh1,/dev/sdb1,w'', ''phy:/dev/OraDataVG/pskh,/dev/sdb2,w'', ''phy:/dev/OraDataVG/temp,/dev/sdb3,w'', ''phy:/dev/sda13,/dev/sdc1,w'' ] > > > > > > > I''m not 100% sure its going to *significantly* improve things, but > > giving the dom-u some swap space may help, unless this one : > > > > ''phy:/dev/sda13,/dev/sdc1,w'' > > > > is handling swap locally, but it looks like it may be the boot vbd, > > unless sda1 also contains the root filesystem for the guest? > > > > Best, > > --Tim > > > > Hello Tim, > > thanks for your answer. > > Every DomU in my solution have a 2 GB Swap > (''phy:/dev/sda13,/dev/sdc1,w'') for him self. > > And every Dom0 and DomU boot over NFS, I use openQRM 3.1.2 from > www.openqrm.org > > Any idea? > > Best > Dietmar Simons > (Neuss, Germany) > > > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > Scanned with Copfilter Version 0.83beta3a (ProxSMTP 1.4) > by Markus Madlener @ http://www.copfilter.org > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Tim Post
2007-Feb-15 11:32 UTC
Re: [Xen-users] Poor performance with Fibre-channel in domU and dom0
On Thu, 2007-02-15 at 08:02 +0100, Simons IT-Service wrote:> > > > Hello Tim, > > thanks for your answer. > > Every DomU in my solution have a 2 GB Swap > (''phy:/dev/sda13,/dev/sdc1,w'') for him self. > > And every Dom0 and DomU boot over NFS, I use openQRM 3.1.2 from > www.openqrm.org > > Any idea? >I''ve never quite gotten QRM + the Xen Plugin working ''quite'' right, sounds like you got a little further with it than I did. Although, I was running it as a VM itself. It sounds like the drivers just aren''t working then, or at least I can''t see any other reasons for it. You may want to post this to xen-devel to see if perhaps there are some patches you could apply to the driver.> Best > Dietmar Simons > (Neuss, Germany) > > > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > Scanned with Copfilter Version 0.83beta3a (ProxSMTP 1.4) > by Markus Madlener @ http://www.copfilter.org_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
info@simons-it-service.de
2007-Feb-15 14:26 UTC
Re: [Xen-users] Poor performance with Fibre-channel in domU and dom0
Hello Jordi, the CPU is a Xeon E5310: Output from cat /proc/cpuinfo ---- schnipp ----- model name : Intel(R) Xeon(R) CPU E5310 @ 1.60GHz ---- schnapp ---- Hope, I could help you. Greetz Dietmar Simons Jordi Segues schrieb:> Hello, > > Could you specify the exact model of your quad-core xeon cpu please? > I''m looking for VT processors well recognized. > Is it a Xeon E5300, or Xeon X5300 family? > > Thanks for your answer, and sorry not to help with your issue. > > Jordi Segues > > > On 2/15/07, Simons IT-Service <info@simons-it-service.de> wrote: >> Am Donnerstag, den 15.02.2007, 12:26 +0800 schrieb Tim Post: >> > On Wed, 2007-02-14 at 17:20 +0100, Simons IT-Service wrote: >> > > Hello, >> > > >> > > I have a server with 4GB of Memory, two Intel QuadCore Xeon CPUs >> and a LSI FibreChannel 4GBit HBA Card connected to a SAN-Box. >> > > >> > > When I install this with a normal Linux (CentOS 4.4) with kernel >> 2.6.9-42.0.3.ELsmp and the mptfc driver, which is deliverd with this >> kernel, everything works fine. >> > > >> > > I can copy a 600 MB file from one directory to an other on the >> same partition in 6 seconds. >> > > >> > > When I install on the same server a Xen-kernel (2.6.16.29-xen) >> with the mptfc driver, which is deliverd with this kernel, the I/O >> performance to the SAN is very,very,very slow. >> > > >> > > When I copy in a domU a 600 MB file from one directory to an >> other on the same partition it take 90 seconds. >> > > >> > > The config file for the disk in my domU is this: >> > > >> > > disk = [ ''phy:/dev/OracleVG/OraAdmin10gR2,/dev/sda1,w'', >> ''phy:/dev/OracleVG/OraDumps10gR2,/dev/sda2,w'', >> ''phy:/dev/OracleVG/OraProduct10gR2,/dev/sda3,w'', >> ''phy:/dev/OracleVG/OraSoftware10gR2,/dev/sda4,w'', >> ''phy:/dev/OraDataVG/pdevkh1,/dev/sdb1,w'', >> ''phy:/dev/OraDataVG/pskh,/dev/sdb2,w'', >> ''phy:/dev/OraDataVG/temp,/dev/sdb3,w'', ''phy:/dev/sda13,/dev/sdc1,w'' ] >> > > >> > >> > I''m not 100% sure its going to *significantly* improve things, but >> > giving the dom-u some swap space may help, unless this one : >> > >> > ''phy:/dev/sda13,/dev/sdc1,w'' >> > >> > is handling swap locally, but it looks like it may be the boot vbd, >> > unless sda1 also contains the root filesystem for the guest? >> > >> > Best, >> > --Tim >> > >> >> Hello Tim, >> >> thanks for your answer. >> >> Every DomU in my solution have a 2 GB Swap >> (''phy:/dev/sda13,/dev/sdc1,w'') for him self. >> >> And every Dom0 and DomU boot over NFS, I use openQRM 3.1.2 from >> www.openqrm.org >> >> Any idea? >> >> Best >> Dietmar Simons >> (Neuss, Germany) >> >> >> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- >> Scanned with Copfilter Version 0.83beta3a (ProxSMTP 1.4) >> by Markus Madlener @ http://www.copfilter.org >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xensource.com >> http://lists.xensource.com/xen-users >> > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users