we have just upgraded to xen4.0+kernel2.6.31.13. however, we fonud
that disk offline in high disk IO(xen4.0+kernel2.6.31.13). but whitout
xen4.0, we only start with kernel2.6.31.13, the disk offline didn''t
happen.
I think it is a bug.
test case:
57 physical machine
we start 5 vm(cpu:2, memory:1024) in a physical machine.at the same
time we do decompression in dom0.
tar xzvf *.tar.gz
tar czvf *.tar.gz *.img
xen4.0+kernel2.6.31.13:
result : disk offline in nearly 50% physical machines(28/57).
kernel2.6.31.13:
result : disk offline didn''t happen(0/57)
our env :
compile env and filesystem :
Redhat AS 5.4
kernel:
pvops_2.6.31.13
hardware ( lspci ):
00:00.0 Host bridge: Intel Corporation 5500 I/O Hub to ESI Port (rev 22)
00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 1 (rev 22)
00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 3 (rev 22)
00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 7 (rev 22)
00:09.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI
Express Root Port 9 (rev 22)
00:10.0 PIC: Intel Corporation 5520/5500/X58 Physical and Link Layer
Registers Port 0 (rev 22)
00:10.1 PIC: Intel Corporation 5520/5500/X58 Routing and Protocol
Layer Registers Port 0 (rev 22)
00:11.0 PIC: Intel Corporation 5520/5500 Physical and Link Layer
Registers Port 1 (rev 22)
00:11.1 PIC: Intel Corporation 5520/5500 Routing & Protocol Layer
Register Port 1 (rev 22)
00:13.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub I/OxAPIC
Interrupt Controller (rev 22)
00:14.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management
Registers (rev 22)
00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O Hub GPIO and Scratch
Pad Registers (rev 22)
00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status
and RAS Registers (rev 22)
00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev 22)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #4
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2
EHCI Controller #2
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2
EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port
SATA IDE Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port
SATA IDE Controller
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
Gigabit Ethernet (rev 20)
01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
Gigabit Ethernet (rev 20)
04:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E
PCI-Express Fusion-MPT SAS (rev 08)
05:00.0 VGA compatible controller: XGI Technology Inc. (eXtreme
Graphics Innovation) Z9s/Z9m (XG21 core)
xm info :
host : r02k08010
release : 2.6.31.13debug
version : #26 SMP Mon Apr 26 16:34:22 CST 2010
machine : x86_64
nr_cpus : 8
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2266
hw_caps :
bfebfbff:28100800:00000000:00001b40:009ce3bd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 24544
free_memory : 1668
node_to_cpu : node0:0-7
node_to_memory : node0:1668
node_to_dma32_mem : node0:839
max_node_id : 0
xen_major : 4
xen_minor : 0
xen_extra : .0
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline :
cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)
cc_compile_by : root
cc_compile_domain :
cc_compile_date : Tue Apr 13 23:04:20 CST 2010
xend_config_format : 4
BTW, the attachment is the dom0 kernel compile config.
Cheers,
wyb
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Apr-26 20:34 UTC
Re: [Xen-devel] disk offline with xen4.0+pvops2.6.31.13
On 04/26/2010 06:42 AM, yingbin wang wrote:> we have just upgraded to xen4.0+kernel2.6.31.13. however, we fonud > that disk offline in high disk IO(xen4.0+kernel2.6.31.13). but whitout > xen4.0, we only start with kernel2.6.31.13, the disk offline didn''t > happen. > I think it is a bug. >What is the last version of Xen which did work. What do you mean by "disk offline"? Are there some specific messages on the kernel console? Thanks, J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
---------- Forwarded message ---------- From: tsk <aixt2006@gmail.com> Date: 2010/4/27 Subject: Re: [Xen-devel] disk offline with xen4.0+pvops2.6.31.13 To: Jeremy Fitzhardinge <jeremy@goop.org> Hi, I''m a friend of yingbin wang, we take some pictures for the disk offline, in the attachment. We did many stress test, the result is: disk offline disappears when there is no xen hypervision in 2.6.31.13, when we use RHEL 5.x kernel+Xen, disk offline disappears too. It seems that redhat should add some patch to solve this problem.. disk offline appears on: linux-2.6.18.8 + Xen-3.4.2 linux-2.6.31.13 + Xen-4.0.0 linux-2.6.32.10 + Xen-3.4.2 Fedora linux-2.6.32 + Xen-4.0.0 Suse linux-2.6.32.11 + Xen-3.4.2 The servers tested above are all with Nehalem architecture CPU. Thanks Tsk 2010/4/27 Jeremy Fitzhardinge <jeremy@goop.org> On 04/26/2010 06:42 AM, yingbin wang wrote:> > we have just upgraded to xen4.0+kernel2.6.31.13. however, we fonud > > that disk offline in high disk IO(xen4.0+kernel2.6.31.13). but whitout > > xen4.0, we only start with kernel2.6.31.13, the disk offline didn''t > > happen. > > I think it is a bug. > > > > What is the last version of Xen which did work. > > What do you mean by "disk offline"? Are there some specific messages on > the kernel console? > > Thanks, > J > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-Apr-27 19:32 UTC
Re: [Xen-devel] disk offline with xen4.0+pvops2.6.31.13
On 04/27/2010 01:10 AM, tsk wrote:> Hi, I''m a friend of yingbin wang, > > we take some pictures for the disk offline, in the attachment. > > We did many stress test, the result is: > disk offline disappears when there is no xen hypervision in 2.6.31.13, > when we use RHEL 5.x kernel+Xen, disk offline disappears too. It seems > that redhat should add some patch to solve this problem.. > > disk offline appears on: > linux-2.6.18.8 + Xen-3.4.2 > linux-2.6.31.13 + Xen-4.0.0 > linux-2.6.32.10 + Xen-3.4.2 > Fedora linux-2.6.32 + Xen-4.0.0 > Suse linux-2.6.32.11 + Xen-3.4.2 > > The servers tested above are all with Nehalem architecture CPU.So you''re saying that any recent Xen with any kernel is failing on fairly recent hardware. Have you tried playing with the iommu boot command line options? Can you post a complete log of Xen''s console output (both hypervisor and kernel console)? What is the hardware platform? Am I right in guessing that you''re seeing this on two distinct machines? Thanks, J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
It seem that we have got a mitigation when enable the mpt_msi_enable_sas option of mptbase module. But not sure if it solve the problem. "So you''re saying that any recent Xen with any kernel is failing on fairly recent hardware. Have you tried playing with the iommu boot command line options?" We have not tried the iommu option. "Can you post a complete log of Xen''s console output (both hypervisor and kernel console)?" It seem that if we do uncompressing stress test in Dom0, offline will appear. "What is the hardware platform? Am I right in guessing that you''re seeing this on two distinct machines?" the detail hardware info of huawei machines is: root@r02k05040 # lspci 00:00.0 Host bridge: Intel Corporation 5500 I/O Hub to ESI Port (rev 13) 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13) 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13) 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13) 00:09.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 9 (rev 13) 00:10.0 PIC: Intel Corporation 5520/5500/X58 Physical and Link Layer Registers Port 0 (rev 13) 00:10.1 PIC: Intel Corporation 5520/5500/X58 Routing and Protocol Layer Registers Port 0 (rev 13) 00:11.0 PIC: Intel Corporation 5520/5500 Physical and Link Layer Registers Port 1 (rev 13) 00:11.1 PIC: Intel Corporation 5520/5500 Routing & Protocol Layer Register Port 1 (rev 13) 00:13.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub I/OxAPIC Interrupt Controller (rev 13) 00:14.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management Registers (rev 13) 00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13) 00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13) 00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev 13) 00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) 00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller 00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) 04:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08) 05:00.0 VGA compatible controller: XGI Technology Inc. (eXtreme Graphics Innovation) Z9s/Z9m (XG21 core) 2010/4/28 Jeremy Fitzhardinge <jeremy@goop.org>> On 04/27/2010 01:10 AM, tsk wrote: > > Hi, I''m a friend of yingbin wang, > > > > we take some pictures for the disk offline, in the attachment. > > > > We did many stress test, the result is: > > disk offline disappears when there is no xen hypervision in 2.6.31.13, > > when we use RHEL 5.x kernel+Xen, disk offline disappears too. It seems > > that redhat should add some patch to solve this problem.. > > > > disk offline appears on: > > linux-2.6.18.8 + Xen-3.4.2 > > linux-2.6.31.13 + Xen-4.0.0 > > linux-2.6.32.10 + Xen-3.4.2 > > Fedora linux-2.6.32 + Xen-4.0.0 > > Suse linux-2.6.32.11 + Xen-3.4.2 > > > > The servers tested above are all with Nehalem architecture CPU. > > So you''re saying that any recent Xen with any kernel is failing on > fairly recent hardware. Have you tried playing with the iommu boot > command line options? > > Can you post a complete log of Xen''s console output (both hypervisor and > kernel console)? > > What is the hardware platform? Am I right in guessing that you''re > seeing this on two distinct machines? > > Thanks, > J >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
yingbin wang
2010-Apr-28 13:27 UTC
Re: [Xen-devel] disk offline with xen4.0+pvops2.6.31.13
we found some info from xen wiki: Enabling MSI/MSI-X for Assigned Devices As of Xen 3.4.0 MSI/MSI-X is always on. This change was made in changeset: 18454:65dc37be0443 In Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter "msi=1". This change was made in changeset: 18127:89d05940cc1c The following snippet from a grub configuration enables MSI/MSI-X kernel xen.gz msi=1 does it mean that we should also Enable MSI Support for SAS in mptbase module? 2010/4/28 tsk <aixt2006@gmail.com>:> It seem that we have got a mitigation when enable the mpt_msi_enable_sas > option of mptbase module. But not sure if it solve the problem. > "So you''re saying that any recent Xen with any kernel is failing on > fairly recent hardware. Have you tried playing with the iommu boot > command line options?" > We have not tried the iommu option. > > "Can you post a complete log of Xen''s console output (both hypervisor and > kernel console)?" > It seem that if we do uncompressing stress test in Dom0, offline will > appear. > "What is the hardware platform? Am I right in guessing that you''re > seeing this on two distinct machines?" > the detail hardware info of huawei machines is: > root@r02k05040 # lspci > 00:00.0 Host bridge: Intel Corporation 5500 I/O Hub to ESI Port (rev 13) > 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root > Port 1 (rev 13) > 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root > Port 3 (rev 13) > 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root > Port 7 (rev 13) > 00:09.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root > Port 9 (rev 13) > 00:10.0 PIC: Intel Corporation 5520/5500/X58 Physical and Link Layer > Registers Port 0 (rev 13) > 00:10.1 PIC: Intel Corporation 5520/5500/X58 Routing and Protocol Layer > Registers Port 0 (rev 13) > 00:11.0 PIC: Intel Corporation 5520/5500 Physical and Link Layer Registers > Port 1 (rev 13) > 00:11.1 PIC: Intel Corporation 5520/5500 Routing & Protocol Layer Register > Port 1 (rev 13) > 00:13.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub I/OxAPIC Interrupt > Controller (rev 13) > 00:14.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management > Registers (rev 13) > 00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O Hub GPIO and Scratch Pad > Registers (rev 13) > 00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status and RAS > Registers (rev 13) > 00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev > 13) > 00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI > Controller #4 > 00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI > Controller #2 > 00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI > Controller #1 > 00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI > Controller #2 > 00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI > Controller #3 > 00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI > Controller #1 > 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) > 00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface > Controller > 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA > IDE Controller > 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller > 00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA > IDE Controller > 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 > Gigabit Ethernet (rev 20) > 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 > Gigabit Ethernet (rev 20) > 04:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E > PCI-Express Fusion-MPT SAS (rev 08) > 05:00.0 VGA compatible controller: XGI Technology Inc. (eXtreme Graphics > Innovation) Z9s/Z9m (XG21 core) > > > > 2010/4/28 Jeremy Fitzhardinge <jeremy@goop.org> >> >> On 04/27/2010 01:10 AM, tsk wrote: >> > Hi, I''m a friend of yingbin wang, >> > >> > we take some pictures for the disk offline, in the attachment. >> > >> > We did many stress test, the result is: >> > disk offline disappears when there is no xen hypervision in 2.6.31.13, >> > when we use RHEL 5.x kernel+Xen, disk offline disappears too. It seems >> > that redhat should add some patch to solve this problem.. >> > >> > disk offline appears on: >> > linux-2.6.18.8 + Xen-3.4.2 >> > linux-2.6.31.13 + Xen-4.0.0 >> > linux-2.6.32.10 + Xen-3.4.2 >> > Fedora linux-2.6.32 + Xen-4.0.0 >> > Suse linux-2.6.32.11 + Xen-3.4.2 >> > >> > The servers tested above are all with Nehalem architecture CPU. >> >> So you''re saying that any recent Xen with any kernel is failing on >> fairly recent hardware. Have you tried playing with the iommu boot >> command line options? >> >> Can you post a complete log of Xen''s console output (both hypervisor and >> kernel console)? >> >> What is the hardware platform? Am I right in guessing that you''re >> seeing this on two distinct machines? >> >> Thanks, >> J > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Apr-28 13:50 UTC
Re: [Xen-devel] disk offline with xen4.0+pvops2.6.31.13
On Wed, Apr 28, 2010 at 09:27:30PM +0800, yingbin wang wrote:> we found some info from xen wiki: > > Enabling MSI/MSI-X for Assigned Devices > As of Xen 3.4.0 MSI/MSI-X is always on. > This change was made in changeset: 18454:65dc37be0443 > In Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter "msi=1". > This change was made in changeset: 18127:89d05940cc1c > The following snippet from a grub configuration enables MSI/MSI-X > kernel xen.gz msi=1 > > does it mean that we should also Enable MSI Support for SAS in mptbase module?First try the iommu option (iommu=off for example) and also the x2apic=off Do include your serial log so we can see what kind of error you are getting. And also include you /proc/interrupts output please. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Zhang, Xiantao
2010-Apr-29 02:56 UTC
RE: [Xen-devel] disk offline with xen4.0+pvops2.6.31.13
Xen hypervisor has the MSI support from Xen 3.3 or newer version, but if you want to use MSI for your device, you have to check its MSI capability first(e.g. lspci -vvv). For the device you are using, you have to enable its MSI through a kernel module parameter because its MSI is set to off by default. Xiantao yingbin wang wrote:> we found some info from xen wiki: > > Enabling MSI/MSI-X for Assigned Devices > As of Xen 3.4.0 MSI/MSI-X is always on. > This change was made in changeset: 18454:65dc37be0443 > In Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter > "msi=1". This change was made in changeset: 18127:89d05940cc1c > The following snippet from a grub configuration enables MSI/MSI-X > kernel xen.gz msi=1 > > does it mean that we should also Enable MSI Support for SAS in > mptbase module? > > 2010/4/28 tsk <aixt2006@gmail.com>: >> It seem that we have got a mitigation when enable the >> mpt_msi_enable_sas option of mptbase module. But not sure if it >> solve the problem. "So you''re saying that any recent Xen with any >> kernel is failing on >> fairly recent hardware. Have you tried playing with the iommu boot >> command line options?" >> We have not tried the iommu option. >> >> "Can you post a complete log of Xen''s console output (both >> hypervisor and kernel console)?" It seem that if we do uncompressing >> stress test in Dom0, offline will appear. "What is the hardware >> platform? Am I right in guessing that you''re >> seeing this on two distinct machines?" >> the detail hardware info of huawei machines is: >> root@r02k05040 # lspci >> 00:00.0 Host bridge: Intel Corporation 5500 I/O Hub to ESI Port (rev >> 13) 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI >> Express Root Port 1 (rev 13) 00:03.0 PCI bridge: Intel Corporation >> 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13) 00:07.0 PCI >> bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root >> Port 7 (rev 13) 00:09.0 PCI bridge: Intel Corporation 5520/5500/X58 >> I/O Hub PCI Express Root Port 9 (rev 13) 00:10.0 PIC: Intel >> Corporation 5520/5500/X58 Physical and Link Layer Registers Port 0 >> (rev 13) 00:10.1 PIC: Intel Corporation 5520/5500/X58 Routing and >> Protocol Layer Registers Port 0 (rev 13) 00:11.0 PIC: Intel >> Corporation 5520/5500 Physical and Link Layer Registers Port 1 (rev >> 13) 00:11.1 PIC: Intel Corporation 5520/5500 Routing & Protocol >> Layer Register Port 1 (rev 13) 00:13.0 PIC: Intel Corporation >> 5520/5500/X58 I/O Hub I/OxAPIC Interrupt Controller (rev 13) 00:14.0 >> PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management >> Registers (rev 13) 00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O >> Hub GPIO and Scratch Pad Registers (rev 13) 00:14.2 PIC: Intel >> Corporation 5520/5500/X58 I/O Hub Control Status and RAS Registers >> (rev 13) 00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub >> Throttle Registers (rev 13) 00:1a.0 USB Controller: Intel >> Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 00:1a.7 >> USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI >> Controller #2 00:1d.0 USB Controller: Intel Corporation 82801JI >> (ICH10 Family) USB UHCI Controller #1 00:1d.1 USB Controller: Intel >> Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 00:1d.2 >> USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI >> Controller #3 00:1d.7 USB Controller: Intel Corporation 82801JI >> (ICH10 Family) USB2 EHCI Controller #1 00:1e.0 PCI bridge: Intel >> Corporation 82801 PCI Bridge (rev 90) 00:1f.0 ISA bridge: Intel >> Corporation 82801JIB (ICH10) LPC Interface Controller 00:1f.2 IDE >> interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE >> Controller 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) >> SMBus Controller 00:1f.5 IDE interface: Intel Corporation 82801JI >> (ICH10 Family) 2 port SATA IDE Controller 01:00.0 Ethernet >> controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit >> Ethernet (rev 20) 01:00.1 Ethernet controller: Broadcom Corporation >> NetXtreme II BCM5709 Gigabit Ethernet (rev 20) 04:00.0 SCSI storage >> controller: LSI Logic / Symbios Logic SAS1068E PCI-Express >> Fusion-MPT SAS (rev 08) 05:00.0 VGA compatible controller: XGI >> Technology Inc. (eXtreme Graphics Innovation) Z9s/Z9m (XG21 core) >> >> >> >> 2010/4/28 Jeremy Fitzhardinge <jeremy@goop.org> >>> >>> On 04/27/2010 01:10 AM, tsk wrote: >>>> Hi, I''m a friend of yingbin wang, >>>> >>>> we take some pictures for the disk offline, in the attachment. >>>> >>>> We did many stress test, the result is: >>>> disk offline disappears when there is no xen hypervision in >>>> 2.6.31.13, when we use RHEL 5.x kernel+Xen, disk offline >>>> disappears too. It seems that redhat should add some patch to >>>> solve this problem.. >>>> >>>> disk offline appears on: >>>> linux-2.6.18.8 + Xen-3.4.2 >>>> linux-2.6.31.13 + Xen-4.0.0 >>>> linux-2.6.32.10 + Xen-3.4.2 >>>> Fedora linux-2.6.32 + Xen-4.0.0 >>>> Suse linux-2.6.32.11 + Xen-3.4.2 >>>> >>>> The servers tested above are all with Nehalem architecture CPU. >>> >>> So you''re saying that any recent Xen with any kernel is failing on >>> fairly recent hardware. Have you tried playing with the iommu boot >>> command line options? >>> >>> Can you post a complete log of Xen''s console output (both >>> hypervisor and kernel console)? >>> >>> What is the hardware platform? Am I right in guessing that you''re >>> seeing this on two distinct machines? >>> >>> Thanks, >>> J >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
OK, we will try these two options.
After enable MSI option, disk offline still appear, the attachment is the
Call Trace.
The content of /proc/interrupts is:
CPU0 CPU1
1: 2 0 xen-pirq-ioapic-edge i8042
4: 9 0 xen-pirq-ioapic-edge serial
8: 0 0 xen-pirq-ioapic-edge rtc0
9: 0 0 xen-pirq-ioapic-level acpi
12: 4 0 xen-pirq-ioapic-edge i8042
16: 34 0 xen-pirq-ioapic-level uhci_hcd:usb3
18: 2 0 xen-pirq-ioapic-level ehci_hcd:usb1,
uhci_hcd:usb6
19: 0 0 xen-pirq-ioapic-level uhci_hcd:usb5, ata_piix,
ata_piix
23: 0 0 xen-pirq-ioapic-level ehci_hcd:usb2,
uhci_hcd:usb4
482: 0 0 xen-dyn-event evtchn:xenstored
483: 47 272 xen-dyn-event evtchn:xenstored
490: 2957623 2181 xen-pirq-msi-x peth0-2
491: 3239519 2907 xen-pirq-msi-x peth0-1
492: 6678288 0 xen-pirq-msi-x peth0-0
493: 134441 80 xen-pirq-msi ioc0
495: 0 0 xen-dyn-virq mce
500: 0 0 xen-dyn-virq pcpu
501: 72 344 xen-dyn-event xenbus
502: 0 43586 xen-dyn-ipi callfuncsingle1
503: 0 0 xen-dyn-virq debug1
504: 0 0 xen-dyn-ipi callfunc1
505: 0 468438 xen-dyn-ipi resched1
506: 0 148914934 xen-dyn-virq timer1
507: 41098 0 xen-dyn-ipi callfuncsingle0
508: 0 0 xen-dyn-virq debug0
509: 0 0 xen-dyn-ipi callfunc0
510: 502029 0 xen-dyn-ipi resched0
511: 148977732 0 xen-dyn-virq timer0
NMI: 0 0 Non-maskable interrupts
LOC: 0 0 Local timer interrupts
SPU: 0 0 Spurious interrupts
CNT: 0 0 Performance counter interrupts
PND: 0 0 Performance pending work
RES: 502029 468438 Rescheduling interrupts
CAL: 41098 43586 Function call interrupts
TLB: 0 0 TLB shootdowns
TRM: 0 0 Thermal event interrupts
THR: 0 0 Threshold APIC interrupts
MCE: 0 0 Machine check exceptions
MCP: 497 497 Machine check polls
ERR: 0
MIS: 0
Thanks.
Tsk
2010/4/28 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> On Wed, Apr 28, 2010 at 09:27:30PM +0800, yingbin wang wrote:
> > we found some info from xen wiki:
> >
> > Enabling MSI/MSI-X for Assigned Devices
> > As of Xen 3.4.0 MSI/MSI-X is always on.
> > This change was made in changeset: 18454:65dc37be0443
> > In Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter
> "msi=1".
> > This change was made in changeset: 18127:89d05940cc1c
> > The following snippet from a grub configuration enables
MSI/MSI-X
> > kernel xen.gz msi=1
> >
> > does it mean that we should also Enable MSI Support for SAS in mptbase
> module?
>
> First try the iommu option (iommu=off for example) and also the x2apic=off
>
> Do include your serial log so we can see what kind of error you are
> getting. And also include you /proc/interrupts output please.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2010-May-03 16:54 UTC
Re: [Xen-devel] disk offline with xen4.0+pvops2.6.31.13
On 05/03/2010 01:07 AM, tsk wrote:> OK, we will try these two options. > > After enable MSI option, disk offline still appear, the attachment is > the Call Trace.Did you try disabling iommu? J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel