This issue was brought up about a year and a half ago. So I''ll bring it up again and see if anything happens. I''ve got an infiniband network and am attempting to pass the infiniband card through the host and give it to the guest. I''m working with standard CentOS 5.2 on both guest and host with their provided xen (3.0.3 ish). I''ve also attempted to install the newest Xen 3.3 and use their standard host kernel and that did the same thing. The guest dmesg output in the guest is similar on both permissive and normal mode. I''m getting issues with detecting the firmware on the card for some reason... Any help would be appreciated. Thanks, - David Brown === GUEST dmesg output ==ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 ====================== === Host modprobe.conf ==alias eth0 bnx2 alias eth1 bnx2 alias scsi_hostadapter cciss options pciback hide=(41:00.0) ==================== === Host lspci output ==# lspci -vs 41:00.0 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ==================== This makes sure it get loaded first off before anything else. === Host mkinitrd cmd ==# mkinitrd -f --with=pciback --preload pciback /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen =================== === Host pciback dmesg ==pciback 0000:41:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: 1) see permissive attribute in sysfs 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ===================== === Host pciback dmesg (after setting it permissive) ==pciback 0000:41:00.0: enabling permissive mode configuration space accesses! pciback 0000:41:00.0: permissive mode is potentially unsafe! pciback: vpci: 0000:41:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ======================================== === Guest lspci output ==# lspci -v 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ==================== _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
subbu kl
2009-Feb-11 14:18 UTC
[Xen-devel] Re: [ofa-general] Fwd: pciback module not working
I am getting the same QUERY_FW failed on RHEL5.2 with xenxen paravirtualized guest with pciback module. No one seems to have tried answering this question on the list, let me ping xen-devel and ofed people again. after executing in dom0 echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind #dmesg ACPI: PCI interrupt for device 0000:0e:00.0 disabled tap tap-1-51712: 2 getting info tap tap-2-51712: 2 getting info pciback 0000:0e:00.0: seizing device PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 ACPI: PCI interrupt for device 0000:0e:00.0 disabled #xm create -c rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. GUEST dmesg: ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 in dom0: Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not ready Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 disabled some more details - [root@p128 ~]# rpm -qa | grep xen kernel-xen-2.6.18-92.1.22.el5 xen-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 [root@p128 ~]# ibv_devinfo hca_id: mthca0 fw_ver: 5.3.0 node_guid: 0002:c902:0022:cd48 sys_image_guid: 0002:c902:0022:cd4b vendor_id: 0x02c9 vendor_part_id: 25218 hw_ver: 0x20 board_id: MT_0370130002 phys_port_cnt: 2 port: 1 state: PORT_INIT (2) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 any help greatly appreciated. ~subbu On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com> wrote:> Okay so my question to the openfabrics guys is, why would the OFED > drivers fail to read the firmware? > > Any thoughts? > > Thanks, > - David Brown > > > ---------- Forwarded message ---------- > From: David Brown <dmlb2000@gmail.com> > Date: Thu, Sep 11, 2008 at 2:24 PM > Subject: pciback module not working > To: xen-users@lists.xensource.com, xen-devel@lists.xensource.com > > > This issue was brought up about a year and a half ago. So I''ll bring > it up again and see if anything happens. > > I''ve got an infiniband network and am attempting to pass the > infiniband card through the host and give it to the guest. > I''m working with standard CentOS 5.2 on both guest and host with their > provided xen (3.0.3 ish). I''ve also attempted to install the newest > Xen 3.3 and use their standard host kernel and that did the same > thing. The guest dmesg output in the guest is similar on both > permissive and normal mode. > > I''m getting issues with detecting the firmware on the card for some > reason... > > Any help would be appreciated. > > Thanks, > - David Brown > > === GUEST dmesg output ==> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) > ib_mthca: Initializing 0000:00:00.0 > PCI: Enabling device 0000:00:00.0 (0000 -> 0002) > PCI: Setting latency timer of device 0000:00:00.0 to 64 > ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. > ib_mthca: probe of 0000:00:00.0 failed with error -11 > ======================> > === Host modprobe.conf ==> alias eth0 bnx2 > alias eth1 bnx2 > alias scsi_hostadapter cciss > options pciback hide=(41:00.0) > ====================> > === Host lspci output ==> # lspci -vs 41:00.0 > 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx > HCA] (rev 20) > Subsystem: Hewlett-Packard Company Unknown device 170a > Flags: fast devsel, IRQ 16 > Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] > Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] > Capabilities: [40] Power Management version 2 > Capabilities: [48] Vital Product Data > Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 > Enable- > Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 > Capabilities: [60] Express Endpoint IRQ 0 > ====================> > This makes sure it get loaded first off before anything else. > === Host mkinitrd cmd ==> # mkinitrd -f --with=pciback --preload pciback > /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen > ===================> > === Host pciback dmesg ==> pciback 0000:41:00.0: Driver tried to write to a read-only > configuration space field at offset 0x44, size 2. This may be > harmless, but if you have problems with your device: > 1) see permissive attribute in sysfs > 2) report problems to the xen-devel mailing list along with details of > your device obtained from lspci. > PCI: Enabling device 0000:41:00.0 (0000 -> 0002) > ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 > PCI: Setting latency timer of device 0000:41:00.0 to 64 > ACPI: PCI interrupt for device 0000:41:00.0 disabled > =====================> > === Host pciback dmesg (after setting it permissive) ==> pciback 0000:41:00.0: enabling permissive mode configuration space > accesses! > pciback 0000:41:00.0: permissive mode is potentially unsafe! > pciback: vpci: 0000:41:00.0: assign to virtual slot 0 > device vif1.0 entered promiscuous mode > ADDRCONF(NETDEV_UP): vif1.0: link is not ready > blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) > PCI: Enabling device 0000:41:00.0 (0000 -> 0002) > ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 > PCI: Setting latency timer of device 0000:41:00.0 to 64 > ACPI: PCI interrupt for device 0000:41:00.0 disabled > ========================================> > === Guest lspci output ==> # lspci -v > 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx > HCA] (rev 20) > Subsystem: Hewlett-Packard Company Unknown device 170a > Flags: fast devsel, IRQ 16 > Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] > Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] > Capabilities: [40] Power Management version 2 > Capabilities: [48] Vital Product Data > Capabilities: [90] Message Signalled Interrupts: 64bit+ > Queue=0/5 Enable- > Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 > Capabilities: [60] Express Endpoint IRQ 0 > ====================> _______________________________________________ > general mailing list > general@lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general >-- . . . s u b b u "You''ve got to be original, because if you''re like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Feb-12 01:18 UTC
RE: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
Seems it is because PCI frontend try to write some configuration space that PCIback has no config_field entry to support it. I think you can firstly try to do as dom0's dmesg suggested: "see permissive attribute in sysfs" (it should be "set permissive attribute...", I think). BTW, where you got following log? That seems suggest config space function not found. PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found." -- Yunhong Jiang ________________________________ From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of subbu kl Sent: 2009年2月11日 22:18 To: David Brown Cc: xen-devel@lists.xensource.com; general@lists.openfabrics.org Subject: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working I am getting the same QUERY_FW failed on RHEL5.2 with xenxen paravirtualized guest with pciback module. No one seems to have tried answering this question on the list, let me ping xen-devel and ofed people again. after executing in dom0 echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind #dmesg ACPI: PCI interrupt for device 0000:0e:00.0 disabled tap tap-1-51712: 2 getting info tap tap-2-51712: 2 getting info pciback 0000:0e:00.0: seizing device PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 ACPI: PCI interrupt for device 0000:0e:00.0 disabled #xm create -c rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. GUEST dmesg: ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 in dom0: Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not ready Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 disabled some more details - [root@p128 ~]# rpm -qa | grep xen kernel-xen-2.6.18-92.1.22.el5 xen-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 [root@p128 ~]# ibv_devinfo hca_id: mthca0 fw_ver: 5.3.0 node_guid: 0002:c902:0022:cd48 sys_image_guid: 0002:c902:0022:cd4b vendor_id: 0x02c9 vendor_part_id: 25218 hw_ver: 0x20 board_id: MT_0370130002 phys_port_cnt: 2 port: 1 state: PORT_INIT (2) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 any help greatly appreciated. ~subbu On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> wrote: Okay so my question to the openfabrics guys is, why would the OFED drivers fail to read the firmware? Any thoughts? Thanks, - David Brown ---------- Forwarded message ---------- From: David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> Date: Thu, Sep 11, 2008 at 2:24 PM Subject: pciback module not working To: xen-users@lists.xensource.com<mailto:xen-users@lists.xensource.com>, xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> This issue was brought up about a year and a half ago. So I'll bring it up again and see if anything happens. I've got an infiniband network and am attempting to pass the infiniband card through the host and give it to the guest. I'm working with standard CentOS 5.2 on both guest and host with their provided xen (3.0.3 ish). I've also attempted to install the newest Xen 3.3 and use their standard host kernel and that did the same thing. The guest dmesg output in the guest is similar on both permissive and normal mode. I'm getting issues with detecting the firmware on the card for some reason... Any help would be appreciated. Thanks, - David Brown === GUEST dmesg output ==ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 ====================== === Host modprobe.conf ==alias eth0 bnx2 alias eth1 bnx2 alias scsi_hostadapter cciss options pciback hide=(41:00.0) ==================== === Host lspci output ==# lspci -vs 41:00.0 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ==================== This makes sure it get loaded first off before anything else. === Host mkinitrd cmd ==# mkinitrd -f --with=pciback --preload pciback /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen =================== === Host pciback dmesg ==pciback 0000:41:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: 1) see permissive attribute in sysfs 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ===================== === Host pciback dmesg (after setting it permissive) ==pciback 0000:41:00.0: enabling permissive mode configuration space accesses! pciback 0000:41:00.0: permissive mode is potentially unsafe! pciback: vpci: 0000:41:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ======================================== === Guest lspci output ==# lspci -v 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ====================_______________________________________________ general mailing list general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
subbu kl
2009-Feb-12 05:52 UTC
Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
no luck ! dmesg in XEN PV guest shows : ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 even after executingh the following in dom0: #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive I am getting the follwing messages on the console as part of the initial bootup messages of the guest: Started domain rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. after executing the following in dom0 : #xm create -c rhel52_64_3 so, problem persisits, ~subbu 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com>> Seems it is because PCI frontend try to write some configuration space > that PCIback has no config_field entry to support it. > I think you can firstly try to do as dom0''s dmesg suggested: "see > permissive attribute in sysfs" (it should be "set permissive attribute...", > I think). > > BTW, where you got following log? That seems suggest config space function > not found. > > PCI: Fatal: No PCI config space access function found > rtc: IRQ 8 is not free. > i8042.c: No controller found." > > -- Yunhong Jiang > > ------------------------------ > *From:* xen-devel-bounces@lists.xensource.com [mailto: > xen-devel-bounces@lists.xensource.com] *On Behalf Of *subbu kl > *Sent:* 2009年2月11日 22:18 > *To:* David Brown > *Cc:* xen-devel@lists.xensource.com; general@lists.openfabrics.org > *Subject:* [Xen-devel] Re: [ofa-general] Fwd: pciback module not working > > I am getting the same QUERY_FW failed on RHEL5.2 with xenxen > paravirtualized guest with pciback module. > > No one seems to have tried answering this question on the list, let me ping > xen-devel and ofed people again. > > after executing in dom0 > echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind > echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot > echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind > > #dmesg > ACPI: PCI interrupt for device 0000:0e:00.0 disabled > tap tap-1-51712: 2 getting info > tap tap-2-51712: 2 getting info > pciback 0000:0e:00.0: seizing device > PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) > ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 > ACPI: PCI interrupt for device 0000:0e:00.0 disabled > > #xm create -c rhel52_64_3 > > PCI: Fatal: No PCI config space access function found > rtc: IRQ 8 is not free. > i8042.c: No controller found. > > > GUEST dmesg: > > ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) > ib_mthca: Initializing 0000:00:00.0 > PCI: Enabling device 0000:00:00.0 (0000 -> 0002) > PCI: Setting latency timer of device 0000:00:00.0 to 64 > ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. > ib_mthca: probe of 0000:00:00.0 failed with error -11 > > in dom0: > Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info > Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to virtual > slot 0 > Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode > Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not ready > Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, protocol > 1 (x86_64-abi) > Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write to > a read-only configuration space field at offset 0x44, size 2. This may be > harmless, but if you have problems with your device: > Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs > Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing > list along with details of your device obtained from lspci. > Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> > 0002) > Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 > (level, low) -> IRQ 16 > Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 > disabled > > > > some more details - [root@p128 ~]# rpm -qa | grep xen > kernel-xen-2.6.18-92.1.22.el5 > xen-3.0.3-64.el5_2.9 > xen-libs-3.0.3-64.el5_2.9 > xen-libs-3.0.3-64.el5_2.9 > > [root@p128 ~]# ibv_devinfo > hca_id: mthca0 > fw_ver: 5.3.0 > node_guid: 0002:c902:0022:cd48 > sys_image_guid: 0002:c902:0022:cd4b > vendor_id: 0x02c9 > vendor_part_id: 25218 > hw_ver: 0x20 > board_id: MT_0370130002 > phys_port_cnt: 2 > port: 1 > state: PORT_INIT (2) > max_mtu: 2048 (4) > active_mtu: 512 (2) > sm_lid: 0 > port_lid: 0 > port_lmc: 0x00 > > port: 2 > state: PORT_DOWN (1) > max_mtu: 2048 (4) > active_mtu: 512 (2) > sm_lid: 0 > port_lid: 0 > port_lmc: 0x00 > > > any help greatly appreciated. > > ~subbu > > On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com> wrote: > >> Okay so my question to the openfabrics guys is, why would the OFED >> drivers fail to read the firmware? >> >> Any thoughts? >> >> Thanks, >> - David Brown >> >> >> ---------- Forwarded message ---------- >> From: David Brown <dmlb2000@gmail.com> >> Date: Thu, Sep 11, 2008 at 2:24 PM >> Subject: pciback module not working >> To: xen-users@lists.xensource.com, xen-devel@lists.xensource.com >> >> >> This issue was brought up about a year and a half ago. So I''ll bring >> it up again and see if anything happens. >> >> I''ve got an infiniband network and am attempting to pass the >> infiniband card through the host and give it to the guest. >> I''m working with standard CentOS 5.2 on both guest and host with their >> provided xen (3.0.3 ish). I''ve also attempted to install the newest >> Xen 3.3 and use their standard host kernel and that did the same >> thing. The guest dmesg output in the guest is similar on both >> permissive and normal mode. >> >> I''m getting issues with detecting the firmware on the card for some >> reason... >> >> Any help would be appreciated. >> >> Thanks, >> - David Brown >> >> === GUEST dmesg output ==>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) >> ib_mthca: Initializing 0000:00:00.0 >> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >> PCI: Setting latency timer of device 0000:00:00.0 to 64 >> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >> ib_mthca: probe of 0000:00:00.0 failed with error -11 >> ======================>> >> === Host modprobe.conf ==>> alias eth0 bnx2 >> alias eth1 bnx2 >> alias scsi_hostadapter cciss >> options pciback hide=(41:00.0) >> ====================>> >> === Host lspci output ==>> # lspci -vs 41:00.0 >> 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >> HCA] (rev 20) >> Subsystem: Hewlett-Packard Company Unknown device 170a >> Flags: fast devsel, IRQ 16 >> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >> Capabilities: [40] Power Management version 2 >> Capabilities: [48] Vital Product Data >> Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 >> Enable- >> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >> Capabilities: [60] Express Endpoint IRQ 0 >> ====================>> >> This makes sure it get loaded first off before anything else. >> === Host mkinitrd cmd ==>> # mkinitrd -f --with=pciback --preload pciback >> /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen >> ===================>> >> === Host pciback dmesg ==>> pciback 0000:41:00.0: Driver tried to write to a read-only >> configuration space field at offset 0x44, size 2. This may be >> harmless, but if you have problems with your device: >> 1) see permissive attribute in sysfs >> 2) report problems to the xen-devel mailing list along with details of >> your device obtained from lspci. >> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >> PCI: Setting latency timer of device 0000:41:00.0 to 64 >> ACPI: PCI interrupt for device 0000:41:00.0 disabled >> =====================>> >> === Host pciback dmesg (after setting it permissive) ==>> pciback 0000:41:00.0: enabling permissive mode configuration space >> accesses! >> pciback 0000:41:00.0: permissive mode is potentially unsafe! >> pciback: vpci: 0000:41:00.0: assign to virtual slot 0 >> device vif1.0 entered promiscuous mode >> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >> blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) >> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >> PCI: Setting latency timer of device 0000:41:00.0 to 64 >> ACPI: PCI interrupt for device 0000:41:00.0 disabled >> ========================================>> >> === Guest lspci output ==>> # lspci -v >> 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >> HCA] (rev 20) >> Subsystem: Hewlett-Packard Company Unknown device 170a >> Flags: fast devsel, IRQ 16 >> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >> Capabilities: [40] Power Management version 2 >> Capabilities: [48] Vital Product Data >> Capabilities: [90] Message Signalled Interrupts: 64bit+ >> Queue=0/5 Enable- >> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >> Capabilities: [60] Express Endpoint IRQ 0 >> ====================>> _______________________________________________ >> general mailing list >> general@lists.openfabrics.org >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> > > > > -- > . . . s u b b u > "You''ve got to be original, because if you''re like someone else, what do > they need you for?" > >-- . . . s u b b u "You''ve got to be original, because if you''re like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Feb-12 06:20 UTC
RE: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
So any changes in dom0's dmesg? ________________________________ From: subbu kl [mailto:subbukl@gmail.com] Sent: 2009年2月12日 13:52 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com; general@lists.openfabrics.org Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working no luck ! dmesg in XEN PV guest shows : ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 even after executingh the following in dom0: #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive I am getting the follwing messages on the console as part of the initial bootup messages of the guest: Started domain rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. after executing the following in dom0 : #xm create -c rhel52_64_3 so, problem persisits, ~subbu 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> Seems it is because PCI frontend try to write some configuration space that PCIback has no config_field entry to support it. I think you can firstly try to do as dom0's dmesg suggested: "see permissive attribute in sysfs" (it should be "set permissive attribute...", I think). BTW, where you got following log? That seems suggest config space function not found. PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found." -- Yunhong Jiang ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of subbu kl Sent: 2009年2月11日 22:18 To: David Brown Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working I am getting the same QUERY_FW failed on RHEL5.2 with xenxen paravirtualized guest with pciback module. No one seems to have tried answering this question on the list, let me ping xen-devel and ofed people again. after executing in dom0 echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind #dmesg ACPI: PCI interrupt for device 0000:0e:00.0 disabled tap tap-1-51712: 2 getting info tap tap-2-51712: 2 getting info pciback 0000:0e:00.0: seizing device PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 ACPI: PCI interrupt for device 0000:0e:00.0 disabled #xm create -c rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. GUEST dmesg: ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 in dom0: Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not ready Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 disabled some more details - [root@p128 ~]# rpm -qa | grep xen kernel-xen-2.6.18-92.1.22.el5 xen-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 [root@p128 ~]# ibv_devinfo hca_id: mthca0 fw_ver: 5.3.0 node_guid: 0002:c902:0022:cd48 sys_image_guid: 0002:c902:0022:cd4b vendor_id: 0x02c9 vendor_part_id: 25218 hw_ver: 0x20 board_id: MT_0370130002 phys_port_cnt: 2 port: 1 state: PORT_INIT (2) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 any help greatly appreciated. ~subbu On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> wrote: Okay so my question to the openfabrics guys is, why would the OFED drivers fail to read the firmware? Any thoughts? Thanks, - David Brown ---------- Forwarded message ---------- From: David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> Date: Thu, Sep 11, 2008 at 2:24 PM Subject: pciback module not working To: xen-users@lists.xensource.com<mailto:xen-users@lists.xensource.com>, xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> This issue was brought up about a year and a half ago. So I'll bring it up again and see if anything happens. I've got an infiniband network and am attempting to pass the infiniband card through the host and give it to the guest. I'm working with standard CentOS 5.2 on both guest and host with their provided xen (3.0.3 ish). I've also attempted to install the newest Xen 3.3 and use their standard host kernel and that did the same thing. The guest dmesg output in the guest is similar on both permissive and normal mode. I'm getting issues with detecting the firmware on the card for some reason... Any help would be appreciated. Thanks, - David Brown === GUEST dmesg output ==ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 ====================== === Host modprobe.conf ==alias eth0 bnx2 alias eth1 bnx2 alias scsi_hostadapter cciss options pciback hide=(41:00.0) ==================== === Host lspci output ==# lspci -vs 41:00.0 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ==================== This makes sure it get loaded first off before anything else. === Host mkinitrd cmd ==# mkinitrd -f --with=pciback --preload pciback /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen =================== === Host pciback dmesg ==pciback 0000:41:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: 1) see permissive attribute in sysfs 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ===================== === Host pciback dmesg (after setting it permissive) ==pciback 0000:41:00.0: enabling permissive mode configuration space accesses! pciback 0000:41:00.0: permissive mode is potentially unsafe! pciback: vpci: 0000:41:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ======================================== === Guest lspci output ==# lspci -v 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ====================_______________________________________________ general mailing list general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
subbu kl
2009-Feb-12 06:42 UTC
Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
oops missed it, well now I dont see that enable permissive...message. here goes the messages what I got in dom0 while booting domU tap tap-1-51712: 2 getting info pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:0e:00.0 to 64 ACPI: PCI interrupt for device 0000:0e:00.0 disabled ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready xenbr0: topology change detected, propagating xenbr0: port 3(vif1.0) entering forwarding state any suspicious message ? any Idea why I get that : PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. message in domU bootup message ? ~subbu On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong <yunhong.jiang@intel.com>wrote:> So any changes in dom0''s dmesg? > > > ------------------------------ > *From:* subbu kl [mailto:subbukl@gmail.com] > *Sent:* 2009年2月12日 13:52 > *To:* Jiang, Yunhong > *Cc:* David Brown; xen-devel@lists.xensource.com; > general@lists.openfabrics.org > *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not > working > > no luck ! > dmesg in XEN PV guest shows : > > ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) > ib_mthca: Initializing 0000:00:00.0 > PCI: Enabling device 0000:00:00.0 (0000 -> 0002) > PCI: Setting latency timer of device 0000:00:00.0 to 64 > ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. > ib_mthca: probe of 0000:00:00.0 failed with error -11 > > even after executingh the following in dom0: > > #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive > > I am getting the follwing messages on the console as part of the initial > bootup messages of the guest: > > Started domain rhel52_64_3 > PCI: Fatal: No PCI config space access function found > rtc: IRQ 8 is not free. > i8042.c: No controller found. > > after executing the following in dom0 : > #xm create -c rhel52_64_3 > > > so, problem persisits, > > ~subbu > > > 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com> > >> Seems it is because PCI frontend try to write some configuration space >> that PCIback has no config_field entry to support it. >> I think you can firstly try to do as dom0''s dmesg suggested: "see >> permissive attribute in sysfs" (it should be "set permissive attribute...", >> I think). >> >> BTW, where you got following log? That seems suggest config space function >> not found. >> >> PCI: Fatal: No PCI config space access function found >> rtc: IRQ 8 is not free. >> i8042.c: No controller found." >> >> -- Yunhong Jiang >> >> ------------------------------ >> *From:* xen-devel-bounces@lists.xensource.com [mailto: >> xen-devel-bounces@lists.xensource.com] *On Behalf Of *subbu kl >> *Sent:* 2009年2月11日 22:18 >> *To:* David Brown >> *Cc:* xen-devel@lists.xensource.com; general@lists.openfabrics.org >> *Subject:* [Xen-devel] Re: [ofa-general] Fwd: pciback module not working >> >> I am getting the same QUERY_FW failed on RHEL5.2 with xenxen >> paravirtualized guest with pciback module. >> >> No one seems to have tried answering this question on the list, let me >> ping xen-devel and ofed people again. >> >> after executing in dom0 >> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind >> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot >> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind >> >> #dmesg >> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >> tap tap-1-51712: 2 getting info >> tap tap-2-51712: 2 getting info >> pciback 0000:0e:00.0: seizing device >> PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) >> ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >> >> #xm create -c rhel52_64_3 >> >> PCI: Fatal: No PCI config space access function found >> rtc: IRQ 8 is not free. >> i8042.c: No controller found. >> >> >> GUEST dmesg: >> >> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >> ib_mthca: Initializing 0000:00:00.0 >> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >> PCI: Setting latency timer of device 0000:00:00.0 to 64 >> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >> ib_mthca: probe of 0000:00:00.0 failed with error -11 >> >> in dom0: >> Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info >> Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to >> virtual slot 0 >> Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode >> Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not >> ready >> Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, protocol >> 1 (x86_64-abi) >> Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write >> to a read-only configuration space field at offset 0x44, size 2. This may be >> harmless, but if you have problems with your device: >> Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs >> Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing >> list along with details of your device obtained from lspci. >> Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> >> 0002) >> Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 >> (level, low) -> IRQ 16 >> Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 >> disabled >> >> >> >> some more details - [root@p128 ~]# rpm -qa | grep xen >> kernel-xen-2.6.18-92.1.22.el5 >> xen-3.0.3-64.el5_2.9 >> xen-libs-3.0.3-64.el5_2.9 >> xen-libs-3.0.3-64.el5_2.9 >> >> [root@p128 ~]# ibv_devinfo >> hca_id: mthca0 >> fw_ver: 5.3.0 >> node_guid: 0002:c902:0022:cd48 >> sys_image_guid: 0002:c902:0022:cd4b >> vendor_id: 0x02c9 >> vendor_part_id: 25218 >> hw_ver: 0x20 >> board_id: MT_0370130002 >> phys_port_cnt: 2 >> port: 1 >> state: PORT_INIT (2) >> max_mtu: 2048 (4) >> active_mtu: 512 (2) >> sm_lid: 0 >> port_lid: 0 >> port_lmc: 0x00 >> >> port: 2 >> state: PORT_DOWN (1) >> max_mtu: 2048 (4) >> active_mtu: 512 (2) >> sm_lid: 0 >> port_lid: 0 >> port_lmc: 0x00 >> >> >> any help greatly appreciated. >> >> ~subbu >> >> On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com> wrote: >> >>> Okay so my question to the openfabrics guys is, why would the OFED >>> drivers fail to read the firmware? >>> >>> Any thoughts? >>> >>> Thanks, >>> - David Brown >>> >>> >>> ---------- Forwarded message ---------- >>> From: David Brown <dmlb2000@gmail.com> >>> Date: Thu, Sep 11, 2008 at 2:24 PM >>> Subject: pciback module not working >>> To: xen-users@lists.xensource.com, xen-devel@lists.xensource.com >>> >>> >>> This issue was brought up about a year and a half ago. So I''ll bring >>> it up again and see if anything happens. >>> >>> I''ve got an infiniband network and am attempting to pass the >>> infiniband card through the host and give it to the guest. >>> I''m working with standard CentOS 5.2 on both guest and host with their >>> provided xen (3.0.3 ish). I''ve also attempted to install the newest >>> Xen 3.3 and use their standard host kernel and that did the same >>> thing. The guest dmesg output in the guest is similar on both >>> permissive and normal mode. >>> >>> I''m getting issues with detecting the firmware on the card for some >>> reason... >>> >>> Any help would be appreciated. >>> >>> Thanks, >>> - David Brown >>> >>> === GUEST dmesg output ==>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) >>> ib_mthca: Initializing 0000:00:00.0 >>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>> ======================>>> >>> === Host modprobe.conf ==>>> alias eth0 bnx2 >>> alias eth1 bnx2 >>> alias scsi_hostadapter cciss >>> options pciback hide=(41:00.0) >>> ====================>>> >>> === Host lspci output ==>>> # lspci -vs 41:00.0 >>> 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>> HCA] (rev 20) >>> Subsystem: Hewlett-Packard Company Unknown device 170a >>> Flags: fast devsel, IRQ 16 >>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>> Capabilities: [40] Power Management version 2 >>> Capabilities: [48] Vital Product Data >>> Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 >>> Enable- >>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>> Capabilities: [60] Express Endpoint IRQ 0 >>> ====================>>> >>> This makes sure it get loaded first off before anything else. >>> === Host mkinitrd cmd ==>>> # mkinitrd -f --with=pciback --preload pciback >>> /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen >>> ===================>>> >>> === Host pciback dmesg ==>>> pciback 0000:41:00.0: Driver tried to write to a read-only >>> configuration space field at offset 0x44, size 2. This may be >>> harmless, but if you have problems with your device: >>> 1) see permissive attribute in sysfs >>> 2) report problems to the xen-devel mailing list along with details of >>> your device obtained from lspci. >>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>> =====================>>> >>> === Host pciback dmesg (after setting it permissive) ==>>> pciback 0000:41:00.0: enabling permissive mode configuration space >>> accesses! >>> pciback 0000:41:00.0: permissive mode is potentially unsafe! >>> pciback: vpci: 0000:41:00.0: assign to virtual slot 0 >>> device vif1.0 entered promiscuous mode >>> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >>> blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) >>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>> ========================================>>> >>> === Guest lspci output ==>>> # lspci -v >>> 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>> HCA] (rev 20) >>> Subsystem: Hewlett-Packard Company Unknown device 170a >>> Flags: fast devsel, IRQ 16 >>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>> Capabilities: [40] Power Management version 2 >>> Capabilities: [48] Vital Product Data >>> Capabilities: [90] Message Signalled Interrupts: 64bit+ >>> Queue=0/5 Enable- >>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>> Capabilities: [60] Express Endpoint IRQ 0 >>> ====================>>> _______________________________________________ >>> general mailing list >>> general@lists.openfabrics.org >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>> >>> To unsubscribe, please visit >>> http://openib.org/mailman/listinfo/openib-general >>> >> >> >> >> -- >> . . . s u b b u >> "You''ve got to be original, because if you''re like someone else, what do >> they need you for?" >> >> > > > -- > . . . s u b b u > "You''ve got to be original, because if you''re like someone else, what do > they need you for?" > >-- . . . s u b b u "You''ve got to be original, because if you''re like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Feb-12 06:56 UTC
RE: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
Sorry that seems the original mail has tried the permissive already :$ How will So how will the card do the QEUREY_FW command?Through config space or through MMIO? Following information is something strange, why all the MMIO range is disabled? Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] As for the following information, I think it should be harmless since domU has no method of config spacess access method. PCI: Fatal: No PCI config space access function found Thanks Yunhong Jiang ________________________________ From: subbu kl [mailto:subbukl@gmail.com] Sent: 2009年2月12日 14:43 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com; general@lists.openfabrics.org Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working oops missed it, well now I dont see that enable permissive...message. here goes the messages what I got in dom0 while booting domU tap tap-1-51712: 2 getting info pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:0e:00.0 to 64 ACPI: PCI interrupt for device 0000:0e:00.0 disabled ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready xenbr0: topology change detected, propagating xenbr0: port 3(vif1.0) entering forwarding state any suspicious message ? any Idea why I get that : PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. message in domU bootup message ? ~subbu On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> wrote: So any changes in dom0's dmesg? ________________________________ From: subbu kl [mailto:subbukl@gmail.com<mailto:subbukl@gmail.com>] Sent: 2009年2月12日 13:52 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working no luck ! dmesg in XEN PV guest shows : ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 even after executingh the following in dom0: #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive I am getting the follwing messages on the console as part of the initial bootup messages of the guest: Started domain rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. after executing the following in dom0 : #xm create -c rhel52_64_3 so, problem persisits, ~subbu 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> Seems it is because PCI frontend try to write some configuration space that PCIback has no config_field entry to support it. I think you can firstly try to do as dom0's dmesg suggested: "see permissive attribute in sysfs" (it should be "set permissive attribute...", I think). BTW, where you got following log? That seems suggest config space function not found. PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found." -- Yunhong Jiang ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of subbu kl Sent: 2009年2月11日 22:18 To: David Brown Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working I am getting the same QUERY_FW failed on RHEL5.2 with xenxen paravirtualized guest with pciback module. No one seems to have tried answering this question on the list, let me ping xen-devel and ofed people again. after executing in dom0 echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind #dmesg ACPI: PCI interrupt for device 0000:0e:00.0 disabled tap tap-1-51712: 2 getting info tap tap-2-51712: 2 getting info pciback 0000:0e:00.0: seizing device PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 ACPI: PCI interrupt for device 0000:0e:00.0 disabled #xm create -c rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. GUEST dmesg: ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 in dom0: Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not ready Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 disabled some more details - [root@p128 ~]# rpm -qa | grep xen kernel-xen-2.6.18-92.1.22.el5 xen-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 [root@p128 ~]# ibv_devinfo hca_id: mthca0 fw_ver: 5.3.0 node_guid: 0002:c902:0022:cd48 sys_image_guid: 0002:c902:0022:cd4b vendor_id: 0x02c9 vendor_part_id: 25218 hw_ver: 0x20 board_id: MT_0370130002 phys_port_cnt: 2 port: 1 state: PORT_INIT (2) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 any help greatly appreciated. ~subbu On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> wrote: Okay so my question to the openfabrics guys is, why would the OFED drivers fail to read the firmware? Any thoughts? Thanks, - David Brown ---------- Forwarded message ---------- From: David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> Date: Thu, Sep 11, 2008 at 2:24 PM Subject: pciback module not working To: xen-users@lists.xensource.com<mailto:xen-users@lists.xensource.com>, xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> This issue was brought up about a year and a half ago. So I'll bring it up again and see if anything happens. I've got an infiniband network and am attempting to pass the infiniband card through the host and give it to the guest. I'm working with standard CentOS 5.2 on both guest and host with their provided xen (3.0.3 ish). I've also attempted to install the newest Xen 3.3 and use their standard host kernel and that did the same thing. The guest dmesg output in the guest is similar on both permissive and normal mode. I'm getting issues with detecting the firmware on the card for some reason... Any help would be appreciated. Thanks, - David Brown === GUEST dmesg output ==ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 ====================== === Host modprobe.conf ==alias eth0 bnx2 alias eth1 bnx2 alias scsi_hostadapter cciss options pciback hide=(41:00.0) ==================== === Host lspci output ==# lspci -vs 41:00.0 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ==================== This makes sure it get loaded first off before anything else. === Host mkinitrd cmd ==# mkinitrd -f --with=pciback --preload pciback /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen =================== === Host pciback dmesg ==pciback 0000:41:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: 1) see permissive attribute in sysfs 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ===================== === Host pciback dmesg (after setting it permissive) ==pciback 0000:41:00.0: enabling permissive mode configuration space accesses! pciback 0000:41:00.0: permissive mode is potentially unsafe! pciback: vpci: 0000:41:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ======================================== === Guest lspci output ==# lspci -v 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ====================_______________________________________________ general mailing list general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
subbu kl
2009-Feb-12 06:58 UTC
Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
So getting PCI config space access in domU will solve the problem ? if so how can I achieve that ? ~subbu On Thu, Feb 12, 2009 at 12:26 PM, Jiang, Yunhong <yunhong.jiang@intel.com>wrote:> Sorry that seems the original mail has tried the permissive already :$ > How will So how will the card do the QEUREY_FW command?Through config space > or through MMIO? Following information is something strange, why all the > MMIO range is disabled? > > Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] > Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] > > As for the following information, I think it should be harmless since domU > has no method of config spacess access method. > PCI: Fatal: No PCI config space access function found > > Thanks > Yunhong Jiang > > ------------------------------ > *From:* subbu kl [mailto:subbukl@gmail.com] > *Sent:* 2009年2月12日 14:43 > > *To:* Jiang, Yunhong > *Cc:* David Brown; xen-devel@lists.xensource.com; > general@lists.openfabrics.org > *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not > working > > oops missed it, > > well now I dont see that enable permissive...message. here goes the > messages what I got in dom0 while booting domU > > tap tap-1-51712: 2 getting info > pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 > device vif1.0 entered promiscuous mode > ADDRCONF(NETDEV_UP): vif1.0: link is not ready > blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) > PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) > ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 > PCI: Setting latency timer of device 0000:0e:00.0 to 64 > ACPI: PCI interrupt for device 0000:0e:00.0 disabled > ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready > xenbr0: topology change detected, propagating > xenbr0: port 3(vif1.0) entering forwarding state > > any suspicious message ? > any Idea why I get that : > PCI: Fatal: No PCI config space access function found > rtc: IRQ 8 is not free. > > message in domU bootup message ? > > ~subbu > > On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong <yunhong.jiang@intel.com>wrote: > >> So any changes in dom0''s dmesg? >> >> >> ------------------------------ >> *From:* subbu kl [mailto:subbukl@gmail.com] >> *Sent:* 2009年2月12日 13:52 >> *To:* Jiang, Yunhong >> *Cc:* David Brown; xen-devel@lists.xensource.com; >> general@lists.openfabrics.org >> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >> working >> >> no luck ! >> dmesg in XEN PV guest shows : >> >> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >> ib_mthca: Initializing 0000:00:00.0 >> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >> PCI: Setting latency timer of device 0000:00:00.0 to 64 >> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >> ib_mthca: probe of 0000:00:00.0 failed with error -11 >> >> even after executingh the following in dom0: >> >> #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive >> >> I am getting the follwing messages on the console as part of the initial >> bootup messages of the guest: >> >> Started domain rhel52_64_3 >> PCI: Fatal: No PCI config space access function found >> rtc: IRQ 8 is not free. >> i8042.c: No controller found. >> >> after executing the following in dom0 : >> #xm create -c rhel52_64_3 >> >> >> so, problem persisits, >> >> ~subbu >> >> >> 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com> >> >>> Seems it is because PCI frontend try to write some configuration space >>> that PCIback has no config_field entry to support it. >>> I think you can firstly try to do as dom0''s dmesg suggested: "see >>> permissive attribute in sysfs" (it should be "set permissive attribute...", >>> I think). >>> >>> BTW, where you got following log? That seems suggest config space >>> function not found. >>> >>> PCI: Fatal: No PCI config space access function found >>> rtc: IRQ 8 is not free. >>> i8042.c: No controller found." >>> >>> -- Yunhong Jiang >>> >>> ------------------------------ >>> *From:* xen-devel-bounces@lists.xensource.com [mailto: >>> xen-devel-bounces@lists.xensource.com] *On Behalf Of *subbu kl >>> *Sent:* 2009年2月11日 22:18 >>> *To:* David Brown >>> *Cc:* xen-devel@lists.xensource.com; general@lists.openfabrics.org >>> *Subject:* [Xen-devel] Re: [ofa-general] Fwd: pciback module not working >>> >>> I am getting the same QUERY_FW failed on RHEL5.2 with xenxen >>> paravirtualized guest with pciback module. >>> >>> No one seems to have tried answering this question on the list, let me >>> ping xen-devel and ofed people again. >>> >>> after executing in dom0 >>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind >>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot >>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind >>> >>> #dmesg >>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>> tap tap-1-51712: 2 getting info >>> tap tap-2-51712: 2 getting info >>> pciback 0000:0e:00.0: seizing device >>> PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) >>> ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>> >>> #xm create -c rhel52_64_3 >>> >>> PCI: Fatal: No PCI config space access function found >>> rtc: IRQ 8 is not free. >>> i8042.c: No controller found. >>> >>> >>> GUEST dmesg: >>> >>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >>> ib_mthca: Initializing 0000:00:00.0 >>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>> >>> in dom0: >>> Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info >>> Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to >>> virtual slot 0 >>> Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode >>> Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not >>> ready >>> Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, >>> protocol 1 (x86_64-abi) >>> Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write >>> to a read-only configuration space field at offset 0x44, size 2. This may be >>> harmless, but if you have problems with your device: >>> Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs >>> Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing >>> list along with details of your device obtained from lspci. >>> Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> >>> 0002) >>> Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI >>> 16 (level, low) -> IRQ 16 >>> Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 >>> disabled >>> >>> >>> >>> some more details - [root@p128 ~]# rpm -qa | grep xen >>> kernel-xen-2.6.18-92.1.22.el5 >>> xen-3.0.3-64.el5_2.9 >>> xen-libs-3.0.3-64.el5_2.9 >>> xen-libs-3.0.3-64.el5_2.9 >>> >>> [root@p128 ~]# ibv_devinfo >>> hca_id: mthca0 >>> fw_ver: 5.3.0 >>> node_guid: 0002:c902:0022:cd48 >>> sys_image_guid: 0002:c902:0022:cd4b >>> vendor_id: 0x02c9 >>> vendor_part_id: 25218 >>> hw_ver: 0x20 >>> board_id: MT_0370130002 >>> phys_port_cnt: 2 >>> port: 1 >>> state: PORT_INIT (2) >>> max_mtu: 2048 (4) >>> active_mtu: 512 (2) >>> sm_lid: 0 >>> port_lid: 0 >>> port_lmc: 0x00 >>> >>> port: 2 >>> state: PORT_DOWN (1) >>> max_mtu: 2048 (4) >>> active_mtu: 512 (2) >>> sm_lid: 0 >>> port_lid: 0 >>> port_lmc: 0x00 >>> >>> >>> any help greatly appreciated. >>> >>> ~subbu >>> >>> On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com> wrote: >>> >>>> Okay so my question to the openfabrics guys is, why would the OFED >>>> drivers fail to read the firmware? >>>> >>>> Any thoughts? >>>> >>>> Thanks, >>>> - David Brown >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: David Brown <dmlb2000@gmail.com> >>>> Date: Thu, Sep 11, 2008 at 2:24 PM >>>> Subject: pciback module not working >>>> To: xen-users@lists.xensource.com, xen-devel@lists.xensource.com >>>> >>>> >>>> This issue was brought up about a year and a half ago. So I''ll bring >>>> it up again and see if anything happens. >>>> >>>> I''ve got an infiniband network and am attempting to pass the >>>> infiniband card through the host and give it to the guest. >>>> I''m working with standard CentOS 5.2 on both guest and host with their >>>> provided xen (3.0.3 ish). I''ve also attempted to install the newest >>>> Xen 3.3 and use their standard host kernel and that did the same >>>> thing. The guest dmesg output in the guest is similar on both >>>> permissive and normal mode. >>>> >>>> I''m getting issues with detecting the firmware on the card for some >>>> reason... >>>> >>>> Any help would be appreciated. >>>> >>>> Thanks, >>>> - David Brown >>>> >>>> === GUEST dmesg output ==>>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) >>>> ib_mthca: Initializing 0000:00:00.0 >>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>> ======================>>>> >>>> === Host modprobe.conf ==>>>> alias eth0 bnx2 >>>> alias eth1 bnx2 >>>> alias scsi_hostadapter cciss >>>> options pciback hide=(41:00.0) >>>> ====================>>>> >>>> === Host lspci output ==>>>> # lspci -vs 41:00.0 >>>> 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>>> HCA] (rev 20) >>>> Subsystem: Hewlett-Packard Company Unknown device 170a >>>> Flags: fast devsel, IRQ 16 >>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>> Capabilities: [40] Power Management version 2 >>>> Capabilities: [48] Vital Product Data >>>> Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 >>>> Enable- >>>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>>> Capabilities: [60] Express Endpoint IRQ 0 >>>> ====================>>>> >>>> This makes sure it get loaded first off before anything else. >>>> === Host mkinitrd cmd ==>>>> # mkinitrd -f --with=pciback --preload pciback >>>> /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen >>>> ===================>>>> >>>> === Host pciback dmesg ==>>>> pciback 0000:41:00.0: Driver tried to write to a read-only >>>> configuration space field at offset 0x44, size 2. This may be >>>> harmless, but if you have problems with your device: >>>> 1) see permissive attribute in sysfs >>>> 2) report problems to the xen-devel mailing list along with details of >>>> your device obtained from lspci. >>>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>>> =====================>>>> >>>> === Host pciback dmesg (after setting it permissive) ==>>>> pciback 0000:41:00.0: enabling permissive mode configuration space >>>> accesses! >>>> pciback 0000:41:00.0: permissive mode is potentially unsafe! >>>> pciback: vpci: 0000:41:00.0: assign to virtual slot 0 >>>> device vif1.0 entered promiscuous mode >>>> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >>>> blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) >>>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>>> ========================================>>>> >>>> === Guest lspci output ==>>>> # lspci -v >>>> 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>>> HCA] (rev 20) >>>> Subsystem: Hewlett-Packard Company Unknown device 170a >>>> Flags: fast devsel, IRQ 16 >>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>> Capabilities: [40] Power Management version 2 >>>> Capabilities: [48] Vital Product Data >>>> Capabilities: [90] Message Signalled Interrupts: 64bit+ >>>> Queue=0/5 Enable- >>>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>>> Capabilities: [60] Express Endpoint IRQ 0 >>>> ====================>>>> _______________________________________________ >>>> general mailing list >>>> general@lists.openfabrics.org >>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>>> >>>> To unsubscribe, please visit >>>> http://openib.org/mailman/listinfo/openib-general >>>> >>> >>> >>> >>> -- >>> . . . s u b b u >>> "You''ve got to be original, because if you''re like someone else, what do >>> they need you for?" >>> >>> >> >> >> -- >> . . . s u b b u >> "You''ve got to be original, because if you''re like someone else, what do >> they need you for?" >> >> > > > -- > . . . s u b b u > "You''ve got to be original, because if you''re like someone else, what do > they need you for?" > >-- . . . s u b b u "You''ve got to be original, because if you''re like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Feb-12 07:00 UTC
RE: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
DomU access config space through pcibackend, so that message is ok. ________________________________ From: subbu kl [mailto:subbukl@gmail.com] Sent: 2009年2月12日 14:59 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com; general@lists.openfabrics.org Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working So getting PCI config space access in domU will solve the problem ? if so how can I achieve that ? ~subbu On Thu, Feb 12, 2009 at 12:26 PM, Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> wrote: Sorry that seems the original mail has tried the permissive already :$ How will So how will the card do the QEUREY_FW command?Through config space or through MMIO? Following information is something strange, why all the MMIO range is disabled? Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] As for the following information, I think it should be harmless since domU has no method of config spacess access method. PCI: Fatal: No PCI config space access function found Thanks Yunhong Jiang ________________________________ From: subbu kl [mailto:subbukl@gmail.com<mailto:subbukl@gmail.com>] Sent: 2009年2月12日 14:43 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working oops missed it, well now I dont see that enable permissive...message. here goes the messages what I got in dom0 while booting domU tap tap-1-51712: 2 getting info pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:0e:00.0 to 64 ACPI: PCI interrupt for device 0000:0e:00.0 disabled ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready xenbr0: topology change detected, propagating xenbr0: port 3(vif1.0) entering forwarding state any suspicious message ? any Idea why I get that : PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. message in domU bootup message ? ~subbu On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> wrote: So any changes in dom0's dmesg? ________________________________ From: subbu kl [mailto:subbukl@gmail.com<mailto:subbukl@gmail.com>] Sent: 2009年2月12日 13:52 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working no luck ! dmesg in XEN PV guest shows : ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 even after executingh the following in dom0: #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive I am getting the follwing messages on the console as part of the initial bootup messages of the guest: Started domain rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. after executing the following in dom0 : #xm create -c rhel52_64_3 so, problem persisits, ~subbu 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> Seems it is because PCI frontend try to write some configuration space that PCIback has no config_field entry to support it. I think you can firstly try to do as dom0's dmesg suggested: "see permissive attribute in sysfs" (it should be "set permissive attribute...", I think). BTW, where you got following log? That seems suggest config space function not found. PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found." -- Yunhong Jiang ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of subbu kl Sent: 2009年2月11日 22:18 To: David Brown Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working I am getting the same QUERY_FW failed on RHEL5.2 with xenxen paravirtualized guest with pciback module. No one seems to have tried answering this question on the list, let me ping xen-devel and ofed people again. after executing in dom0 echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind #dmesg ACPI: PCI interrupt for device 0000:0e:00.0 disabled tap tap-1-51712: 2 getting info tap tap-2-51712: 2 getting info pciback 0000:0e:00.0: seizing device PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 ACPI: PCI interrupt for device 0000:0e:00.0 disabled #xm create -c rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. GUEST dmesg: ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 in dom0: Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not ready Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 disabled some more details - [root@p128 ~]# rpm -qa | grep xen kernel-xen-2.6.18-92.1.22.el5 xen-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 [root@p128 ~]# ibv_devinfo hca_id: mthca0 fw_ver: 5.3.0 node_guid: 0002:c902:0022:cd48 sys_image_guid: 0002:c902:0022:cd4b vendor_id: 0x02c9 vendor_part_id: 25218 hw_ver: 0x20 board_id: MT_0370130002 phys_port_cnt: 2 port: 1 state: PORT_INIT (2) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 any help greatly appreciated. ~subbu On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> wrote: Okay so my question to the openfabrics guys is, why would the OFED drivers fail to read the firmware? Any thoughts? Thanks, - David Brown ---------- Forwarded message ---------- From: David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> Date: Thu, Sep 11, 2008 at 2:24 PM Subject: pciback module not working To: xen-users@lists.xensource.com<mailto:xen-users@lists.xensource.com>, xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> This issue was brought up about a year and a half ago. So I'll bring it up again and see if anything happens. I've got an infiniband network and am attempting to pass the infiniband card through the host and give it to the guest. I'm working with standard CentOS 5.2 on both guest and host with their provided xen (3.0.3 ish). I've also attempted to install the newest Xen 3.3 and use their standard host kernel and that did the same thing. The guest dmesg output in the guest is similar on both permissive and normal mode. I'm getting issues with detecting the firmware on the card for some reason... Any help would be appreciated. Thanks, - David Brown === GUEST dmesg output ==ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 ====================== === Host modprobe.conf ==alias eth0 bnx2 alias eth1 bnx2 alias scsi_hostadapter cciss options pciback hide=(41:00.0) ==================== === Host lspci output ==# lspci -vs 41:00.0 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ==================== This makes sure it get loaded first off before anything else. === Host mkinitrd cmd ==# mkinitrd -f --with=pciback --preload pciback /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen =================== === Host pciback dmesg ==pciback 0000:41:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: 1) see permissive attribute in sysfs 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ===================== === Host pciback dmesg (after setting it permissive) ==pciback 0000:41:00.0: enabling permissive mode configuration space accesses! pciback 0000:41:00.0: permissive mode is potentially unsafe! pciback: vpci: 0000:41:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ======================================== === Guest lspci output ==# lspci -v 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ====================_______________________________________________ general mailing list general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
subbu kl
2009-Feb-12 07:45 UTC
Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
so back to square one ? Why QUERY_FW should fail in domU ? ~subbu On Thu, Feb 12, 2009 at 12:30 PM, Jiang, Yunhong <yunhong.jiang@intel.com>wrote:> DomU access config space through pcibackend, so that message is ok. > > ------------------------------ > *From:* subbu kl [mailto:subbukl@gmail.com] > *Sent:* 2009年2月12日 14:59 > > *To:* Jiang, Yunhong > *Cc:* David Brown; xen-devel@lists.xensource.com; > general@lists.openfabrics.org > *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not > working > > So getting PCI config space access in domU will solve the problem ? if so > how can I achieve that ? > > ~subbu > > On Thu, Feb 12, 2009 at 12:26 PM, Jiang, Yunhong <yunhong.jiang@intel.com>wrote: > >> Sorry that seems the original mail has tried the permissive already :$ >> How will So how will the card do the QEUREY_FW command?Through config >> space or through MMIO? Following information is something strange, why all >> the MMIO range is disabled? >> >> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >> >> As for the following information, I think it should be harmless since domU >> has no method of config spacess access method. >> PCI: Fatal: No PCI config space access function found >> >> Thanks >> Yunhong Jiang >> >> ------------------------------ >> *From:* subbu kl [mailto:subbukl@gmail.com] >> *Sent:* 2009年2月12日 14:43 >> >> *To:* Jiang, Yunhong >> *Cc:* David Brown; xen-devel@lists.xensource.com; >> general@lists.openfabrics.org >> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >> working >> >> oops missed it, >> >> well now I dont see that enable permissive...message. here goes the >> messages what I got in dom0 while booting domU >> >> tap tap-1-51712: 2 getting info >> pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 >> device vif1.0 entered promiscuous mode >> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >> blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) >> PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) >> ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >> PCI: Setting latency timer of device 0000:0e:00.0 to 64 >> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >> ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready >> xenbr0: topology change detected, propagating >> xenbr0: port 3(vif1.0) entering forwarding state >> >> any suspicious message ? >> any Idea why I get that : >> PCI: Fatal: No PCI config space access function found >> rtc: IRQ 8 is not free. >> >> message in domU bootup message ? >> >> ~subbu >> >> On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong <yunhong.jiang@intel.com >> > wrote: >> >>> So any changes in dom0''s dmesg? >>> >>> >>> ------------------------------ >>> *From:* subbu kl [mailto:subbukl@gmail.com] >>> *Sent:* 2009年2月12日 13:52 >>> *To:* Jiang, Yunhong >>> *Cc:* David Brown; xen-devel@lists.xensource.com; >>> general@lists.openfabrics.org >>> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>> working >>> >>> no luck ! >>> dmesg in XEN PV guest shows : >>> >>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >>> ib_mthca: Initializing 0000:00:00.0 >>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>> >>> even after executingh the following in dom0: >>> >>> #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive >>> >>> I am getting the follwing messages on the console as part of the initial >>> bootup messages of the guest: >>> >>> Started domain rhel52_64_3 >>> PCI: Fatal: No PCI config space access function found >>> rtc: IRQ 8 is not free. >>> i8042.c: No controller found. >>> >>> after executing the following in dom0 : >>> #xm create -c rhel52_64_3 >>> >>> >>> so, problem persisits, >>> >>> ~subbu >>> >>> >>> 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com> >>> >>>> Seems it is because PCI frontend try to write some configuration space >>>> that PCIback has no config_field entry to support it. >>>> I think you can firstly try to do as dom0''s dmesg suggested: "see >>>> permissive attribute in sysfs" (it should be "set permissive attribute...", >>>> I think). >>>> >>>> BTW, where you got following log? That seems suggest config space >>>> function not found. >>>> >>>> PCI: Fatal: No PCI config space access function found >>>> rtc: IRQ 8 is not free. >>>> i8042.c: No controller found." >>>> >>>> -- Yunhong Jiang >>>> >>>> ------------------------------ >>>> *From:* xen-devel-bounces@lists.xensource.com [mailto: >>>> xen-devel-bounces@lists.xensource.com] *On Behalf Of *subbu kl >>>> *Sent:* 2009年2月11日 22:18 >>>> *To:* David Brown >>>> *Cc:* xen-devel@lists.xensource.com; general@lists.openfabrics.org >>>> *Subject:* [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>>> working >>>> >>>> I am getting the same QUERY_FW failed on RHEL5.2 with xenxen >>>> paravirtualized guest with pciback module. >>>> >>>> No one seems to have tried answering this question on the list, let me >>>> ping xen-devel and ofed people again. >>>> >>>> after executing in dom0 >>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind >>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot >>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind >>>> >>>> #dmesg >>>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>>> tap tap-1-51712: 2 getting info >>>> tap tap-2-51712: 2 getting info >>>> pciback 0000:0e:00.0: seizing device >>>> PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) >>>> ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>>> >>>> #xm create -c rhel52_64_3 >>>> >>>> PCI: Fatal: No PCI config space access function found >>>> rtc: IRQ 8 is not free. >>>> i8042.c: No controller found. >>>> >>>> >>>> GUEST dmesg: >>>> >>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >>>> ib_mthca: Initializing 0000:00:00.0 >>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>> >>>> in dom0: >>>> Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info >>>> Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to >>>> virtual slot 0 >>>> Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode >>>> Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not >>>> ready >>>> Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, >>>> protocol 1 (x86_64-abi) >>>> Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write >>>> to a read-only configuration space field at offset 0x44, size 2. This may be >>>> harmless, but if you have problems with your device: >>>> Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs >>>> Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing >>>> list along with details of your device obtained from lspci. >>>> Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> >>>> 0002) >>>> Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI >>>> 16 (level, low) -> IRQ 16 >>>> Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 >>>> disabled >>>> >>>> >>>> >>>> some more details - [root@p128 ~]# rpm -qa | grep xen >>>> kernel-xen-2.6.18-92.1.22.el5 >>>> xen-3.0.3-64.el5_2.9 >>>> xen-libs-3.0.3-64.el5_2.9 >>>> xen-libs-3.0.3-64.el5_2.9 >>>> >>>> [root@p128 ~]# ibv_devinfo >>>> hca_id: mthca0 >>>> fw_ver: 5.3.0 >>>> node_guid: 0002:c902:0022:cd48 >>>> sys_image_guid: 0002:c902:0022:cd4b >>>> vendor_id: 0x02c9 >>>> vendor_part_id: 25218 >>>> hw_ver: 0x20 >>>> board_id: MT_0370130002 >>>> phys_port_cnt: 2 >>>> port: 1 >>>> state: PORT_INIT (2) >>>> max_mtu: 2048 (4) >>>> active_mtu: 512 (2) >>>> sm_lid: 0 >>>> port_lid: 0 >>>> port_lmc: 0x00 >>>> >>>> port: 2 >>>> state: PORT_DOWN (1) >>>> max_mtu: 2048 (4) >>>> active_mtu: 512 (2) >>>> sm_lid: 0 >>>> port_lid: 0 >>>> port_lmc: 0x00 >>>> >>>> >>>> any help greatly appreciated. >>>> >>>> ~subbu >>>> >>>> On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com>wrote: >>>> >>>>> Okay so my question to the openfabrics guys is, why would the OFED >>>>> drivers fail to read the firmware? >>>>> >>>>> Any thoughts? >>>>> >>>>> Thanks, >>>>> - David Brown >>>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: David Brown <dmlb2000@gmail.com> >>>>> Date: Thu, Sep 11, 2008 at 2:24 PM >>>>> Subject: pciback module not working >>>>> To: xen-users@lists.xensource.com, xen-devel@lists.xensource.com >>>>> >>>>> >>>>> This issue was brought up about a year and a half ago. So I''ll bring >>>>> it up again and see if anything happens. >>>>> >>>>> I''ve got an infiniband network and am attempting to pass the >>>>> infiniband card through the host and give it to the guest. >>>>> I''m working with standard CentOS 5.2 on both guest and host with their >>>>> provided xen (3.0.3 ish). I''ve also attempted to install the newest >>>>> Xen 3.3 and use their standard host kernel and that did the same >>>>> thing. The guest dmesg output in the guest is similar on both >>>>> permissive and normal mode. >>>>> >>>>> I''m getting issues with detecting the firmware on the card for some >>>>> reason... >>>>> >>>>> Any help would be appreciated. >>>>> >>>>> Thanks, >>>>> - David Brown >>>>> >>>>> === GUEST dmesg output ==>>>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) >>>>> ib_mthca: Initializing 0000:00:00.0 >>>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>>> ======================>>>>> >>>>> === Host modprobe.conf ==>>>>> alias eth0 bnx2 >>>>> alias eth1 bnx2 >>>>> alias scsi_hostadapter cciss >>>>> options pciback hide=(41:00.0) >>>>> ====================>>>>> >>>>> === Host lspci output ==>>>>> # lspci -vs 41:00.0 >>>>> 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>>>> HCA] (rev 20) >>>>> Subsystem: Hewlett-Packard Company Unknown device 170a >>>>> Flags: fast devsel, IRQ 16 >>>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] >>>>> [size=1M] >>>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>>> Capabilities: [40] Power Management version 2 >>>>> Capabilities: [48] Vital Product Data >>>>> Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 >>>>> Enable- >>>>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>>>> Capabilities: [60] Express Endpoint IRQ 0 >>>>> ====================>>>>> >>>>> This makes sure it get loaded first off before anything else. >>>>> === Host mkinitrd cmd ==>>>>> # mkinitrd -f --with=pciback --preload pciback >>>>> /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen >>>>> ===================>>>>> >>>>> === Host pciback dmesg ==>>>>> pciback 0000:41:00.0: Driver tried to write to a read-only >>>>> configuration space field at offset 0x44, size 2. This may be >>>>> harmless, but if you have problems with your device: >>>>> 1) see permissive attribute in sysfs >>>>> 2) report problems to the xen-devel mailing list along with details of >>>>> your device obtained from lspci. >>>>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>>>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>>>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>>>> =====================>>>>> >>>>> === Host pciback dmesg (after setting it permissive) ==>>>>> pciback 0000:41:00.0: enabling permissive mode configuration space >>>>> accesses! >>>>> pciback 0000:41:00.0: permissive mode is potentially unsafe! >>>>> pciback: vpci: 0000:41:00.0: assign to virtual slot 0 >>>>> device vif1.0 entered promiscuous mode >>>>> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >>>>> blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) >>>>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>>>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>>>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>>>> ========================================>>>>> >>>>> === Guest lspci output ==>>>>> # lspci -v >>>>> 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>>>> HCA] (rev 20) >>>>> Subsystem: Hewlett-Packard Company Unknown device 170a >>>>> Flags: fast devsel, IRQ 16 >>>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] >>>>> [size=1M] >>>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>>> Capabilities: [40] Power Management version 2 >>>>> Capabilities: [48] Vital Product Data >>>>> Capabilities: [90] Message Signalled Interrupts: 64bit+ >>>>> Queue=0/5 Enable- >>>>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>>>> Capabilities: [60] Express Endpoint IRQ 0 >>>>> ====================>>>>> _______________________________________________ >>>>> general mailing list >>>>> general@lists.openfabrics.org >>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>>>> >>>>> To unsubscribe, please visit >>>>> http://openib.org/mailman/listinfo/openib-general >>>>> >>>> >>>> >>>> >>>> -- >>>> . . . s u b b u >>>> "You''ve got to be original, because if you''re like someone else, what do >>>> they need you for?" >>>> >>>> >>> >>> >>> -- >>> . . . s u b b u >>> "You''ve got to be original, because if you''re like someone else, what do >>> they need you for?" >>> >>> >> >> >> -- >> . . . s u b b u >> "You''ve got to be original, because if you''re like someone else, what do >> they need you for?" >> >> > > > -- > . . . s u b b u > "You''ve got to be original, because if you''re like someone else, what do > they need you for?" > >-- . . . s u b b u "You''ve got to be original, because if you''re like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jiang, Yunhong
2009-Feb-12 08:01 UTC
RE: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
Can you please share more information how will the ib_mthca do QUERY_FW? Through config space access? Through MMIO access? I think more information will be helpful. The only thing seems strange to me is, from "Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M]" , seems the MMIO is disabled? Thanks Yunhong Jiang ________________________________ From: subbu kl [mailto:subbukl@gmail.com] Sent: 2009年2月12日 15:46 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com; general@lists.openfabrics.org Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working so back to square one ? Why QUERY_FW should fail in domU ? ~subbu On Thu, Feb 12, 2009 at 12:30 PM, Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> wrote: DomU access config space through pcibackend, so that message is ok. ________________________________ From: subbu kl [mailto:subbukl@gmail.com<mailto:subbukl@gmail.com>] Sent: 2009年2月12日 14:59 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working So getting PCI config space access in domU will solve the problem ? if so how can I achieve that ? ~subbu On Thu, Feb 12, 2009 at 12:26 PM, Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> wrote: Sorry that seems the original mail has tried the permissive already :$ How will So how will the card do the QEUREY_FW command?Through config space or through MMIO? Following information is something strange, why all the MMIO range is disabled? Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] As for the following information, I think it should be harmless since domU has no method of config spacess access method. PCI: Fatal: No PCI config space access function found Thanks Yunhong Jiang ________________________________ From: subbu kl [mailto:subbukl@gmail.com<mailto:subbukl@gmail.com>] Sent: 2009年2月12日 14:43 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working oops missed it, well now I dont see that enable permissive...message. here goes the messages what I got in dom0 while booting domU tap tap-1-51712: 2 getting info pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:0e:00.0 to 64 ACPI: PCI interrupt for device 0000:0e:00.0 disabled ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready xenbr0: topology change detected, propagating xenbr0: port 3(vif1.0) entering forwarding state any suspicious message ? any Idea why I get that : PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. message in domU bootup message ? ~subbu On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> wrote: So any changes in dom0's dmesg? ________________________________ From: subbu kl [mailto:subbukl@gmail.com<mailto:subbukl@gmail.com>] Sent: 2009年2月12日 13:52 To: Jiang, Yunhong Cc: David Brown; xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working no luck ! dmesg in XEN PV guest shows : ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 even after executingh the following in dom0: #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive I am getting the follwing messages on the console as part of the initial bootup messages of the guest: Started domain rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. after executing the following in dom0 : #xm create -c rhel52_64_3 so, problem persisits, ~subbu 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com<mailto:yunhong.jiang@intel.com>> Seems it is because PCI frontend try to write some configuration space that PCIback has no config_field entry to support it. I think you can firstly try to do as dom0's dmesg suggested: "see permissive attribute in sysfs" (it should be "set permissive attribute...", I think). BTW, where you got following log? That seems suggest config space function not found. PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found." -- Yunhong Jiang ________________________________ From: xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com> [mailto:xen-devel-bounces@lists.xensource.com<mailto:xen-devel-bounces@lists.xensource.com>] On Behalf Of subbu kl Sent: 2009年2月11日 22:18 To: David Brown Cc: xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com>; general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> Subject: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working I am getting the same QUERY_FW failed on RHEL5.2 with xenxen paravirtualized guest with pciback module. No one seems to have tried answering this question on the list, let me ping xen-devel and ofed people again. after executing in dom0 echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind #dmesg ACPI: PCI interrupt for device 0000:0e:00.0 disabled tap tap-1-51712: 2 getting info tap tap-2-51712: 2 getting info pciback 0000:0e:00.0: seizing device PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 ACPI: PCI interrupt for device 0000:0e:00.0 disabled #xm create -c rhel52_64_3 PCI: Fatal: No PCI config space access function found rtc: IRQ 8 is not free. i8042.c: No controller found. GUEST dmesg: ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 in dom0: Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not ready Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device 0000:0e:00.0 disabled some more details - [root@p128 ~]# rpm -qa | grep xen kernel-xen-2.6.18-92.1.22.el5 xen-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 xen-libs-3.0.3-64.el5_2.9 [root@p128 ~]# ibv_devinfo hca_id: mthca0 fw_ver: 5.3.0 node_guid: 0002:c902:0022:cd48 sys_image_guid: 0002:c902:0022:cd4b vendor_id: 0x02c9 vendor_part_id: 25218 hw_ver: 0x20 board_id: MT_0370130002 phys_port_cnt: 2 port: 1 state: PORT_INIT (2) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 port: 2 state: PORT_DOWN (1) max_mtu: 2048 (4) active_mtu: 512 (2) sm_lid: 0 port_lid: 0 port_lmc: 0x00 any help greatly appreciated. ~subbu On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> wrote: Okay so my question to the openfabrics guys is, why would the OFED drivers fail to read the firmware? Any thoughts? Thanks, - David Brown ---------- Forwarded message ---------- From: David Brown <dmlb2000@gmail.com<mailto:dmlb2000@gmail.com>> Date: Thu, Sep 11, 2008 at 2:24 PM Subject: pciback module not working To: xen-users@lists.xensource.com<mailto:xen-users@lists.xensource.com>, xen-devel@lists.xensource.com<mailto:xen-devel@lists.xensource.com> This issue was brought up about a year and a half ago. So I'll bring it up again and see if anything happens. I've got an infiniband network and am attempting to pass the infiniband card through the host and give it to the guest. I'm working with standard CentOS 5.2 on both guest and host with their provided xen (3.0.3 ish). I've also attempted to install the newest Xen 3.3 and use their standard host kernel and that did the same thing. The guest dmesg output in the guest is similar on both permissive and normal mode. I'm getting issues with detecting the firmware on the card for some reason... Any help would be appreciated. Thanks, - David Brown === GUEST dmesg output ==ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) ib_mthca: Initializing 0000:00:00.0 PCI: Enabling device 0000:00:00.0 (0000 -> 0002) PCI: Setting latency timer of device 0000:00:00.0 to 64 ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. ib_mthca: probe of 0000:00:00.0 failed with error -11 ====================== === Host modprobe.conf ==alias eth0 bnx2 alias eth1 bnx2 alias scsi_hostadapter cciss options pciback hide=(41:00.0) ==================== === Host lspci output ==# lspci -vs 41:00.0 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ==================== This makes sure it get loaded first off before anything else. === Host mkinitrd cmd ==# mkinitrd -f --with=pciback --preload pciback /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen =================== === Host pciback dmesg ==pciback 0000:41:00.0: Driver tried to write to a read-only configuration space field at offset 0x44, size 2. This may be harmless, but if you have problems with your device: 1) see permissive attribute in sysfs 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ===================== === Host pciback dmesg (after setting it permissive) ==pciback 0000:41:00.0: enabling permissive mode configuration space accesses! pciback 0000:41:00.0: permissive mode is potentially unsafe! pciback: vpci: 0000:41:00.0: assign to virtual slot 0 device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) PCI: Enabling device 0000:41:00.0 (0000 -> 0002) ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:41:00.0 to 64 ACPI: PCI interrupt for device 0000:41:00.0 disabled ======================================== === Guest lspci output ==# lspci -v 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20) Subsystem: Hewlett-Packard Company Unknown device 170a Flags: fast devsel, IRQ 16 Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Capabilities: [60] Express Endpoint IRQ 0 ====================_______________________________________________ general mailing list general@lists.openfabrics.org<mailto:general@lists.openfabrics.org> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" -- . . . s u b b u "You've got to be original, because if you're like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
subbu kl
2009-Feb-12 08:20 UTC
Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
did a quick search, I believe its MMIO, as it is in file - http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/drivers/infiniband/hw/mthca/mthca_main.c mthca_QUERY_FW <http://www.cs.fsu.edu/%7Ebaker/devices/lxr/http/ident?i=mthca_QUERY_FW>() is resulting into mthca_QUERY_FW() which inturn will result into mthca_cmd_post_dbell()/mthca_cmd_post_hcr() which inturn results into __raw_writel((__force u32) cpu_to_be32(in_param >> 32), ptr + offs[0]); in the file - http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/drivers/infiniband/hw/mthca/mthca_cmd.c OFED people should be more helpful here to comment if I have missed out something. Roland any clue? ~subbu On Thu, Feb 12, 2009 at 1:31 PM, Jiang, Yunhong <yunhong.jiang@intel.com>wrote:> Can you please share more information how will the ib_mthca do QUERY_FW? > Through config space access? Through MMIO access? I think more information > will be helpful. The only thing seems strange to me is, from "Memory at > fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M]" , seems the MMIO > is disabled? > > Thanks > Yunhong Jiang > > ------------------------------ > *From:* subbu kl [mailto:subbukl@gmail.com] > *Sent:* 2009年2月12日 15:46 > > *To:* Jiang, Yunhong > *Cc:* David Brown; xen-devel@lists.xensource.com; > general@lists.openfabrics.org > *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not > working > > so back to square one ? > Why QUERY_FW should fail in domU ? > > ~subbu > > On Thu, Feb 12, 2009 at 12:30 PM, Jiang, Yunhong <yunhong.jiang@intel.com>wrote: > >> DomU access config space through pcibackend, so that message is ok. >> >> ------------------------------ >> *From:* subbu kl [mailto:subbukl@gmail.com] >> *Sent:* 2009年2月12日 14:59 >> >> *To:* Jiang, Yunhong >> *Cc:* David Brown; xen-devel@lists.xensource.com; >> general@lists.openfabrics.org >> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >> working >> >> So getting PCI config space access in domU will solve the problem ? if >> so how can I achieve that ? >> >> ~subbu >> >> On Thu, Feb 12, 2009 at 12:26 PM, Jiang, Yunhong <yunhong.jiang@intel.com >> > wrote: >> >>> Sorry that seems the original mail has tried the permissive already :$ >>> How will So how will the card do the QEUREY_FW command?Through config >>> space or through MMIO? Following information is something strange, why all >>> the MMIO range is disabled? >>> >>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>> >>> As for the following information, I think it should be harmless since >>> domU has no method of config spacess access method. >>> PCI: Fatal: No PCI config space access function found >>> >>> Thanks >>> Yunhong Jiang >>> >>> ------------------------------ >>> *From:* subbu kl [mailto:subbukl@gmail.com] >>> *Sent:* 2009年2月12日 14:43 >>> >>> *To:* Jiang, Yunhong >>> *Cc:* David Brown; xen-devel@lists.xensource.com; >>> general@lists.openfabrics.org >>> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>> working >>> >>> oops missed it, >>> >>> well now I dont see that enable permissive...message. here goes the >>> messages what I got in dom0 while booting domU >>> >>> tap tap-1-51712: 2 getting info >>> pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 >>> device vif1.0 entered promiscuous mode >>> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >>> blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) >>> PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) >>> ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>> PCI: Setting latency timer of device 0000:0e:00.0 to 64 >>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>> ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready >>> xenbr0: topology change detected, propagating >>> xenbr0: port 3(vif1.0) entering forwarding state >>> >>> any suspicious message ? >>> any Idea why I get that : >>> PCI: Fatal: No PCI config space access function found >>> rtc: IRQ 8 is not free. >>> >>> message in domU bootup message ? >>> >>> ~subbu >>> >>> On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong < >>> yunhong.jiang@intel.com> wrote: >>> >>>> So any changes in dom0''s dmesg? >>>> >>>> >>>> ------------------------------ >>>> *From:* subbu kl [mailto:subbukl@gmail.com] >>>> *Sent:* 2009年2月12日 13:52 >>>> *To:* Jiang, Yunhong >>>> *Cc:* David Brown; xen-devel@lists.xensource.com; >>>> general@lists.openfabrics.org >>>> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>>> working >>>> >>>> no luck ! >>>> dmesg in XEN PV guest shows : >>>> >>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >>>> ib_mthca: Initializing 0000:00:00.0 >>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>> >>>> even after executingh the following in dom0: >>>> >>>> #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive >>>> >>>> I am getting the follwing messages on the console as part of the initial >>>> bootup messages of the guest: >>>> >>>> Started domain rhel52_64_3 >>>> PCI: Fatal: No PCI config space access function found >>>> rtc: IRQ 8 is not free. >>>> i8042.c: No controller found. >>>> >>>> after executing the following in dom0 : >>>> #xm create -c rhel52_64_3 >>>> >>>> >>>> so, problem persisits, >>>> >>>> ~subbu >>>> >>>> >>>> 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com> >>>> >>>>> Seems it is because PCI frontend try to write some configuration >>>>> space that PCIback has no config_field entry to support it. >>>>> I think you can firstly try to do as dom0''s dmesg suggested: "see >>>>> permissive attribute in sysfs" (it should be "set permissive attribute...", >>>>> I think). >>>>> >>>>> BTW, where you got following log? That seems suggest config space >>>>> function not found. >>>>> >>>>> PCI: Fatal: No PCI config space access function found >>>>> rtc: IRQ 8 is not free. >>>>> i8042.c: No controller found." >>>>> >>>>> -- Yunhong Jiang >>>>> >>>>> ------------------------------ >>>>> *From:* xen-devel-bounces@lists.xensource.com [mailto: >>>>> xen-devel-bounces@lists.xensource.com] *On Behalf Of *subbu kl >>>>> *Sent:* 2009年2月11日 22:18 >>>>> *To:* David Brown >>>>> *Cc:* xen-devel@lists.xensource.com; general@lists.openfabrics.org >>>>> *Subject:* [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>>>> working >>>>> >>>>> I am getting the same QUERY_FW failed on RHEL5.2 with xenxen >>>>> paravirtualized guest with pciback module. >>>>> >>>>> No one seems to have tried answering this question on the list, let me >>>>> ping xen-devel and ofed people again. >>>>> >>>>> after executing in dom0 >>>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind >>>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot >>>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind >>>>> >>>>> #dmesg >>>>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>>>> tap tap-1-51712: 2 getting info >>>>> tap tap-2-51712: 2 getting info >>>>> pciback 0000:0e:00.0: seizing device >>>>> PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) >>>>> ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>>>> >>>>> #xm create -c rhel52_64_3 >>>>> >>>>> PCI: Fatal: No PCI config space access function found >>>>> rtc: IRQ 8 is not free. >>>>> i8042.c: No controller found. >>>>> >>>>> >>>>> GUEST dmesg: >>>>> >>>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >>>>> ib_mthca: Initializing 0000:00:00.0 >>>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>>> >>>>> in dom0: >>>>> Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info >>>>> Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to >>>>> virtual slot 0 >>>>> Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode >>>>> Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not >>>>> ready >>>>> Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, >>>>> protocol 1 (x86_64-abi) >>>>> Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to >>>>> write to a read-only configuration space field at offset 0x44, size 2. This >>>>> may be harmless, but if you have problems with your device: >>>>> Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs >>>>> Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel >>>>> mailing list along with details of your device obtained from lspci. >>>>> Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> >>>>> 0002) >>>>> Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI >>>>> 16 (level, low) -> IRQ 16 >>>>> Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device >>>>> 0000:0e:00.0 disabled >>>>> >>>>> >>>>> >>>>> some more details - [root@p128 ~]# rpm -qa | grep xen >>>>> kernel-xen-2.6.18-92.1.22.el5 >>>>> xen-3.0.3-64.el5_2.9 >>>>> xen-libs-3.0.3-64.el5_2.9 >>>>> xen-libs-3.0.3-64.el5_2.9 >>>>> >>>>> [root@p128 ~]# ibv_devinfo >>>>> hca_id: mthca0 >>>>> fw_ver: 5.3.0 >>>>> node_guid: 0002:c902:0022:cd48 >>>>> sys_image_guid: 0002:c902:0022:cd4b >>>>> vendor_id: 0x02c9 >>>>> vendor_part_id: 25218 >>>>> hw_ver: 0x20 >>>>> board_id: MT_0370130002 >>>>> phys_port_cnt: 2 >>>>> port: 1 >>>>> state: PORT_INIT (2) >>>>> max_mtu: 2048 (4) >>>>> active_mtu: 512 (2) >>>>> sm_lid: 0 >>>>> port_lid: 0 >>>>> port_lmc: 0x00 >>>>> >>>>> port: 2 >>>>> state: PORT_DOWN (1) >>>>> max_mtu: 2048 (4) >>>>> active_mtu: 512 (2) >>>>> sm_lid: 0 >>>>> port_lid: 0 >>>>> port_lmc: 0x00 >>>>> >>>>> >>>>> any help greatly appreciated. >>>>> >>>>> ~subbu >>>>> >>>>> On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com>wrote: >>>>> >>>>>> Okay so my question to the openfabrics guys is, why would the OFED >>>>>> drivers fail to read the firmware? >>>>>> >>>>>> Any thoughts? >>>>>> >>>>>> Thanks, >>>>>> - David Brown >>>>>> >>>>>> >>>>>> ---------- Forwarded message ---------- >>>>>> From: David Brown <dmlb2000@gmail.com> >>>>>> Date: Thu, Sep 11, 2008 at 2:24 PM >>>>>> Subject: pciback module not working >>>>>> To: xen-users@lists.xensource.com, xen-devel@lists.xensource.com >>>>>> >>>>>> >>>>>> This issue was brought up about a year and a half ago. So I''ll bring >>>>>> it up again and see if anything happens. >>>>>> >>>>>> I''ve got an infiniband network and am attempting to pass the >>>>>> infiniband card through the host and give it to the guest. >>>>>> I''m working with standard CentOS 5.2 on both guest and host with their >>>>>> provided xen (3.0.3 ish). I''ve also attempted to install the newest >>>>>> Xen 3.3 and use their standard host kernel and that did the same >>>>>> thing. The guest dmesg output in the guest is similar on both >>>>>> permissive and normal mode. >>>>>> >>>>>> I''m getting issues with detecting the firmware on the card for some >>>>>> reason... >>>>>> >>>>>> Any help would be appreciated. >>>>>> >>>>>> Thanks, >>>>>> - David Brown >>>>>> >>>>>> === GUEST dmesg output ==>>>>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) >>>>>> ib_mthca: Initializing 0000:00:00.0 >>>>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>>>> ======================>>>>>> >>>>>> === Host modprobe.conf ==>>>>>> alias eth0 bnx2 >>>>>> alias eth1 bnx2 >>>>>> alias scsi_hostadapter cciss >>>>>> options pciback hide=(41:00.0) >>>>>> ====================>>>>>> >>>>>> === Host lspci output ==>>>>>> # lspci -vs 41:00.0 >>>>>> 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>>>>> HCA] (rev 20) >>>>>> Subsystem: Hewlett-Packard Company Unknown device 170a >>>>>> Flags: fast devsel, IRQ 16 >>>>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] >>>>>> [size=1M] >>>>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>>>> Capabilities: [40] Power Management version 2 >>>>>> Capabilities: [48] Vital Product Data >>>>>> Capabilities: [90] Message Signalled Interrupts: 64bit+ >>>>>> Queue=0/5 Enable- >>>>>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>>>>> Capabilities: [60] Express Endpoint IRQ 0 >>>>>> ====================>>>>>> >>>>>> This makes sure it get loaded first off before anything else. >>>>>> === Host mkinitrd cmd ==>>>>>> # mkinitrd -f --with=pciback --preload pciback >>>>>> /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen >>>>>> ===================>>>>>> >>>>>> === Host pciback dmesg ==>>>>>> pciback 0000:41:00.0: Driver tried to write to a read-only >>>>>> configuration space field at offset 0x44, size 2. This may be >>>>>> harmless, but if you have problems with your device: >>>>>> 1) see permissive attribute in sysfs >>>>>> 2) report problems to the xen-devel mailing list along with details of >>>>>> your device obtained from lspci. >>>>>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>>>>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>>>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>>>>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>>>>> =====================>>>>>> >>>>>> === Host pciback dmesg (after setting it permissive) ==>>>>>> pciback 0000:41:00.0: enabling permissive mode configuration space >>>>>> accesses! >>>>>> pciback 0000:41:00.0: permissive mode is potentially unsafe! >>>>>> pciback: vpci: 0000:41:00.0: assign to virtual slot 0 >>>>>> device vif1.0 entered promiscuous mode >>>>>> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >>>>>> blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) >>>>>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>>>>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>>>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>>>>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>>>>> ========================================>>>>>> >>>>>> === Guest lspci output ==>>>>>> # lspci -v >>>>>> 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>>>>> HCA] (rev 20) >>>>>> Subsystem: Hewlett-Packard Company Unknown device 170a >>>>>> Flags: fast devsel, IRQ 16 >>>>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] >>>>>> [size=1M] >>>>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>>>> Capabilities: [40] Power Management version 2 >>>>>> Capabilities: [48] Vital Product Data >>>>>> Capabilities: [90] Message Signalled Interrupts: 64bit+ >>>>>> Queue=0/5 Enable- >>>>>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>>>>> Capabilities: [60] Express Endpoint IRQ 0 >>>>>> ====================>>>>>> _______________________________________________ >>>>>> general mailing list >>>>>> general@lists.openfabrics.org >>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>>>>> >>>>>> To unsubscribe, please visit >>>>>> http://openib.org/mailman/listinfo/openib-general >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> . . . s u b b u >>>>> "You''ve got to be original, because if you''re like someone else, what >>>>> do they need you for?" >>>>> >>>>> >>>> >>>> >>>> -- >>>> . . . s u b b u >>>> "You''ve got to be original, because if you''re like someone else, what do >>>> they need you for?" >>>> >>>> >>> >>> >>> -- >>> . . . s u b b u >>> "You''ve got to be original, because if you''re like someone else, what do >>> they need you for?" >>> >>> >> >> >> -- >> . . . s u b b u >> "You''ve got to be original, because if you''re like someone else, what do >> they need you for?" >> >> > > > -- > . . . s u b b u > "You''ve got to be original, because if you''re like someone else, what do > they need you for?" > >-- . . . s u b b u "You''ve got to be original, because if you''re like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
subbu kl
2009-Feb-16 09:51 UTC
Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not working
anyone any clue on this ? As I am seeing the same issue with centos 5.2 HVM guest also with xen 3.4 unstable ! ~subbu On Thu, Feb 12, 2009 at 1:50 PM, subbu kl <subbukl@gmail.com> wrote:> did a quick search, > I believe its MMIO, as it is > > in file - http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/drivers/infiniband/hw/mthca/mthca_main.c <http://www.cs.fsu.edu/%7Ebaker/devices/lxr/http/source/linux/drivers/infiniband/hw/mthca/mthca_main.c> > mthca_QUERY_FW <http://www.cs.fsu.edu/%7Ebaker/devices/lxr/http/ident?i=mthca_QUERY_FW>() is resulting into > > mthca_QUERY_FW() which inturn will result into mthca_cmd_post_dbell()/mthca_cmd_post_hcr() which inturn results into > __raw_writel((__force u32) cpu_to_be32(in_param >> 32), ptr + offs[0]); > > > in the file - http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/drivers/infiniband/hw/mthca/mthca_cmd.c <http://www.cs.fsu.edu/%7Ebaker/devices/lxr/http/source/linux/drivers/infiniband/hw/mthca/mthca_cmd.c> > > OFED people should be more helpful here to comment if I have missed out > something. Roland any clue? > > ~subbu > > > On Thu, Feb 12, 2009 at 1:31 PM, Jiang, Yunhong <yunhong.jiang@intel.com>wrote: > >> Can you please share more information how will the ib_mthca do QUERY_FW? >> Through config space access? Through MMIO access? I think more information >> will be helpful. The only thing seems strange to me is, from "Memory at >> fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M]" , seems the MMIO >> is disabled? >> >> Thanks >> Yunhong Jiang >> >> ------------------------------ >> *From:* subbu kl [mailto:subbukl@gmail.com] >> *Sent:* 2009年2月12日 15:46 >> >> *To:* Jiang, Yunhong >> *Cc:* David Brown; xen-devel@lists.xensource.com; >> general@lists.openfabrics.org >> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >> working >> >> so back to square one ? >> Why QUERY_FW should fail in domU ? >> >> ~subbu >> >> On Thu, Feb 12, 2009 at 12:30 PM, Jiang, Yunhong <yunhong.jiang@intel.com >> > wrote: >> >>> DomU access config space through pcibackend, so that message is ok. >>> >>> ------------------------------ >>> *From:* subbu kl [mailto:subbukl@gmail.com] >>> *Sent:* 2009年2月12日 14:59 >>> >>> *To:* Jiang, Yunhong >>> *Cc:* David Brown; xen-devel@lists.xensource.com; >>> general@lists.openfabrics.org >>> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>> working >>> >>> So getting PCI config space access in domU will solve the problem ? if >>> so how can I achieve that ? >>> >>> ~subbu >>> >>> On Thu, Feb 12, 2009 at 12:26 PM, Jiang, Yunhong < >>> yunhong.jiang@intel.com> wrote: >>> >>>> Sorry that seems the original mail has tried the permissive already :$ >>>> How will So how will the card do the QEUREY_FW command?Through config >>>> space or through MMIO? Following information is something strange, why all >>>> the MMIO range is disabled? >>>> >>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M] >>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>> >>>> As for the following information, I think it should be harmless since >>>> domU has no method of config spacess access method. >>>> PCI: Fatal: No PCI config space access function found >>>> >>>> Thanks >>>> Yunhong Jiang >>>> >>>> ------------------------------ >>>> *From:* subbu kl [mailto:subbukl@gmail.com] >>>> *Sent:* 2009年2月12日 14:43 >>>> >>>> *To:* Jiang, Yunhong >>>> *Cc:* David Brown; xen-devel@lists.xensource.com; >>>> general@lists.openfabrics.org >>>> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>>> working >>>> >>>> oops missed it, >>>> >>>> well now I dont see that enable permissive...message. here goes the >>>> messages what I got in dom0 while booting domU >>>> >>>> tap tap-1-51712: 2 getting info >>>> pciback: vpci: 0000:0e:00.0: assign to virtual slot 0 >>>> device vif1.0 entered promiscuous mode >>>> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >>>> blktap: ring-ref 9, event-channel 9, protocol 1 (x86_64-abi) >>>> PCI: Enabling device 0000:0e:00.0 (0000 -> 0002) >>>> ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>> PCI: Setting latency timer of device 0000:0e:00.0 to 64 >>>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>>> ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready >>>> xenbr0: topology change detected, propagating >>>> xenbr0: port 3(vif1.0) entering forwarding state >>>> >>>> any suspicious message ? >>>> any Idea why I get that : >>>> PCI: Fatal: No PCI config space access function found >>>> rtc: IRQ 8 is not free. >>>> >>>> message in domU bootup message ? >>>> >>>> ~subbu >>>> >>>> On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong < >>>> yunhong.jiang@intel.com> wrote: >>>> >>>>> So any changes in dom0''s dmesg? >>>>> >>>>> >>>>> ------------------------------ >>>>> *From:* subbu kl [mailto:subbukl@gmail.com] >>>>> *Sent:* 2009年2月12日 13:52 >>>>> *To:* Jiang, Yunhong >>>>> *Cc:* David Brown; xen-devel@lists.xensource.com; >>>>> general@lists.openfabrics.org >>>>> *Subject:* Re: [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>>>> working >>>>> >>>>> no luck ! >>>>> dmesg in XEN PV guest shows : >>>>> >>>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >>>>> ib_mthca: Initializing 0000:00:00.0 >>>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>>> >>>>> even after executingh the following in dom0: >>>>> >>>>> #echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/permissive >>>>> >>>>> I am getting the follwing messages on the console as part of the >>>>> initial bootup messages of the guest: >>>>> >>>>> Started domain rhel52_64_3 >>>>> PCI: Fatal: No PCI config space access function found >>>>> rtc: IRQ 8 is not free. >>>>> i8042.c: No controller found. >>>>> >>>>> after executing the following in dom0 : >>>>> #xm create -c rhel52_64_3 >>>>> >>>>> >>>>> so, problem persisits, >>>>> >>>>> ~subbu >>>>> >>>>> >>>>> 2009/2/12 Jiang, Yunhong <yunhong.jiang@intel.com> >>>>> >>>>>> Seems it is because PCI frontend try to write some configuration >>>>>> space that PCIback has no config_field entry to support it. >>>>>> I think you can firstly try to do as dom0''s dmesg suggested: "see >>>>>> permissive attribute in sysfs" (it should be "set permissive attribute...", >>>>>> I think). >>>>>> >>>>>> BTW, where you got following log? That seems suggest config space >>>>>> function not found. >>>>>> >>>>>> PCI: Fatal: No PCI config space access function found >>>>>> rtc: IRQ 8 is not free. >>>>>> i8042.c: No controller found." >>>>>> >>>>>> -- Yunhong Jiang >>>>>> >>>>>> ------------------------------ >>>>>> *From:* xen-devel-bounces@lists.xensource.com [mailto: >>>>>> xen-devel-bounces@lists.xensource.com] *On Behalf Of *subbu kl >>>>>> *Sent:* 2009年2月11日 22:18 >>>>>> *To:* David Brown >>>>>> *Cc:* xen-devel@lists.xensource.com; general@lists.openfabrics.org >>>>>> *Subject:* [Xen-devel] Re: [ofa-general] Fwd: pciback module not >>>>>> working >>>>>> >>>>>> I am getting the same QUERY_FW failed on RHEL5.2 with xenxen >>>>>> paravirtualized guest with pciback module. >>>>>> >>>>>> No one seems to have tried answering this question on the list, let me >>>>>> ping xen-devel and ofed people again. >>>>>> >>>>>> after executing in dom0 >>>>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/ib_mthca/unbind >>>>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/new_slot >>>>>> echo -n 0000:0e:00.0 > /sys/bus/pci/drivers/pciback/bind >>>>>> >>>>>> #dmesg >>>>>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>>>>> tap tap-1-51712: 2 getting info >>>>>> tap tap-2-51712: 2 getting info >>>>>> pciback 0000:0e:00.0: seizing device >>>>>> PCI: Enabling device 0000:0e:00.0 (0140 -> 0142) >>>>>> ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>>>> ACPI: PCI interrupt for device 0000:0e:00.0 disabled >>>>>> >>>>>> #xm create -c rhel52_64_3 >>>>>> >>>>>> PCI: Fatal: No PCI config space access function found >>>>>> rtc: IRQ 8 is not free. >>>>>> i8042.c: No controller found. >>>>>> >>>>>> >>>>>> GUEST dmesg: >>>>>> >>>>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4, 2008) >>>>>> ib_mthca: Initializing 0000:00:00.0 >>>>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>>>> >>>>>> in dom0: >>>>>> Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2 getting info >>>>>> Feb 11 19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to >>>>>> virtual slot 0 >>>>>> Feb 11 19:44:37 p128 kernel: device vif3.0 entered promiscuous mode >>>>>> Feb 11 19:44:37 p128 kernel: ADDRCONF(NETDEV_UP): vif3.0: link is not >>>>>> ready >>>>>> Feb 11 19:44:39 p128 kernel: blktap: ring-ref 9, event-channel 9, >>>>>> protocol 1 (x86_64-abi) >>>>>> Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0: Driver tried to >>>>>> write to a read-only configuration space field at offset 0x44, size 2. This >>>>>> may be harmless, but if you have problems with your device: >>>>>> Feb 11 19:44:48 p128 kernel: 1) see permissive attribute in sysfs >>>>>> Feb 11 19:44:48 p128 kernel: 2) report problems to the xen-devel >>>>>> mailing list along with details of your device obtained from lspci. >>>>>> Feb 11 19:44:48 p128 kernel: PCI: Enabling device 0000:0e:00.0 (0000 >>>>>> -> 0002) >>>>>> Feb 11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] -> >>>>>> GSI 16 (level, low) -> IRQ 16 >>>>>> Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device >>>>>> 0000:0e:00.0 disabled >>>>>> >>>>>> >>>>>> >>>>>> some more details - [root@p128 ~]# rpm -qa | grep xen >>>>>> kernel-xen-2.6.18-92.1.22.el5 >>>>>> xen-3.0.3-64.el5_2.9 >>>>>> xen-libs-3.0.3-64.el5_2.9 >>>>>> xen-libs-3.0.3-64.el5_2.9 >>>>>> >>>>>> [root@p128 ~]# ibv_devinfo >>>>>> hca_id: mthca0 >>>>>> fw_ver: 5.3.0 >>>>>> node_guid: 0002:c902:0022:cd48 >>>>>> sys_image_guid: 0002:c902:0022:cd4b >>>>>> vendor_id: 0x02c9 >>>>>> vendor_part_id: 25218 >>>>>> hw_ver: 0x20 >>>>>> board_id: MT_0370130002 >>>>>> phys_port_cnt: 2 >>>>>> port: 1 >>>>>> state: PORT_INIT (2) >>>>>> max_mtu: 2048 (4) >>>>>> active_mtu: 512 (2) >>>>>> sm_lid: 0 >>>>>> port_lid: 0 >>>>>> port_lmc: 0x00 >>>>>> >>>>>> port: 2 >>>>>> state: PORT_DOWN (1) >>>>>> max_mtu: 2048 (4) >>>>>> active_mtu: 512 (2) >>>>>> sm_lid: 0 >>>>>> port_lid: 0 >>>>>> port_lmc: 0x00 >>>>>> >>>>>> >>>>>> any help greatly appreciated. >>>>>> >>>>>> ~subbu >>>>>> >>>>>> On Sat, Oct 18, 2008 at 4:54 AM, David Brown <dmlb2000@gmail.com>wrote: >>>>>> >>>>>>> Okay so my question to the openfabrics guys is, why would the OFED >>>>>>> drivers fail to read the firmware? >>>>>>> >>>>>>> Any thoughts? >>>>>>> >>>>>>> Thanks, >>>>>>> - David Brown >>>>>>> >>>>>>> >>>>>>> ---------- Forwarded message ---------- >>>>>>> From: David Brown <dmlb2000@gmail.com> >>>>>>> Date: Thu, Sep 11, 2008 at 2:24 PM >>>>>>> Subject: pciback module not working >>>>>>> To: xen-users@lists.xensource.com, xen-devel@lists.xensource.com >>>>>>> >>>>>>> >>>>>>> This issue was brought up about a year and a half ago. So I''ll bring >>>>>>> it up again and see if anything happens. >>>>>>> >>>>>>> I''ve got an infiniband network and am attempting to pass the >>>>>>> infiniband card through the host and give it to the guest. >>>>>>> I''m working with standard CentOS 5.2 on both guest and host with >>>>>>> their >>>>>>> provided xen (3.0.3 ish). I''ve also attempted to install the newest >>>>>>> Xen 3.3 and use their standard host kernel and that did the same >>>>>>> thing. The guest dmesg output in the guest is similar on both >>>>>>> permissive and normal mode. >>>>>>> >>>>>>> I''m getting issues with detecting the firmware on the card for some >>>>>>> reason... >>>>>>> >>>>>>> Any help would be appreciated. >>>>>>> >>>>>>> Thanks, >>>>>>> - David Brown >>>>>>> >>>>>>> === GUEST dmesg output ==>>>>>>> ib_mthca: Mellanox InfiniBand HCA driver v1.0 (February 28, 2008) >>>>>>> ib_mthca: Initializing 0000:00:00.0 >>>>>>> PCI: Enabling device 0000:00:00.0 (0000 -> 0002) >>>>>>> PCI: Setting latency timer of device 0000:00:00.0 to 64 >>>>>>> ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting. >>>>>>> ib_mthca: probe of 0000:00:00.0 failed with error -11 >>>>>>> ======================>>>>>>> >>>>>>> === Host modprobe.conf ==>>>>>>> alias eth0 bnx2 >>>>>>> alias eth1 bnx2 >>>>>>> alias scsi_hostadapter cciss >>>>>>> options pciback hide=(41:00.0) >>>>>>> ====================>>>>>>> >>>>>>> === Host lspci output ==>>>>>>> # lspci -vs 41:00.0 >>>>>>> 41:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>>>>>> HCA] (rev 20) >>>>>>> Subsystem: Hewlett-Packard Company Unknown device 170a >>>>>>> Flags: fast devsel, IRQ 16 >>>>>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] >>>>>>> [size=1M] >>>>>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>>>>> Capabilities: [40] Power Management version 2 >>>>>>> Capabilities: [48] Vital Product Data >>>>>>> Capabilities: [90] Message Signalled Interrupts: 64bit+ >>>>>>> Queue=0/5 Enable- >>>>>>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>>>>>> Capabilities: [60] Express Endpoint IRQ 0 >>>>>>> ====================>>>>>>> >>>>>>> This makes sure it get loaded first off before anything else. >>>>>>> === Host mkinitrd cmd ==>>>>>>> # mkinitrd -f --with=pciback --preload pciback >>>>>>> /boot/initrd-2.6.18-92.1.10.el5xen.img 2.6.18-92.1.10.el5xen >>>>>>> ===================>>>>>>> >>>>>>> === Host pciback dmesg ==>>>>>>> pciback 0000:41:00.0: Driver tried to write to a read-only >>>>>>> configuration space field at offset 0x44, size 2. This may be >>>>>>> harmless, but if you have problems with your device: >>>>>>> 1) see permissive attribute in sysfs >>>>>>> 2) report problems to the xen-devel mailing list along with details >>>>>>> of >>>>>>> your device obtained from lspci. >>>>>>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>>>>>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>>>>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>>>>>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>>>>>> =====================>>>>>>> >>>>>>> === Host pciback dmesg (after setting it permissive) ==>>>>>>> pciback 0000:41:00.0: enabling permissive mode configuration space >>>>>>> accesses! >>>>>>> pciback 0000:41:00.0: permissive mode is potentially unsafe! >>>>>>> pciback: vpci: 0000:41:00.0: assign to virtual slot 0 >>>>>>> device vif1.0 entered promiscuous mode >>>>>>> ADDRCONF(NETDEV_UP): vif1.0: link is not ready >>>>>>> blkback: ring-ref 9, event-channel 28, protocol 1 (x86_64-abi) >>>>>>> PCI: Enabling device 0000:41:00.0 (0000 -> 0002) >>>>>>> ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16 >>>>>>> PCI: Setting latency timer of device 0000:41:00.0 to 64 >>>>>>> ACPI: PCI interrupt for device 0000:41:00.0 disabled >>>>>>> ========================================>>>>>>> >>>>>>> === Guest lspci output ==>>>>>>> # lspci -v >>>>>>> 00:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx >>>>>>> HCA] (rev 20) >>>>>>> Subsystem: Hewlett-Packard Company Unknown device 170a >>>>>>> Flags: fast devsel, IRQ 16 >>>>>>> Memory at fdc00000 (64-bit, non-prefetchable) [disabled] >>>>>>> [size=1M] >>>>>>> Memory at fd000000 (64-bit, prefetchable) [disabled] [size=8M] >>>>>>> Capabilities: [40] Power Management version 2 >>>>>>> Capabilities: [48] Vital Product Data >>>>>>> Capabilities: [90] Message Signalled Interrupts: 64bit+ >>>>>>> Queue=0/5 Enable- >>>>>>> Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 >>>>>>> Capabilities: [60] Express Endpoint IRQ 0 >>>>>>> ====================>>>>>>> _______________________________________________ >>>>>>> general mailing list >>>>>>> general@lists.openfabrics.org >>>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general >>>>>>> >>>>>>> To unsubscribe, please visit >>>>>>> http://openib.org/mailman/listinfo/openib-general >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> . . . s u b b u >>>>>> "You''ve got to be original, because if you''re like someone else, what >>>>>> do they need you for?" >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> . . . s u b b u >>>>> "You''ve got to be original, because if you''re like someone else, what >>>>> do they need you for?" >>>>> >>>>> >>>> >>>> >>>> -- >>>> . . . s u b b u >>>> "You''ve got to be original, because if you''re like someone else, what do >>>> they need you for?" >>>> >>>> >>> >>> >>> -- >>> . . . s u b b u >>> "You''ve got to be original, because if you''re like someone else, what do >>> they need you for?" >>> >>> >> >> >> -- >> . . . s u b b u >> "You''ve got to be original, because if you''re like someone else, what do >> they need you for?" >> >> > > > -- > . . . s u b b u > "You''ve got to be original, because if you''re like someone else, what do > they need you for?" >-- . . . s u b b u "You''ve got to be original, because if you''re like someone else, what do they need you for?" _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel