I am trying to setup SRIOV, on xen 4.0 with debian for ixgbe links, and
having issue.
I first got this message, the virtual device is up but it does not
send/receive anything
-pciback: vpci: 0000:0f:10.0: assign to virtual slot 0
[ 5845.667153] pciback 0000:0f:10.0: Driver tried to write to a read-only
configuration space field at offset 0x72, size 2. This may be harmless, but
if you have problems with your device:
[ 5845.667155] 1) see permissive attribute in sysfs
[ 5845.667156] 2) report problems to the xen-devel mailing list along with
details of your device obtained from lspci.
then I followed
http://technical.bestgrid.org/index.php/Xen:_assigning_PCI_devices_to_a_domain
to change pci permssions,
now pci permission message goes away, but now i get the message below
[14580.450249] xen-pciback: vpci: 0000:0f:10.0: assign to virtual slot 0
[14580.469992] pciback 0000:0f:10.0: enabling device (0000 -> 0002)
[14611.960465] ixgbe 0000:0f:00.0: eth2: Set MAC msg received from VF 0
[14611.979470] ixgbe 0000:0f:00.0: eth2: VF 0 Hung DMA
and I am still not able to use device, even if device says it is up. seems
like VF driver is hung, - i am wondering if there is any way to fix it.
I used the following fix in /etc/xen/xend-pci-quirks.sxp
(ixgbe
(pci_ids
(''8086:10ed''
)
)
(pci_config_space_fields
(''00000072:2:00000000'' #reported by pci
)
)
)
_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
I tried with other VFs too, everytime i seem to get the message [ 2999.007530] pciback 0000:0f:11.6: enabling device (0000 -> 0002) [ 3031.448507] ixgbe 0000:0f:00.0: eth2: Set MAC msg received from VF 7 [ 3031.474977] ixgbe 0000:0f:00.0: eth2: VF 7 Hung DMA Not sure, what is going wrong. thanks, Ashok On Thu, Oct 4, 2012 at 1:21 PM, Ashok Anand <ashok.anand@gmail.com> wrote:> I am trying to setup SRIOV, on xen 4.0 with debian for ixgbe links, and > having issue. > > I first got this message, the virtual device is up but it does not > send/receive anything > > -pciback: vpci: 0000:0f:10.0: assign to virtual slot 0 > [ 5845.667153] pciback 0000:0f:10.0: Driver tried to write to a read-only > configuration space field at offset 0x72, size 2. This may be harmless, but > if you have problems with your device: > [ 5845.667155] 1) see permissive attribute in sysfs > [ 5845.667156] 2) report problems to the xen-devel mailing list along with > details of your device obtained from lspci. > > > then I followed > http://technical.bestgrid.org/index.php/Xen:_assigning_PCI_devices_to_a_domain > > to change pci permssions, > now pci permission message goes away, but now i get the message below > > [14580.450249] xen-pciback: vpci: 0000:0f:10.0: assign to virtual slot 0 > [14580.469992] pciback 0000:0f:10.0: enabling device (0000 -> 0002) > [14611.960465] ixgbe 0000:0f:00.0: eth2: Set MAC msg received from VF 0 > [14611.979470] ixgbe 0000:0f:00.0: eth2: VF 0 Hung DMA > > and I am still not able to use device, even if device says it is up. seems > like VF driver is hung, - i am wondering if there is any way to fix it. > > I used the following fix in /etc/xen/xend-pci-quirks.sxp > (ixgbe > (pci_ids > (''8086:10ed'' > ) > ) > > (pci_config_space_fields > (''00000072:2:00000000'' #reported by pci > ) > ) > ) > > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Have U tried to boot machine WITHOUT IOMMU (i.e. switching OFF DMA in BIOS)? We had issues with AMD based DELL servers (R815) running Debian+Xen when DMA (IOMMU) was ON and also NUMA ON (kernel with NUMA flags ON). It seems some (many??) AMD based platforms may have not full support for DMA/IOMMU (somethimes it works but quite often not) GB> I tried with other VFs too, everytime i seem to get the message > > [ 2999.007530] pciback 0000:0f:11.6: enabling device (0000 -> 0002) > [ 3031.448507] ixgbe 0000:0f:00.0: eth2: Set MAC msg received from VF 7 > [ 3031.474977] ixgbe 0000:0f:00.0: eth2: VF 7 Hung DMA > > Not sure, what is going wrong. > > thanks, > Ashok > > On Thu, Oct 4, 2012 at 1:21 PM, Ashok Anand <ashok.anand@gmail.com> wrote: > >> I am trying to setup SRIOV, on xen 4.0 with debian for ixgbe links, and >> having issue. >> >> I first got this message, the virtual device is up but it does not >> send/receive anything >> >> -pciback: vpci: 0000:0f:10.0: assign to virtual slot 0 >> [ 5845.667153] pciback 0000:0f:10.0: Driver tried to write to a read-only >> configuration space field at offset 0x72, size 2. This may be harmless, but >> if you have problems with your device: >> [ 5845.667155] 1) see permissive attribute in sysfs >> [ 5845.667156] 2) report problems to the xen-devel mailing list along with >> details of your device obtained from lspci. >> >> >> then I followed >> http://technical.bestgrid.org/index.php/Xen:_assigning_PCI_devices_to_a_domain >> >> to change pci permssions, >> now pci permission message goes away, but now i get the message below >> >> [14580.450249] xen-pciback: vpci: 0000:0f:10.0: assign to virtual slot 0 >> [14580.469992] pciback 0000:0f:10.0: enabling device (0000 -> 0002) >> [14611.960465] ixgbe 0000:0f:00.0: eth2: Set MAC msg received from VF 0 >> [14611.979470] ixgbe 0000:0f:00.0: eth2: VF 0 Hung DMA >> >> and I am still not able to use device, even if device says it is up. seems >> like VF driver is hung, - i am wondering if there is any way to fix it. >> >> I used the following fix in /etc/xen/xend-pci-quirks.sxp >> (ixgbe >> (pci_ids >> (''8086:10ed'' >> ) >> ) >> >> (pci_config_space_fields >> (''00000072:2:00000000'' #reported by pci >> ) >> ) >> ) >> >> >> >> >> >> > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users
thanks for your response.. i just tried with iommu=soft,earlyprintk=xen for kernel boot parameters in guest VM - that seemed to work. On Thu, Oct 4, 2012 at 3:22 PM, <G.Bakalarski@icm.edu.pl> wrote:> > Have U tried to boot machine WITHOUT IOMMU (i.e. switching OFF DMA in > BIOS)? > > We had issues with AMD based DELL servers (R815) running Debian+Xen > when DMA (IOMMU) was ON and also NUMA ON (kernel with NUMA flags ON). > It seems some (many??) AMD based platforms may have not full support > for DMA/IOMMU (somethimes it works but quite often not) > > GB > > > I tried with other VFs too, everytime i seem to get the message > > > > [ 2999.007530] pciback 0000:0f:11.6: enabling device (0000 -> 0002) > > [ 3031.448507] ixgbe 0000:0f:00.0: eth2: Set MAC msg received from VF 7 > > [ 3031.474977] ixgbe 0000:0f:00.0: eth2: VF 7 Hung DMA > > > > Not sure, what is going wrong. > > > > thanks, > > Ashok > > > > On Thu, Oct 4, 2012 at 1:21 PM, Ashok Anand <ashok.anand@gmail.com> > wrote: > > > >> I am trying to setup SRIOV, on xen 4.0 with debian for ixgbe links, and > >> having issue. > >> > >> I first got this message, the virtual device is up but it does not > >> send/receive anything > >> > >> -pciback: vpci: 0000:0f:10.0: assign to virtual slot 0 > >> [ 5845.667153] pciback 0000:0f:10.0: Driver tried to write to a > read-only > >> configuration space field at offset 0x72, size 2. This may be harmless, > but > >> if you have problems with your device: > >> [ 5845.667155] 1) see permissive attribute in sysfs > >> [ 5845.667156] 2) report problems to the xen-devel mailing list along > with > >> details of your device obtained from lspci. > >> > >> > >> then I followed > >> > http://technical.bestgrid.org/index.php/Xen:_assigning_PCI_devices_to_a_domain > >> > >> to change pci permssions, > >> now pci permission message goes away, but now i get the message below > >> > >> [14580.450249] xen-pciback: vpci: 0000:0f:10.0: assign to virtual slot 0 > >> [14580.469992] pciback 0000:0f:10.0: enabling device (0000 -> 0002) > >> [14611.960465] ixgbe 0000:0f:00.0: eth2: Set MAC msg received from VF 0 > >> [14611.979470] ixgbe 0000:0f:00.0: eth2: VF 0 Hung DMA > >> > >> and I am still not able to use device, even if device says it is up. > seems > >> like VF driver is hung, - i am wondering if there is any way to fix it. > >> > >> I used the following fix in /etc/xen/xend-pci-quirks.sxp > >> (ixgbe > >> (pci_ids > >> (''8086:10ed'' > >> ) > >> ) > >> > >> (pci_config_space_fields > >> (''00000072:2:00000000'' #reported by pci > >> ) > >> ) > >> ) > >> > >> > >> > >> > >> > >> > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xen.org > > http://lists.xen.org/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users