libvirt users - Aug 2013 - libvirt-qemu/kvm passthru w/Intel 82598EB

Hi,
 I am trying to configure two identical systems with the following setup.

dom0: Centos 6 OS on Apple Xserve i7 server with PCI cards for 4GB FC
(LSI FC949ES) and 10GB Ethernet (Intel 82598EB)

dom1: Centos 5 OS  with passthru mode of PCI 4GB FC and 10GB Ethernet
(ixgbe and mptfc blacklisted in dom0)

Everything starts ok, but networking in dom1 using the 10GB interface
has extremely high rtt w/ping tests (20-30ms avg).

I've tried a few tweaks I've discovered via googling, but nothing seems
to help. (run libvirt as root, set privilege mode, etc)

For validation, I have the same card in Dell hosts running Centos 6
natively and they get an avg rtt of .06ms or better. Furthemore the Dell
will contact the dom1 but the connection has rtt avg times of about 6ms.

With the ethernet interface on the motherboard of the Apple Xserve, I
will see rtt times of .06ms within the dom1's, so I think this points to
a problem with the Intel card passthru usage, perhaps interrupts handling?.

Does anyone know if I need to set any special options for the ixgbe
driver under libvirt?

TIA,

-- 
Laurence Schuler (Larry)                       Laurence.Schuler@nasa.gov
Systems Support                                       ADNET Systems, Inc
Scientific Visualization Studio                 http://svs.gsfc.nasa.gov
NASA/Goddard Space Flight Center, Code 606.4       phone: 1-301-286-1799
Greenbelt, MD 20771                                  fax: 1-301-286-1634
Note: I am not a government employee and have no authority to obligate
any federal, state or local government to perform any action or payment.FC949ES

On Fri, Aug 30, 2013 at 4:07 PM, laurence.schuler
<laurence.schuler@nasa.gov
> wrote:
> Hi,
>  I am trying to configure two identical systems with the following setup.
>
> dom0: Centos 6 OS on Apple Xserve i7 server with PCI cards for 4GB FC
> (LSI FC949ES) and 10GB Ethernet (Intel 82598EB)
>
> dom1: Centos 5 OS  with passthru mode of PCI 4GB FC and 10GB Ethernet
> (ixgbe and mptfc blacklisted in dom0)
>
> Everything starts ok, but networking in dom1 using the 10GB interface
> has extremely high rtt w/ping tests (20-30ms avg).
>
> I've tried a few tweaks I've discovered via googling, but nothing
seems
> to help. (run libvirt as root, set privilege mode, etc)
>
> For validation, I have the same card in Dell hosts running Centos 6
> natively and they get an avg rtt of .06ms or better. Furthemore the Dell
> will contact the dom1 but the connection has rtt avg times of about 6ms.
>
> With the ethernet interface on the motherboard of the Apple Xserve, I
> will see rtt times of .06ms within the dom1's, so I think this points
to
> a problem with the Intel card passthru usage, perhaps interrupts handling?.
>
> Does anyone know if I need to set any special options for the ixgbe
> driver under libvirt?
>
> TIA,
>

Larry,

While I don't know of any specifics for your card that libvirt could help
with, it doesn't mean there aren't any. You might have a better time
asking
on the qemu-users ML at
https://lists.nongnu.org/mailman/listinfo/qemu-discuss but you are dealing
with a very specific card so your range of people with knowledge of that
card might be small.

--
Doug


>
> --
> Laurence Schuler (Larry)                       Laurence.Schuler@nasa.gov
> Systems Support                                       ADNET Systems, Inc
> Scientific Visualization Studio                 http://svs.gsfc.nasa.gov
> NASA/Goddard Space Flight Center, Code 606.4       phone: 1-301-286-1799
> Greenbelt, MD 20771                                  fax: 1-301-286-1634
> Note: I am not a government employee and have no authority to obligate
> any federal, state or local government to perform any action or
> payment.FC949ES
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users@redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
>


-- 
Doug Goldstein