Matthew Hook
2013-Oct-01 10:08 UTC
Re: Does anybody in the Xen community have any experience with the xen-pciback driver and MSI-X??
Greetings Andrew, I have managed to get SR-IOV working to a degree under Xen but it required quite a late kernel build. It seems to kinda work with kernel 3.11.2. I also get this error. pciback 0000:06:00.1: seizing device Oct 1 01:10:56 xen4 kernel: [ 77.196621] pciback 0000:06:00.1: enabling device (0000 -> 0002) Oct 1 01:10:56 xen4 kernel: [ 77.196711] pciback 0000:06:00.1: MSI-X preparation failed (-38) The other thing I found was that you need to use the latest OFED stack 3.5. If you''re using Ubuntu there are some PPA''s for that. I got these errors in the opensm log which I can''t find any information about. Oct 01 02:07:37 965995 [4590C700] 0x01 -> validate_requested_mgid: ERR 1B01: Wrong MGID Prefix 0x8000 must be 0xFF Oct 01 02:07:37 966024 [4590C700] 0x01 -> mcmr_rcv_create_new_mgrp: ERR 1B22: Invalid requested MGID Oct 01 02:07:37 966346 [4A115700] 0x01 -> validate_requested_mgid: ERR 1B01: Wrong MGID Prefix 0x8000 must be 0xFF Oct 01 02:07:37 966394 [4A115700] 0x01 -> mcmr_rcv_create_new_mgrp: ERR 1B22: Invalid requested MGID I wonder if this is related to msi-x and pciback. Regards, Matt> Greetings Xen user community, > > I am interested in using Mellanox ConnectX cards with SR-IOV capabilitiesto passthrough pci-e Virtual Functions (VFs) to Xen guests. The hope is to allow for the use of InfiniBand directly within virtual machines and thereby enable a plethora of high performance computing applications that already leverage InfiniBand interconnects. However, I have run into some issues using the xen-pciback driver and its initialization of MSI-X as required for VFs in Xen. The hardware used is Mellanox Connect X3 MT27500 VPI pci-express cards set up in InfiniBand mode in HP blades with Intel Xeon E5-2670 CPUs and 42GB of memory. SR-IOV is enabled in the system BIOS along with VT-X, and of course VT-d. _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users