Yaron Illouz
2015-Jan-28 16:06 UTC
Re: [libvirt-users] Sr-iov passthrough - no packet arrive to guest
I can see from different post that if working with sr-iov, i should work
with vlan
Is this an obligation to work with vlan if working with sr-iov?
If not according to which parameter will the different vf get the
traffic.
Let's say i declare max_vfs=7, how will the traffic be seperated between
the vm?
However till i get an answer i tried to work with vlan
And i still don't get traffic in vm, but i see packet arriving in the
physical interface of the host
I tried to do the following to define the vlan
In virsh edit
<interface type='hostdev' managed='yes'>
<mac address='52:54:00:60:7f:57'/>
<source>
<address type='pci' domain='0x0000'
bus='0x15' slot='0x10'
function='0x0'/>
</source>
<vlan>
<tag id='44'/>
</vlan>
<alias name='hostdev0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x07'
function='0x0'/>
</interface>
In host ip link
23: eth6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq
state UP qlen 1000
link/ether 38:ea:a7:8f:fa:98 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 52:54:00:60:7f:57, vlan 44
In guest
modprob 8021q
eth1 Link encap:Ethernet HWaddr 52:54:00:60:7F:57
inet6 addr: fe80::5054:ff:fe60:7f57/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:1156 (1.1 KiB)
eth1.44 Link encap:Ethernet HWaddr 52:54:00:60:7F:57
inet6 addr: fe80::5054:ff:fe60:7f57/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
________________________________
From: Yaron Illouz
Sent: Tuesday, January 27, 2015 9:56 PM
To: 'libvirt-users@redhat.com'
Subject: Sr-iov passthrough - no packet arrive to guest
Hi
I am trying to do passthrough to guest using sriov and libvirt
I tried to use both techniques from here
http://wiki.libvirt.org/page/Networking#Assignment_with_.3Chostdev.3E
<http://wiki.libvirt.org/page/Networking#Assignment_with_.3Chostdev.3E>
Assignment from a pool of SRIOV VFs in a libvirt <network> definition
Assignment with <interface type='hostdev'> (SRIOV devices only)
<network>
<name>passthrough-eth6</name>
<forward mode='hostdev' managed='yes'>
<pf dev='eth6'/>
</forward>
</network>
I launch the vm, there is a vm with virt-install .... --network
network=passthrough-eth6
I inject traffic with a generator so I defined it in promisc mode
without an ip address.(ifconfig eth6 promisc up)
In my physical device eth6 i can see RX increasing. But in the guest
side nothing arrive in the RX? Why?
I don't know in which log to look at to understand the problem
I defined the network-script/eth1 file in the guest is defined as
static (not dhcp) as the interface it is connected to in host have no
ip.
I have a another interface on the host connected via ovs and libvirt and
it is working fine - you can see it in log as vnet0(thanks to this
mailing-list help).
In the guest part I attached the log but I think interesting part is
Jan 27 17:03:21 localhost kernel: ixgbevf 0000:00:04.0: NIC Link is Up,
10 Gbps
I also attached dmesg output from guest
Here is the log from the host side, I can't see an error.
/var/log/message
Jan 27 12:02:29 1235 avahi-daemon[2598]: Withdrawing address record for
fe80::fc54:ff:fe92:7352 on vnet0.
Jan 27 12:02:29 1235 kernel: device vnet0 left promiscuous mode
Jan 27 12:02:30 1235 kernel: ixgbe 0000:15:00.0: setting MAC
a2:fc:a8:89:2a:8f on VF 0
Jan 27 12:02:30 1235 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl
--timeout=5 -- --if-exists del-port vnet0
Jan 27 12:02:30 1235 ovs-vswitchd: 00180|bridge|ERR|bridge virbr0:
mirror mirror-1-to-3 does not specify output; ignoring
Jan 27 12:02:31 1235 ntpd[2905]: Deleting interface #28 vnet0,
fe80::fc54:ff:fe92:7352#123, interface stats: received=0, sent=0,
dropped=0, active_time=981 secs
Jan 27 12:02:31 1235 ntpd[2905]: peers refreshed
Jan 27 12:02:43 1235 kernel: ixgbe 0000:15:00.0: Reload the VF driver to
make this change effective.
Jan 27 12:02:43 1235 kernel: kvm: 7957: cpu0 unhandled rdmsr: 0x345
Jan 27 12:02:43 1235 kernel: kvm: 7957: cpu0 unhandled wrmsr: 0x680 data
0
...
Jan 27 12:02:51 1235 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl
--timeout=5 -- --may-exist add-port br3 vnet0 -- set Interface vnet0
"external-ids:attached-mac=\"52:54:00:43:BF:8C\"" -- set
Interface vnet0
"external-ids:iface-id=\"309ed431-417f-2843-89b8-179a066aa595\""
-- set
Interface vnet0
"external-ids:vm-id=\"870e3b65-953b-01ee-84a3-c2d05300c2d9\""
-- set
Interface vnet0 external-ids:iface-status=active
Jan 27 12:02:51 1235 ovs-vswitchd: 00183|bridge|ERR|bridge virbr0:
mirror mirror-1-to-3 does not specify output; ignoring
Jan 27 12:02:51 1235 kernel: device vnet0 entered promiscuous mode
Jan 27 12:02:51 1235 kernel: ixgbe 0000:15:00.0: setting MAC
52:54:00:d9:f1:bd on VF 0
Jan 27 12:02:51 1235 kernel: ixgbe 0000:15:00.0: Reload the VF driver to
make this change effective.
Jan 27 12:02:51 1235 kernel: pci-stub 0000:15:10.0: enabling device
(0000 -> 0002)
Jan 27 12:02:52 1235 avahi-daemon[2598]: Registering new address record
for fe80::fc54:ff:fe43:bf8c on vnet0.*.
Jan 27 12:02:54 1235 ntpd[2905]: Listen normally on 29 vnet0
fe80::fc54:ff:fe43:bf8c UDP 123
Jan 27 12:02:54 1235 ntpd[2905]: peers refreshed
Jan 27 12:03:17 1235 kernel: __ratelimit: 23 callbacks suppressed
Jan 27 12:03:17 1235 kernel: kvm: 8006: cpu0 disabled perfctr wrmsr:
0xc1 data 0xabcd
Jan 27 12:03:20 1235 kernel: ixgbe 0000:15:00.0: eth6: VF Reset msg
received from vf 0
And this is the /var/log/libvirt/qemu/probe8.log from the host
2015-01-27 17:02:51.762+0000: starting up
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none
/usr/libexec/qemu-kvm -name probe8 -S -M rhel6.6.0 -enable-kvm -m 4096
-realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid
870e3b65-953b-01ee-84a3-c2d05300c2d9 -nodefconfig -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/probe8.monitor,server,n
owait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7
-device
ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,ad
dr=0x5 -device
ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1
-device
ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -drive
file=/data/KVM/comp_VNF_MaveriQ_Probe2-cloud8.qcow2,if=none,id=drive-ide
0-0-0,format=qcow2,cache=none -device
ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
-drive
file=/var/lib/libvirt/images/configuration8.iso,if=none,id=drive-virtio-
disk0,format=raw,cache=none -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=v
irtio-disk0 -netdev tap,fd=25,id=hostnet0 -device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:43:bf:8c,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -vnc 0.0.0.0:0 -vga cirrus
-device
pci-assign,host=15:10.0,id=hostdev0,configfd=26,bus=pci.0,addr=0x4
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg
timestamp=on
char device redirected to /dev/pts/1
I can see that virtual function succeeded because I got the following
output
virsh net-dumpxml passthrough-eth6
<network connections='1'>
<name>passthrough-eth6</name>
<uuid>f8fd37cd-3215-e4b0-17b1-0b5515140db9</uuid>
<forward mode='hostdev' managed='yes'>
<pf dev='eth6'/>
<address type='pci' domain='0x0000' bus='0x15'
slot='0x10'
function='0x0'/>
<address type='pci' domain='0x0000' bus='0x15'
slot='0x10'
function='0x2'/>
<address type='pci' domain='0x0000' bus='0x15'
slot='0x10'
function='0x4'/>
<address type='pci' domain='0x0000' bus='0x15'
slot='0x10'
function='0x6'/>
...
</forward>
</network>
lspci | grep Intel
15:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
15:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
15:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
15:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
....
And Intel-IOMMU: enabled in the dmesg
>From host dmesg
ixgbe 0000:15:00.0: setting MAC 52:54:00:97:c6:86 on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
device vnet0 entered promiscuous mode
pci-stub 0000:15:10.0: enabling device (0000 -> 0002)
assign device 0:15:10.0
vnet0: no IPv6 routers present
__ratelimit: 23 callbacks suppressed
kvm: 6896: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 3
pci-stub 0000:15:10.6: claimed by stub
device vnet0 left promiscuous mode
pci-stub 0000:15:10.0: restoring config space at offset 0x1 (was
0x100000, writing 0x100004)
ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
kvm: 7207: cpu0 unhandled rdmsr: 0x345
kvm: 7207: cpu0 unhandled wrmsr: 0x680 data 0
...
ixgbe 0000:15:00.0: setting MAC 52:54:00:16:b6:0c on VF 3
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
device vnet0 entered promiscuous mode
pci-stub 0000:15:10.6: enabling device (0000 -> 0002)
assign device 0:15:10.6
vnet0: no IPv6 routers present
__ratelimit: 24 callbacks suppressed
kvm: 7254: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 3
pci-stub 0000:15:10.6: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.6: irq 63 for MSI/MSI-X
pci-stub 0000:15:10.6: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.6: irq 63 for MSI/MSI-X
device vnet0 left promiscuous mode
pci-stub 0000:15:10.6: restoring config space at offset 0x1 (was
0x100000, writing 0x100004)
ixgbe 0000:15:00.0: setting MAC ba:c1:bd:19:c1:d5 on VF 3
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
device vnet0 entered promiscuous mode
ixgbe 0000:15:00.0: setting MAC 52:54:00:e2:39:62 on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
pci-stub 0000:15:10.0: enabling device (0000 -> 0002)
assign device 0:15:10.0
vnet0: no IPv6 routers present
kvm: 7639: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X
device vnet0 left promiscuous mode
pci-stub 0000:15:10.0: restoring config space at offset 0x1 (was
0x100000, writing 0x100004)
ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
kvm: 7957: cpu0 unhandled rdmsr: 0x345
kvm: 7957: cpu0 unhandled wrmsr: 0x680 data 0
...
device vnet0 entered promiscuous mode
ixgbe 0000:15:00.0: setting MAC 52:54:00:d9:f1:bd on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
pci-stub 0000:15:10.0: enabling device (0000 -> 0002)
assign device 0:15:10.0
vnet0: no IPv6 routers present
__ratelimit: 23 callbacks suppressed
kvm: 8006: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
...
Yaron Illouz
2015-Jan-28 23:27 UTC
Re: [libvirt-users] Sr-iov passthrough - no packet arrive to guest
I read a bit more about the card that i am using and
The problem was that it filtered packet because it a trace that i replay
and not real traffic
So i changed the destination mac adress in the pcap to be the same as vm
and all packet got into the vm.
________________________________
From: libvirt-users-bounces@redhat.com
[mailto:libvirt-users-bounces@redhat.com] On Behalf Of Yaron Illouz
Sent: Wednesday, January 28, 2015 6:06 PM
To: libvirt-users@redhat.com
Subject: Re: [libvirt-users] Sr-iov passthrough - no packet arrive to
guest
I can see from different post that if working with sr-iov, i should work
with vlan
Is this an obligation to work with vlan if working with sr-iov?
If not according to which parameter will the different vf get the
traffic.
Let's say i declare max_vfs=7, how will the traffic be seperated between
the vm?
However till i get an answer i tried to work with vlan
And i still don't get traffic in vm, but i see packet arriving in the
physical interface of the host
I tried to do the following to define the vlan
In virsh edit
<interface type='hostdev' managed='yes'>
<mac address='52:54:00:60:7f:57'/>
<source>
<address type='pci' domain='0x0000'
bus='0x15' slot='0x10'
function='0x0'/>
</source>
<vlan>
<tag id='44'/>
</vlan>
<alias name='hostdev0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x07'
function='0x0'/>
</interface>
In host ip link
23: eth6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq
state UP qlen 1000
link/ether 38:ea:a7:8f:fa:98 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 52:54:00:60:7f:57, vlan 44
In guest
modprob 8021q
eth1 Link encap:Ethernet HWaddr 52:54:00:60:7F:57
inet6 addr: fe80::5054:ff:fe60:7f57/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:1156 (1.1 KiB)
eth1.44 Link encap:Ethernet HWaddr 52:54:00:60:7F:57
inet6 addr: fe80::5054:ff:fe60:7f57/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
________________________________
From: Yaron Illouz
Sent: Tuesday, January 27, 2015 9:56 PM
To: 'libvirt-users@redhat.com'
Subject: Sr-iov passthrough - no packet arrive to guest
Hi
I am trying to do passthrough to guest using sriov and libvirt
I tried to use both techniques from here
http://wiki.libvirt.org/page/Networking#Assignment_with_.3Chostdev.3E
<http://wiki.libvirt.org/page/Networking#Assignment_with_.3Chostdev.3E>
Assignment from a pool of SRIOV VFs in a libvirt <network> definition
Assignment with <interface type='hostdev'> (SRIOV devices only)
<network>
<name>passthrough-eth6</name>
<forward mode='hostdev' managed='yes'>
<pf dev='eth6'/>
</forward>
</network>
I launch the vm, there is a vm with virt-install .... --network
network=passthrough-eth6
I inject traffic with a generator so I defined it in promisc mode
without an ip address.(ifconfig eth6 promisc up)
In my physical device eth6 i can see RX increasing. But in the guest
side nothing arrive in the RX? Why?
I don't know in which log to look at to understand the problem
I defined the network-script/eth1 file in the guest is defined as
static (not dhcp) as the interface it is connected to in host have no
ip.
I have a another interface on the host connected via ovs and libvirt and
it is working fine - you can see it in log as vnet0(thanks to this
mailing-list help).
In the guest part I attached the log but I think interesting part is
Jan 27 17:03:21 localhost kernel: ixgbevf 0000:00:04.0: NIC Link is Up,
10 Gbps
I also attached dmesg output from guest
Here is the log from the host side, I can't see an error.
/var/log/message
Jan 27 12:02:29 1235 avahi-daemon[2598]: Withdrawing address record for
fe80::fc54:ff:fe92:7352 on vnet0.
Jan 27 12:02:29 1235 kernel: device vnet0 left promiscuous mode
Jan 27 12:02:30 1235 kernel: ixgbe 0000:15:00.0: setting MAC
a2:fc:a8:89:2a:8f on VF 0
Jan 27 12:02:30 1235 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl
--timeout=5 -- --if-exists del-port vnet0
Jan 27 12:02:30 1235 ovs-vswitchd: 00180|bridge|ERR|bridge virbr0:
mirror mirror-1-to-3 does not specify output; ignoring
Jan 27 12:02:31 1235 ntpd[2905]: Deleting interface #28 vnet0,
fe80::fc54:ff:fe92:7352#123, interface stats: received=0, sent=0,
dropped=0, active_time=981 secs
Jan 27 12:02:31 1235 ntpd[2905]: peers refreshed
Jan 27 12:02:43 1235 kernel: ixgbe 0000:15:00.0: Reload the VF driver to
make this change effective.
Jan 27 12:02:43 1235 kernel: kvm: 7957: cpu0 unhandled rdmsr: 0x345
Jan 27 12:02:43 1235 kernel: kvm: 7957: cpu0 unhandled wrmsr: 0x680 data
0
...
Jan 27 12:02:51 1235 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl
--timeout=5 -- --may-exist add-port br3 vnet0 -- set Interface vnet0
"external-ids:attached-mac=\"52:54:00:43:BF:8C\"" -- set
Interface vnet0
"external-ids:iface-id=\"309ed431-417f-2843-89b8-179a066aa595\""
-- set
Interface vnet0
"external-ids:vm-id=\"870e3b65-953b-01ee-84a3-c2d05300c2d9\""
-- set
Interface vnet0 external-ids:iface-status=active
Jan 27 12:02:51 1235 ovs-vswitchd: 00183|bridge|ERR|bridge virbr0:
mirror mirror-1-to-3 does not specify output; ignoring
Jan 27 12:02:51 1235 kernel: device vnet0 entered promiscuous mode
Jan 27 12:02:51 1235 kernel: ixgbe 0000:15:00.0: setting MAC
52:54:00:d9:f1:bd on VF 0
Jan 27 12:02:51 1235 kernel: ixgbe 0000:15:00.0: Reload the VF driver to
make this change effective.
Jan 27 12:02:51 1235 kernel: pci-stub 0000:15:10.0: enabling device
(0000 -> 0002)
Jan 27 12:02:52 1235 avahi-daemon[2598]: Registering new address record
for fe80::fc54:ff:fe43:bf8c on vnet0.*.
Jan 27 12:02:54 1235 ntpd[2905]: Listen normally on 29 vnet0
fe80::fc54:ff:fe43:bf8c UDP 123
Jan 27 12:02:54 1235 ntpd[2905]: peers refreshed
Jan 27 12:03:17 1235 kernel: __ratelimit: 23 callbacks suppressed
Jan 27 12:03:17 1235 kernel: kvm: 8006: cpu0 disabled perfctr wrmsr:
0xc1 data 0xabcd
Jan 27 12:03:20 1235 kernel: ixgbe 0000:15:00.0: eth6: VF Reset msg
received from vf 0
And this is the /var/log/libvirt/qemu/probe8.log from the host
2015-01-27 17:02:51.762+0000: starting up
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none
/usr/libexec/qemu-kvm -name probe8 -S -M rhel6.6.0 -enable-kvm -m 4096
-realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid
870e3b65-953b-01ee-84a3-c2d05300c2d9 -nodefconfig -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/probe8.monitor,server,n
owait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7
-device
ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,ad
dr=0x5 -device
ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1
-device
ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -drive
file=/data/KVM/comp_VNF_MaveriQ_Probe2-cloud8.qcow2,if=none,id=drive-ide
0-0-0,format=qcow2,cache=none -device
ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
-drive
file=/var/lib/libvirt/images/configuration8.iso,if=none,id=drive-virtio-
disk0,format=raw,cache=none -device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=v
irtio-disk0 -netdev tap,fd=25,id=hostnet0 -device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:43:bf:8c,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -vnc 0.0.0.0:0 -vga cirrus
-device
pci-assign,host=15:10.0,id=hostdev0,configfd=26,bus=pci.0,addr=0x4
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg
timestamp=on
char device redirected to /dev/pts/1
I can see that virtual function succeeded because I got the following
output
virsh net-dumpxml passthrough-eth6
<network connections='1'>
<name>passthrough-eth6</name>
<uuid>f8fd37cd-3215-e4b0-17b1-0b5515140db9</uuid>
<forward mode='hostdev' managed='yes'>
<pf dev='eth6'/>
<address type='pci' domain='0x0000' bus='0x15'
slot='0x10'
function='0x0'/>
<address type='pci' domain='0x0000' bus='0x15'
slot='0x10'
function='0x2'/>
<address type='pci' domain='0x0000' bus='0x15'
slot='0x10'
function='0x4'/>
<address type='pci' domain='0x0000' bus='0x15'
slot='0x10'
function='0x6'/>
...
</forward>
</network>
lspci | grep Intel
15:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
15:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
15:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
15:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
....
And Intel-IOMMU: enabled in the dmesg
>From host dmesg
ixgbe 0000:15:00.0: setting MAC 52:54:00:97:c6:86 on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
device vnet0 entered promiscuous mode
pci-stub 0000:15:10.0: enabling device (0000 -> 0002)
assign device 0:15:10.0
vnet0: no IPv6 routers present
__ratelimit: 23 callbacks suppressed
kvm: 6896: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 3
pci-stub 0000:15:10.6: claimed by stub
device vnet0 left promiscuous mode
pci-stub 0000:15:10.0: restoring config space at offset 0x1 (was
0x100000, writing 0x100004)
ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
kvm: 7207: cpu0 unhandled rdmsr: 0x345
kvm: 7207: cpu0 unhandled wrmsr: 0x680 data 0
...
ixgbe 0000:15:00.0: setting MAC 52:54:00:16:b6:0c on VF 3
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
device vnet0 entered promiscuous mode
pci-stub 0000:15:10.6: enabling device (0000 -> 0002)
assign device 0:15:10.6
vnet0: no IPv6 routers present
__ratelimit: 24 callbacks suppressed
kvm: 7254: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 3
pci-stub 0000:15:10.6: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.6: irq 63 for MSI/MSI-X
pci-stub 0000:15:10.6: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.6: irq 63 for MSI/MSI-X
device vnet0 left promiscuous mode
pci-stub 0000:15:10.6: restoring config space at offset 0x1 (was
0x100000, writing 0x100004)
ixgbe 0000:15:00.0: setting MAC ba:c1:bd:19:c1:d5 on VF 3
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
device vnet0 entered promiscuous mode
ixgbe 0000:15:00.0: setting MAC 52:54:00:e2:39:62 on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
pci-stub 0000:15:10.0: enabling device (0000 -> 0002)
assign device 0:15:10.0
vnet0: no IPv6 routers present
kvm: 7639: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X
device vnet0 left promiscuous mode
pci-stub 0000:15:10.0: restoring config space at offset 0x1 (was
0x100000, writing 0x100004)
ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
kvm: 7957: cpu0 unhandled rdmsr: 0x345
kvm: 7957: cpu0 unhandled wrmsr: 0x680 data 0
...
device vnet0 entered promiscuous mode
ixgbe 0000:15:00.0: setting MAC 52:54:00:d9:f1:bd on VF 0
ixgbe 0000:15:00.0: Reload the VF driver to make this change effective.
pci-stub 0000:15:10.0: enabling device (0000 -> 0002)
assign device 0:15:10.0
vnet0: no IPv6 routers present
__ratelimit: 23 callbacks suppressed
kvm: 8006: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd
ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0
pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X
...
Reasonably Related Threads
- Sr-iov passthrough - no packet arrive to guest
- NIC Stability Problems Under Xen 4.4 / CentOS 6 / Linux 3.18
- "failed to setup INTx fd: Operation not permitted" error when using PCI passthrough
- NIC Stability Problems Under Xen 4.4 / CentOS 6 / Linux 3.18
- NIC Stability Problems Under Xen 4.4 / CentOS 6 / Linux 3.18