libvirt users - May 2016 - Guest still receive packet with vlan tag (SR-IOV transparent VLAN)

Hello everyone,

I'm having problem with SR-IOV transparent VLAN.
My host is Fedora 22 and my guest is: pfSense 2.3. The NIC is Intel 82576.

I'm using pfSense Packet Capture to check the packet and find that inbound
packet still have the vlan tag. The outbound packet seem fine because the
router can receive DHCP discover packet from the guest and then send DHCP
offer and ARP request packet to the guest (which still have the vlan tag so
the guest does not response).

Loading Ubuntu LiveCD in the same VM does not have this issue so I do not
know what is the cause? Any idea to diagnose?


Another problem I have is libvirt does not correctly restore the vf mac
address to all zero and then some occasion, there will be duplicate mac
address in different vfs which will trigger  spoof checking.
I found the related mail here
https://www.redhat.com/archives/libvir-list/2015-December/msg00478.html
which point to this bug https://bugzilla.redhat.com/show_bug.cgi?id=1302166
. The bug only mention to mellanox cards. Do anyone have the same problem
(with igb/igbvf driver) ? Or is it fixed in upstream?

On Sat, May 14, 2016 at 3:00 AM, Laine Stump <laine@laine.org>
wrote:
> On 05/13/2016 02:21 PM, Long Hoang wrote:
>
> Hello everyone,
>
> I'm having problem with SR-IOV transparent VLAN.
> My host is Fedora 22 and my guest is: pfSense 2.3. The NIC is Intel 82576.
>
> I'm using pfSense Packet Capture to check the packet and find that
inbound
> packet still have the vlan tag. The outbound packet seem fine because the
> router can receive DHCP discover packet from the guest and then send DHCP
> offer and ARP request packet to the guest (which still have the vlan tag so
> the guest does not response).
>
> Loading Ubuntu LiveCD in the same VM does not have this issue so I do not
> know what is the cause? Any idea to diagnose?
>
>
> This is extremely odd. AFAIK, the vlan tag is stripped off on the card
> before it ever gets to the OS in the guest. Are you certain that the guest
> has no knowledge of the vlan tag, and that the tag is set in the card
itself
> (look at the output of "ip link show" for the PF and see if that
VF has a
> vlan tag listed).
>
>
Thank you for your reply. I've try clean install but the same issue.
And "ip link show" show the vlan also.
After dig a bit, I found some interesting link:
https://access.redhat.com/solutions/1126973
https://reviews.freebsd.org/D4788

It seem that VLAN stripping will depend on RXDCTL.VME flag that the
guest driver need to set. And FreeBSD driver does not set it so
FreeBSD guest will still see the VLAN tag. They only mention intel
82599 (ixgb/ixgbe) but I check intel 82576 datasheet and the similar
flag is CTRL.VME. But the 82576 sr-iov driver guide mention the
VMOLR.STRVLAN flag (?)

http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/82576eb-gigabit-ethernet-controller-datasheet.pdf
http://www.intel.com/content/dam/doc/design-guide/82576-sr-iov-driver-companion-guide.pdf
>
>
> Another problem I have is libvirt does not correctly restore the vf mac
> address to all zero and then some occasion, there will be duplicate mac
> address in different vfs which will trigger  spoof checking.
> I found the related mail here
> https://www.redhat.com/archives/libvir-list/2015-December/msg00478.html
> which point to this bug https://bugzilla.redhat.com/show_bug.cgi?id=1302166
> . The bug only mention to mellanox cards. Do anyone have the same problem
> (with igb/igbvf driver) ? Or is it fixed in upstream?
>
>
> The same problem exists in the igb (82576), ixgb and enic drivers (at
> least). I've meant to file a bug against the kernel, but keep getting
> sidetracked while I'm half-finished writing it.
>
> You can kind of work around the problem by setting all your VF macs to
> 00:00:00:00:00:01 at boot time:
>
>       for vf in $(seq 0 6)
>       do
>           ip link set $PF vf 6 mac 00:00:00:00:00:01
>       done
>
> Then when it's resetting the MAC, it will reset to a value the kernel
> accepts.
>
>
Thank you for your work around.