Jason Tubnor
2020-Sep-29 01:10 UTC
Fragmentation of em(4) traffic (potentially others) when using vlanhwtag 12.2-BETA1
On Fri, 18 Sep 2020 at 19:52, Jason Tubnor <jason at tubnor.net> wrote:> ---------- Forwarded message --------- > From: Jason Tubnor <jason at tubnor.net> > Date: Thu, 17 Sep 2020 at 10:49 > Subject: Fragmentation of em(4) traffic (potentially others) when using > vlanhwtag 12.2-BETA1 > To: freebsd-stable <freebsd-stable at freebsd.org> > > > Hi, > > Doing some edge case testing with 12.2-BETA1, I have come across > fragmentation issues when vlanhwtag is enabled on the em(4) network device > (device listed below), causing IPSec traffic moving through a bhyve > 'router-on-a-stick' to tank on send. When you disable (-vlanhwtag), this > issue goes away and full speed send is possible. > > Here is a snip of the traffic leaving for the internet after traversing > several VLANs through a bhyve router with vlanhwtag enabled: > > 10:09:54.173412 Dest.443 > Source.60901: . ack 12384 win 492 <nop,nop,sack > 2 {10944:12235} {15373:18253} > (DF) > 10:09:54.173936 Source.60901 > Dest.443: . 38413:39873(1460) ack 3837 win > 1027 (frag 20228:1480 at 0+) > 10:09:54.173937 Source > Dest: (frag 20228:1420 at 1480) > > pciconf -vlbc > > em0 at pci0:0:25:0: class=0x020000 card=0x308617aa chip=0x15028086 > rev=0x04 hdr=0x00 > vendor = 'Intel Corporation' > device = '82579LM Gigabit Network Connection (Lewisville)' > class = network > subclass = ethernet > bar [10] = type Memory, range 32, base 0xf7d00000, size 131072, > enabled > bar [14] = type Memory, range 32, base 0xf7d39000, size 4096, enabled > bar [18] = type I/O Port, range 32, base 0xf080, size 32, enabled > cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message > cap 13[e0] = PCI Advanced Features: FLR TP > > >Further investigation with help from grehan@ this is a LRO issue and not a vlanhwtag issue. When a vlan interface is added to a bridge, LRO isn't automatically disabled. LRO behaviour changed between 11 and 12 and this is why we haven't seen this issue on our 11.x devices where bridges are extensively used with bhyve/tap. Can this be addressed prior to the 12.2 release please. Happy to test any patches. Thanks, Jason.