freebsd stable - Apr 2019 - em performs worse than igb (latency wise) in 12?

Hello,

I've upgraded from Stable 11 to Stable 12, and noticed that igb has been 
removed and is now part of em driver. However, the performance seems a 
lot worse. This is using HP 366FLR which is just HP's version of Intel 
i350 I believe.

I've tried incorporating the sysctl settings I used in 11 into 12:

dev.igb.0.fc=0
dev.igb.0.iflib.rx_budget=65535
hw.igb.rx_process_limit="-1"
hw.igb.num_queues="2"
hw.igb.rxd="2048"? # (default 1024)
hw.igb.txd="2048"? # (default 1024)
hw.igb.max_interrupt_rate="32000" # (default 8000)

hw.em.rx_process_limit="-1"
hw.em.num_queues="2"
hw.em.rxd="2048"? # (default 1024)
hw.em.txd="2048"? # (default 1024)
hw.em.max_interrupt_rate="32000"

But that didn't seem to help.

Switching back to 11, of course restored the performance.

The performance hit I'm talking about seems to be with latency, not 
throughput.

To give you an example, this server is serving a website that is 
monitored from different locations and all locations the latency has 
increased by about 30-50% using stable 12.

Any ideas as to how to improve latency in 12? Can you compile igb driver 
in 12? Has something else other than the network driver changed in 12 
that could cause an increase in latency?

On 05/04/2019 16:01, Kris von Mach wrote:
> I've upgraded from Stable 11 to Stable 12, and noticed that igb has
been
> removed and is now part of em driver. However, the performance seems a
> lot worse. This is using HP 366FLR which is just HP's version of Intel
> i350 I believe.
Something odd going on there there - I am using 12-STABLE and I have igb 
just fine, and it attaches to the same hardware that 11 did:

...

[petefrench at dilbert ~]$ ssh turpentine
Last login: Fri Apr  5 18:52:50 2019 from 2a02:b90:3002:411::6
FreeBSD 12.0-STABLE r343538 GENERIC

Baby baby turpentine...

[webadmin at turpentine ~]$ ifconfig igb0
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether ac:1f:6b:46:5e:32
	inet 10.32.10.5 netmask 0xffff0000 broadcast 10.32.255.255
	inet6 fe80::ae1f:6bff:fe46:5e32%igb0 prefixlen 64 scopeid 0x1
	inet6 2a02:1658:1:2:e550::5 prefixlen 64
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>


Do you have a custom kernel, and if so did you see this note in UPDATING?

20170109:
         The igb(4), em(4) and lem(4) ethernet drivers are now 
implemented via
         IFLIB.  If you have a custom kernel configuration that excludes 
em(4)
         but you use igb(4), you need to re-add em(4) to your custom 
configuration.