Harry Schmalzbauer
2016-Jan-29 18:42 UTC
svn commit: r294958 - in stable/10: share/man/man4 sys/dev/e1000 sys/dev/ixgb sys/dev/netmap
Bez?glich Mike Tancsa's Nachricht vom 29.01.2016 19:08 (localtime):> On 1/27/2016 5:31 PM, Marius Strobl wrote: >> Author: marius >> Date: Wed Jan 27 22:31:08 2016 >> New Revision: 294958 >> URL: https://svnweb.freebsd.org/changeset/base/294958 >> >> Log: >> Sync the e1000 drivers with what's in head as of r294327, modulo parts >> that don't apply to stable/10 (driver API, if_inc_counter(), RSS changes > > Hi, > I am seeing some timeouts since upgrading to this rev. I am running > r295008, i386. onboard NIC > > Manufacturer: Supermicro > Product Name: PDSMi > > > em0: Watchdog timeout Queue[0]-- resetting > Interface is RUNNING and ACTIVE > em0: TX Queue 0 ------ > em0: hw tdh = 946, hw tdt = 159 > em0: Tx Queue Status = -2147483648 > em0: TX descriptors avail = 786 > em0: Tx Descriptors avail failure = 0 > em0: RX Queue 0 ------ > em0: hw rdh = 401, hw rdt = 400 > em0: RX discarded packets = 0 > em0: RX Next to Check = 401 > em0: RX Next to Refresh = 400 > em0: link state changed to DOWN > em0: link state changed to UP > em0: Watchdog timeout Queue[0]-- resetting > Interface is RUNNING and ACTIVE > em0: TX Queue 0 ------ > em0: hw tdh = 87, hw tdt = 378 > em0: Tx Queue Status = -2147483648 > em0: TX descriptors avail = 720 > em0: Tx Descriptors avail failure = 0 > em0: RX Queue 0 ------ > em0: hw rdh = 740, hw rdt = 739 > em0: RX discarded packets = 0 > em0: RX Next to Check = 741 > em0: RX Next to Refresh = 740 > em0: link state changed to DOWN > em0: link state changed to UP > Limiting open port RST response from 292 to 200 packets/sec > em0: Watchdog timeout Queue[0]-- resetting > Interface is RUNNING and ACTIVE > em0: TX Queue 0 ------ > em0: hw tdh = 611, hw tdt = 840 > em0: Tx Queue Status = -2147483648 > em0: TX descriptors avail = 773 > em0: Tx Descriptors avail failure = 0 > em0: RX Queue 0 ------ > em0: hw rdh = 660, hw rdt = 659 > em0: RX discarded packets = 0 > em0: RX Next to Check = 660 > em0: RX Next to Refresh = 659 > em0: link state changed to DOWN > em0: link state changed to UP > > > > # pciconf -lBvcb em0 > em0 at pci0:13:0:0: class=0x020000 card=0x108c15d9 chip=0x108c8086 > rev=0x03 hdr=0x00 > vendor = 'Intel Corporation' > device = '82573E Gigabit Ethernet Controller (Copper)'I guess you haven't compiled the kernel with EM_MULTIQUEUE. I can't remember if 82573 is supposed to be able to handle 2 queues. I couldn't help solving your problem anyways, but I found default number of rx/tx descriptors somewhen increased from 1024 to 4096 for my 82574. What does hw.em.txd read with your 82573? Before my EM-MULTIQUEUE problem vanished, reducing hw.em.txd (and rxd) to 256 relaxed the timeout problem a lot. Seems your interface is recovering after watchdog-reset? Mine stayed unusable unitl I triggered ifconfig down/up. Have you checked if disabling TSO changes anything? Probably checking if hw.em.enable_msix changes symptoms could also narrow down the root cause. Hope your problem also vanishes soon :-) -Harry
Mike Tancsa
2016-Jan-29 20:41 UTC
possible em regression (was Re: svn commit: r294958 - in stable/10: share/man/man4 sys/dev/e1000 sys/dev/ixgb sys/dev/netmap)
On 1/29/2016 1:42 PM, Harry Schmalzbauer wrote:>> # pciconf -lBvcb em0 >> em0 at pci0:13:0:0: class=0x020000 card=0x108c15d9 chip=0x108c8086 >> rev=0x03 hdr=0x00 >> vendor = 'Intel Corporation' >> device = '82573E Gigabit Ethernet Controller (Copper)' > > I guess you haven't compiled the kernel with EM_MULTIQUEUE. I can't > remember if 82573 is supposed to be able to handle 2 queues. I couldn't > help solving your problem anyways, but I found default number of rx/tx > descriptors somewhen increased from 1024 to 4096 for my 82574. > What does hw.em.txd read with your 82573? > Before my EM-MULTIQUEUE problem vanished, reducing hw.em.txd (and rxd) > to 256 relaxed the timeout problem a lot. > Seems your interface is recovering after watchdog-reset? Mine stayed > unusable unitl I triggered ifconfig down/up. > Have you checked if disabling TSO changes anything? > Probably checking if hw.em.enable_msix changes symptoms could also > narrow down the root cause. >No multi queue. Stock GENERIC kernel with a couple of things removed. hw.em are just the defaults. I will try without TSO % ifconfig em0 em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO> 21,22c21,22 < cpu I486_CPU < cpu I586_CPU ---> #cpu I486_CPU > #cpu I586_CPU24c24 < ident GENERIC ---> ident vinyl34c34 < options SCTP # Stream Control Transmission Protocol ---> #options SCTP # Stream Control TransmissionProtocol 375a376,383> > device pf > device pflog > options QUOTA > > options ACCEPT_FILTER_HTTP > options ACCEPT_FILTER_DATA >% sysctl -a dev.em.0 dev.em.0.interrupts.rx_overrun: 0 dev.em.0.interrupts.rx_desc_min_thresh: 0 dev.em.0.interrupts.tx_queue_min_thresh: 2 dev.em.0.interrupts.tx_queue_empty: 0 dev.em.0.interrupts.tx_abs_timer: 371 dev.em.0.interrupts.tx_pkt_timer: 359 dev.em.0.interrupts.rx_abs_timer: 0 dev.em.0.interrupts.rx_pkt_timer: 299 dev.em.0.interrupts.asserts: 9741226 dev.em.0.mac_stats.tso_ctx_fail: 0 dev.em.0.mac_stats.tso_txd: 854147 dev.em.0.mac_stats.tx_frames_1024_1522: 4927464 dev.em.0.mac_stats.tx_frames_512_1023: 127639 dev.em.0.mac_stats.tx_frames_256_511: 1116852 dev.em.0.mac_stats.tx_frames_128_255: 133036 dev.em.0.mac_stats.tx_frames_65_127: 4915970 dev.em.0.mac_stats.tx_frames_64: 110245 dev.em.0.mac_stats.mcast_pkts_txd: 10 dev.em.0.mac_stats.bcast_pkts_txd: 17 dev.em.0.mac_stats.good_pkts_txd: 11331206 dev.em.0.mac_stats.total_pkts_txd: 11331206 dev.em.0.mac_stats.good_octets_txd: 8363930804 dev.em.0.mac_stats.good_octets_recvd: 2034174419 dev.em.0.mac_stats.rx_frames_1024_1522: 773394 dev.em.0.mac_stats.rx_frames_512_1023: 113644 dev.em.0.mac_stats.rx_frames_256_511: 1702858 dev.em.0.mac_stats.rx_frames_128_255: 78132 dev.em.0.mac_stats.rx_frames_65_127: 4889577 dev.em.0.mac_stats.rx_frames_64: 1334131 dev.em.0.mac_stats.mcast_pkts_recvd: 0 dev.em.0.mac_stats.bcast_pkts_recvd: 26788 dev.em.0.mac_stats.good_pkts_recvd: 8891736 dev.em.0.mac_stats.total_pkts_recvd: 8891736 dev.em.0.mac_stats.xoff_txd: 0 dev.em.0.mac_stats.xoff_recvd: 0 dev.em.0.mac_stats.xon_txd: 0 dev.em.0.mac_stats.xon_recvd: 0 dev.em.0.mac_stats.coll_ext_errs: 0 dev.em.0.mac_stats.alignment_errs: 0 dev.em.0.mac_stats.crc_errs: 0 dev.em.0.mac_stats.recv_errs: 0 dev.em.0.mac_stats.recv_jabber: 0 dev.em.0.mac_stats.recv_oversize: 0 dev.em.0.mac_stats.recv_fragmented: 0 dev.em.0.mac_stats.recv_undersize: 0 dev.em.0.mac_stats.recv_no_buff: 0 dev.em.0.mac_stats.missed_packets: 0 dev.em.0.mac_stats.defer_count: 0 dev.em.0.mac_stats.sequence_errors: 0 dev.em.0.mac_stats.symbol_errors: 0 dev.em.0.mac_stats.collision_count: 0 dev.em.0.mac_stats.late_coll: 0 dev.em.0.mac_stats.multiple_coll: 0 dev.em.0.mac_stats.single_coll: 0 dev.em.0.mac_stats.excess_coll: 0 dev.em.0.queue_rx_0.rx_irq: 0 dev.em.0.queue_rx_0.rxd_tail: 76 dev.em.0.queue_rx_0.rxd_head: 77 dev.em.0.queue_tx_0.no_desc_avail: 0 dev.em.0.queue_tx_0.tx_irq: 0 dev.em.0.queue_tx_0.txd_tail: 493 dev.em.0.queue_tx_0.txd_head: 493 dev.em.0.fc_low_water: 8740 dev.em.0.fc_high_water: 10240 dev.em.0.rx_control: 67141634 dev.em.0.device_control: 1075053128 dev.em.0.watchdog_timeouts: 3 dev.em.0.rx_overruns: 0 dev.em.0.tx_dma_fail: 0 dev.em.0.mbuf_defrag_fail: 0 dev.em.0.link_irq: 0 dev.em.0.dropped: 0 dev.em.0.eee_control: 1 dev.em.0.rx_processing_limit: 100 dev.em.0.itr: 488 dev.em.0.tx_abs_int_delay: 66 dev.em.0.rx_abs_int_delay: 66 dev.em.0.tx_int_delay: 66 dev.em.0.rx_int_delay: 0 dev.em.0.fc: 3 dev.em.0.debug: -1 dev.em.0.nvm: -1 dev.em.0.%parent: pci13 dev.em.0.%pnpinfo: vendor=0x8086 device=0x108c subvendor=0x15d9 subdevice=0x108c class=0x020000 dev.em.0.%location: slot=0 function=0 dev.em.0.%driver: em dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.4.2 % sysctl -A hw.em hw.em.eee_setting: 1 hw.em.rx_process_limit: 100 hw.em.enable_msix: 1 hw.em.sbp: 0 hw.em.smart_pwr_down: 0 hw.em.txd: 1024 hw.em.rxd: 1024 hw.em.rx_abs_int_delay: 66 hw.em.tx_abs_int_delay: 66 hw.em.rx_int_delay: 0 hw.em.tx_int_delay: 66 hw.em.disable_crc_stripping: 0 -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike at sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/