Monitoring a tcpdump between two systems, a FreeBSD 9.1 system has the following interface: msk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=c011b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINKSTATE> ether 00:11:2f:2a:c7:03 inet 10.0.1.199 netmask 0xffffff00 broadcast 10.0.1.255 inet6 fe80::211:2fff:fe2a:c703%msk0 prefixlen 64 scopeid 0x1 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (100baseTX <full-duplex,flowcontrol,rxpause,txpause>) status: active It sent the following packet: (data content abbreviated) 02:14:42.081617 IP 10.0.1.199.443 > 10.0.1.2.61258: Flags [P.], seq 930:4876, ack 846, win 1040, options [nop,nop,TS val 401838072 ecr 920110183], length 3946 0x0000: 4500 0f9e ea89 4000 4006 2a08 0a00 01c7 E..... at .@.*..... 0x0010: 0a00 0102 01bb ef4a ece1 680b ae37 1bbc .......J..h..7.. 0x0020: 8018 0410 3407 0000 0101 080a 17f3 8ff8 ....4...??. The indicated packet length is 3946 and the load of data shown is that size. The MTU on both interfaces is 1500. The receiving system received 3 packets. There is a router and switch between them. One of them fragmented that packet. This is part of a SSL/TLS exchange and one side or the other is hanging on this and just dropping the connection. I suspect the packet size is the issue. ssldump complains about the packet too and stops monitoring. Could this possibly be related to the hardware checksums?
13.02.2013 17:25, Doug Hardie ?????:> Monitoring a tcpdump between two systems, a FreeBSD 9.1 system has the following interface: > > msk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=c011b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINKSTATE> > ether 00:11:2f:2a:c7:03 > inet 10.0.1.199 netmask 0xffffff00 broadcast 10.0.1.255 > inet6 fe80::211:2fff:fe2a:c703%msk0 prefixlen 64 scopeid 0x1 > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect (100baseTX <full-duplex,flowcontrol,rxpause,txpause>) > status: active > > > It sent the following packet: (data content abbreviated) > > 02:14:42.081617 IP 10.0.1.199.443 > 10.0.1.2.61258: Flags [P.], seq 930:4876, ack 846, win 1040, options [nop,nop,TS val 401838072 ecr 920110183], length 3946 > 0x0000: 4500 0f9e ea89 4000 4006 2a08 0a00 01c7 E..... at .@.*..... > 0x0010: 0a00 0102 01bb ef4a ece1 680b ae37 1bbc .......J..h..7.. > 0x0020: 8018 0410 3407 0000 0101 080a 17f3 8ff8 ....4...??. > > > The indicated packet length is 3946 and the load of data shown is that size. The MTU on both interfaces is 1500. The receiving system received 3 packets. There is a router and switch between them. One of them fragmented that packet. This is part of a SSL/TLS exchange and one side or the other is hanging on this and just dropping the connection. I suspect the packet size is the issue. ssldump complains about the packet too and stops monitoring. Could this possibly be related to the hardware checksums?You have TSO enabled on the interface, so large outgoing TCP packet is pretty normal. It will be split by the NIC. Disable TSO with ifconfig if it interferes with your ssldump. Eugene Grosbein
On 13 February 2013, at 02:29, Eugene Grosbein <egrosbein at rdtc.ru> wrote:> 13.02.2013 17:25, Doug Hardie ?????: >> Monitoring a tcpdump between two systems, a FreeBSD 9.1 system has the following interface: >> >> msk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 >> options=c011b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINKSTATE> >> ether 00:11:2f:2a:c7:03 >> inet 10.0.1.199 netmask 0xffffff00 broadcast 10.0.1.255 >> inet6 fe80::211:2fff:fe2a:c703%msk0 prefixlen 64 scopeid 0x1 >> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >> media: Ethernet autoselect (100baseTX <full-duplex,flowcontrol,rxpause,txpause>) >> status: active >> >> >> It sent the following packet: (data content abbreviated) >> >> 02:14:42.081617 IP 10.0.1.199.443 > 10.0.1.2.61258: Flags [P.], seq 930:4876, ack 846, win 1040, options [nop,nop,TS val 401838072 ecr 920110183], length 3946 >> 0x0000: 4500 0f9e ea89 4000 4006 2a08 0a00 01c7 E..... at .@.*..... >> 0x0010: 0a00 0102 01bb ef4a ece1 680b ae37 1bbc .......J..h..7.. >> 0x0020: 8018 0410 3407 0000 0101 080a 17f3 8ff8 ....4...??. >> >> >> The indicated packet length is 3946 and the load of data shown is that size. The MTU on both interfaces is 1500. The receiving system received 3 packets. There is a router and switch between them. One of them fragmented that packet. This is part of a SSL/TLS exchange and one side or the other is hanging on this and just dropping the connection. I suspect the packet size is the issue. ssldump complains about the packet too and stops monitoring. Could this possibly be related to the hardware checksums? > > You have TSO enabled on the interface, so large outgoing TCP packet is pretty normal. > It will be split by the NIC. Disable TSO with ifconfig if it interferes with your ssldump.Thanks. Now all the packets are 1500 or under. They all are received with a SSL header.
On Wed, Feb 13, 2013 at 01:57:38PM -0800, Doug Hardie wrote:> > On 13 February 2013, at 02:29, Eugene Grosbein <egrosbein at rdtc.ru> wrote: > > > 13.02.2013 17:25, Doug Hardie ??????????: > >> Monitoring a tcpdump between two systems, a FreeBSD 9.1 system has the following interface: > >> > >> msk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > >> options=c011b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINKSTATE> > >> ether 00:11:2f:2a:c7:03 > >> inet 10.0.1.199 netmask 0xffffff00 broadcast 10.0.1.255 > >> inet6 fe80::211:2fff:fe2a:c703%msk0 prefixlen 64 scopeid 0x1 > >> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > >> media: Ethernet autoselect (100baseTX <full-duplex,flowcontrol,rxpause,txpause>) > >> status: active > >> > >> > >> It sent the following packet: (data content abbreviated) > >> > >> 02:14:42.081617 IP 10.0.1.199.443 > 10.0.1.2.61258: Flags [P.], seq 930:4876, ack 846, win 1040, options [nop,nop,TS val 401838072 ecr 920110183], length 3946 > >> 0x0000: 4500 0f9e ea89 4000 4006 2a08 0a00 01c7 E..... at .@.*..... > >> 0x0010: 0a00 0102 01bb ef4a ece1 680b ae37 1bbc .......J..h..7.. > >> 0x0020: 8018 0410 3407 0000 0101 080a 17f3 8ff8 ....4...??????. > >> > >> > >> The indicated packet length is 3946 and the load of data shown is that size. The MTU on both interfaces is 1500. The receiving system received 3 packets. There is a router and switch between them. One of them fragmented that packet. This is part of a SSL/TLS exchange and one side or the other is hanging on this and just dropping the connection. I suspect the packet size is the issue. ssldump complains about the packet too and stops monitoring. Could this possibly be related to the hardware checksums? > > > > You have TSO enabled on the interface, so large outgoing TCP packet is pretty normal. > > It will be split by the NIC. Disable TSO with ifconfig if it interferes with your ssldump. > > Thanks. Now all the packets are 1500 or under. They all are received with a SSL header.If disabling TSO on msk(4) fixed the issue of the remote end dropping/ignoring the packet, that sounds like a bug in msk(4). Yong-Hyeon, do you have any recent msk(4) patches relating to TSO? -- | Jeremy Chadwick jdc at koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |