Xen users - May 2010 - Do bridges or vif defragment IP-packets?

We have a SLES11 system with XEN 3.3 and configured an internal bridge
"intbr1" with interfaces "vif2.1" and "vif9.1" to
two Linux-guests:

# ifconfig (reduced output)

intbr1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:169 errors:0 dropped:0 overruns:0 frame:0
TX packets:85 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1458368 (1.3 Mb) TX bytes:28192 (27.5 Kb)

vif2.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:600 errors:0 dropped:0 overruns:0 frame:0
TX packets:612 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:751740 (734.1 Kb) TX bytes:757842 (740.0 Kb)

vif9.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:198 errors:0 dropped:0 overruns:0 frame:0
TX packets:202 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:260680 (254.5 Kb) TX bytes:263558 (257.3 Kb)

# brctl show (reduced output)
bridge name bridge id             STP enabled interfaces
intbr1           8000.feffffffffff   no                vif2.1
                                                                  vif9.1
                            
When we send an ICMP-packet with 6400 data-bytes (ping -s 6400 -c 1
192.168.39.1) from Linux-guest connected to "vif9.1" to
the other linux-guest connected to "vif2.1" we see with tcpdump
connected to intbr1, that some instance (intbr1? vif9.1) defragmented
the 5 ip-packets from vif9.1 into 1 ip-packet, though mtu-size for the
vif-interfaces, intbr1 and eth-interfaces of the guests
are all set to 1500 bytes:

# tcpdump -i intbr1 -v
tcpdump: WARNING: intbr1: no IPv4 address assigned
tcpdump: listening on intbr1, link-type EN10MB (Ethernet), capture size 96 bytes
14:33:12.853741 IP (tos 0x0, ttl 64, id 62828, offset 0, flags [none], proto
ICMP (1), length 6428) 192.168.39.2 > 192.168.39.1: ICMP echo request, id
2327, seq 1, length 6408
14:33:12.853956 IP (tos 0x0, ttl 64, id 62329, offset 0, flags [none], proto
ICMP (1), length 6428) 192.168.39.1 > 192.168.39.2: ICMP echo reply, id 2327,
seq 1, length 6408
14:33:17.851500 arp who-has 192.168.39.1 tell 192.168.39.2
14:33:17.851581 arp reply 192.168.39.1 is-at 00:16:3e:02:24:db (oui Unknown)

# tcpdump -i vif2.1 -v
tcpdump: WARNING: vif2.1: no IPv4 address assigned
tcpdump: listening on vif2.1, link-type EN10MB (Ethernet), capture size 96 bytes
14:33:12.853798 IP (tos 0x0, ttl 64, id 62828, offset 0, flags [+], proto ICMP
(1), length 1500) 192.168.39.2 > 192.168.39.1: ICMP echo request, id 2327,
seq 1, length 1480
14:33:12.853801 IP (tos 0x0, ttl 64, id 62828, offset 1480, flags [+], proto
ICMP (1), length 1500) 192.168.39.2 > 192.168.39.1: icmp
14:33:12.853803 IP (tos 0x0, ttl 64, id 62828, offset 2960, flags [+], proto
ICMP (1), length 1500) 192.168.39.2 > 192.168.39.1: icmp
14:33:12.853805 IP (tos 0x0, ttl 64, id 62828, offset 4440, flags [+], proto
ICMP (1), length 1500) 192.168.39.2 > 192.168.39.1: icmp
14:33:12.853806 IP (tos 0x0, ttl 64, id 62828, offset 5920, flags [none], proto
ICMP (1), length 508) 192.168.39.2 > 192.168.39.1: icmp
14:33:12.853948 IP (tos 0x0, ttl 64, id 62329, offset 0, flags [+], proto ICMP
(1), length 1500) 192.168.39.1 > 192.168.39.2: ICMP echo reply, id 2327, seq
1, length 1480
14:33:12.853948 IP (tos 0x0, ttl 64, id 62329, offset 1480, flags [+], proto
ICMP (1), length 1500) 192.168.39.1 > 192.168.39.2: icmp
14:33:12.853948 IP (tos 0x0, ttl 64, id 62329, offset 2960, flags [+], proto
ICMP (1), length 1500) 192.168.39.1 > 192.168.39.2: icmp
14:33:12.853949 IP (tos 0x0, ttl 64, id 62329, offset 4440, flags [+], proto
ICMP (1), length 1500) 192.168.39.1 > 192.168.39.2: icmp
14:33:12.853956 IP (tos 0x0, ttl 64, id 62329, offset 5920, flags [none], proto
ICMP (1), length 508) 192.168.39.1 > 192.168.39.2: icmp
14:33:17.851517 arp who-has 192.168.39.1 tell 192.168.39.2
14:33:17.851581 arp reply 192.168.39.1 is-at 00:16:3e:02:24:db (oui Unknown)

# tcpdump -i vif9.1 -v
tcpdump: WARNING: vif9.1: no IPv4 address assigned
tcpdump: listening on vif9.1, link-type EN10MB (Ethernet), capture size 96 bytes
14:33:12.853739 IP (tos 0x0, ttl 64, id 62828, offset 0, flags [+], proto ICMP
(1), length 1500) 192.168.39.2 > 192.168.39.1: ICMP echo request, id 2327,
seq 1, length 1480
14:33:12.853740 IP (tos 0x0, ttl 64, id 62828, offset 1480, flags [+], proto
ICMP (1), length 1500) 192.168.39.2 > 192.168.39.1: icmp
14:33:12.853740 IP (tos 0x0, ttl 64, id 62828, offset 2960, flags [+], proto
ICMP (1), length 1500) 192.168.39.2 > 192.168.39.1: icmp
14:33:12.853741 IP (tos 0x0, ttl 64, id 62828, offset 4440, flags [+], proto
ICMP (1), length 1500) 192.168.39.2 > 192.168.39.1: icmp
14:33:12.853741 IP (tos 0x0, ttl 64, id 62828, offset 5920, flags [none], proto
ICMP (1), length 508) 192.168.39.2 > 192.168.39.1: icmp
14:33:12.853967 IP (tos 0x0, ttl 64, id 62329, offset 0, flags [+], proto ICMP
(1), length 1500) 192.168.39.1 > 192.168.39.2: ICMP echo reply, id 2327, seq
1, length 1480
14:33:12.853968 IP (tos 0x0, ttl 64, id 62329, offset 1480, flags [+], proto
ICMP (1), length 1500) 192.168.39.1 > 192.168.39.2: icmp
14:33:12.853969 IP (tos 0x0, ttl 64, id 62329, offset 2960, flags [+], proto
ICMP (1), length 1500) 192.168.39.1 > 192.168.39.2: icmp
14:33:12.853970 IP (tos 0x0, ttl 64, id 62329, offset 4440, flags [+], proto
ICMP (1), length 1500) 192.168.39.1 > 192.168.39.2: icmp
14:33:12.853970 IP (tos 0x0, ttl 64, id 62329, offset 5920, flags [none], proto
ICMP (1), length 508) 192.168.39.1 > 192.168.39.2: icmp
14:33:17.851500 arp who-has 192.168.39.1 tell 192.168.39.2
14:33:17.851612 arp reply 192.168.39.1 is-at 00:16:3e:02:24:db (oui Unknown)

We also tried to set tso and gso off with ethtool in all interfaces (intbr1,
vif2.1, vif9.1 and ethx on guest systems), but with no effect.

Does anyone have any idea what instance defragments the ip-packets and why and
how we can force this instance not to do this?

Thanks!

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> When we send an ICMP-packet with 6400 data-bytes (ping -s 6400 -c 1
> 192.168.39.1) from Linux-guest connected to "vif9.1" to
> the other linux-guest connected to "vif2.1" we see with tcpdump
connected to
> intbr1, that some instance (intbr1? vif9.1) defragmented
> the 5 ip-packets from vif9.1 into 1 ip-packet, though mtu-size for the
vif-
> interfaces, intbr1 and eth-interfaces of the guests
> are all set to 1500 bytes:
What did you expect to happen? If the MTU is 1500 bytes then by
definition, no larger packet will be allowed through (TCP is an
exception to this where LSO/TSO/GSO is supported)

If the ''do not fragment'' flag is not set on the icmp packet
then the IP
layer on the sending host will fragment the packet at the IP level.

If the ''do not fragment'' flag is set then the sending host
will respond
with an icmp ''fragmentation required'' packet.

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi James,
> What did you expect to happen? If the MTU is 1500 bytes then by
> definition, no larger packet will be allowed through (TCP is an
> exception to this where LSO/TSO/GSO is supported)
> 
I expected, that tcpdump listening to intbr1, shows also 5 ip-packets coming
from vif9.1 and
5 ip-packets coming from vif2.1

Dieter

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users