Hi, I'm working on setting up a bridge on with embedded Linux boards running kernel version 2.6.20.4. I seem to be having some issues. I can reliably ping from the root bridge to the other bridge. However when I try and ping from the non-root bridge to the root bridge we get around an 83% packet loss. Our topology is as follows: ************************************************************ * STP Topology: * * +---------+ +---------+ * | BR-1 | | BR-2 | * | eth0 [=]---------[HUB]-------------[=] eth0 | * | | | | * | eth1 [=]---------[HUB]-------------[=] eth1 | * | [ ] [ ] | * | | | | * +---------+ +---------+ * ************************************************************* In this case BR-1 ends up being our root bridge. The bridges on the two boards are setup as follows: BR-1: ---------------------------------------------------------------------------- ifconfig eth0 0.0.0.0 down ifconfig eth1 0.0.0.0 down /mnt/nand/currload/sbin/brctl addbr br0 /mnt/nand/currload/sbin/brctl stp br0 on /mnt/nand/currload/sbin/brctl addif br0 eth0 /mnt/nand/currload/sbin/brctl addif br0 eth1 ifconfig eth0 up ifconfig eth1 up ifconfig br0 up ifconfig br0 192.168.1.2 ---------------------------------------------------------------------------- BR-2: ---------------------------------------------------------------------------- ifconfig eth0 0.0.0.0 down ifconfig eth1 0.0.0.0 down /mnt/nand/currload/sbin/brctl addbr br0 /mnt/nand/currload/sbin/brctl stp br0 on /mnt/nand/currload/sbin/brctl addif br0 eth0 /mnt/nand/currload/sbin/brctl addif br0 eth1 ifconfig eth0 up ifconfig eth1 up ifconfig br0 up ifconfig br0 192.168.1.1 ---------------------------------------------------------------------------- Once the bridges are allowed to come up and sniff out the network using BPDU frames showstp reports: BR-1: ---------------------------------------------------------------------------- # brctl showstp br0 br0 bridge id 8000.0050c263d001 designated root 8000.0050c263d001 root port 0 path cost 0 max age 20.00 bridge max age 20.00 hello time 2.00 bridge hello time 2.00 forward delay 15.00 bridge forward delay 15.00 ageing time 300.00 hello timer 1.55 tcn timer 0.00 topology change timer 0.00 gc timer 0.05 flags eth0 (1) port id 8001 state forwarding designated root 8000.0050c263d001 path cost 100 designated bridge 8000.0050c263d001 message age timer 0.00 designated port 8001 forward delay timer 0.00 designated cost 0 hold timer 0.54 flags eth1 (2) port id 8002 state forwarding designated root 8000.0050c263d001 path cost 100 designated bridge 8000.0050c263d001 message age timer 0.00 designated port 8002 forward delay timer 0.00 designated cost 0 hold timer 0.54 flags ---------------------------------------------------------------------------- BR-2: ---------------------------------------------------------------------------- # brctl showstp br0 br0 bridge id 8000.0050c263d065 designated root 8000.0050c263d001 root port 1 path cost 100 max age 20.00 bridge max age 20.00 hello time 2.00 bridge hello time 2.00 forward delay 15.00 bridge forward delay 15.00 ageing time 300.00 hello timer 0.00 tcn timer 0.00 topology change timer 0.00 gc timer 0.06 flags eth0 (1) port id 8001 state forwarding designated root 8000.0050c263d001 path cost 100 designated bridge 8000.0050c263d001 message age timer 18.39 designated port 8001 forward delay timer 0.00 designated cost 0 hold timer 0.00 flags eth1 (2) port id 8002 state blocking designated root 8000.0050c263d001 path cost 100 designated bridge 8000.0050c263d001 message age timer 18.39 designated port 8002 forward delay timer 0.00 designated cost 0 hold timer 0.00 flags ---------------------------------------------------------------------------- From what I understand this is what should happen. The other bridge is receiving BPDU frames on both interfaces and blocks one interface to prevent creating a loop. We have sniffed packets externally at the hubs with other machines and with tcpdump on the boards directly. STP frames are being sent by the root bridge, and if we change topology of the network there is a topology change frame sent, and the bridge starts seting BPDU frames with the topology change flag set for a while after. It appears that STP is doing what it's supposed to do and adapting to changes in the network. However an issue appears once we starting trying to ping between the bridges. When we ping from the root bridge things seem to work fine. Here is a TCP dump from the two boards. BR-1: ---------------------------------------------------------------------------- ~ # tcpdump -i eth0 tcpdump: WARNING: eth0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 68 bytes 00:41:11.211037 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 10240, length 64 00:41:11.212685 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 10240, length 64 00:41:12.020297 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:41:12.221029 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 10496, length 64 00:41:12.222643 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 10496, length 64 00:41:13.231001 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 10752, length 64 00:41:13.232655 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 10752, length 64 00:41:14.020297 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:41:14.241019 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 11008, length 64 00:41:14.242641 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 11008, length 64 00:41:15.200218 ARP, Request who-has 192.168.1.1 tell 192.168.1.2, length 28 00:41:15.201127 ARP, Reply 192.168.1.1 is-at 00:50:c2:63:d0:65 (oui Unknown), length 46 00:41:15.251017 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 11264, length 64 00:41:15.252624 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 11264, length 64 00:41:16.020299 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 ---------------------------------------------------------------------------- BR-2: ---------------------------------------------------------------------------- ~ # tcpdump -i eth0 tcpdump: WARNING: eth0: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 68 bytes 00:40:32.651507 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 1024, length 64 00:40:32.652541 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 1024, length 64 00:40:33.661415 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 1280, length 64 00:40:33.662478 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 1280, length 64 00:40:33.820673 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:40:34.671433 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 1536, length 64 00:40:34.672511 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 1536, length 64 00:40:35.630208 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 28 00:40:35.630929 ARP, Reply 192.168.1.2 is-at 00:50:c2:63:d0:01 (oui Unknown), length 46 00:40:35.681407 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 1792, length 64 00:40:35.682379 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 1792, length 64 00:40:35.820659 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:40:36.691328 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 2048, length 64 00:40:36.692389 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 2048, length 64 00:40:37.701391 IP 192.168.1.2 > 192.168.1.1: ICMP echo request, id 49412, seq 2304, length 64 00:40:37.702417 IP 192.168.1.1 > 192.168.1.2: ICMP echo reply, id 49412, seq 2304, length 64 ---------------------------------------------------------------------------- Looks link a normal tcpdump of pinging interleaved with STP frames. Note I didn't end up getting the ARP transaction captured at the beginning of the session. When I bring from the non-root bridge to the root bridge around 83% of the pings do not work. Looking at packets dumped by tcpdump reveals some strangeness. As soon as a STP frame is received the no more ICMP echo requests are sent out, after a few seconds the board will ARP for the IP address it is trying to ping, get a few pings through, and then stop again when an STP frame is received. Here are the tcpdumps: BR-1: ---------------------------------------------------------------------------- 00:35:55.379503 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46 00:35:55.380110 ARP, Reply 192.168.1.2 is-at 00:50:c2:63:d0:01 (oui Unknown), length 28 00:35:55.380968 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 41984, length 64 00:35:55.382151 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 41984, length 64 00:35:56.020435 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:35:58.020299 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:00.020301 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:00.380208 ARP, Request who-has 192.168.1.1 tell 192.168.1.2, length 28 00:36:00.381120 ARP, Reply 192.168.1.1 is-at 00:50:c2:63:d0:65 (oui Unknown), length 46 00:36:00.429901 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 43264, length 64 00:36:00.430958 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 43264, length 64 00:36:01.439996 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 43520, length 64 00:36:01.441037 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 43520, length 64 00:36:02.020303 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:04.020303 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:06.020295 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:08.020299 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:10.020293 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:12.020297 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:14.020305 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:16.020297 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:18.020305 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:20.020301 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:22.020299 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:24.020289 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:26.020295 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:28.020328 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:30.020293 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:32.020297 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:34.020287 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:36.020291 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:38.020293 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:38.819580 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 46 00:36:38.820192 ARP, Reply 192.168.1.2 is-at 00:50:c2:63:d0:01 (oui Unknown), length 28 00:36:38.821074 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 52992, length 64 00:36:38.822232 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 52992, length 64 00:36:39.820108 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 53248, length 64 00:36:39.821169 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 53248, length 64 00:36:40.020316 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:42.020297 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:43.820230 ARP, Request who-has 192.168.1.1 tell 192.168.1.2, length 28 00:36:43.821200 ARP, Reply 192.168.1.1 is-at 00:50:c2:63:d0:65 (oui Unknown), length 46 00:36:43.870043 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 54272, length 64 00:36:43.871023 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 54272, length 64 00:36:44.020314 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:46.020295 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:48.020299 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 00:36:50.020291 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 35 ---------------------------------------------------------------------------- BR-2: ---------------------------------------------------------------------------- 00:34:26.280368 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 28 00:34:26.281413 ARP, Reply 192.168.1.2 is-at 00:50:c2:63:d0:01 (oui Unknown), length 46 00:34:26.281746 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 19968, length 64 00:34:26.283187 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 19968, length 64 00:34:27.280872 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 20224, length 64 00:34:27.282375 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 20224, length 64 00:34:27.821850 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:29.821852 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:31.281787 ARP, Request who-has 192.168.1.1 tell 192.168.1.2, length 46 00:34:31.282204 ARP, Reply 192.168.1.1 is-at 00:50:c2:63:d0:65 (oui Unknown), length 28 00:34:31.320885 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 21248, length 64 00:34:31.322307 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 21248, length 64 00:34:31.821836 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:33.821996 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:35.821988 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:37.821990 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:39.821978 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:41.821962 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:43.821960 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:45.821960 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:47.821797 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:49.821791 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:51.821789 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:53.821775 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:55.821777 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:57.821923 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:34:59.821913 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:01.821897 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:03.821895 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:05.821897 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:07.821742 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:09.720316 ARP, Request who-has 192.168.1.2 tell 192.168.1.1, length 28 00:35:09.721145 ARP, Reply 192.168.1.2 is-at 00:50:c2:63:d0:01 (oui Unknown), length 46 00:35:09.721480 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 30976, length 64 00:35:09.722966 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 30976, length 64 00:35:09.821787 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:11.821736 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:13.821875 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:14.721818 ARP, Request who-has 192.168.1.1 tell 192.168.1.2, length 46 00:35:14.722244 ARP, Reply 192.168.1.1 is-at 00:50:c2:63:d0:65 (oui Unknown), length 28 00:35:14.770793 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 32256, length 64 00:35:14.772444 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 32256, length 64 00:35:15.780868 IP 192.168.1.1 > 192.168.1.2: ICMP echo request, id 42500, seq 32512, length 64 00:35:15.782596 IP 192.168.1.2 > 192.168.1.1: ICMP echo reply, id 42500, seq 32512, length 64 00:35:15.821868 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:17.821864 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:19.821856 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 00:35:21.821836 STP 802.1d, Config, Flags [none], bridge-id 8000.00:50:c2:63:d0:01.8001, length 43 ---------------------------------------------------------------------------- Why is this happening? Why would receiving an STP frame cause the board to stop sending the ICMP echo requests and then have to request ARP information again. Is there something that I'm missing in setting up the bridges? I'm not sure if the drivers we are using (and ethernet devices for that matter) have been used in STP enabled bridges or not and if there are issues with that. I'm going to add debug to the kernel to see if I can find out exactly what is going on. If anyone has any information about what might be going on here I would greatly appreciate it. I am new to bridges and STP, so I'm unsure where this issue lies. I am also going to try to get my hands on two Linux PCs with multiple NICs so that I can try creating STP enabled bridges with them to see if we have similar issues. Sorry for the long email. If there is any other information that would help debugging this issue I can provide it. Thanks, Andrew McKay Iders Inc.