Moritz Bunkus
2006-Jan-18 10:59 UTC
[Xen-users] network: bridging problem accessing domU from dom0
Hey,
I''m trying to set up the following with Xen 3.0:
http://www.linet-services.de/~mbunkus/xen-test-setup.png
(tried to do with ASCII graphics but failed ;))
The idea is:
* The dom0 is accessible via its eth0.
* The dom0''s eth1 is bridged into the "router" domU.
* The "router" domU has a second interface which is bridged inside the
dom0 to the "server" domUs.
This setup works nearly perfectly so far. I''m using custom vif-scripts
for setting this up.
Now my problem: I cannot use TCP apps like SSH or a web browser (w3m,
wget...) from the dom0 to one of the domUs, but pinging works.
I''m on the dom0. I ping 172.16.2.1 (the "server1" domU) and
receive
replies. I start tcpdump on the "server1" domU and see the pings and
the
replies with the proper addresses (172.16.2.253 <-> 172.16.2.1).
Then I try to ssh from dom0 to "server1". In the tcpdump I see the
following:
- The handshake is OK: SYN, SYN+ACK, ACK are all sent and received. The
TCP connection is established.
- The "server1" sends the server "greeting" which is ACKed
by the dom0.
- The dom0 now sends its first "real" data packet -- but from this
point
on no packets are ever ACKed by the "server1" domU. However, the
tcpdump
running on the dumU does see those packets! So they are arriving
there, but they somehow seem not to be picked up by the kernel... And
I have no clue whatsoever why this happens.
Some output from my dom0:
ls-bs-vm-xenh1:/etc/xen# ifconfig
br-rou-ex Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1392 (1.3 KiB) TX bytes:0 (0.0 b)
br-rou-sr Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:172.16.2.253 Bcast:172.16.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:42 errors:0 dropped:0 overruns:0 frame:0
TX packets:27 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1553 (1.5 KiB) TX bytes:2776 (2.7 KiB)
eth0 Link encap:Ethernet HWaddr 00:0A:5E:52:6D:50
inet addr:172.16.0.2 Bcast:172.16.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5036 errors:0 dropped:0 overruns:0 frame:0
TX packets:2846 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:563869 (550.6 KiB) TX bytes:361819 (353.3 KiB)
Interrupt:16 Base address:0x2000
eth1 Link encap:Ethernet HWaddr 00:0D:61:B1:85:A1
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3719 errors:0 dropped:0 overruns:0 frame:0
TX packets:10307 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1166977 (1.1 MiB) TX bytes:3298596 (3.1 MiB)
Interrupt:17 Base address:0xc400
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:44 errors:0 dropped:0 overruns:0 frame:0
TX packets:44 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4536 (4.4 KiB) TX bytes:4536 (4.4 KiB)
vif-rou-e Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:311 errors:0 dropped:0 overruns:0 frame:0
TX packets:424 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:45591 (44.5 KiB) TX bytes:40995 (40.0 KiB)
vif-rou-s Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:43 errors:0 dropped:0 overruns:0 frame:0
TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2183 (2.1 KiB) TX bytes:4162 (4.0 KiB)
(Interface names cut short by ifconfig)
ls-bs-vm-xenh1:/etc/xen# brctl show
bridge name bridge id STP enabled interfaces
br-rou-ext 8000.feffffffffff no eth1
vif-rou-ext
br-rou-srv 8000.feffffffffff no vif-rou-srv
(again layout messed up by long interface names)
Any idea how I can solve or at least debug this?
Thanks.
Mosu
--
LINET Services GbR
Gotenweg 15 Tel.: 0531-280 191 71
38106 Braunschweig Fax.: 0531-280 191 72
http://www.linet-services.de
mailto:info@linet-services.de
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Jan Niehusmann
2006-Jan-18 15:27 UTC
[Xen-users] Re: network: bridging problem accessing domU from dom0
Moritz Bunkus wrote:> - The dom0 now sends its first "real" data packet -- but from this point > on no packets are ever ACKed by the "server1" domU. However, the tcpdump > running on the dumU does see those packets! So they are arriving > there, but they somehow seem not to be picked up by the kernel... And > I have no clue whatsoever why this happens.This very much looks like the bad tcp checksum problem. You can verify this with tcpdump -vv, which should show you if the checksum is invalid. If this is really your problem, you could try turning off tcp checksum offloading with ''ethtool -K eth0 tx off'' (replace eth0 with the respective interface). Unfortunately, the loopback-device used in dom0 to create dom0''s virtual interfaces doesn''t support this yet, unlike the netback/netfront interface connecting domUs to dom0. I sent a patch to xen-devel to add this feature to dom0 a few days ago. Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Moritz Bunkus
2006-Jan-19 08:12 UTC
Re: [Xen-users] Re: network: bridging problem accessing domU from dom0
Hey, On Wednesday 18 January 2006 16:27, Jan Niehusmann wrote:> This very much looks like the bad tcp checksum problem. You can verify > this with tcpdump -vv, which should show you if the checksum is > invalid.Nope, the checksum is OK: 09:11:53.042543 IP (tos 0x0, ttl 64, id 41457, offset 0, flags [DF], length: 60) 172.16.2.253.2834 > 172.16.2.254.22: S [tcp sum ok] 2415494891:2415494891 (0) win 5840 <mss 1460,sackOK,timestamp 10010 0,nop,wscale 2> 09:11:53.042619 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], length: 60) 172.16.2.254.22 > 172.16.2.253.2834: S [tcp sum ok] 2400049154:2400049154(0) ack 2415494892 win 5792 <mss 1460,sackOK,timestamp 4294909055 10010,nop,wscale 2> 09:11:53.043082 IP (tos 0x0, ttl 64, id 41458, offset 0, flags [DF], length: 52) 172.16.2.253.2834 > 172.16.2.254.22: . [tcp sum ok] 1:1(0) ack 1 win 1460 <nop,nop,timestamp 10010 4294909055> 09:11:53.044739 IP (tos 0x0, ttl 64, id 50300, offset 0, flags [DF], length: 93) 172.16.2.254.22 > 172.16.2.253.2834: P 1:42(41) ack 1 win 1448 <nop,nop,timestamp 4294909055 10010> 09:11:53.045249 IP (tos 0x0, ttl 64, id 41459, offset 0, flags [DF], length: 52) 172.16.2.253.2834 > 172.16.2.254.22: . [tcp sum ok] 1:1(0) ack 42 win 1460 <nop,nop,timestamp 10010 4294909055> 09:11:53.045262 IP (tos 0x0, ttl 64, id 41460, offset 0, flags [DF], length: 93) 172.16.2.253.2834 > 172.16.2.254.22: P 1:42(41) ack 42 win 1460 <nop,nop,timestamp 10010 4294909055> 09:11:53.245937 IP (tos 0x0, ttl 64, id 41461, offset 0, flags [DF], length: 93) 172.16.2.253.2834 > 172.16.2.254.22: P 1:42(41) ack 42 win 1460 <nop,nop,timestamp 10061 4294909055> .... Any other ideas? Mosu -- LINET Services GbR Gotenweg 15 Tel.: 0531-280 191 71 38106 Braunschweig Fax.: 0531-280 191 72 http://www.linet-services.de mailto:info@linet-services.de _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users