[Xen-users] Strange network problem (Etch)

Heikki Levanto
Thu, 04 Oct 2007 11:41:05 -0000

Hi,

I am trying to install xen, and am running into a strange network
problem. When I enable the bridge, I my machine looses its network
connectivity - except to the local net.

I can access the machine from another box on the same net, ping and ssh
work fine. But any attempt to contact the machine from the outside, or
to contact anything outside that net from the box itself fails
miserably.

The machine is some sort of HP, located at a co-lo facility. It has two
net cards (tg3), each with its own address (once upon a time we had two
machines...)

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:11:0A:2E:DA:C5
          inet addr:xx.xxx.xx.136  Bcast:xx.xxx.xx.255 Mask:255.255.255.0
          inet6 addr: fe80::211:aff:fe2e:dac5/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:11468 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:832588 (813.0 KiB)  TX bytes:492 (492.0 b)
          Interrupt:17

eth1      Link encap:Ethernet  HWaddr 00:11:0A:2E:CA:9B
          inet addr:xx.xxx.xx.64  Bcast:xx.xxx.xx.255 Mask:255.255.255.0
          inet6 addr: fe80::211:aff:fe2e:ca9b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:21802 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13124 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2345492 (2.2 MiB)  TX bytes:13005551 (12.4 MiB)
          Interrupt:18

My routing table is simple:
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
xx.xxx.xx.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
xx.xxx.xx.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         xx.xxx.xx.1     0.0.0.0         UG    0      0        0 eth1

I have tried to enable the network bridge in /etc/xen/xend-config.sxp
(here without all the comments):

# cat xend-config.sxp | sed ''/^#.*$/d'' | sed
''/^$/d''
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 196)
(dom0-cpus 0)


When I run
   /etc/init.d/xend restart
I get the new network devices all right:


# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:11:0A:2E:DA:C5
          inet addr:xx.xxx.xx.136  Bcast:xx.xxx.xx.255 Mask:255.255.255.0
          inet6 addr: fe80::211:aff:fe2e:dac5/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:15219 errors:0 dropped:0 overruns:0 frame:0
          TX packets:120 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1103031 (1.0 MiB)  TX bytes:18022 (17.5 KiB)
          Interrupt:17

eth1      Link encap:Ethernet  HWaddr 00:11:0A:2E:CA:9B
          inet addr:xx.xxx.xx.64  Bcast:xx.xxx.xx.255 Mask:255.255.255.0
          inet6 addr: fe80::211:aff:fe2e:ca9b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:155 errors:0 dropped:0 overruns:0 frame:0
          TX packets:206 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:9300 (9.0 KiB)  TX bytes:16314 (15.9 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:23283 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23283 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:55074193 (52.5 MiB)  TX bytes:55074193 (52.5 MiB)

peth1     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:221 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:17223 (16.8 KiB)  TX bytes:654 (654.0 b)
          Interrupt:18

vif0.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:206 errors:0 dropped:0 overruns:0 frame:0
          TX packets:155 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:16314 (15.9 KiB)  TX bytes:9300 (9.0 KiB)

xenbr1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:163 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:7993 (7.8 KiB)  TX bytes:0 (0.0 b)

My route looks much like it did before:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
xx.xxx.xx.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
xx.xxx.xx.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
0.0.0.0         xx.xxx.xx.1     0.0.0.0         UG    0      0        0 eth1

As far as I can see, the bridge is there all right:
# brctl show
bridge name     bridge id               STP enabled     interfaces
xenbr1          8000.feffffffffff       no              vif0.1
                                                        peth1


Now I can ping the other machine I have on that net:
ping falaffel
PING falaffel.xxx.xx (xx.xxx.xx.140) 56(84) bytes of data.
64 bytes from falaffel.xxx.xx (xx.xxx.xx.140): icmp_seq=1 ttl=64 time=3.62 ms
64 bytes from falaffel.xxx.xx (xx.xxx.xx.140): icmp_seq=2 ttl=64 time=0.359 ms
64 bytes from falaffel.xxx.xx (xx.xxx.xx.140): icmp_seq=3 ttl=64 time=0.357 ms
64 bytes from falaffel.xxx.xx (xx.xxx.xx.140): icmp_seq=4 ttl=64 time=0.357 ms
--- falaffel.indexdata.dk ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.357/1.173/3.620/1.412 ms

But trying to ping a machine on the outside net fails miserably:

kafta:~# ping bagel
PING bagel.xxx.xx (yyy.yyy.yy.11) 56(84) bytes of data.

--- bagel.xxx.xx ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3008ms


I tried the trick with
# ethtool -K eth0 tx off
# ethtool -K eth1 tx off

but that does not help anything.


Actually, I would like to set things up a bit differently, but first I
wanted to get this setup to work, to be sure I understand what is going
on. I post another message about the config I really want. But I still
need to get this problem solved first!

Any and all help would be appreciated.  I hope I have provided all
necesary information, but if not, just ask, and I will gladly give more.
I am under some pressure to get this up and running soon!

Regards

   Heikki Levanto
   sysadmin, Index Data



--
Heikki Levanto    heikki at indexdata dot dk   "In Murphy We Turst"