thr3ads.net - Xen users - [Xen-users] Strange network problem (Etch) [Oct 2007]

If this information is useful, please help other people find it:
Share via:

Heikki Levanto

2007-Oct-04 11:41 UTC

[Xen-users] Strange network problem (Etch)

Hi,

I am trying to install xen, and am running into a strange network
problem. When I enable the bridge, I my machine looses its network
connectivity - except to the local net.

I can access the machine from another box on the same net, ping and ssh
work fine. But any attempt to contact the machine from the outside, or
to contact anything outside that net from the box itself fails
miserably.

The machine is some sort of HP, located at a co-lo facility. It has two
net cards (tg3), each with its own address (once upon a time we had two
machines...)

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:11:0A:2E:DA:C5
          inet addr:xx.xxx.xx.136  Bcast:xx.xxx.xx.255 Mask:255.255.255.0
          inet6 addr: fe80::211:aff:fe2e:dac5/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:11468 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:832588 (813.0 KiB)  TX bytes:492 (492.0 b)
          Interrupt:17

eth1      Link encap:Ethernet  HWaddr 00:11:0A:2E:CA:9B
          inet addr:xx.xxx.xx.64  Bcast:xx.xxx.xx.255 Mask:255.255.255.0
          inet6 addr: fe80::211:aff:fe2e:ca9b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:21802 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13124 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2345492 (2.2 MiB)  TX bytes:13005551 (12.4 MiB)
          Interrupt:18

My routing table is simple:
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
xx.xxx.xx.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
xx.xxx.xx.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         xx.xxx.xx.1     0.0.0.0         UG    0      0        0 eth1

I have tried to enable the network bridge in /etc/xen/xend-config.sxp
(here without all the comments):

# cat xend-config.sxp | sed ''/^#.*$/d'' | sed
''/^$/d''
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 196)
(dom0-cpus 0)


When I run
   /etc/init.d/xend restart
I get the new network devices all right:


# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:11:0A:2E:DA:C5
          inet addr:xx.xxx.xx.136  Bcast:xx.xxx.xx.255 Mask:255.255.255.0
          inet6 addr: fe80::211:aff:fe2e:dac5/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:15219 errors:0 dropped:0 overruns:0 frame:0
          TX packets:120 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1103031 (1.0 MiB)  TX bytes:18022 (17.5 KiB)
          Interrupt:17

eth1      Link encap:Ethernet  HWaddr 00:11:0A:2E:CA:9B
          inet addr:xx.xxx.xx.64  Bcast:xx.xxx.xx.255 Mask:255.255.255.0
          inet6 addr: fe80::211:aff:fe2e:ca9b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:155 errors:0 dropped:0 overruns:0 frame:0
          TX packets:206 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:9300 (9.0 KiB)  TX bytes:16314 (15.9 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:23283 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23283 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:55074193 (52.5 MiB)  TX bytes:55074193 (52.5 MiB)

peth1     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:221 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:17223 (16.8 KiB)  TX bytes:654 (654.0 b)
          Interrupt:18

vif0.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:206 errors:0 dropped:0 overruns:0 frame:0
          TX packets:155 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:16314 (15.9 KiB)  TX bytes:9300 (9.0 KiB)

xenbr1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:163 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:7993 (7.8 KiB)  TX bytes:0 (0.0 b)

My route looks much like it did before:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
xx.xxx.xx.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
xx.xxx.xx.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
0.0.0.0         xx.xxx.xx.1     0.0.0.0         UG    0      0        0 eth1

As far as I can see, the bridge is there all right:
# brctl show
bridge name     bridge id               STP enabled     interfaces
xenbr1          8000.feffffffffff       no              vif0.1
                                                        peth1


Now I can ping the other machine I have on that net:
ping falaffel
PING falaffel.xxx.xx (xx.xxx.xx.140) 56(84) bytes of data.
64 bytes from falaffel.xxx.xx (xx.xxx.xx.140): icmp_seq=1 ttl=64 time=3.62 ms
64 bytes from falaffel.xxx.xx (xx.xxx.xx.140): icmp_seq=2 ttl=64 time=0.359 ms
64 bytes from falaffel.xxx.xx (xx.xxx.xx.140): icmp_seq=3 ttl=64 time=0.357 ms
64 bytes from falaffel.xxx.xx (xx.xxx.xx.140): icmp_seq=4 ttl=64 time=0.357 ms
--- falaffel.indexdata.dk ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.357/1.173/3.620/1.412 ms

But trying to ping a machine on the outside net fails miserably:

kafta:~# ping bagel
PING bagel.xxx.xx (yyy.yyy.yy.11) 56(84) bytes of data.

--- bagel.xxx.xx ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3008ms


I tried the trick with
# ethtool -K eth0 tx off
# ethtool -K eth1 tx off

but that does not help anything.


Actually, I would like to set things up a bit differently, but first I
wanted to get this setup to work, to be sure I understand what is going
on. I post another message about the config I really want. But I still
need to get this problem solved first!

Any and all help would be appreciated.  I hope I have provided all
necesary information, but if not, just ask, and I will gladly give more.
I am under some pressure to get this up and running soon!

Regards

   Heikki Levanto
   sysadmin, Index Data



-- 
Heikki Levanto    heikki at indexdata dot dk   "In Murphy We Turst"

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Heikki Levanto

2007-Oct-04 14:18 UTC

head link

Re: [Xen-users] Strange network problem (Etch)

I have not solved this problem yet, but I found out that the vif-nat way
of doing things nicely sidestpes the problem, so suddenly it is not
nearly as pressing.


Thanks anyway

   Heikki

-- 
Heikki Levanto    heikki at indexdata dot dk   "In Murphy We Turst"

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Jens Seidel

2007-Oct-04 15:06 UTC

head link

Re: [Xen-users] Strange network problem (Etch)

On Thu, Oct 04, 2007 at 04:18:33PM +0200, Heikki Levanto
wrote:> I have not solved this problem yet, but I found out that the vif-nat way
> of doing things nicely sidestpes the problem, so suddenly it is not
> nearly as pressing.
I had very similar network problems using Xen 3.1 (my dom0 network did
no longer worked). In Xen 3.0.3 my dom0 network worked well but I had
trouble getting an ethernet device in a domU.

Using vif-nat is also my solution ...

Jens

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Geoff Kirk

2007-Oct-04 15:22 UTC

head link

Re: [Xen-users] Strange network problem (Etch)

Let me try to explain what i did recently to setup some domU''s behind a
domU acting as a gateway between 2 bridges to play with shorewall.

Using the scripts here

http://renial.net/weblog/2007/02/27/xen-vlan/

i put them in /etc/xen/scripts and edited 
/etc/xen/scripts/network-multi-vlan to look like

dir=$(dirname "$0")
"$dir/network-bridge" "$@" vifnum=0
"$dir/network-bridge-vlan" "$@" vlan=2

brctl show then gives
bridge name     bridge id               STP enabled     interfaces
vlanbr2         8000.feffffffffff       no              eth0.2
xenbr0          8000.feffffffffff       no              vif0.0
                                                        peth0

I then setup a domU to act as a gateway between xenbr0 and vlanbr2 in 
domU config like this

vif = [ ''mac=00:16:3E:00:00:26, bridge=xenbr0, 
vifname=gateway.0'',''mac=00:16:3E:00:02:10, bridge=vlanbr2, 
vifname=gateway.1'' ]

and created a couple of other domU''s on vlanbr2. To give this.

vlanbr2         8000.feffffffffff       no              eth0.2
                                                        gateway.1
                                                        dom1.0
                                                        dom2.0
xenbr0          8000.feffffffffff       no              vif0.0
                                                        peth0
                                                        gateway.0

At this point i found an issue that anyone on my physical lan could ping 
anything on vlanbr0 simply by creating a vlan on the same network. Which 
i found by removing eth0.2 from vlanbr2 with "brctl delif vlanbr2 
eth0.2" made them unreachable from anything not on the bridge and now 
all traffic going to the vlanbr2 bridge has to be port forwarded through 
the gateway domU. Enabling ip forwarding of course and NAT on the out 
going interface. Don''t forget also domU''s on the bridge will
need to
know the gateway address to get outside eg.

route add default gw 10.0.0.1 dev eth0

regards
Geoff

Jens Seidel wrote:> On Thu, Oct 04, 2007 at 04:18:33PM +0200, Heikki Levanto wrote:
>   
>> I have not solved this problem yet, but I found out that the vif-nat
way
>> of doing things nicely sidestpes the problem, so suddenly it is not
>> nearly as pressing.
>>     
>
> I had very similar network problems using Xen 3.1 (my dom0 network did
> no longer worked). In Xen 3.0.3 my dom0 network worked well but I had
> trouble getting an ethernet device in a domU.
>
> Using vif-nat is also my solution ...
>
> Jens
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>   

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Xen users - Oct 2007 - Strange network problem (Etch)

[Xen-users] Strange network problem (Etch)

Re: [Xen-users] Strange network problem (Etch)

Re: [Xen-users] Strange network problem (Etch)

Re: [Xen-users] Strange network problem (Etch)