thr3ads.net - Xen users - [Xen-users] DOM0 networking - dead to the outside world [Mar 2007]

If this information is useful, please help other people find it:
Share via:

Alan Pearson

2007-Mar-22 20:35 UTC

[Xen-users] DOM0 networking - dead to the outside world

Hi Folks,

I''ve been using Xen 3.0.2-2 for quite a while now on various RHEL 4.3  
(host + guests, using xensource RPMs)  systems, successfully.

Now I have the pleasure installing Xen  3.0.4.1 on RHEL4.4 (host +  
guests, using xensource RPMs) system, and I''m having a nightmare with  
Dom0 networking.

The DomUs can see the outside world AND the Dom0 eth0 address, but  
Dom0 cannot see the outside world nor the outside world see it.
Dom0 can ping the guests fine, and the guests can ping Dom0.

For the purposes of this discussion, outside world means ''same subnet  
on the physical interface''

Dom0 networking works fine before network-bridge is started, after it  
starts, no more Dom0 networking.

I''ve really no idea what is wrong, and hours and hours playing with  
Linux bridging has got me nowhere ;-(
There doesn''t seem to be any iptables rules stopping traffic, and  
selinux is disabled.


The output of the brctl show & ifconfig commands looks identical on  
the xen3.0.4.1 to the working 3.0.2-2 systems, so I''m lost.
Running tcpdump -s0 -L -A   produces no output on any interface.


Any help at all, gratefully received, and if I have not provided  
enough information, please ask.



Some Info :

vif5.0 is the interface associated with my DomU.
The 172.16.5.X subnet is the one giving the trouble (eth1 is just  
used for accessing the box for the moment until I can get it working,  
so the 10.2.2.X addresses can be ignored.)





[root@xen-srv2 /]# brctl show
bridge name     bridge id               STP enabled     interfaces
xenbr0          8000.feffffffffff       no              peth0
                                                         vif0.0
                                                         vif5.0




[root@xen-srv2 /]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:14:5E:DE:BB:DA
           inet addr:172.16.5.33  Bcast:172.16.5.63  Mask: 
255.255.255.224
           inet6 addr: fe80::214:5eff:fede:bbda/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:27 errors:0 dropped:0 overruns:0 frame:0
           TX packets:106 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:2016 (1.9 KiB)  TX bytes:5172 (5.0 KiB)

eth1      Link encap:Ethernet  HWaddr 00:14:5E:DE:BB:DC
           inet addr:10.2.2.1  Bcast:10.255.255.255  Mask:255.0.0.0
           inet6 addr: fe80::214:5eff:fede:bbdc/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:92809 errors:0 dropped:0 overruns:0 frame:0
           TX packets:122427 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:6684042 (6.3 MiB)  TX bytes:20822432 (19.8 MiB)
           Interrupt:16 Memory:ce000000-ce011100

lo        Link encap:Local Loopback
           inet addr:127.0.0.1  Mask:255.0.0.0
           inet6 addr: ::1/128 Scope:Host
           UP LOOPBACK RUNNING  MTU:16436  Metric:1
           RX packets:449 errors:0 dropped:0 overruns:0 frame:0
           TX packets:449 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:42171 (41.1 KiB)  TX bytes:42171 (41.1 KiB)

peth0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
           UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
           RX packets:29 errors:0 dropped:0 overruns:0 frame:0
           TX packets:150 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:2464 (2.4 KiB)  TX bytes:20184 (19.7 KiB)
           Interrupt:19 Memory:c8000000-c8011100

vif0.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
           UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
           RX packets:106 errors:0 dropped:0 overruns:0 frame:0
           TX packets:27 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:5172 (5.0 KiB)  TX bytes:2016 (1.9 KiB)

vif5.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
           UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
           RX packets:26 errors:0 dropped:0 overruns:0 frame:0
           TX packets:107 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:32
           RX bytes:1784 (1.7 KiB)  TX bytes:5496 (5.3 KiB)

xenbr0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
           UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
           RX packets:110 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:3674 (3.5 KiB)  TX bytes:0 (0.0 b)





---
AlanP



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

jez

2007-Mar-23 17:32 UTC

head link

Re: [Xen-users] DOM0 networking - dead to the outside world

On Thu, Mar 22, 2007 at 08:35:03PM +0000, Alan Pearson
wrote:> 
> Now I have the pleasure installing Xen  3.0.4.1 on RHEL4.4 (host +  
> guests, using xensource RPMs) system, and I''m having a nightmare
with
> Dom0 networking.
> 
> The DomUs can see the outside world AND the Dom0 eth0 address, but  
> Dom0 cannot see the outside world nor the outside world see it.
> Dom0 can ping the guests fine, and the guests can ping Dom0.
> 
> For the purposes of this discussion, outside world means ''same
subnet
> on the physical interface''
> 
> Dom0 networking works fine before network-bridge is started, after it  
> starts, no more Dom0 networking.
> 
> I''ve really no idea what is wrong, and hours and hours playing
with
> Linux bridging has got me nowhere ;-(
> There doesn''t seem to be any iptables rules stopping traffic, and
> selinux is disabled.
> 
> 
> The output of the brctl show & ifconfig commands looks identical on  
> the xen3.0.4.1 to the working 3.0.2-2 systems, so I''m lost.
> Running tcpdump -s0 -L -A   produces no output on any interface.
> 
Hi Alan

can''t tell you what the problem is, but a similar problem popped up
last
week and the proposed solution was to create your own bridge. The start
of the thread (subject: dom0 networking disabled) is here:

  http://lists.xensource.com/archives/html/xen-users/2007-03/msg00636.html

A possible RedHat configuration is here:

  http://lists.xensource.com/archives/html/xen-users/2007-03/msg00697.html

HTH

jez

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Alan Pearson

2007-Mar-23 18:01 UTC

head link

Re: [Xen-users] DOM0 networking - dead to the outside world

Hi Jez


Something wierd going on, I don''t get the bridge device brought up (I
presume this should be done with service network start).

So after service network start I get this :

[root@xen-srv2 network-scripts]# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:14:5E:DE:BB:DA
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:34 errors:0 dropped:0 overruns:0 frame:0
          TX packets:51 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2366 (2.3 KiB)  TX bytes:14320 (13.9 KiB)
          Interrupt:19 Memory:c8000000-c8011100

eth1      Link encap:Ethernet  HWaddr 00:14:5E:DE:BB:DC
          inet addr:10.2.2.1  Bcast:10.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::214:5eff:fede:bbdc/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:96692 errors:0 dropped:0 overruns:0 frame:0
          TX packets:124994 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7027654 (6.7 MiB)  TX bytes:21223048 (20.2 MiB)
          Interrupt:16 Memory:ce000000-ce011100

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1652 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1652 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:151876 (148.3 KiB)  TX bytes:151876 (148.3 KiB)

sit0      Link encap:IPv6-in-IPv4
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

veth0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          BROADCAST NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:42 errors:0 dropped:0 overruns:0 frame:0
          TX packets:450 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3908 (3.8 KiB)  TX bytes:19116 (18.6 KiB)

veth1     Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:74 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6468 (6.3 KiB)  TX bytes:0 (0.0 b)

veth2     Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

veth3     Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif0.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:450 errors:0 dropped:0 overruns:0 frame:0
          TX packets:42 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:19116 (18.6 KiB)  TX bytes:3908 (3.8 KiB)

vif0.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:74 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:6468 (6.3 KiB)

vif0.2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif0.3    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif5.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3387 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:2008 (1.9 KiB)  TX bytes:148960 (145.4 KiB)





No ip address on eth0 and no xbr0 device.

?


[root@xen-srv2 network-scripts]# brctl show
bridge name     bridge id               STP enabled     interfaces
[root@xen-srv2 network-scripts]#



So am I doing something wrong ? I''d expect to see a bridge device with
an
IP address ?

-- 
AlanP

On Fri, March 23, 2007 5:32 pm, jez wrote:> On Thu, Mar 22, 2007 at 08:35:03PM +0000, Alan Pearson wrote:
>>
>> Now I have the pleasure installing Xen  3.0.4.1 on RHEL4.4 (host +
>> guests, using xensource RPMs) system, and I''m having a
nightmare with
>> Dom0 networking.
>>
>> The DomUs can see the outside world AND the Dom0 eth0 address, but
>> Dom0 cannot see the outside world nor the outside world see it.
>> Dom0 can ping the guests fine, and the guests can ping Dom0.
>>
>> For the purposes of this discussion, outside world means ''same
subnet
>> on the physical interface''
>>
>> Dom0 networking works fine before network-bridge is started, after it
>> starts, no more Dom0 networking.
>>
>> I''ve really no idea what is wrong, and hours and hours playing
with
>> Linux bridging has got me nowhere ;-(
>> There doesn''t seem to be any iptables rules stopping traffic,
and
>> selinux is disabled.
>>
>>
>> The output of the brctl show & ifconfig commands looks identical on
>> the xen3.0.4.1 to the working 3.0.2-2 systems, so I''m lost.
>> Running tcpdump -s0 -L -A   produces no output on any interface.
>>
>
> Hi Alan
>
> can''t tell you what the problem is, but a similar problem popped
up last
> week and the proposed solution was to create your own bridge. The start
> of the thread (subject: dom0 networking disabled) is here:
>
>   http://lists.xensource.com/archives/html/xen-users/2007-03/msg00636.html
>
> A possible RedHat configuration is here:
>
>   http://lists.xensource.com/archives/html/xen-users/2007-03/msg00697.html
>
> HTH
>
> jez
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

jez

2007-Mar-23 20:16 UTC

head link

Re: [Xen-users] DOM0 networking - dead to the outside world

On Fri, Mar 23, 2007 at 06:01:21PM -0000, Alan Pearson
wrote:> 
> Something wierd going on, I don''t get the bridge device brought up
(I
> presume this should be done with service network start).
> 
> So after service network start I get this :
> 
> [root@xen-srv2 network-scripts]# ifconfig -a
> eth0      Link encap:Ethernet  HWaddr 00:14:5E:DE:BB:DA
>           UP BROADCAST RUNNING  MTU:1500  Metric:1
>           RX packets:34 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:51 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:2366 (2.3 KiB)  TX bytes:14320 (13.9 KiB)
>           Interrupt:19 Memory:c8000000-c8011100
> 
<snip/>> 
> No ip address on eth0 and no xbr0 device.
> 
> ?
> 
> [root@xen-srv2 network-scripts]# brctl show
> bridge name     bridge id               STP enabled     interfaces
> [root@xen-srv2 network-scripts]#
> 
> So am I doing something wrong ? I''d expect to see a bridge device
with an
> IP address ?
> 
No Alan, I doubt you are doing anything wrong. I tested those startup
configurations on an FC5 box. Now that I think about it it''s quite
possible that this configuration is not supported on the more conservative 
RHEL4.3. You could try grepping the network startup scripts for 
"BRIDGE" to see if this stuff is supported:

    grep -i bridge /etc/sysconfig/network-scripts/*

As another alternative you could write your own shell script to replace
the network-bridge script that will set up your bridge for you. It''s
not
complicated to do, it''s just not something you want to try when you are
logged into the server remotely.

Assuming eth0 had an address 172.16.1.1, you could create a bridge called 
xbr0 as follows (note that this is from memory not a cut and paste from
a script, so treat with due caution):

[First a little something for those who don''t RTFP]
NOTE: DO *NOT* TYPE THE FOLLOWING AT THE COMMAND LINE WHEN YOU ARE 
      LOGGED IN REMOTELY VIA "eth0"

    brctl addbr xbr0
    ifconfig eth0 0.0.0.0 up
    brctl addif xbr0 eth0
    ifconfig xbr0 172.16.1.1 netmask 255.255.255.0 

It''s that simple. However, it''s worthwhile adding a few
timeout values
or your bridge will sit around studying it''s surroundings for the first
15 seconds or so. So a better version is:

    brctl addbr xbr0
    brctl setfd xbr0 0
    brctl sethello xbr0 0
    brctl stp xbr0 off
    ifconfig eth0 0.0.0.0 up
    brctl addif xbr0 eth0
    ifconfig xbr0 172.16.1.1 netmask 255.255.255.0 

The advantage of creating your own bridges IMHO, is that you can easily 
understand the traffic flow. I personally find the networking 
infrastucture that Xen creates by default to be difficult to understand 
conceptually and difficult to debug in practice. Maybe that''s just 
me though.

jez

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Nico Kadel-Garcia

2007-Mar-24 11:54 UTC

head link

Re: [Xen-users] DOM0 networking - dead to the outside world

> On Thu, Mar 22, 2007 at 08:35:03PM +0000, Alan Pearson wrote:
>   
>> Now I have the pleasure installing Xen  3.0.4.1 on RHEL4.4 (host +  
>> guests, using xensource RPMs) system, and I''m having a
nightmare with
>> Dom0 networking.
>>
>> The DomUs can see the outside world AND the Dom0 eth0 address, but  
>> Dom0 cannot see the outside world nor the outside world see it.
>> Dom0 can ping the guests fine, and the guests can ping Dom0.
>>
>> For the purposes of this discussion, outside world means ''same
subnet
>> on the physical interface''
>>
>> Dom0 networking works fine before network-bridge is started, after it  
>> starts, no more Dom0 networking.
>>
>> I''ve really no idea what is wrong, and hours and hours playing
with
>> Linux bridging has got me nowhere ;-(
>> There doesn''t seem to be any iptables rules stopping traffic,
and
>> selinux is disabled.
>>
>>
>> The output of the brctl show & ifconfig commands looks identical on
>> the xen3.0.4.1 to the working 3.0.2-2 systems, so I''m lost.
>> Running tcpdump -s0 -L -A   produces no output on any interface.
>>
>>     It''s probably iptables: I''ve not seen a good write-up on how
to deal
with this, but if you don''t need or want iptables filtering Dom0, you 
can disable it.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James Youngman

2007-Mar-24 21:13 UTC

head link

Re: [Xen-users] DOM0 networking - dead to the outside world

On 3/22/07, Alan Pearson <alandpearson@yahoo.com> wrote:
> The DomUs can see the outside world AND the Dom0 eth0 address, but
> Dom0 cannot see the outside world nor the outside world see it.
> Dom0 can ping the guests fine, and the guests can ping Dom0.
Did you try turning off TCP checksum offloading with ethtool?
> The output of the brctl show & ifconfig commands looks identical on
> the xen3.0.4.1 to the working 3.0.2-2 systems, so I''m lost.
> Running tcpdump -s0 -L -A   produces no output on any interface.
What about "tcpdump -n -v -v"?

James.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Alan Pearson

2007-Mar-25 17:47 UTC

head link

Re: [Xen-users] DOM0 networking - dead to the outside world

On 24 Mar 2007, at 11:54, Nico Kadel-Garcia wrote:
> SNIP
> It''s probably iptables: I''ve not seen a good write-up on
how to
> deal with this, but if you don''t need or want iptables filtering  
> Dom0, you can disable it.


No, iptables is set to default forward everything, and I''ve even put  
explicit forward rules in, no dice.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Alan Pearson

2007-Mar-25 17:50 UTC

head link

Re: [Xen-users] DOM0 networking - dead to the outside world

On 23 Mar 2007, at 17:32, jez wrote:
> Hi Alan
>
> can''t tell you what the problem is, but a similar problem popped
up
> last
> week and the proposed solution was to create your own bridge. The  
> start
> of the thread (subject: dom0 networking disabled) is here:
>
>   http://lists.xensource.com/archives/html/xen-users/2007-03/ 
> msg00636.html
>
> A possible RedHat configuration is here:
>
>   http://lists.xensource.com/archives/html/xen-users/2007-03/ 
> msg00697.html
>
> HTH
>
> jez
>

Now, although I''ve yet to get it working at box startup, I can tell  
you that creating my own bridge as per Jez''s recommendations works.
I.e. the bridge itself has an IP address.

So it seems like the vif0.0 or veth0  aren''t getting to peth0 and  
vice versa. Really strange.

I''ll go with Jez''s workaround, many thanks Jez, and post
tomorrow the
final config that will work at box startup !

Alan

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Xen users - Mar 2007 - DOM0 networking - dead to the outside world

[Xen-users] DOM0 networking - dead to the outside world

Re: [Xen-users] DOM0 networking - dead to the outside world

Re: [Xen-users] DOM0 networking - dead to the outside world

Re: [Xen-users] DOM0 networking - dead to the outside world

Re: [Xen-users] DOM0 networking - dead to the outside world

Re: [Xen-users] DOM0 networking - dead to the outside world

Re: [Xen-users] DOM0 networking - dead to the outside world

Re: [Xen-users] DOM0 networking - dead to the outside world