thr3ads.net - Xen users - [Xen-users] Xen 3.0.0 AMD64 - Can ping through bridge/vif - but only host (domain0) can use XenU guests services [Dec 2005]

If this information is useful, please help other people find it:
Share via:

Kevin Jackson

2005-Dec-23 10:11 UTC

[Xen-users] Xen 3.0.0 AMD64 - Can ping through bridge/vif - but only host (domain0) can use XenU guests services

Dear all,
I had a working Xen 2.0.0 32-Bit setup on RHEL 4 with one server (Opteron)
hosting multiple Xen guests.  Networking works fully and I''m able to
use
these Xen guests like any otehr server as expected.

I''ve now got a second server setup using Xen 3.0.0 running under 64-Bit
RHEL.

This Domain0 Xen server has its networking setup using VLANs on eth0 - so
eth0 doesn''t have a network address.

eth0
eth0.4086 172.18.138.48/255.255.0.0
eth0.4092 172.19.138.48/255.255.0.0

I have 2 bridges set up
bridge name     bridge id               STP enabled     interfaces
xen-br-back             8000.00093d10d5fc       no              eth0.4092
                                                        vif1.1
xen-br-front            8000.00093d10d5fc       no              eth0.4086
                                                        vif1.0


The guests have eth0 bridged through vif1.0 and packets leave the Domain0
server through eth0.4086 to other machines on the real network.
Same with eth1, but for the second interface.
They have a network address like:  eth0 172.18.135.10/255.255.0.0 and have a
default route through the Domain0''s eth0.4086 interface [with
forwarding
enabled on this box].

I can ping to and from the Xen guests from any machine on the phyiscal
network, but - I can''t, for example, SSH from the physical network to
this
Xen guest and vice versa.
But I can go from the Domain0 server and ssh to the Xen box.  So I think
I''ve some network level issue with the actual Domain0 server.

What is interesting is that I can - say - ssh from 172.19.138.1 (physical
box on same network as Xen guest) to 172.19.135.10 (Xen Guest) -- and can
see a netstat ESTABLISHED connection and eventually it times out.


Can anybody shed any light on what is going on?
I had a problem getting this to work with VLANs - I edited
/etc/xen/scripts/network-bridge script ops_start() function to not
''exit 1''
if Link $vdev is missing as this gets created fine (it seems!) and make sure
the script executes:

# old style without ${vdev}
transfer_addrs  ${netdev} ${bridge}
transfer_routes ${netdev} ${bridge}

as it was taking down my VLANs and not bringing them up again!

Domain0 (with one XenU Guest running):

Linux NLWAPP049 2.6.12.6-xen0 #3 Fri Dec 9 11:33:59 GMT 2005 x86_64 x86_64
x86_64 GNU/Linux

eth0      Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:812699 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11094 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:59039386 (56.3 MiB)  TX bytes:1438590 (1.3 MiB)
          Interrupt:25

eth0.4086 Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          inet addr:172.18.138.49  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:515633 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1192 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:24148579 (23.0 MiB)  TX bytes:115282 (112.5 KiB)

eth0.4092 Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          inet addr:172.19.138.49  Bcast:172.19.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:295755 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9925 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:16590729 (15.8 MiB)  TX bytes:1214086 (1.1 MiB)

eth1      Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FD
          inet addr:172.20.138.49  Bcast:172.20.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:110431 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:12261024 (11.6 MiB)  TX bytes:128 (128.0 b)
          Interrupt:26

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:217 errors:0 dropped:0 overruns:0 frame:0
          TX packets:217 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1150665 (1.0 MiB)  TX bytes:1150665 (1.0 MiB)

vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7 errors:0 dropped:0 overruns:0 frame:0
          TX packets:705 errors:0 dropped:13 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:486 (486.0 b)  TX bytes:42945 (41.9 KiB)

vif2.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:101 errors:0 dropped:0 overruns:0 frame:0
          TX packets:454 errors:0 dropped:137 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:7854 (7.6 KiB)  TX bytes:33697 (32.9 KiB)

xen-br-ba Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          inet addr:172.19.138.49  Bcast:172.19.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:292321 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7858 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:15423109 (14.7 MiB)  TX bytes:1056275 (1.0 MiB)

xen-br-fr Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
          inet addr:172.18.138.49  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:515367 errors:0 dropped:0 overruns:0 frame:0
          TX packets:688 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:24120245 (23.0 MiB)  TX bytes:44070 (43.0 KiB)



Bridge info

bridge name     bridge id               STP enabled     interfaces
xen-br-back             8000.00093d10d5fc       no              eth0.4092
                                                        vif2.1
xen-br-front            8000.00093d10d5fc       no              eth0.4086
                                                        vif2.0

net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 1
kernel.sysrq = 0
kernel.core_uses_pid = 1



Guest xenU (NLWXEN010)

Linux NLWXEN010 2.6.12.6-xenU #1 SMP Thu Dec 8 15:15:30 GMT 2005 x86_64
x86_64 x86_64 GNU/Linux

eth0      Link encap:Ethernet  HWaddr 00:16:3E:67:B0:35
          inet addr:172.18.135.10  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1252 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:76284 (74.4 KiB)  TX bytes:708 (708.0 b)

eth1      Link encap:Ethernet  HWaddr 00:16:3E:20:6E:75
          inet addr:172.19.135.10  Bcast:255.255.0.0  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:758 errors:0 dropped:0 overruns:0 frame:0
          TX packets:123 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:55771 (54.4 KiB)  TX bytes:9526 (9.3 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:560 (560.0 b)  TX bytes:560 (560.0 b)


/etc/xen/NLWXEN010
kernel = "/boot/vmlinuz-2.6-xenU"
memory = 512
name = "NLWXEN010"
nics = 1
ip = "172.18.135.10"
vif = [
''bridge=xen-br-front'',''bridge=xen-br-back'' ]
gateway = "172.18.138.49"
netmask = "255.255.0.0"
disk = [''file:/xen/NLWXEN010_rootfs01,sda1,w'',
''file:/xen/NLWXEN010_swap01,sda2,w'']
root = "/dev/sda1 ro"


Regards,

Kev
--
Kevin Jackson


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Kevin Jackson

2006-Jan-03 15:03 UTC

head link

Re: [Xen-users] Xen 3.0.0 AMD64 - Can ping through bridge/vif - butonly host (domain0) can use XenU guests services

On 1/3/06, Kevin Jackson <uksysadmin@gmail.com>
wrote:>
> Greg,
> I have indeed got it going by disabling the checksum on the DomU Guest
> interfaces (ethtool -K eth0 tx off).  It''s a workaround for now
which has
> allowed me getting all my guest OSes running and working with the rest of
> the network.  Its not the most ideal workaround - but usable.  Noting that
> Bugzilla entry at least highlights this as a known issue.
>
> Many thanks,
>
> Kev
>
> On 12/28/05, Greg Brackley <
> greg.brackley-sender-3f9d00@lucidsolutions.co.nz > wrote:
> >
> > I ran into a similar issue. I have made a few posts on this issue to
the
> > devel (and users) list. My workaround was to disable checksum offload
in
> > domU machines. I haven''t found an acceptable work-a-round for
dom0, and
> > instead provide non-VLAN frames on another physical interface for
dom0. Bug
> > #143 [1] might be related.
> >
> > Please let me know if you make any progress.
> >
> > Regards,
> >
> > Greg :-)
> >
> >
> > [1] http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=143
> >
> > ----- Original Message -----
> >  *From:* Kevin Jackson <uksysadmin@gmail.com>
> > *To:* xen-users@lists.xensource.com
> > *Sent:* Friday, December 23, 2005 11:11 PM
> > *Subject:* [Xen-users] Xen 3.0.0 AMD64 - Can ping through bridge/vif -
> > butonly host (domain0) can use XenU guests services
> >
> > Dear all,
> > I had a working Xen 2.0.0 32-Bit setup on RHEL 4 with one server
> > (Opteron) hosting multiple Xen guests.  Networking works fully and
I''m able
> > to use these Xen guests like any otehr server as expected.
> >
> > I''ve now got a second server setup using Xen 3.0.0 running
under 64-Bit
> > RHEL.
> >
> > This Domain0 Xen server has its networking setup using VLANs on eth0 -
> > so eth0 doesn''t have a network address.
> >
> > eth0
> > eth0.4086 172.18.138.48/255.255.0.0
> > eth0.4092 172.19.138.48/255.255.0.0
> >
> > I have 2 bridges set up
> > bridge name     bridge id               STP enabled     interfaces
> > xen-br-back             8000.00093d10d5fc       no
> > eth0.4092
> >                                                         vif1.1
> > xen-br-front            8000.00093d10d5fc       no
> > eth0.4086
> >                                                         vif1.0
> >
> >
> > The guests have eth0 bridged through vif1.0 and packets leave the
> > Domain0 server through eth0.4086 to other machines on the real
network.
> > Same with eth1, but for the second interface.
> > They have a network address like:  eth0 172.18.135.10/255.255.0.0 and
> > have a default route through the Domain0''s eth0.4086
interface [with
> > forwarding enabled on this box].
> >
> > I can ping to and from the Xen guests from any machine on the phyiscal
> > network, but - I can''t, for example, SSH from the physical
network to this
> > Xen guest and vice versa.
> > But I can go from the Domain0 server and ssh to the Xen box.  So I
think
> > I''ve some network level issue with the actual Domain0 server.
> >
> > What is interesting is that I can - say - ssh from
172.19.138.1(physical box on same network as Xen guest) to
> > 172.19.135.10 (Xen Guest) -- and can see a netstat ESTABLISHED
> > connection and eventually it times out.
> >
> >
> > Can anybody shed any light on what is going on?
> > I had a problem getting this to work with VLANs - I edited
> > /etc/xen/scripts/network-bridge script ops_start() function to not
''exit 1''
> > if Link $vdev is missing as this gets created fine (it seems!) and
make sure
> > the script executes:
> >
> > # old style without ${vdev}
> > transfer_addrs  ${netdev} ${bridge}
> > transfer_routes ${netdev} ${bridge}
> >
> > as it was taking down my VLANs and not bringing them up again!
> >
> > Domain0 (with one XenU Guest running):
> >
> > Linux NLWAPP049 2.6.12.6-xen0 #3 Fri Dec 9 11:33:59 GMT 2005 x86_64
> > x86_64 x86_64 GNU/Linux
> >
> > eth0      Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:812699 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:11094 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:59039386 (56.3 MiB)  TX bytes:1438590 (1.3 MiB)
> >           Interrupt:25
> >
> > eth0.4086 Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
> >           inet addr:172.18.138.49  Bcast: 172.18.255.255  Mask:
> > 255.255.0.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:515633 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:1192 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:24148579 (23.0 MiB)  TX bytes:115282 (112.5 KiB)
> >
> > eth0.4092 Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
> >           inet addr:172.19.138.49  Bcast: 172.19.255.255  Mask:
> > 255.255.0.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:295755 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:9925 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:16590729 (15.8 MiB)  TX bytes:1214086 (1.1 MiB)
> >
> > eth1      Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FD
> >           inet addr:172.20.138.49  Bcast: 172.20.255.255  Mask:
> > 255.255.0.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:110431 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:12261024 (11.6 MiB)  TX bytes:128 (128.0 b)
> >           Interrupt:26
> >
> > lo        Link encap:Local Loopback
> >           inet addr:127.0.0.1  Mask: 255.0.0.0
> >           UP LOOPBACK RUNNING  MTU:16436  Metric:1
> >           RX packets:217 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:217 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:1150665 (1.0 MiB)  TX bytes:1150665 (1.0 MiB)
> >
> > vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:7 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:705 errors:0 dropped:13 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:486 (486.0 b)  TX bytes:42945 (41.9 KiB)
> >
> > vif2.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:101 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:454 errors:0 dropped:137 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:7854 (7.6 KiB)  TX bytes:33697 (32.9 KiB)
> >
> > xen-br-ba Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
> >           inet addr:172.19.138.49  Bcast: 172.19.255.255  Mask:
> > 255.255.0.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:292321 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:7858 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:15423109 (14.7 MiB)  TX bytes:1056275 (1.0 MiB)
> >
> > xen-br-fr Link encap:Ethernet  HWaddr 00:09:3D:10:D5:FC
> >           inet addr:172.18.138.49  Bcast: 172.18.255.255  Mask:
> > 255.255.0.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:515367 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:688 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:24120245 (23.0 MiB)  TX bytes:44070 (43.0 KiB)
> >
> >
> >
> > Bridge info
> >
> > bridge name     bridge id               STP enabled     interfaces
> > xen-br-back             8000.00093d10d5fc       no
> > eth0.4092
> >                                                         vif2.1
> > xen-br-front            8000.00093d10d5fc       no
> > eth0.4086
> >                                                         vif2.0
> >
> > net.ipv4.ip_forward = 1
> > net.ipv4.conf.default.rp_filter = 1
> > net.ipv4.conf.default.accept_source_route = 1
> > kernel.sysrq = 0
> > kernel.core_uses_pid = 1
> >
> >
> >
> > Guest xenU (NLWXEN010)
> >
> > Linux NLWXEN010 2.6.12.6-xenU #1 SMP Thu Dec 8 15:15:30 GMT 2005
x86_64
> > x86_64 x86_64 GNU/Linux
> >
> > eth0      Link encap:Ethernet  HWaddr 00:16:3E:67:B0:35
> >           inet addr:172.18.135.10  Bcast: 172.18.255.255  Mask:
> > 255.255.0.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:1252 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:76284 (74.4 KiB)  TX bytes:708 (708.0 b)
> >
> > eth1      Link encap:Ethernet  HWaddr 00:16:3E:20:6E:75
> >           inet addr:172.19.135.10  Bcast: 255.255.0.0 
Mask:255.255.0.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:758 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:123 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:55771 (54.4 KiB)  TX bytes:9526 (9.3 KiB)
> >
> > lo        Link encap:Local Loopback
> >           inet addr:127.0.0.1  Mask: 255.0.0.0
> >           UP LOOPBACK RUNNING  MTU:16436  Metric:1
> >           RX packets:8 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:560 (560.0 b)  TX bytes:560 (560.0 b)
> >
> >
> > /etc/xen/NLWXEN010
> > kernel = "/boot/vmlinuz-2.6-xenU"
> > memory = 512
> > name = "NLWXEN010"
> > nics = 1
> > ip = "172.18.135.10"
> > vif = [
''bridge=xen-br-front'',''bridge=xen-br-back'' ]
> > gateway = "172.18.138.49"
> > netmask = " 255.255.0.0"
> > disk = [''file:/xen/NLWXEN010_rootfs01,sda1,w'',
> > ''file:/xen/NLWXEN010_swap01,sda2,w'']
> > root = "/dev/sda1 ro"
> >
> >
> > Regards,
> >
> > Kev
> > --
> > Kevin Jackson
> >
> >  ------------------------------
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@lists.xensource.com
> > http://lists.xensource.com/xen-users
> >
> >
>
>
> --
> Kevin Jackson
> Red Hat Certified Engineer
> www.uksysadmin.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Xen users - Dec 2005 - Xen 3.0.0 AMD64 - Can ping through bridge/vif - but only host (domain0) can use XenU guests services

[Xen-users] Xen 3.0.0 AMD64 - Can ping through bridge/vif - but only host (domain0) can use XenU guests services

Re: [Xen-users] Xen 3.0.0 AMD64 - Can ping through bridge/vif - butonly host (domain0) can use XenU guests services