cyber@MidnightFantasy.Com
2009-Jun-15 18:41 UTC
[Xen-users] DomU''s unable to connect to vnet0 / bridge
I''m trying to get some XEN VMs working on my CentOS 5.3 server. The
VM''s
are installed and everything is working but for the network. I believe my
problem is with the bridge. I have two ethernet adapters.
eth0 - external NIC running routable IP
eth1 - internal NIC running 10.0.xxx.xxx network
The internal machines behind the server are all NAT''d to the eth1 which
has an ip of 10.0.0.1. I have a vanilla install of CentOS 5.3 with the
XEN bits. Here is a list of the installed XEN and support packages:
[root@cerberus ~]# rpm -qa |grep -i xen
kernel-xen-2.6.18-92.1.17.el5
kernel-xen-2.6.18-92.1.22.el5
xen-libs-3.0.3-80.el5_3.2
xen-3.0.3-80.el5_3.2
kernel-xen-2.6.18-128.1.10.el5
[root@cerberus scripts]# rpm -qa |grep -i libvirt
libvirt-0.3.3-14.el5_3.1
libvirt-python-0.3.3-14.el5_3.1
[root@cerberus init.d]# rpm -qa |grep -i dnsmasq
dnsmasq-2.45-1.el5_2.1
I''m running the 2.6.18-128.1.10.el5xen kernel:
[root@cerberus init.d]# uname -r
2.6.18-128.1.10.el5xen
I only have one static routable IP address, and will be port forwarding on
the firewall (IPTables) to the VMs for the services the VMs will be
running. My hope is to have the VMs running on 10.0.2.XXX but that''s
not
a requirement. It seems that the CentOS distro is setup for
192.168.122.xxx so if that''s needed, I''ll deal with it.
Here is the route tables:
[root@cerberus ~]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
66.14.92.0 * 255.255.255.0 U 0 0 0 eth0
192.168.122.0 * 255.255.255.0 U 0 0 0 vnet0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0
10.0.0.0 * 255.0.0.0 U 0 0 0 eth1
default L408.AUSTTX-DSL 0.0.0.0 UG 0 0 0 eth0
I tried to change over the bridge by changing all the eth0 references in
the /etc/xen/scripts directory to use eth1:
[root@cerberus scripts]# grep eth1 *
network-bridge-bonding:netdev=${netdev:-eth1}
network-nat:netdev=${netdev:-eth1}
vif-common.sh: local nd=${netdev:-eth1}
but this isn''t working. :( I have Dom1 and Dom2 up and running, and
setup as 192.168.122.2 and 192.168.122.3 but they are unable to ping or
connect to 192.168.122.1. Dom0 is able to see and connect to the vnet0:
[root@cerberus scripts]# ping -c2 192.168.122.1
PING 192.168.122.1 (192.168.122.1) 56(84) bytes of data.
64 bytes from 192.168.122.1: icmp_seq=1 ttl=64 time=0.127 ms
64 bytes from 192.168.122.1: icmp_seq=2 ttl=64 time=0.108 ms
--- 192.168.122.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.108/0.117/0.127/0.014 ms
The last time I used these VMs was back when this was still a CentOS 5.0
(perhaps 5.1) box, but it seems that somewhere along the way the RPM
updates that brought it up to 5.3 did something and I''m unable to get
things working. :(
I had some problems with named, dhcpd, dnsmasq, and libvirtd having some
conflicts such that dnsmasq wouldn''t start due to the ports being used,
but I set named and dhcpd to only listen on eth0 and eth1, and dnsmasq to
only listen on vnet0, so all four services are functional (near as I can
tell) now.
I''ve tried setting the DomU''s as static, no joy.
I''ve tried using the
dnsmasq dhcp, also to no avail. I spent the weekend searching the
archives, googleing by brains out, and trying experiment after experiment,
to get one of my DomU''s to connect to either 192.168.122.1 or 10.0.0.1.
Any assistance here would be GREATLY appreciated... even if it were just
an example working configuration that allows a DomU to connect to an
internal private network on eth1.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Sreeharsha Totakura
2009-Jun-16 11:31 UTC
Re: [Xen-users] DomU''s unable to connect to vnet0 / bridge
perhaps you should configure the bridge as a trusted interface in firewall
settings.
Regards,
Sree Harsha Totakura
Member, Open Source Platform - TEG
Tata Consultancy Services
Ph:- +91-22-67782038
Mailto: sreeharsha.totakura@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Outsourcing
____________________________________________
cyber@MidnightFan
tasy.Com
Sent by: To
xen-users-bounces xen-users@lists.xensource.com
@lists.xensource. cc
com
Subject
[Xen-users] DomU''s unable to
06/16/2009 12:11 connect to vnet0 / bridge
AM
I''m trying to get some XEN VMs working on my CentOS 5.3 server. The
VM''s
are installed and everything is working but for the network. I believe my
problem is with the bridge. I have two ethernet adapters.
eth0 - external NIC running routable IP
eth1 - internal NIC running 10.0.xxx.xxx network
The internal machines behind the server are all NAT''d to the eth1 which
has an ip of 10.0.0.1. I have a vanilla install of CentOS 5.3 with the
XEN bits. Here is a list of the installed XEN and support packages:
[root@cerberus ~]# rpm -qa |grep -i xen
kernel-xen-2.6.18-92.1.17.el5
kernel-xen-2.6.18-92.1.22.el5
xen-libs-3.0.3-80.el5_3.2
xen-3.0.3-80.el5_3.2
kernel-xen-2.6.18-128.1.10.el5
[root@cerberus scripts]# rpm -qa |grep -i libvirt
libvirt-0.3.3-14.el5_3.1
libvirt-python-0.3.3-14.el5_3.1
[root@cerberus init.d]# rpm -qa |grep -i dnsmasq
dnsmasq-2.45-1.el5_2.1
I''m running the 2.6.18-128.1.10.el5xen kernel:
[root@cerberus init.d]# uname -r
2.6.18-128.1.10.el5xen
I only have one static routable IP address, and will be port forwarding on
the firewall (IPTables) to the VMs for the services the VMs will be
running. My hope is to have the VMs running on 10.0.2.XXX but that''s
not
a requirement. It seems that the CentOS distro is setup for
192.168.122.xxx so if that''s needed, I''ll deal with it.
Here is the route tables:
[root@cerberus ~]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
66.14.92.0 * 255.255.255.0 U 0 0 0
eth0
192.168.122.0 * 255.255.255.0 U 0 0 0
vnet0
169.254.0.0 * 255.255.0.0 U 0 0 0
eth0
10.0.0.0 * 255.0.0.0 U 0 0 0
eth1
default L408.AUSTTX-DSL 0.0.0.0 UG 0 0 0
eth0
I tried to change over the bridge by changing all the eth0 references in
the /etc/xen/scripts directory to use eth1:
[root@cerberus scripts]# grep eth1 *
network-bridge-bonding:netdev=${netdev:-eth1}
network-nat:netdev=${netdev:-eth1}
vif-common.sh: local nd=${netdev:-eth1}
but this isn''t working. :( I have Dom1 and Dom2 up and running, and
setup as 192.168.122.2 and 192.168.122.3 but they are unable to ping or
connect to 192.168.122.1. Dom0 is able to see and connect to the vnet0:
[root@cerberus scripts]# ping -c2 192.168.122.1
PING 192.168.122.1 (192.168.122.1) 56(84) bytes of data.
64 bytes from 192.168.122.1: icmp_seq=1 ttl=64 time=0.127 ms
64 bytes from 192.168.122.1: icmp_seq=2 ttl=64 time=0.108 ms
--- 192.168.122.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.108/0.117/0.127/0.014 ms
The last time I used these VMs was back when this was still a CentOS 5.0
(perhaps 5.1) box, but it seems that somewhere along the way the RPM
updates that brought it up to 5.3 did something and I''m unable to get
things working. :(
I had some problems with named, dhcpd, dnsmasq, and libvirtd having some
conflicts such that dnsmasq wouldn''t start due to the ports being used,
but I set named and dhcpd to only listen on eth0 and eth1, and dnsmasq to
only listen on vnet0, so all four services are functional (near as I can
tell) now.
I''ve tried setting the DomU''s as static, no joy.
I''ve tried using the
dnsmasq dhcp, also to no avail. I spent the weekend searching the
archives, googleing by brains out, and trying experiment after experiment,
to get one of my DomU''s to connect to either 192.168.122.1 or 10.0.0.1.
Any assistance here would be GREATLY appreciated... even if it were just
an example working configuration that allows a DomU to connect to an
internal private network on eth1.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
ForwardSourceID:NT000047B6
=====-----=====-----====Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
cyber@MidnightFantasy.Com
2009-Jun-16 19:56 UTC
Re: [Xen-users] DomU''s unable to connect to vnet0 / bridge
>>I''m trying to get some XEN VMs working on my CentOS 5.3 server. The >>VM''s are installed and everything is working but for the network. I >>believe my problem is with the bridge. I have two ethernet adapters. >>eth0 - external NIC running routable IP >>eth1 - internal NIC running 10.0.xxx.xxx network>perhaps you should configure the bridge as a trusted interface in >firewall settings.I''m working now. I talked with one of the NOC nerds here at work and he had JUST fought through the same problems the previous weekend so had the solution at hand. In the end, it seems that the XEN bridge scripts for RHEL 5.3 will ONLY work for ETH0. I made a number of changes along the way, so I''m not 100% sure exactly which modifications were required to get things working, but this is the best guess at the mods made that got me working: 1) Switched over my named.conf to only listen on lo, eth0, and eth1. 2) Set dnsmasq.conf to only listen on vnet1 and exclude lo, eth0, and eth1. 3) Set the network-bridge script to have hardcoded values for: vifnum=1 bridge=xenbr1 netdev=eth1 4) Uncomment the line for the network-bridge device in the xend-config.sxp script: [root@cerberus xen]# grep eth1 xend-config.sxp # To use a different one (e.g. eth1) use (network-script ''network-bridge netdev=eth1'') 5) Replace all references from eth0 to eth1 in /etc/xen/scripts: [root@cerberus scripts]# grep eth1 /etc/xen/scripts/* /etc/xen/scripts/network-bridge: netdev=eth1 /etc/xen/scripts/network-bridge-bonding: netdev=${netdev:-eth1} /etc/xen/scripts/network-nat: netdev=${netdev:-eth1} /etc/xen/scripts/vif-common.sh: local nd=${netdev:-eth1} 6) Give up on trying to use dnsmasq or dhcpd for DHCP on the VMs and just hardcode them for static IP''s in 10.0.2.xxx space. It seems my attempts to use 192.168.122.xxx for the VM space (as explained to me) was unnecessary, as I could ignore that and just use my 10-net for the VMs... I''ve got one CentOS and one Windows Server 2003 VMs currently running, and all seems to be well. I would have been nice if there was some sort of how-to doc on the wiki (I searched and was unable to find anything) for switching over from eth0 over to an eth1 config, but maybe that''s not as common of a configuration as I thought it might have been. {shrug}>From what I hear, RHEL / CentOS will be moving away from XEN and movingover to KVM (this was new news for me, certainly way old for y''all tho I''m guessing) so who knows... maybe with the 5.4 updates (or maybe RHEL 6) it''ll all be moot anyway... I certainly like the performance of XEN and am not looking forward to a VM akin to VMWare (which it seems KVM is more akin to) but my guess is I''ve got a few years at least before I need to worry about how that''s all going to play out and I''ll deal with it all on my next server buildout... which with any luck, will not be for at least 4-5 years. :) -- - Matt Schreiner o Cyber@MidnightFantasy.Com =\> MidnightFantasy Photography ...O=>\O... http://www.MidnightFantasy.Com When the last moment comes, and my life flashes before my eyes, it''s gonna be one hell of a show! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha
2009-Jun-17 01:22 UTC
Re: [Xen-users] DomU''s unable to connect to vnet0 / bridge
On Wed, Jun 17, 2009 at 2:56 AM, <cyber@midnightfantasy.com> wrote:> I''m working now. I talked with one of the NOC nerds here at work and he had > JUST fought through the same problems the previous weekend so had the > solution at hand. In the end, it seems that the XEN bridge scripts for RHEL > 5.3 will ONLY work for ETH0.If it''s true, that would be a bug, and you should use RHN to report that. After all, that''s what you''re paying RHEL support for isn''t it?> I made a number of changes along the way, so > I''m not 100% sure exactly which modifications were required to get things > working, but this is the best guess at the mods made that got me working: > 1) Switched over my named.conf to only listen on lo, eth0, and eth1. > > 2) Set dnsmasq.conf to only listen on vnet1 and exclude lo, eth0, and eth1. >I''m actually confused to read this. What''s vnet1? xen on RHEL only create xenbr0 and virbr0 by default. Did you create your own bridge?> 3) Set the network-bridge script to have hardcoded values for: > vifnum=1 > bridge=xenbr1 > netdev=eth1 > > 4) Uncomment the line for the network-bridge device in the > xend-config.sxp script: > [root@cerberus xen]# grep eth1 xend-config.sxp > # To use a different one (e.g. eth1) use > (network-script ''network-bridge netdev=eth1'')This should be the right way to create bridge over eth1. But I see you''ve hardcoded the values anyway :)> 6) Give up on trying to use dnsmasq or dhcpd for DHCP on the VMs and just > hardcode them for static IP''s in 10.0.2.xxx space. > > It seems my attempts to use 192.168.122.xxx for the VM space (as explained > to me) was unnecessary, as I could ignore that and just use my 10-net for > the VMs...192.168.122.0/24 is the default address space for virbr0. If you want to use it, you can simply put your domUs on virbr0 and it should just work without any additional config. dnsmasq already configure as DNS and DHCP server, and iptables NAT rule already created.> > I''ve got one CentOS and one Windows Server 2003 VMs currently running, and > all seems to be well. I would have been nice if there was some sort of > how-to doc on the wiki (I searched and was unable to find anything) for > switching over from eth0 over to an eth1 config, but maybe that''s not as > common of a configuration as I thought it might have been. {shrug}http://wiki.xensource.com/xenwiki/XenNetworking ... and since you use RHEL reading their doc wont hurt either. In my case I gave up xen''s default bridge a long time ago (had some problems back in the days of Xen 3.0), so I create my bridges manually using /etc/sysconfig/network-scripts/ifcfg-*> >> From what I hear, RHEL / CentOS will be moving away from XEN and moving > > over to KVM (this was new news for me, certainly way old for y''all tho I''m > guessing) so who knows... maybe with the 5.4 updates (or maybe RHEL 6)RH should still support Xen for the lifetime of RHEL 5.x. Don''t know about RHEL6 though.> it''ll all be moot anyway... I certainly like the performance of XEN and am > not looking forward to a VM akin to VMWare (which it seems KVM is more akin > to)If I understand correctly, several reasons for the move is that : - KVM is more integrated with Linux (whereas Xen dom0 support is not yet integrated in vanilla kernel) - newer pv_ops kernel should be able to detect KVM environment, and avoid using "expensive" instructions, which should lead to comparable performance (cpu-wise) to that of Xen PV. - PV drivers should reduce I/O performance penalty, which (ideally) should lead to comparable performance (I/O-wise) to that of Xen PV - CPUs with hardware virtualization support is quite common nowadays In any case you could still use Xen as long as you like. Note that opensolaris also uses Xen, so that could be an alternative if you want OS-supported Xen version in the future. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Sreeharsha Totakura
2009-Jun-17 04:44 UTC
Re: [Xen-users] DomU''s unable to connect to vnet0 / bridge
> "Fajar A. Nugraha" <fajar@fajar.net> > Sent by: xen-users-bounces@lists.xensource.com > > 06/17/2009 06:52 AM > > To > > Xen User-List <xen-users@lists.xensource.com> > > cc > > Subject > > Re: [Xen-users] DomU''s unable to connect to vnet0 / bridge > > On Wed, Jun 17, 2009 at 2:56 AM, <cyber@midnightfantasy.com> wrote: > > I''m working now. I talked with one of the NOC nerds here at work andhe had> > JUST fought through the same problems the previous weekend so had the > > solution at hand. In the end, it seems that the XEN bridge scripts forRHEL> > 5.3 will ONLY work for ETH0. > > If it''s true, that would be a bug, and you should use RHN to report > that. After all, that''s what you''re paying RHEL support for isn''t it? > > > I made a number of changes along the way, so > > I''m not 100% sure exactly which modifications were required to getthings> > working, but this is the best guess at the mods made that got meworking:> > 1) Switched over my named.conf to only listen on lo, eth0, and eth1. > > > > 2) Set dnsmasq.conf to only listen on vnet1 and exclude lo, eth0,andeth1.> > > > I''m actually confused to read this. What''s vnet1? xen on RHEL only > create xenbr0 and virbr0 by default. Did you create your own bridge? >I guess the virt-manager on RHEL creates a virtual interface vnetX for every vitual machine you start. If you have two virtual machines, you will have two vnet interfaces, vnet1, vnet2. xenbr0 is used to bridge these virtual interfaces with the physical network through eth0/eth1 -Harsha =====-----=====-----====Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users