Hi,
I have been using xen with debian etch during two years. I had some problems
during the initial steps, but after that the system was working without any
problem.
Last week I upgrade the debian Lenny and my domU stopped to work (network)
and I''m really lost.
My configuration is simple. I have a box with two nics. In one nic I have dom0
(attached to a network) and in the other nix I have domU (attached to another
network).
domU boot, up the interfaces but no network. No packed throw the bridge.
my brctl show is:
bridge name bridge id STP enabled interfaces
eth0 8000.001422757b2c no peth0
eth1 8000.000423a8b773 no peth1
vif6.0
the important part of my sxp file is:
(network-script ''network-bridge netdev=eth1 bridge=eth1'')
in domU the interface is eth1
on of the things that I don''t like is that doing a tcpdump in the
interface
peth1, I got that there''s traffic, but:
14:50:04.222421 STP 802.1d, Config, Flags [none], bridge-id
806f.00:13:1a:a9:b3:00.8007, length 43
message-age 1.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay
15.00s
pay attention with the bridge-id 806f.00:13:1a:a9:b3:00.8007, not showed in
brctl.
So, please any idea what I''m doing wrong, or what''s happening?
I''m running:
ii xen-hypervisor-3.2-1-i386 3.2.1-2
ii xen-linux-system-2.6.26-2-xen-686 2.6.26-15lenny2
ii xen-utils-3.2-1 3.2.1-2
ii xen-utils-common 3.2.0-2
ii xenman 0.5-2.1
ii xenstore-utils 3.2.1-2
ii xenwatch 0.5.2-1
best regards,
Leo
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
On Fri, May 22, 2009 at 7:57 PM, Leopold Palomo-Avellaneda <leo@alaxarxa.net> wrote:> Last week I upgrade the debian Lenny and my domU stopped to work (network) > and I''m really lost.Hope you have a backup. It''s always a good idea to have tests first before implementing them in production.> domU boot, up the interfaces but no network. No packed throw the bridge. > > my brctl show is: > > bridge name bridge id STP enabled interfaces > eth0 8000.001422757b2c no peth0 > eth1 8000.000423a8b773 no peth1 > vif6.0Start with the basics : - does eth1 work when it''s not in a bridge (i.e. before starting xend) - does eth1 (the bridge) work when you put ip address there - is eth1 up -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
A Divendres 22 Maig 2009, Fajar A. Nugraha va escriure:> On Fri, May 22, 2009 at 7:57 PM, Leopold Palomo-Avellaneda > > <leo@alaxarxa.net> wrote: > > Last week I upgrade the debian Lenny and my domU stopped to work > > (network) and I''m really lost. > > Hope you have a backup. It''s always a good idea to have tests first > before implementing them in production.it''s not dramatic, just a bit of time, but I have backup of the important things.> > domU boot, up the interfaces but no network. No packed throw the bridge. > > > > my brctl show is: > > > > bridge name bridge id STP enabled interfaces > > eth0 8000.001422757b2c no peth0 > > eth1 8000.000423a8b773 no peth1 > > vif6.0 > > Start with the basics : > - does eth1 work when it''s not in a bridge (i.e. before starting xend)yes, I tested it. I make it work without xend. I have tried with the interface down and up, with the same result.> - does eth1 (the bridge) work when you put ip address therewell, in dom0 it works, with bridge and without bridge, but not in the other side of the bridge (domU)> - is eth1 upI have tested it with eth1 up and down. Regards, Leo _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, May 24, 2009 at 2:15 AM, Leopold Palomo Avellaneda <leo@alaxarxa.net> wrote:>> - does eth1 (the bridge) work when you put ip address there > > well, in dom0 it works, with bridge and without bridgeat least that worked. Now : - check iptables rules. Is it blocking packets? - try putting IP address on dom0''s eth1 (the bridge) and domUs eth0, see if you can ping dom0 from domU. While trying that, on another terminal do tcpdump on dom0 to snoop packets on the vif (vif6.0 on your example) and on eth1. This should determine whether the problem is on the bridge setup, NIC, or switch. As a side note, you should be able to use your old, working dom0 kernel (from Etch?) on newer Xen versions, so you might want to try that as well. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
A Diumenge 24 Maig 2009, Fajar A. Nugraha va escriure:> On Sun, May 24, 2009 at 2:15 AM, Leopold Palomo Avellaneda > > <leo@alaxarxa.net> wrote: > >> - does eth1 (the bridge) work when you put ip address there > > > > well, in dom0 it works, with bridge and without bridge > > at least that worked. Now : > - check iptables rules. Is it blocking packets?I didn''t modify any iptables. The server is under a firewall and there''s no blocking packets.> - try putting IP address on dom0''s eth1 (the bridge) and domUs eth0, > see if you can ping dom0 from domU. While trying that, on another > terminal do tcpdump on dom0 to snoop packets on the vif (vif6.0 on > your example) and on eth1. This should determine whether the problem > is on the bridge setup, NIC, or switch.ok, I found some interested result. I have configured eth1 in dom0. So, I have assigned an IP and I have ifup the interface. I have configured the interface in domU. I have had some little trouble because if I put the mac address the then I got a duplicate mac address, so I solved it with just only: vif = [''bridge=eth1''] Then I have rebooted domU and the network have been working. So, I have stopped the domU and then I have de-configured eth1 (ifdown the interface ) and then the network have stopped. So, I guest that the problem is that, there''s have been some changes in xen sources and I _must_ have the interface that acts as a bridge up, whenever the IP uses, if not, the bridge doesn''t work. Could you confirm this?> As a side note, you should be able to use your old, working dom0 > kernel (from Etch?) on newer Xen versions, so you might want to try > that as well.I would prefer that no. It''s better, IMHO as debian user, to follow the kernel versions of the stable series. Regards, Leo PS OTOH, I think that some people (as me) are suffering crashes from xen [1] , but this is another history. [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=524571 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I wish I could provide a better description than this but this is the best I have right now. I have a dual cpu quadcore optron system with 32g of ram. The base os is Centos 5.3 with updates as of a week or so ago(just checking again I found a new set this morning). I have several other systems with more or less the same config that have been running for months on end with no problems. I get the following from xm list. Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 1000 8 r----- 518.4 druid 3 511 1 r----- 14.1 imap 4 2048 1 -b---- 24.4 iwt 5 511 1 -b---- 29.7 l2tp1 6 1000 1 -b---- 21.5 oxmail 7 2000 1 -b---- 30.6 oxruby 8 1400 1 r----- 21.4 radius 9 256 1 -b---- 37.8 svnserver 10 500 1 ------ 19.1 vm-web-1 11 2000 1 -b---- 17.1 Note the svnserver status. On the next xm list it went back to the usual blocked but I have seen this same status for all the DomU''s. At this point the DomU''s are hung. I have also seen times that are very large -ve numbers. Trying to destroy the DomU sometimes ends in a Zombie. Restarting xend ends up with lots of zombie processes. Some time the Dom0 networking will hang also althought I can still get access via the serial console. I see nothing in /var/log/messages and the xen logs have not been much help. The system will run normally for several days before this problem crops up. Any kind of help or pointers of where to start looking would be greatly appreciated. -- Alvin Starr || voice: (416)585-9971x690 Interlink Connectivity || fax: (416)585-9974 alvin@iplink.net || _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha
2009-May-26 02:32 UTC
Re: [Xen-users] hard to pin down DomU/0 hanging problem.
On Mon, May 25, 2009 at 8:06 PM, Alvin Starr <alvin@iplink.net> wrote:> The base os is Centos 5.3 with updates as of a week or so ago(just checking > again I found a new set this morning).Try updating to latest version, shutdown all domUs, and don''t forget to restart dom0 afterwards. By default RHEL/Centos tries to save domU state on dom0 shutdown, which might cause some problems on xen upgrades, so make sure you shut down all domUs before rebooting.> Note the svnserver status.I''ve seen it some times, but it bevahes normally on my system. Is that particular domU the only one that''s having problems, or is it also the same for all domUs? -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, May 25, 2009 at 5:23 PM, Leopold Palomo-Avellaneda <leo@alaxarxa.net> wrote:> So, I guest that the problem is that, there''s have been some changes in xen > sources and I _must_ have the interface that acts as a bridge up, whenever > the IP uses, if not, the bridge doesn''t work. > > Could you confirm this?The bridge and physical interface MUST be up. On your case, the bridge is eth1 and the physical device is peth1. It does NOT need to have an IP address though. If I''m not mistaken earlier versions of Xen uses xenbr0/xenbr1 as bridge name, so that might be the change you get.> > >> As a side note, you should be able to use your old, working dom0 >> kernel (from Etch?) on newer Xen versions, so you might want to try >> that as well. > > I would prefer that no. It''s better, IMHO as debian user, to follow the kernel > versions of the stable series.If it works, that would be best.> PS OTOH, I think that some people (as me) are suffering crashes from xen [1] , > but this is another history. > > [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=524571... but it doesn''t work, you can use the old version as a workaround :) If that still doesn''t work, kernel 2.6.18 from xen.org might give better results. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
A Dimarts 26 Maig 2009, Fajar A. Nugraha va escriure:> On Mon, May 25, 2009 at 5:23 PM, Leopold Palomo-Avellaneda > > <leo@alaxarxa.net> wrote: > > So, I guest that the problem is that, there''s have been some changes in > > xen sources and I _must_ have the interface that acts as a bridge up, > > whenever the IP uses, if not, the bridge doesn''t work. > > > > Could you confirm this? > > The bridge and physical interface MUST be up. On your case, the bridge > is eth1 and the physical device is peth1. It does NOT need to have an > IP address though. > > If I''m not mistaken earlier versions of Xen uses xenbr0/xenbr1 as > bridge name, so that might be the change you get.well, I have not be able (or i don''t know how ...) to have an interface up without an ip ... in the last version of xen, my interfaces file had no entry for eth1 (peth1). Yes, this is also I changethat I don''t understand.> >> As a side note, you should be able to use your old, working dom0 > >> kernel (from Etch?) on newer Xen versions, so you might want to try > >> that as well. > > > > I would prefer that no. It''s better, IMHO as debian user, to follow the > > kernel versions of the stable series. > > If it works, that would be best.sure ....> > PS OTOH, I think that some people (as me) are suffering crashes from xen > > [1] , but this is another history. > > > > [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=524571 > > ... but it doesn''t work, you can use the old version as a workaround :) > If that still doesn''t work, kernel 2.6.18 from xen.org might give > better results.yes .... I would prefer a modern kernel but .... Thanks for all. Regards, Leo _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tue, May 26, 2009 at 3:16 PM, Leopold Palomo-Avellaneda <leo@alaxarxa.net> wrote:>> The bridge and physical interface MUST be up. On your case, the bridge >> is eth1 and the physical device is peth1. It does NOT need to have an >> IP address though. >> >> If I''m not mistaken earlier versions of Xen uses xenbr0/xenbr1 as >> bridge name, so that might be the change you get. > > well, I have not be able (or i don''t know how ...) to have an interface up > without an ip ... in the last version of xen, my interfaces file had no entry > for eth1 (peth1).Perhaps something like this will work auto eth1 iface eth1 inet manual> > Yes, this is also I changethat I don''t understand.Xen''s default bridging has changed several times, and it has caused some problems for me in the past as well. Plus I needed to use vlans (have a bridge for each vlan). So I disabled xend''s network scripts (on xend-config.sxp) and setup vlans and bridges manually from the OS. I''m using RHEL5.3 btw, which is stable so far, and I can complain to RH''s support (although there was never a need to) if I get problems. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
A Dimarts 26 Maig 2009, Fajar A. Nugraha va escriure:> On Tue, May 26, 2009 at 3:16 PM, Leopold Palomo-Avellaneda > > <leo@alaxarxa.net> wrote: > >> The bridge and physical interface MUST be up. On your case, the bridge > >> is eth1 and the physical device is peth1. It does NOT need to have an > >> IP address though. > >> > >> If I''m not mistaken earlier versions of Xen uses xenbr0/xenbr1 as > >> bridge name, so that might be the change you get. > > > > well, I have not be able (or i don''t know how ...) to have an interface > > up without an ip ... in the last version of xen, my interfaces file had > > no entry for eth1 (peth1). > > Perhaps something like this will work > > auto eth1 > iface eth1 inet manualnops, It didn''t work ... I simply put a private IP, but it should work without a IP.> > Yes, this is also I changethat I don''t understand. > > Xen''s default bridging has changed several times, and it has caused > some problems for me in the past as well. Plus I needed to use vlans > (have a bridge for each vlan). So I disabled xend''s network scripts > (on xend-config.sxp) and setup vlans and bridges manually from the OS. > I''m using RHEL5.3 btw, which is stable so far, and I can complain to > RH''s support (although there was never a need to) if I get problems.Ok, thanks. Leo _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Alvin Starr
2009-May-28 23:05 UTC
Re: [Xen-users] hard to pin down DomU/0 hanging problem.(found)
I think I found the problem. It seems the the sky2 network drivers do not like working with the bridge code. The thing that supprises me is that the kernel panic messages did not show up in dmesg but did as messages over the serial port. Fajar A. Nugraha wrote:> On Mon, May 25, 2009 at 8:06 PM, Alvin Starr <alvin@iplink.net> wrote: > >> The base os is Centos 5.3 with updates as of a week or so ago(just checking >> again I found a new set this morning). >> > > Try updating to latest version, shutdown all domUs, and don''t forget > to restart dom0 afterwards. > By default RHEL/Centos tries to save domU state on dom0 shutdown, > which might cause some problems on xen upgrades, so make sure you shut > down all domUs before rebooting. > > >> Note the svnserver status. >> > > I''ve seen it some times, but it bevahes normally on my system. > Is that particular domU the only one that''s having problems, or is it > also the same for all domUs? > >-- Alvin Starr || voice: (416)585-9971x690 Interlink Connectivity || fax: (416)585-9974 alvin@iplink.net || _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users