Hallo, we have a XEN server with Redhat 5.2 and XEN 3.0.3. We have two problems. We build an Oracle RAC with two XEN guest Redhat 4.4. When we add more then 3 GB of RAM we cannot work with shared disks between the guests. Looks like a XEN bug. We have to reduce the RAM to max. 3 GB. Major problem is that under high load we loose some of the network interfaces and cannot get them working unless we reboot the guest. We need RH 4.4 as guest operating system. The guests are HVM. We have two bridges defined for the networker communication. It seem one bridge is no longer functional. We have a XEN IA64 server. Any idea how to solve this problem? regards Bernd _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
I saw a problem of losing network bridges before that was related to problems with spanning tree. You may be experiencing a similar problem Rob Aronson Practice Manager, Novacoast USA On Sun, Nov 30, 2008 at 12:13 AM, Bernd Gotschy < Bernd.Gotschy@inforsacom.com> wrote:> Hallo, > > we have a XEN server with Redhat 5.2 and XEN 3.0.3. We have two > problems. > We build an Oracle RAC with two XEN guest Redhat 4.4. When we add more > then 3 GB of RAM we cannot work with shared disks between the guests. > Looks like a XEN bug. We have to reduce the RAM to max. 3 GB. > Major problem is that under high load we loose some of the network > interfaces and cannot get them working unless we reboot the guest. > We need RH 4.4 as guest operating system. The guests are HVM. > We have two bridges defined for the networker communication. It > seem one bridge is no longer functional. We have a XEN IA64 server. > > Any idea how to solve this problem? > > regards > Bernd > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
> On Sun, Nov 30, 2008 at 12:13 AM, Bernd Gotschy > <Bernd.Gotschy@inforsacom.com <mailto:Bernd.Gotschy@inforsacom.com>> wrote: > Major problem is that under high load we loose some of the network > interfaces and cannot get them working unless we reboot the guest.We have find out quite the opposite way. If one of the VM has no activity, the ARP vs IP cache of most switch will discard the MAC of the VMs. So we have made a very simple python script that we deploy on all of our Xen servers, that simply does a ping of all VMs it finds in the /etc/xen/auto folder. This way, the ARP cache gets refreshed, and everything is working fine. Maybe this is what you are experiencing, no? Anyway, bellow is the script, I hope that helps (take care, the procs line might appear on 2 lines as I didn''t do an attachment). Thomas #!/usr/bin/env python import glob import re import subprocess import os import sys pathspec = "/etc/xen/auto/*" regexp = re.compile(r"vif.*?=.*?ip=([0-9\. ]+)''") files = glob.glob(pathspec) contents = ( file(t).read(-1) for t in files ) def ips(contents): matches = regexp.findall("\n".join(list(contents))) for match in matches: mips = match.split(" ") for ip in mips: yield ip devnull = file("/dev/null","w") procs = ( subprocess.Popen(["ping",ip,"-c","1"],stdin=devnull,stdout=devnull,stderr=devnull) for ip in ips(contents) ) returncodes = ( proc.wait() for proc in list(procs) ) sys.exit(sum(returncodes)) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
How often do you run this script? Federico -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Thomas Goirand Sent: Monday, December 01, 2008 6:41 AM Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] Stability of XEN under high load> On Sun, Nov 30, 2008 at 12:13 AM, Bernd Gotschy > <Bernd.Gotschy@inforsacom.com <mailto:Bernd.Gotschy@inforsacom.com>>wrote:> Major problem is that under high load we loose some of the network > interfaces and cannot get them working unless we reboot the guest.We have find out quite the opposite way. If one of the VM has no activity, the ARP vs IP cache of most switch will discard the MAC of the VMs. So we have made a very simple python script that we deploy on all of our Xen servers, that simply does a ping of all VMs it finds in the /etc/xen/auto folder. This way, the ARP cache gets refreshed, and everything is working fine. Maybe this is what you are experiencing, no? Anyway, bellow is the script, I hope that helps (take care, the procs line might appear on 2 lines as I didn''t do an attachment). Thomas #!/usr/bin/env python import glob import re import subprocess import os import sys pathspec = "/etc/xen/auto/*" regexp = re.compile(r"vif.*?=.*?ip=([0-9\. ]+)''") files = glob.glob(pathspec) contents = ( file(t).read(-1) for t in files ) def ips(contents): matches = regexp.findall("\n".join(list(contents))) for match in matches: mips = match.split(" ") for ip in mips: yield ip devnull = file("/dev/null","w") procs = ( subprocess.Popen(["ping",ip,"-c","1"],stdin=devnull,stdout=devnull,stderr=de vnull) for ip in ips(contents) ) returncodes = ( proc.wait() for proc in list(procs) ) sys.exit(sum(returncodes)) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
I get all these errors when I run the script. I named it "refresh.py". It is located in /root. Am I wrong? linux-tlxs:~ # ./refresh.py Traceback (most recent call last): File "./refresh.py", line 25, in ? returncodes = ( proc.wait() for proc in list(procs) ) File "./refresh.py", line 23, in <generator expression> procs (subprocess.Popen(["ping",ip,"-c","1"],stdin=devnull,stdout=devnull,stderr=d evnull) File "./refresh.py", line 17, in ips matches = regexp.findall("\n".join(list(contents))) File "./refresh.py", line 14, in <generator expression> contents = ( file(t).read(-1) for t in files ) IOError: [Errno 21] Is a directory -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Thomas Goirand Sent: Monday, December 01, 2008 6:41 AM Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] Stability of XEN under high load> On Sun, Nov 30, 2008 at 12:13 AM, Bernd Gotschy > <Bernd.Gotschy@inforsacom.com <mailto:Bernd.Gotschy@inforsacom.com>>wrote:> Major problem is that under high load we loose some of the network > interfaces and cannot get them working unless we reboot the guest.We have find out quite the opposite way. If one of the VM has no activity, the ARP vs IP cache of most switch will discard the MAC of the VMs. So we have made a very simple python script that we deploy on all of our Xen servers, that simply does a ping of all VMs it finds in the /etc/xen/auto folder. This way, the ARP cache gets refreshed, and everything is working fine. Maybe this is what you are experiencing, no? Anyway, bellow is the script, I hope that helps (take care, the procs line might appear on 2 lines as I didn''t do an attachment). Thomas #!/usr/bin/env python import glob import re import subprocess import os import sys pathspec = "/etc/xen/auto/*" regexp = re.compile(r"vif.*?=.*?ip=([0-9\. ]+)''") files = glob.glob(pathspec) contents = ( file(t).read(-1) for t in files ) def ips(contents): matches = regexp.findall("\n".join(list(contents))) for match in matches: mips = match.split(" ") for ip in mips: yield ip devnull = file("/dev/null","w") procs = ( subprocess.Popen(["ping",ip,"-c","1"],stdin=devnull,stdout=devnull,stderr=de vnull) for ip in ips(contents) ) returncodes = ( proc.wait() for proc in list(procs) ) sys.exit(sum(returncodes)) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Thomas, thanks for your input, but we are sure we do not have an arp problem. The phenomenon is that suddenly under heavy load a vif stops working - no connection to others on the bridge is possible any more then, it does not send out anything (not even arp requests are sent, then). Since this is happening under heavy load the mac of this vif is well listed in all concerned arp caches of others on the bridge. In the mean time we found out that we can trigger this phenomenon with certain tests that create massive network I/O. Is there any known bugs with xen 3.0.3 on ia64 concerning network load? Dirk --On 1. Dezember 2008 19:41:02 +0800 Thomas Goirand <thomas@goirand.fr> wrote:>> On Sun, Nov 30, 2008 at 12:13 AM, Bernd Gotschy >> <Bernd.Gotschy@inforsacom.com <mailto:Bernd.Gotschy@inforsacom.com>> >> wrote: Major problem is that under high load we loose some of the >> network interfaces and cannot get them working unless we reboot the >> guest. > > We have find out quite the opposite way. If one of the VM has no > activity, the ARP vs IP cache of most switch will discard the MAC of the > VMs. So we have made a very simple python script that we deploy on all > of our Xen servers, that simply does a ping of all VMs it finds in the > /etc/xen/auto folder. This way, the ARP cache gets refreshed, and > everything is working fine. Maybe this is what you are experiencing, no? > Anyway, bellow is the script, I hope that helps (take care, the procs > line might appear on 2 lines as I didn''t do an attachment). > > Thomas > ># !/usr/bin/env python > > import glob > import re > import subprocess > import os > import sys > > pathspec = "/etc/xen/auto/*" > regexp = re.compile(r"vif.*?=.*?ip=([0-9\. ]+)''") > > files = glob.glob(pathspec) > > contents = ( file(t).read(-1) for t in files ) > > def ips(contents): > matches = regexp.findall("\n".join(list(contents))) > for match in matches: > mips = match.split(" ") > for ip in mips: yield ip > > devnull = file("/dev/null","w") > procs = ( > subprocess.Popen(["ping",ip,"-c","1"],stdin=devnull,stdout=devnull,stderr > =devnull) for ip in ips(contents) ) > > returncodes = ( proc.wait() for proc in list(procs) ) > > sys.exit(sum(returncodes)) > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > lists.xensource.com/xen-users-------------------------------------------------------------- Dirk H. Schulz IT Systems Service Wiesenweg 12, 85567 Grafing Tel. 0 80 92/86 25 68 Fax. 0 80 92/86 25 72 -------------------------------------------------------------- Technik vom Feinsten - und das nötige Tuning _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Venefax wrote:> How often do you run this script? > FedericoEvery 3 minutes, as most switches have a timeout of 300 seconds for the ARP cache. Thomas _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
On Fri, Dec 5, 2008 at 5:01 PM, piavka <piavka@cs.bgu.ac.il> wrote:> > Hi, > > Can someone please post a working xml config file for a paravirt-ops domU. > > Thanks > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > lists.xensource.com/xen-users >sure, google.com -- Kind Regards Rudi Ahlers _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Hi, Can someone please post a working xml config file for a paravirt-ops domU. Thanks _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users