Yuvraj Agarwal
2011-Feb-08 17:02 UTC
[Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
Hi All, I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am having some trouble starting XEN. I am using the latest mercurial checkout (4.0.2-rc2) and the only change I made was to Config.mk to enable building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML version of xenstored since it is supposed to be much better/faster when starting up a larger number of domUs. The system starts up, and I can see xend started up, but when I do "xm list" or "xm info" I get an error message that xend has not started up? root@EM:/etc# xm list Error: Unable to connect to xend: No such file or directory. Is xend running? root@EM:/etc# root@EM:/etc# ps -efw | grep xen root 88 2 0 Feb07 ? 00:00:00 [xenwatch] root 89 2 0 Feb07 ? 00:00:00 [xenbus] root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python /usr/sbin/xend start root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python /usr/sbin/xend start root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen root@EM:/etc# root@EM:/etc# uname -a Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux root@EM:/etc# I did check and found two difference with my earlier installation of Xen 4.0.0 with the same kernel (although with the standard xenstored). 1. The xen 4.0 installation does show a "xenstored" process also running in addition to these ones. 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that doesn''t seem to exist on my system? Any pointers to fix this issue? Thanks! Yuvraj _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Vincent Hanquez
2011-Feb-09 08:22 UTC
Re: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
On 08/02/11 17:02, Yuvraj Agarwal wrote:> Hi All, > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > having some trouble starting XEN. I am using the latest mercurial checkout > (4.0.2-rc2) and the only change I made was to Config.mk to enable building > the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using > the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen > 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML > version of xenstored since it is supposed to be much better/faster when > starting up a larger number of domUs. > > The system starts up, and I can see xend started up, but when I do "xm > list" or "xm info" I get an error message that xend has not started up? > > root@EM:/etc# xm list > Error: Unable to connect to xend: No such file or directory. Is xend > running? > root@EM:/etc# > > root@EM:/etc# ps -efw | grep xen > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > /usr/sbin/xend start > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > /usr/sbin/xend start > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > root@EM:/etc# > > root@EM:/etc# uname -a > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux > root@EM:/etc# > > I did check and found two difference with my earlier installation of Xen > 4.0.0 with the same kernel (although with the standard xenstored). > 1. The xen 4.0 installation does show a "xenstored" process also running > in addition to these ones. > 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that > doesn''t seem to exist on my system? > > Any pointers to fix this issue?The reason xend died is that xenstored hasn''t started (or something happened to it), you need to figure out why. You can start xenstored directly on the cmdline with the no daemonize flag, to see if the program stays running, and if not you''ll see the reason why it dies. -- Vincent _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Yuvraj Agarwal
2011-Feb-09 18:26 UTC
RE: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
I did some more debugging and based on some of the instructions on http://wiki.xen.org/xenwiki/XenParavirtOps I found out that I may have been missing the "gntdev" module. I rebuilt the kernel and added that in (part of the kernel rather than as a separate module), but still have the same issues. Adding some more debugging information: root@nor:/home/yuvraj# cat /proc/misc 54 network_throughput 55 network_latency 56 cpu_dma_latency 57 device-mapper 221 mpt2ctl 144 nvram 228 hpet 58 blktap-control 59 gntdev 60 evtchn 61 autofs 231 snapshot 184 microcode 227 mcelog 62 rfkill 63 vga_arbiter root@nor:/home/yuvraj# ll /dev/xen/ total 0 drwxr-xr-x 2 root root 80 2011-02-08 22:07 ./ drwxr-xr-x 17 root root 6120 2011-02-08 22:07 ../ crw-r----- 1 root root 10, 63 2011-02-08 22:07 eventchn crw-rw---- 1 root root 10, 60 2011-02-08 22:07 evtchn root@nor:/home/yuvraj# Seems like I am missing the /dev/xen/gntdev entries? Shouldn''t these be created automatically? Thanks Yuvraj -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal Sent: Tuesday, February 08, 2011 9:02 AM To: xen-devel@lists.xensource.com Subject: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? Hi All, I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am having some trouble starting XEN. I am using the latest mercurial checkout (4.0.2-rc2) and the only change I made was to Config.mk to enable building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML version of xenstored since it is supposed to be much better/faster when starting up a larger number of domUs. The system starts up, and I can see xend started up, but when I do "xm list" or "xm info" I get an error message that xend has not started up? root@EM:/etc# xm list Error: Unable to connect to xend: No such file or directory. Is xend running? root@EM:/etc# root@EM:/etc# ps -efw | grep xen root 88 2 0 Feb07 ? 00:00:00 [xenwatch] root 89 2 0 Feb07 ? 00:00:00 [xenbus] root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python /usr/sbin/xend start root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python /usr/sbin/xend start root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen root@EM:/etc# root@EM:/etc# uname -a Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux root@EM:/etc# I did check and found two difference with my earlier installation of Xen 4.0.0 with the same kernel (although with the standard xenstored). 1. The xen 4.0 installation does show a "xenstored" process also running in addition to these ones. 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that doesn''t seem to exist on my system? Any pointers to fix this issue? Thanks! Yuvraj _______________________________________________ Xen-devel mailing list <mailto:Xen-devel@lists.xensource.com> Xen-devel@lists.xensource.com <http://lists.xensource.com/xen-devel> http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Yuvraj Agarwal
2011-Feb-10 04:39 UTC
RE: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
Hi All, I did some further debugging and am attaching some more information here. Any help is appreciated of course. Out of curiosity, are we amongst the few people who are trying to use the OCAML version of xenstored? If I change the Config.mk to disable building the OCAML version of xenstored xend starts up and I can do "xm info" and everything works fine. If I build the OCAML version of xenstored instead, I get the "xend is not started error" even though I see two xend processes. Furthermore, if I run the OCAML xentored with an strace, I get the following error: "strace -f /usr/bin/xenstored --no-fork" open("/dev/xen/eventchn", O_RDWR) = 6 open("/proc/xen/xsd_port", O_RDONLY) = 7 read(7, "145", 20) = 3 close(7) = 0 open("/proc/xen/xsd_kva", O_RDWR) = 7 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0) = 0x7f2a4f682000 close(7) = 0 ioctl(6, EVIOCGVERSION, 0x7fff86ad7320) = -1 ENOTTY (Inappropriate ioctl for device) write(2, "Fatal error: exception Failure(\""..., 64Fatal error: exception Failure("ioctl bind_interdomain failed") ) = 64 exit_group(2) = ? thanks Yuvraj From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal Sent: Wednesday, February 09, 2011 10:26 AM To: xen-devel@lists.xensource.com Cc: Asim Subject: RE: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? I did some more debugging and based on some of the instructions on http://wiki.xen.org/xenwiki/XenParavirtOps I found out that I may have been missing the "gntdev" module. I rebuilt the kernel and added that in (part of the kernel rather than as a separate module), but still have the same issues. Adding some more debugging information: root@nor:/home/yuvraj# cat /proc/misc 54 network_throughput 55 network_latency 56 cpu_dma_latency 57 device-mapper 221 mpt2ctl 144 nvram 228 hpet 58 blktap-control 59 gntdev 60 evtchn 61 autofs 231 snapshot 184 microcode 227 mcelog 62 rfkill 63 vga_arbiter root@nor:/home/yuvraj# ll /dev/xen/ total 0 drwxr-xr-x 2 root root 80 2011-02-08 22:07 ./ drwxr-xr-x 17 root root 6120 2011-02-08 22:07 ../ crw-r----- 1 root root 10, 63 2011-02-08 22:07 eventchn crw-rw---- 1 root root 10, 60 2011-02-08 22:07 evtchn root@nor:/home/yuvraj# Seems like I am missing the /dev/xen/gntdev entries? Shouldn''t these be created automatically? Thanks Yuvraj -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal Sent: Tuesday, February 08, 2011 9:02 AM To: xen-devel@lists.xensource.com Subject: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? Hi All, I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am having some trouble starting XEN. I am using the latest mercurial checkout (4.0.2-rc2) and the only change I made was to Config.mk to enable building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML version of xenstored since it is supposed to be much better/faster when starting up a larger number of domUs. The system starts up, and I can see xend started up, but when I do "xm list" or "xm info" I get an error message that xend has not started up? root@EM:/etc# xm list Error: Unable to connect to xend: No such file or directory. Is xend running? root@EM:/etc# root@EM:/etc# ps -efw | grep xen root 88 2 0 Feb07 ? 00:00:00 [xenwatch] root 89 2 0 Feb07 ? 00:00:00 [xenbus] root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python /usr/sbin/xend start root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python /usr/sbin/xend start root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen root@EM:/etc# root@EM:/etc# uname -a Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux root@EM:/etc# I did check and found two difference with my earlier installation of Xen 4.0.0 with the same kernel (although with the standard xenstored). 1. The xen 4.0 installation does show a "xenstored" process also running in addition to these ones. 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that doesn''t seem to exist on my system? Any pointers to fix this issue? Thanks! Yuvraj _______________________________________________ Xen-devel mailing list <mailto:Xen-devel@lists.xensource.com> Xen-devel@lists.xensource.com <http://lists.xensource.com/xen-devel> http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2011-Feb-10 07:11 UTC
Re: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
On Wed, Feb 09, 2011 at 08:39:37PM -0800, Yuvraj Agarwal wrote:> Hi All, > > > > I did some further debugging and am attaching some more information here. > Any help is appreciated of course. Out of curiosity, are we amongst the > few people who are trying to use the OCAML version of xenstored? >At least Citrix XenServer and XCP do use oxenstored.. -- Pasi> If I change the Config.mk to disable building the OCAML version of > xenstored xend starts up and I can do "xm info" and everything works fine. > > > > If I build the OCAML version of xenstored instead, I get the "xend is not > started error" even though I see two xend processes. > > Furthermore, if I run the OCAML xentored with an strace, I get the > following error: > > > > "strace -f /usr/bin/xenstored --no-fork" > > > > open("/dev/xen/eventchn", O_RDWR) = 6 > > open("/proc/xen/xsd_port", O_RDONLY) = 7 > > read(7, "145", 20) = 3 > > close(7) = 0 > > open("/proc/xen/xsd_kva", O_RDWR) = 7 > > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0) = 0x7f2a4f682000 > > close(7) = 0 > > ioctl(6, EVIOCGVERSION, 0x7fff86ad7320) = -1 ENOTTY (Inappropriate ioctl > for device) > > write(2, "Fatal error: exception Failure(\""..., 64Fatal error: exception > Failure("ioctl bind_interdomain failed") > > ) = 64 > > exit_group(2) = ? > > > > thanks > > Yuvraj > > > > > > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal > Sent: Wednesday, February 09, 2011 10:26 AM > To: xen-devel@lists.xensource.com > Cc: Asim > Subject: RE: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not > starting? > > > > I did some more debugging and based on some of the instructions on > [1]http://wiki.xen.org/xenwiki/XenParavirtOps I found out that I may have > been missing the "gntdev" module. > > I rebuilt the kernel and added that in (part of the kernel rather than as > a separate module), but still have the same issues. Adding some more > debugging information: > > > > root@nor:/home/yuvraj# cat /proc/misc > > 54 network_throughput > > 55 network_latency > > 56 cpu_dma_latency > > 57 device-mapper > > 221 mpt2ctl > > 144 nvram > > 228 hpet > > 58 blktap-control > > 59 gntdev > > 60 evtchn > > 61 autofs > > 231 snapshot > > 184 microcode > > 227 mcelog > > 62 rfkill > > 63 vga_arbiter > > root@nor:/home/yuvraj# ll /dev/xen/ > > total 0 > > drwxr-xr-x 2 root root 80 2011-02-08 22:07 ./ > > drwxr-xr-x 17 root root 6120 2011-02-08 22:07 ../ > > crw-r----- 1 root root 10, 63 2011-02-08 22:07 eventchn > > crw-rw---- 1 root root 10, 60 2011-02-08 22:07 evtchn > > root@nor:/home/yuvraj# > > > > Seems like I am missing the /dev/xen/gntdev entries? Shouldn''t these be > created automatically? > > Thanks > > Yuvraj > > > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal > Sent: Tuesday, February 08, 2011 9:02 AM > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not > starting? > > > > Hi All, > > > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > having some trouble starting XEN. I am using the latest mercurial checkout > > (4.0.2-rc2) and the only change I made was to Config.mk to enable building > the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using > the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen > > 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML > version of xenstored since it is supposed to be much better/faster when > starting up a larger number of domUs. > > > > The system starts up, and I can see xend started up, but when I do "xm > list" or "xm info" I get an error message that xend has not started up? > > > > root@EM:/etc# xm list > > Error: Unable to connect to xend: No such file or directory. Is xend > running? > > root@EM:/etc# > > > > root@EM:/etc# ps -efw | grep xen > > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > > /usr/sbin/xend start > > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > > /usr/sbin/xend start > > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > > root@EM:/etc# > > > > root@EM:/etc# uname -a > > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux > root@EM:/etc# > > > > I did check and found two difference with my earlier installation of Xen > > 4.0.0 with the same kernel (although with the standard xenstored). > > 1. The xen 4.0 installation does show a "xenstored" process also running > in addition to these ones. > > 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that > doesn''t seem to exist on my system? > > > > Any pointers to fix this issue? > > > > Thanks! > > Yuvraj > > > > _______________________________________________ > > Xen-devel mailing list > > [2]Xen-devel@lists.xensource.com > > [3]http://lists.xensource.com/xen-devel > > References > > Visible links > 1. http://wiki.xen.org/xenwiki/XenParavirtOps > 2. mailto:Xen-devel@lists.xensource.com > 3. http://lists.xensource.com/xen-devel> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Feb-10 08:42 UTC
Re: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
On Wed, 2011-02-09 at 08:22 +0000, Vincent Hanquez wrote:> On 08/02/11 17:02, Yuvraj Agarwal wrote: > > Hi All, > > > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > > having some trouble starting XEN. I am using the latest mercurial checkout > > (4.0.2-rc2) and the only change I made was to Config.mk to enable building > > the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using > > the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen > > 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML > > version of xenstored since it is supposed to be much better/faster when > > starting up a larger number of domUs. > > > > The system starts up, and I can see xend started up, but when I do "xm > > list" or "xm info" I get an error message that xend has not started up? > > > > root@EM:/etc# xm list > > Error: Unable to connect to xend: No such file or directory. Is xend > > running? > > root@EM:/etc# > > > > root@EM:/etc# ps -efw | grep xen > > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > > /usr/sbin/xend start > > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > > /usr/sbin/xend start > > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > > root@EM:/etc# > > > > root@EM:/etc# uname -a > > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux > > root@EM:/etc# > > > > I did check and found two difference with my earlier installation of Xen > > 4.0.0 with the same kernel (although with the standard xenstored). > > 1. The xen 4.0 installation does show a "xenstored" process also running > > in addition to these ones. > > 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that > > doesn''t seem to exist on my system? > > > > Any pointers to fix this issue? > > The reason xend died is that xenstored hasn''t started (or something > happened to it), you need to figure out why. > > You can start xenstored directly on the cmdline with the no daemonize > flag, to see if the program stays running, and if not you''ll see > the reason why it dies.Xen 4.0 appears to use oxenstored from http://xenbits.xensource.com/ext/xen-ocaml-tools.hg (it only moved in-tree in the 4.1 dev cycle). IIRC the ocaml evtchn bindings used to blow away the /dev/evtchn node and replace it with their own, with hardcoded major:minor or 10:61. However that "61" is not really a static number and can vary depending on kernel version / configuration. Judging from Yuvraj''s debugging his evtchn node has ended up at minor 60. Looks like http://xenbits.xensource.com/xen-unstable.hg?rev/e7b55cc5533a needs porting to xen-ocaml-tools.hg. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Yuvraj Agarwal
2011-Feb-10 17:53 UTC
RE: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
Hi Ian and Vincent, Thank you for your responses - There are several possible issues it seems: ** I actually have both /dev/xen/eventchn and /dev/xen/evtchn but no /dev/xen/gntdev currently? root@EM:/home/yuvraj# ll /dev/xen/ total 0 drwxr-xr-x 2 root root 80 2011-02-10 09:18 ./ drwxr-xr-x 17 root root 6120 2011-02-10 09:18 ../ crw-r----- 1 root root 10, 63 2011-02-10 09:18 eventchn crw-rw---- 1 root root 10, 60 2011-02-10 09:18 evtchn root@EM:/home/yuvraj# ** It also looks like the minor numbers are mixed up, for example /proc/misc shows minor number 63 also for vga_arbitrer? Not sure if that matters. root@EM:/home/yuvraj# cat /proc/misc < snip > 59 gntdev 60 evtchn < snip > 63 vga_arbiter root@EM:/home/yuvraj# ** I also tried to start xenstored without the daemonize flag as Vincent mentioned, and posted the results in a separate email, (re-posting it here). root@EM:/home/yuvraj# strace -f /usr/bin/xenstored --no-fork <snip> open("/dev/xen/eventchn", O_RDWR) = 6 open("/proc/xen/xsd_port", O_RDONLY) = 7 read(7, "145", 20) = 3 close(7) = 0 open("/proc/xen/xsd_kva", O_RDWR) = 7 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0) = 0x7f5bbd2ef000 close(7) = 0 ioctl(6, EVIOCGVERSION, 0x7fffd0cd3e30) = -1 ENOTTY (Inappropriate ioctl for device) write(2, "Fatal error: exception Failure(\""..., 64Fatal error: exception Failure("ioctl bind_interdomain failed") ) = 64 exit_group(2) = ? Is there a way I can fix (e.g. a patch I can apply) this on my end? I can try it out and post the results back so that this makes it into the tree, and helps anyone else using the OCAML version of xenstored. Thanks! Yuvraj -----Original Message----- From: Ian Campbell [mailto:Ian.Campbell@citrix.com] Sent: Thursday, February 10, 2011 12:43 AM To: Vincent Hanquez Cc: Yuvraj Agarwal; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? On Wed, 2011-02-09 at 08:22 +0000, Vincent Hanquez wrote:> On 08/02/11 17:02, Yuvraj Agarwal wrote: > > Hi All, > > > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > > having some trouble starting XEN. I am using the latest mercurial > > checkout > > (4.0.2-rc2) and the only change I made was to Config.mk to enable > > building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?> > y". I am using the same kernel I built earlier for a similar Ubuntu > > 10.04-64bit / Xen > > 4.0.0 installation (version 2.6.32.12). I would like to use the > > OCAML version of xenstored since it is supposed to be much > > better/faster when starting up a larger number of domUs. > > > > The system starts up, and I can see xend started up, but when I do > > "xm list" or "xm info" I get an error message that xend has not started > > up? > > > > root@EM:/etc# xm list > > Error: Unable to connect to xend: No such file or directory. Is xend > > running? > > root@EM:/etc# > > > > root@EM:/etc# ps -efw | grep xen > > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > > /usr/sbin/xend start > > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > > /usr/sbin/xend start > > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > > root@EM:/etc# > > > > root@EM:/etc# uname -a > > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 > > GNU/Linux root@EM:/etc# > > > > I did check and found two difference with my earlier installation of > > Xen > > 4.0.0 with the same kernel (although with the standard xenstored). > > 1. The xen 4.0 installation does show a "xenstored" process also > > running in addition to these ones. > > 2. "xend" seems to be looking for a directory, " > > /var/lib/xenstored" that doesn''t seem to exist on my system? > > > > Any pointers to fix this issue? > > The reason xend died is that xenstored hasn''t started (or something > happened to it), you need to figure out why. > > You can start xenstored directly on the cmdline with the no daemonize > flag, to see if the program stays running, and if not you''ll see the > reason why it dies.Xen 4.0 appears to use oxenstored from http://xenbits.xensource.com/ext/xen-ocaml-tools.hg (it only moved in-tree in the 4.1 dev cycle). IIRC the ocaml evtchn bindings used to blow away the /dev/evtchn node and replace it with their own, with hardcoded major:minor or 10:61. However that "61" is not really a static number and can vary depending on kernel version / configuration. Judging from Yuvraj''s debugging his evtchn node has ended up at minor 60. Looks like http://xenbits.xensource.com/xen-unstable.hg?rev/e7b55cc5533a needs porting to xen-ocaml-tools.hg. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
John McCullough
2011-Feb-10 17:56 UTC
Re: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
ocaml-xenstored made 10 63 eventchn. On Thu, Feb 10, 2011 at 9:53 AM, Yuvraj Agarwal <yuvraj@cs.ucsd.edu> wrote:> Hi Ian and Vincent, > > Thank you for your responses - There are several possible issues it seems: > > ** I actually have both /dev/xen/eventchn and /dev/xen/evtchn but no > /dev/xen/gntdev currently? > > root@EM:/home/yuvraj# ll /dev/xen/ > total 0 > drwxr-xr-x 2 root root 80 2011-02-10 09:18 ./ > drwxr-xr-x 17 root root 6120 2011-02-10 09:18 ../ > crw-r----- 1 root root 10, 63 2011-02-10 09:18 eventchn > crw-rw---- 1 root root 10, 60 2011-02-10 09:18 evtchn > root@EM:/home/yuvraj# > > ** It also looks like the minor numbers are mixed up, for example > /proc/misc > shows minor number 63 also for vga_arbitrer? Not sure if that matters. > > root@EM:/home/yuvraj# cat /proc/misc > < snip > > 59 gntdev > 60 evtchn > < snip > > 63 vga_arbiter > root@EM:/home/yuvraj# > > ** I also tried to start xenstored without the daemonize flag as Vincent > mentioned, and posted the results in a separate email, (re-posting it > here). > > root@EM:/home/yuvraj# strace -f /usr/bin/xenstored --no-fork > > <snip> > open("/dev/xen/eventchn", O_RDWR) = 6 > open("/proc/xen/xsd_port", O_RDONLY) = 7 > read(7, "145", 20) = 3 > close(7) = 0 > open("/proc/xen/xsd_kva", O_RDWR) = 7 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0) = 0x7f5bbd2ef000 > close(7) = 0 > ioctl(6, EVIOCGVERSION, 0x7fffd0cd3e30) = -1 ENOTTY (Inappropriate ioctl > for > device) > write(2, "Fatal error: exception Failure(\""..., 64Fatal error: exception > Failure("ioctl bind_interdomain failed") > ) = 64 > exit_group(2) = ? > > Is there a way I can fix (e.g. a patch I can apply) this on my end? I can > try it out and post the results back so that this makes it into the tree, > and helps anyone else using the OCAML version of xenstored. > > Thanks! > Yuvraj > > > -----Original Message----- > From: Ian Campbell [mailto:Ian.Campbell@citrix.com] > Sent: Thursday, February 10, 2011 12:43 AM > To: Vincent Hanquez > Cc: Yuvraj Agarwal; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not > starting? > > On Wed, 2011-02-09 at 08:22 +0000, Vincent Hanquez wrote: > > On 08/02/11 17:02, Yuvraj Agarwal wrote: > > > Hi All, > > > > > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > > > having some trouble starting XEN. I am using the latest mercurial > > > checkout > > > (4.0.2-rc2) and the only change I made was to Config.mk to enable > > > building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?> > > y". I am using the same kernel I built earlier for a similar Ubuntu > > > 10.04-64bit / Xen > > > 4.0.0 installation (version 2.6.32.12). I would like to use the > > > OCAML version of xenstored since it is supposed to be much > > > better/faster when starting up a larger number of domUs. > > > > > > The system starts up, and I can see xend started up, but when I do > > > "xm list" or "xm info" I get an error message that xend has not started > > > up? > > > > > > root@EM:/etc# xm list > > > Error: Unable to connect to xend: No such file or directory. Is xend > > > running? > > > root@EM:/etc# > > > > > > root@EM:/etc# ps -efw | grep xen > > > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > > > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > > > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > > > /usr/sbin/xend start > > > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > > > /usr/sbin/xend start > > > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > > > root@EM:/etc# > > > > > > root@EM:/etc# uname -a > > > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 > > > GNU/Linux root@EM:/etc# > > > > > > I did check and found two difference with my earlier installation of > > > Xen > > > 4.0.0 with the same kernel (although with the standard xenstored). > > > 1. The xen 4.0 installation does show a "xenstored" process also > > > running in addition to these ones. > > > 2. "xend" seems to be looking for a directory, " > > > /var/lib/xenstored" that doesn''t seem to exist on my system? > > > > > > Any pointers to fix this issue? > > > > The reason xend died is that xenstored hasn''t started (or something > > happened to it), you need to figure out why. > > > > You can start xenstored directly on the cmdline with the no daemonize > > flag, to see if the program stays running, and if not you''ll see the > > reason why it dies. > > Xen 4.0 appears to use oxenstored from > http://xenbits.xensource.com/ext/xen-ocaml-tools.hg (it only moved in-tree > in the 4.1 dev cycle). > > IIRC the ocaml evtchn bindings used to blow away the /dev/evtchn node and > replace it with their own, with hardcoded major:minor or 10:61. > However that "61" is not really a static number and can vary depending on > kernel version / configuration. > > Judging from Yuvraj''s debugging his evtchn node has ended up at minor 60. > > Looks like http://xenbits.xensource.com/xen-unstable.hg?rev/e7b55cc5533a > needs porting to xen-ocaml-tools.hg. > > Ian. > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Feb-10 18:08 UTC
RE: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting?
On Thu, 2011-02-10 at 17:53 +0000, Yuvraj Agarwal wrote:> Hi Ian and Vincent, > > Thank you for your responses - There are several possible issues it seems:No, just one -- the event channel one.> ** I actually have both /dev/xen/eventchn and /dev/xen/evtchn > but no /dev/xen/gntdev currently?AFAIK oxenstored does not use gntdev.> root@EM:/home/yuvraj# > > ** It also looks like the minor numbers are mixed up, for example /proc/misc > shows minor number 63 also for vga_arbitrer? Not sure if that matters.The misc device minor numbers are completely dynamic so becoming "mixed up" is normal. The bug is that this version of oxenstored hardcodes a specific minor number, which is wrong. It also uses the device name /dev/xen/eventchn which is different to the /dev/xen/evtchn that the system creates, but that''s a minor aside.> root@EM:/home/yuvraj# cat /proc/misc > < snip > > 59 gntdev > 60 evtchn > < snip > > 63 vga_arbiter > root@EM:/home/yuvraj#[...]> root@EM:/home/yuvraj# strace -f /usr/bin/xenstored --no-fork > > <snip> > open("/dev/xen/eventchn", O_RDWR) = 6So here you open /dev/xen/eventchn, which oxenstored created as 10:63. But, as shown in /proc/misc above, the event channel driver actually ends up with minor 60 on this kernel and so the device you have just opened is the vga_arbiter, whatever that is. Hence...> ioctl(6, EVIOCGVERSION, 0x7fffd0cd3e30) = -1 ENOTTY (Inappropriate ioctl for > device)... this fails, since that ioctl is not valid for the vga_arbiter.> Is there a way I can fix (e.g. a patch I can apply) this on my end?Yes. Apply the patch I linked to in my previous mail (e.g. http://xenbits.xensource.com/xen-unstable.hg?rev/e7b55cc5533a) to the tree under tools/ocaml-xenstored. You may need to adjust the paths in the (hopefully) obvious way. This patch causes oxenstored to use the exiting evtchn device node which is correctly created by udev with the correct minor number. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Seemingly Similar Threads
- Xenstore daemon is not running after boot; although can be started manually later. Any pointers why ?
- Xenstore daemon is not running after boot; although can be started manually later. Any pointers why ?
- /proc/xen/xenbus supports watch?
- [OCAML 0/7] V4 or so of the xen ocaml packaging patches
- Upgrade to xen 4 Error: Device 0 (vif) could not be connected. Hotplug scripts not working.