Jason Vas Dias
2013-Feb-21 17:29 UTC
help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?
Good day -
This is my first post to this list , and I''m new to Xen - any help on
this issue would be much appreciated .
I downloaded, built and installed xen-4.2.1 (hypervisor and tools) on
an x86_64 ArchLinux box
updated to latest software as of today.
I am trying to bring up a Linux guest from a remote iSCSI disk.
The iSCSI-initiator (open-iscsi) logs in to the remote target OK and
makes it available as /dev/sde ,
on which is a bootable linux distribution . containing /boot/vmlinuz* etc.
I can mount this iSCSI disk rw OK on the domain_0 host - so presumably
I should be able to
boot up a guest domain from it ?
I modified the domain_0 host''s boot arguments to boot Xen OK :
multiboot /boot/xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all
module /boot/vmlinuz-linux root=/dev/sda2 ro debug
module /boot/initramfs-linux.img
and made an /etc/xen/${domain}.cfg file:
#
# xl.cfg(1) file for booting the iSCSI share:
#
name="$domain"
#vcpus=1
#uuid="ceb083a2-feaa-45b0-afe2-3bd44eb4d8c3"
memory=1024
on_poweroff="destroy"
on_reboot="destroy"
disk=["/dev/sde,raw,hda,rw"]
vif=["mac=00:09:3d:13:0d:57,model=tg3,bridge=xenbr0"]
kernel="/boot/vmlinuz-2.6.39-300.26.1.el6uek.x86_64"
ramdisk="/boot/initramfs-2.6.39-300.26.1.el6uek.x86_64.img"
Is the disk line OK ? Anything else wrong with this file ?
But attempting to create the domain , as root, fails:
# xl -vvv create -c /etc/xen/${domain}.cfg
Parsing config from ${domain}.cfg
libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x119c890:
create: how=(nil) callback=(nil) poller=0x119c920
libxl: error: libxl_create.c:420:libxl__domain_make: domain creation fail
libxl: error: libxl_create.c:646:initiate_domain_create: cannot make domain: -3
libxl: error: libxl.c:1394:libxl__destroy_domid: non-existant domain -1
libxl: error: libxl.c:1358:domain_destroy_callback: unable to destroy
guest with domid 4294967295
libxl: error: libxl_create.c:1153:domcreate_destruction_cb: unable to
destroy domain 4294967295 following failed creation
libxl: debug: libxl_event.c:1499:libxl__ao_complete: ao 0x119c890:
complete, rc=-3
libxl: debug: libxl_create.c:1187:do_domain_create: ao 0x119c890:
inprogress: poller=0x119c920, flags=ic
libxl: debug: libxl_event.c:1471:libxl__ao__destroy: ao 0x119c890: destroy
xc: debug: hypercall buffer: total allocations:5 total releases:5
xc: debug: hypercall buffer: current allocations:0 maximum allocations:2
xc: debug: hypercall buffer: cache current size:2
xc: debug: hypercall buffer: cache hits:3 misses:2 toobig:0
Even though I''ve enabled all possible log messages in Xen with
''
loglvl=all guest_loglvl=all'' (or have I?)
I don''t see any log messages about this failed attempt to create a
domain at all , in dmesg output or
in the system log.
Tracing this command in gdb shows the hypervisor command is returning
-1, with errno set to 1 (EPERM) :
libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x622890:
create: how=(nil) callback=(nil) poller=0x622920
Breakpoint 2, xc_domain_create (xch=0x622240, ssidref=0,
handle=handle@entry=0x7fffffffe1b0
"\316\260\203\242\376\252E\260\257\342;\324N\264\330\303\031",
flags=flags@entry=0, pdomid=pdomid@entry=0x7fffffffe28c) at xc_domain.c:33
33 {
(gdb) n
37 domctl.cmd = XEN_DOMCTL_createdomain;
(gdb)
38 domctl.domain = (domid_t)*pdomid;
(gdb)
39 domctl.u.createdomain.ssidref = ssidref;
(gdb)
40 domctl.u.createdomain.flags = flags;
(gdb)
41 memcpy(domctl.u.createdomain.handle, handle,
sizeof(xen_domain_handle_t));
(gdb)
42 if ( (err = do_domctl(xch, &domctl)) != 0 )
(gdb) s
do_domctl (domctl=0x7fffffffe0a0, xch=0x622240) at xc_private.h:239
239 DECLARE_HYPERCALL_BOUNCE(domctl, sizeof(*domctl),
XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
(gdb) n
241 domctl->interface_version = XEN_DOMCTL_INTERFACE_VERSION;
(gdb)
243 if ( xc_hypercall_bounce_pre(xch, domctl) )
(gdb)
249 hypercall.op = __HYPERVISOR_domctl;
(gdb)
250 hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(domctl);
(gdb)
252 if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
(gdb) s
do_xen_hypercall (xch=xch@entry=0x622240,
hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:246
246 return xch->ops->u.privcmd.hypercall(xch, xch->ops_handle,
hypercall);
(gdb) s
linux_privcmd_hypercall (xch=<optimized out>, h=9,
hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
116 return ioctl(fd, IOCTL_PRIVCMD_HYPERCALL, hypercall);
(gdb) fin
Run till exit from #0 linux_privcmd_hypercall (xch=<optimized out>,
h=9, hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
do_xen_hypercall (xch=xch@entry=0x622240,
hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
247 }
Value returned is $3 = -1
(gdb) fin
Run till exit from #0 do_xen_hypercall (xch=xch@entry=0x622240,
hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
0x00007ffff775b340 in do_domctl (domctl=0x7fffffffe0a0, xch=0x622240)
at xc_private.h:252
252 if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
Value returned is $4 = -1
(gdb) p errno
$5 = 1
Any ideas how to get this working, or on where I could get better
diagnostics ?
I''m guessing this is caused by this code in
arch/x86/platform_hypercall.c :
ret_t do_platform_op(XEN_GUEST_HANDLE(xen_platform_op_t) u_xenpf_op)
{
ret_t ret = 0;
struct xen_platform_op curop, *op = &curop;
if ( !IS_PRIV(current->domain) )
return -EPERM;
So how do I make the IS_PRIV macro return non-zero for my domain_0 ,
ie. set d->is_privileged /
make my domain0 into a "privileged" domain ? What is meant by
"privileged" in this context ?
There seems to be no documentation on this issue in any manual page
installed by xen''s ''make install'' ,
or anywhere else I could find.
Thanks in advance for any replies,
Regards, Jason
Jason Vas Dias
2013-Feb-21 18:53 UTC
Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?
Aha! Finally I discovered ''xl dmesg'' and saw
''Xen does not allow DomU creation on this CPU for security
reasons.''
o why not give an option to make Xen write to the standard kernel
log, so I could have found this message with ''dmesg'' ?
It is far from obvious from the documentation that this the only
way of seeing Xen log messages .
It appears the dom0 CPU has AMD Errata 121 - see:
http://lists.xen.org/archives/html/xen-announce/2012-06/msg00002.html
But since both my dom0 kernel (3.7.9-1-ARCH) and my domU kernel
(2.6.39-300.26.1.el6uek) apparently have the fix for this
errata , I added ''allow_unsafe'' to the Xen dom0 boot command
line,
and I''m now able to progress beyond the EPERM issue .
It would have been nice if Xen or xl could have emitted some log
message other than ''cannot make domain: -3'', particularly
when it was Xen itself that was refusing to allow the domain creation
to proceed .
It would also have been nice if Xen could have recognized that it was
running kernels with the fix for this issue, and NOT silently
disabled domain creation.
Regards,
Jason
On Thu, Feb 21, 2013 at 5:29 PM, Jason Vas Dias
<jason.vas.dias@gmail.com> wrote:> Good day -
>
> This is my first post to this list , and I''m new to Xen - any help
on
> this issue would be much appreciated .
>
> I downloaded, built and installed xen-4.2.1 (hypervisor and tools) on
> an x86_64 ArchLinux box
> updated to latest software as of today.
>
> I am trying to bring up a Linux guest from a remote iSCSI disk.
>
> The iSCSI-initiator (open-iscsi) logs in to the remote target OK and
> makes it available as /dev/sde ,
> on which is a bootable linux distribution . containing /boot/vmlinuz* etc.
> I can mount this iSCSI disk rw OK on the domain_0 host - so presumably
> I should be able to
> boot up a guest domain from it ?
>
> I modified the domain_0 host''s boot arguments to boot Xen OK :
>
> multiboot /boot/xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all
> module /boot/vmlinuz-linux root=/dev/sda2 ro debug
> module /boot/initramfs-linux.img
>
> and made an /etc/xen/${domain}.cfg file:
> #
> # xl.cfg(1) file for booting the iSCSI share:
> #
> name="$domain"
> #vcpus=1
> #uuid="ceb083a2-feaa-45b0-afe2-3bd44eb4d8c3"
> memory=1024
> on_poweroff="destroy"
> on_reboot="destroy"
> disk=["/dev/sde,raw,hda,rw"]
> vif=["mac=00:09:3d:13:0d:57,model=tg3,bridge=xenbr0"]
> kernel="/boot/vmlinuz-2.6.39-300.26.1.el6uek.x86_64"
> ramdisk="/boot/initramfs-2.6.39-300.26.1.el6uek.x86_64.img"
>
> Is the disk line OK ? Anything else wrong with this file ?
>
> But attempting to create the domain , as root, fails:
> # xl -vvv create -c /etc/xen/${domain}.cfg
> Parsing config from ${domain}.cfg
> libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x119c890:
> create: how=(nil) callback=(nil) poller=0x119c920
> libxl: error: libxl_create.c:420:libxl__domain_make: domain creation fail
> libxl: error: libxl_create.c:646:initiate_domain_create: cannot make
domain: -3
> libxl: error: libxl.c:1394:libxl__destroy_domid: non-existant domain -1
> libxl: error: libxl.c:1358:domain_destroy_callback: unable to destroy
> guest with domid 4294967295
> libxl: error: libxl_create.c:1153:domcreate_destruction_cb: unable to
> destroy domain 4294967295 following failed creation
> libxl: debug: libxl_event.c:1499:libxl__ao_complete: ao 0x119c890:
> complete, rc=-3
> libxl: debug: libxl_create.c:1187:do_domain_create: ao 0x119c890:
> inprogress: poller=0x119c920, flags=ic
> libxl: debug: libxl_event.c:1471:libxl__ao__destroy: ao 0x119c890: destroy
> xc: debug: hypercall buffer: total allocations:5 total releases:5
> xc: debug: hypercall buffer: current allocations:0 maximum allocations:2
> xc: debug: hypercall buffer: cache current size:2
> xc: debug: hypercall buffer: cache hits:3 misses:2 toobig:0
>
> Even though I''ve enabled all possible log messages in Xen with
''
> loglvl=all guest_loglvl=all'' (or have I?)
> I don''t see any log messages about this failed attempt to create a
> domain at all , in dmesg output or
> in the system log.
>
> Tracing this command in gdb shows the hypervisor command is returning
> -1, with errno set to 1 (EPERM) :
>
> libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x622890:
> create: how=(nil) callback=(nil) poller=0x622920
>
> Breakpoint 2, xc_domain_create (xch=0x622240, ssidref=0,
> handle=handle@entry=0x7fffffffe1b0
> "\316\260\203\242\376\252E\260\257\342;\324N\264\330\303\031",
> flags=flags@entry=0, pdomid=pdomid@entry=0x7fffffffe28c) at
xc_domain.c:33
> 33 {
> (gdb) n
> 37 domctl.cmd = XEN_DOMCTL_createdomain;
> (gdb)
> 38 domctl.domain = (domid_t)*pdomid;
> (gdb)
> 39 domctl.u.createdomain.ssidref = ssidref;
> (gdb)
> 40 domctl.u.createdomain.flags = flags;
> (gdb)
> 41 memcpy(domctl.u.createdomain.handle, handle,
> sizeof(xen_domain_handle_t));
> (gdb)
> 42 if ( (err = do_domctl(xch, &domctl)) != 0 )
> (gdb) s
> do_domctl (domctl=0x7fffffffe0a0, xch=0x622240) at xc_private.h:239
> 239 DECLARE_HYPERCALL_BOUNCE(domctl, sizeof(*domctl),
> XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
> (gdb) n
> 241 domctl->interface_version = XEN_DOMCTL_INTERFACE_VERSION;
> (gdb)
> 243 if ( xc_hypercall_bounce_pre(xch, domctl) )
> (gdb)
> 249 hypercall.op = __HYPERVISOR_domctl;
> (gdb)
> 250 hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(domctl);
> (gdb)
> 252 if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
> (gdb) s
> do_xen_hypercall (xch=xch@entry=0x622240,
> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:246
> 246 return xch->ops->u.privcmd.hypercall(xch,
xch->ops_handle,
> hypercall);
> (gdb) s
> linux_privcmd_hypercall (xch=<optimized out>, h=9,
> hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
> 116 return ioctl(fd, IOCTL_PRIVCMD_HYPERCALL, hypercall);
> (gdb) fin
> Run till exit from #0 linux_privcmd_hypercall (xch=<optimized out>,
> h=9, hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
> do_xen_hypercall (xch=xch@entry=0x622240,
> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
> 247 }
> Value returned is $3 = -1
> (gdb) fin
> Run till exit from #0 do_xen_hypercall (xch=xch@entry=0x622240,
> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
> 0x00007ffff775b340 in do_domctl (domctl=0x7fffffffe0a0, xch=0x622240)
> at xc_private.h:252
> 252 if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
> Value returned is $4 = -1
> (gdb) p errno
> $5 = 1
>
> Any ideas how to get this working, or on where I could get better
> diagnostics ?
>
> I''m guessing this is caused by this code in
arch/x86/platform_hypercall.c :
>
> ret_t do_platform_op(XEN_GUEST_HANDLE(xen_platform_op_t) u_xenpf_op)
> {
> ret_t ret = 0;
> struct xen_platform_op curop, *op = &curop;
>
> if ( !IS_PRIV(current->domain) )
> return -EPERM;
>
> So how do I make the IS_PRIV macro return non-zero for my domain_0 ,
> ie. set d->is_privileged /
> make my domain0 into a "privileged" domain ? What is meant by
> "privileged" in this context ?
>
> There seems to be no documentation on this issue in any manual page
> installed by xen''s ''make install'' ,
> or anywhere else I could find.
>
> Thanks in advance for any replies,
> Regards, Jason
Jason Vas Dias
2013-Feb-21 19:24 UTC
Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?
I''m beginning to conclude that Xen is not of sufficient code quality to be useful. Firstly, my ''xl create'' command returns a 0 exit status, meaning success, but yet no domain is created or can be attached to or listed with ''xl list'' : # xl create my_domain.cfg Parsing config from my_domain.cfg Daemon running with PID 1436 # echo $? 0 # xl list Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 35.5 This appears to be due to this error message in ''xl dmesg'' output : (XEN) Dom5 may compromise security on this CPU. (XEN) traps.c:2584:d5 Domain attempted WRMSR 00000000c0010004 from 0x0000000000000000 to 0x000000000000abcd. From reading the source, the traps.c message appears to indicate an error condition, but you would not know this from the log message. Anyone know how to avoid this unhandled WRMSR ? Code that silently ignores and does not report error conditions does not get my vote of confidence. Can anyone convince me that Xen is of sufficient quality to be used ? Regards, Jason On Thu, Feb 21, 2013 at 6:53 PM, Jason Vas Dias <jason.vas.dias@gmail.com> wrote:> Aha! Finally I discovered ''xl dmesg'' and saw > ''Xen does not allow DomU creation on this CPU for security reasons.'' > > o why not give an option to make Xen write to the standard kernel > log, so I could have found this message with ''dmesg'' ? > It is far from obvious from the documentation that this the only > way of seeing Xen log messages . > > It appears the dom0 CPU has AMD Errata 121 - see: > http://lists.xen.org/archives/html/xen-announce/2012-06/msg00002.html > > But since both my dom0 kernel (3.7.9-1-ARCH) and my domU kernel > (2.6.39-300.26.1.el6uek) apparently have the fix for this > errata , I added ''allow_unsafe'' to the Xen dom0 boot command line, > and I''m now able to progress beyond the EPERM issue . > It would have been nice if Xen or xl could have emitted some log > message other than ''cannot make domain: -3'', particularly > when it was Xen itself that was refusing to allow the domain creation > to proceed . > It would also have been nice if Xen could have recognized that it was > running kernels with the fix for this issue, and NOT silently > disabled domain creation. > > Regards, > Jason > > On Thu, Feb 21, 2013 at 5:29 PM, Jason Vas Dias > <jason.vas.dias@gmail.com> wrote: >> Good day - >> >> This is my first post to this list , and I''m new to Xen - any help on >> this issue would be much appreciated . >> >> I downloaded, built and installed xen-4.2.1 (hypervisor and tools) on >> an x86_64 ArchLinux box >> updated to latest software as of today. >> >> I am trying to bring up a Linux guest from a remote iSCSI disk. >> >> The iSCSI-initiator (open-iscsi) logs in to the remote target OK and >> makes it available as /dev/sde , >> on which is a bootable linux distribution . containing /boot/vmlinuz* etc. >> I can mount this iSCSI disk rw OK on the domain_0 host - so presumably >> I should be able to >> boot up a guest domain from it ? >> >> I modified the domain_0 host''s boot arguments to boot Xen OK : >> >> multiboot /boot/xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all >> module /boot/vmlinuz-linux root=/dev/sda2 ro debug >> module /boot/initramfs-linux.img >> >> and made an /etc/xen/${domain}.cfg file: >> # >> # xl.cfg(1) file for booting the iSCSI share: >> # >> name="$domain" >> #vcpus=1 >> #uuid="ceb083a2-feaa-45b0-afe2-3bd44eb4d8c3" >> memory=1024 >> on_poweroff="destroy" >> on_reboot="destroy" >> disk=["/dev/sde,raw,hda,rw"] >> vif=["mac=00:09:3d:13:0d:57,model=tg3,bridge=xenbr0"] >> kernel="/boot/vmlinuz-2.6.39-300.26.1.el6uek.x86_64" >> ramdisk="/boot/initramfs-2.6.39-300.26.1.el6uek.x86_64.img" >> >> Is the disk line OK ? Anything else wrong with this file ? >> >> But attempting to create the domain , as root, fails: >> # xl -vvv create -c /etc/xen/${domain}.cfg >> Parsing config from ${domain}.cfg >> libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x119c890: >> create: how=(nil) callback=(nil) poller=0x119c920 >> libxl: error: libxl_create.c:420:libxl__domain_make: domain creation fail >> libxl: error: libxl_create.c:646:initiate_domain_create: cannot make domain: -3 >> libxl: error: libxl.c:1394:libxl__destroy_domid: non-existant domain -1 >> libxl: error: libxl.c:1358:domain_destroy_callback: unable to destroy >> guest with domid 4294967295 >> libxl: error: libxl_create.c:1153:domcreate_destruction_cb: unable to >> destroy domain 4294967295 following failed creation >> libxl: debug: libxl_event.c:1499:libxl__ao_complete: ao 0x119c890: >> complete, rc=-3 >> libxl: debug: libxl_create.c:1187:do_domain_create: ao 0x119c890: >> inprogress: poller=0x119c920, flags=ic >> libxl: debug: libxl_event.c:1471:libxl__ao__destroy: ao 0x119c890: destroy >> xc: debug: hypercall buffer: total allocations:5 total releases:5 >> xc: debug: hypercall buffer: current allocations:0 maximum allocations:2 >> xc: debug: hypercall buffer: cache current size:2 >> xc: debug: hypercall buffer: cache hits:3 misses:2 toobig:0 >> >> Even though I''ve enabled all possible log messages in Xen with '' >> loglvl=all guest_loglvl=all'' (or have I?) >> I don''t see any log messages about this failed attempt to create a >> domain at all , in dmesg output or >> in the system log. >> >> Tracing this command in gdb shows the hypervisor command is returning >> -1, with errno set to 1 (EPERM) : >> >> libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x622890: >> create: how=(nil) callback=(nil) poller=0x622920 >> >> Breakpoint 2, xc_domain_create (xch=0x622240, ssidref=0, >> handle=handle@entry=0x7fffffffe1b0 >> "\316\260\203\242\376\252E\260\257\342;\324N\264\330\303\031", >> flags=flags@entry=0, pdomid=pdomid@entry=0x7fffffffe28c) at xc_domain.c:33 >> 33 { >> (gdb) n >> 37 domctl.cmd = XEN_DOMCTL_createdomain; >> (gdb) >> 38 domctl.domain = (domid_t)*pdomid; >> (gdb) >> 39 domctl.u.createdomain.ssidref = ssidref; >> (gdb) >> 40 domctl.u.createdomain.flags = flags; >> (gdb) >> 41 memcpy(domctl.u.createdomain.handle, handle, >> sizeof(xen_domain_handle_t)); >> (gdb) >> 42 if ( (err = do_domctl(xch, &domctl)) != 0 ) >> (gdb) s >> do_domctl (domctl=0x7fffffffe0a0, xch=0x622240) at xc_private.h:239 >> 239 DECLARE_HYPERCALL_BOUNCE(domctl, sizeof(*domctl), >> XC_HYPERCALL_BUFFER_BOUNCE_BOTH); >> (gdb) n >> 241 domctl->interface_version = XEN_DOMCTL_INTERFACE_VERSION; >> (gdb) >> 243 if ( xc_hypercall_bounce_pre(xch, domctl) ) >> (gdb) >> 249 hypercall.op = __HYPERVISOR_domctl; >> (gdb) >> 250 hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(domctl); >> (gdb) >> 252 if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 ) >> (gdb) s >> do_xen_hypercall (xch=xch@entry=0x622240, >> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:246 >> 246 return xch->ops->u.privcmd.hypercall(xch, xch->ops_handle, >> hypercall); >> (gdb) s >> linux_privcmd_hypercall (xch=<optimized out>, h=9, >> hypercall=0x7fffffffe070) at xc_linux_osdep.c:116 >> 116 return ioctl(fd, IOCTL_PRIVCMD_HYPERCALL, hypercall); >> (gdb) fin >> Run till exit from #0 linux_privcmd_hypercall (xch=<optimized out>, >> h=9, hypercall=0x7fffffffe070) at xc_linux_osdep.c:116 >> do_xen_hypercall (xch=xch@entry=0x622240, >> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247 >> 247 } >> Value returned is $3 = -1 >> (gdb) fin >> Run till exit from #0 do_xen_hypercall (xch=xch@entry=0x622240, >> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247 >> 0x00007ffff775b340 in do_domctl (domctl=0x7fffffffe0a0, xch=0x622240) >> at xc_private.h:252 >> 252 if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 ) >> Value returned is $4 = -1 >> (gdb) p errno >> $5 = 1 >> >> Any ideas how to get this working, or on where I could get better >> diagnostics ? >> >> I''m guessing this is caused by this code in arch/x86/platform_hypercall.c : >> >> ret_t do_platform_op(XEN_GUEST_HANDLE(xen_platform_op_t) u_xenpf_op) >> { >> ret_t ret = 0; >> struct xen_platform_op curop, *op = &curop; >> >> if ( !IS_PRIV(current->domain) ) >> return -EPERM; >> >> So how do I make the IS_PRIV macro return non-zero for my domain_0 , >> ie. set d->is_privileged / >> make my domain0 into a "privileged" domain ? What is meant by >> "privileged" in this context ? >> >> There seems to be no documentation on this issue in any manual page >> installed by xen''s ''make install'' , >> or anywhere else I could find. >> >> Thanks in advance for any replies, >> Regards, Jason
Casey DeLorme
2013-Feb-21 21:30 UTC
Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?
Hello Jason, Looking at your configuration, was the kernel built to be PV Compatible ( http://sanifool.wordpress.com/2013/01/09/booting-up-a-domu-on-xen-4-2/). The disk line looks fine provided you wanted to pass the whole disk to your PVM (including it''s partition table etc). Also if you have it mounted while trying to launch the PVM that may throw warnings or errors. Is it possible you are missing the bootloader ( http://xenbits.xen.org/docs/4.2-testing/man/xl.cfg.5.html)? I don''t believe it would affect your ability to launch the PVM, but the I am unfamiliar with your vif device model, and the mac address isn''t in the suggested Xen range ( http://xenbits.xen.org/docs/4.2-testing/misc/xl-network-configuration.html). If you are looking for more detailed diagnostics, you might check that the xen-debug kernel flag is set (and rebuild your ramdisk or kernel). Logging must vary by distribution as I almost never see non-kernel software log its errors to dmesg (/var/log/messages), but most will place logs into the same directory and organize them by file or folder. In Xen''s case it uses your toolstacks command xl dmesg (/var/logs/xen/). --- On Thu, Feb 21, 2013 at 2:24 PM, Jason Vas Dias <jason.vas.dias@gmail.com>wrote:> I''m beginning to conclude that Xen is not of sufficient code quality > to be useful. > > Firstly, my ''xl create'' command returns a 0 exit status, meaning > success, but yet no domain > is created or can be attached to or listed with ''xl list'' : > > # xl create my_domain.cfg > Parsing config from my_domain.cfg > Daemon running with PID 1436 > # echo $? > 0 > # xl list > Name ID Mem VCPUs State > Time(s) > Domain-0 0 1024 2 r----- > 35.5 > > This appears to be due to this error message in ''xl dmesg'' output : > > (XEN) Dom5 may compromise security on this CPU. > (XEN) traps.c:2584:d5 Domain attempted WRMSR 00000000c0010004 from > 0x0000000000000000 to 0x000000000000abcd. > > From reading the source, the traps.c message appears to indicate an > error condition, but you would not know this from the log message. > > Have you checked all of the log files in /var/logs/xen/?Pretty sure that so long as the configuration itself doesn''t contain syntactical errors and the command to qemu goes through that xl will assume it did its job. Is it possible xl create is launching the domains just fine (hence code 0), and they are failing at a different stage (eg qemu during boot due to configuration problems)? Anyone know how to avoid this unhandled WRMSR ?> > Code that silently ignores and does not report error conditions does > not get my vote of confidence. > > Can anyone convince me that Xen is of sufficient quality to be used ? > > Plenty of well known companies are using Xen, but I don''t believe they areusing the very latest stable release (4.2.1). AFAIK 4.2.1 is not available as a package in any stable operating systems yet. Stable release of code does not guarantee it will be compatible without plenty of configuration based on your choice of Dom0. However I am under the assumption that you are not using it for production purposes given your choices (Xen 4.2.1 and ArchLinux don''t exactly point to concerns with stability or desirable ease of operation as a priority). If that is the case then as a personal user I can say that I have had great success using Xen 4.2 on Debain Wheezy for several months now. However I have not dabbled in many of the areas you are (such as iSCSI, PVM''s and programmatic domain control). However, even with my limited experience, I can easily conclude that because Xen can be used for such a wide variety of purposes it becomes that much harder to ensure functionality as factors that can influence success tend to cover a wide range (host hardware, Xen version, installed packages and package versions, Dom0 distro and version, etc). Hope these answers will help you identify the problem. ~Casey _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Jason Vas Dias
2013-Feb-22 19:12 UTC
Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?
Hi Casey - many thanks for your informative response ! - I''ve now got it working , with a number of caveats / gotchas, detailed below: On Thu, Feb 21, 2013 at 9:30 PM, Casey DeLorme <cdelorme@gmail.com> wrote:> > Hello Jason, > > Looking at your configuration, was the kernel built to be PV Compatible > (http://sanifool.wordpress.com/2013/01/09/booting-up-a-domu-on-xen-4-2/). >Yes, the configuration arguments of the latest ArchLinux kernel ( 3.7.9-1-ARCH) check out for Xen .> The disk line looks fine provided you wanted to pass the whole disk to your > PVM (including it''s partition table etc). Also if you have it mounted while > trying to launch the PVM that may throw warnings or errors. > > Is it possible you are missing the bootloader > (http://xenbits.xen.org/docs/4.2-testing/man/xl.cfg.5.html)? >Yes, that was confusing - I first gave xl just ''kernel="/boot/vmlinuz..." , but then xl complained it couldn''t find the kernel, so I mounted the iSCSI share and gave xl the full path to the kernel ; but then I saw your post and specified the "bootloader="pygrub"'' setting, which did not work until the fix for this bug is built into libxl : Bug 1844 - xl coredumps whenever any "bootloader=" setting is used in an xl.cfg file Having fixed that, then I came up against a number of disk issues - No setting for the disk ''vdev'' field seemed to have any effect - ie. sda , sda6, hda, sdc (the real device name) - the guest OS could not see that device. Happily, with the dracut ''rdshell'' argument supported by the guest OS, I was able to get it into initramfs debug mode, and saw the disk on /dev/xvdc - this appears to be the only usable setting for the root= kernel argument - it is now booting from this. Perhaps this is just because I''m using the default initramfs file and did not build one with the extra devices ?> I don''t believe it would affect your ability to launch the PVM, but the I am > unfamiliar with your vif device model, and the mac address isn''t in the > suggested Xen range > (http://xenbits.xen.org/docs/4.2-testing/misc/xl-network-configuration.html). >Yes, I should have read that document more closely - the setting that works is vif=["bridge=eth0"] Then, with the default ''peth0'' bridge created by default on ArchLinux, I get : # brctl show bridge name bridge id STP enabled interfaces eth0 8000.00093d130d57 no peth0 vif22.0> > If you are looking for more detailed diagnostics, you might check that the > xen-debug kernel flag is set (and rebuild your ramdisk or kernel). > > > Logging must vary by distribution as I almost never see non-kernel software > log its errors to dmesg (/var/log/messages), but most will place logs into > the same directory and organize them by file or folder. In Xen''s case it > uses your toolstacks command xl dmesg (/var/logs/xen/). >Yes, but my point was that xl should not silently fail because of the errata-121 issue without telling the user - perhaps, if xl finds domain creation failed, it could automatically do its dmesg stuff and dump the Xen log messages to stderr ? Also, in many cases when domain creation failed because of the above issues, xl returned a 0 exit status - it really should be returning >0 in these cases.> --- > > > On Thu, Feb 21, 2013 at 2:24 PM, Jason Vas Dias <jason.vas.dias@gmail.com> > wrote: >> >> I''m beginning to conclude that Xen is not of sufficient code quality >> to be useful. >> >> Firstly, my ''xl create'' command returns a 0 exit status, meaning >> success, but yet no domain >> is created or can be attached to or listed with ''xl list'' : >> >> # xl create my_domain.cfg >> Parsing config from my_domain.cfg >> Daemon running with PID 1436 >> # echo $? >> 0 >> # xl list >> Name ID Mem VCPUs State >> Time(s) >> Domain-0 0 1024 2 r----- >> 35.5 >> >> This appears to be due to this error message in ''xl dmesg'' output : >> >> (XEN) Dom5 may compromise security on this CPU. >> (XEN) traps.c:2584:d5 Domain attempted WRMSR 00000000c0010004 from >> 0x0000000000000000 to 0x000000000000abcd. >> >> From reading the source, the traps.c message appears to indicate an >> error condition, but you would not know this from the log message. >> > Have you checked all of the log files in /var/logs/xen/? > > Pretty sure that so long as the configuration itself doesn''t contain > syntactical errors and the command to qemu goes through that xl will assume > it did its job. > > Is it possible xl create is launching the domains just fine (hence code 0), > and they are failing at a different stage (eg qemu during boot due to > configuration problems)?But then xl should not be returning ''0'' .> >> Anyone know how to avoid this unhandled WRMSR ? >> >> Code that silently ignores and does not report error conditions does >> not get my vote of confidence. >> >> Can anyone convince me that Xen is of sufficient quality to be used ? >> > Plenty of well known companies are using Xen, but I don''t believe they are > using the very latest stable release (4.2.1). AFAIK 4.2.1 is not available > as a package in any stable operating systems yet. Stable release of code > does not guarantee it will be compatible without plenty of configuration > based on your choice of Dom0. >OK - I prefer to use latest available versions of everything so at least I know all known bugs are fixed, and I can help find new bugs.> However I am under the assumption that you are not using it for production > purposes given your choices (Xen 4.2.1 and ArchLinux don''t exactly point to > concerns with stability or desirable ease of operation as a priority). If > that is the case then as a personal user I can say that I have had great > success using Xen 4.2 on Debain Wheezy for several months now. However I > have not dabbled in many of the areas you are (such as iSCSI, PVM''s and > programmatic domain control). > > However, even with my limited experience, I can easily conclude that because > Xen can be used for such a wide variety of purposes it becomes that much > harder to ensure functionality as factors that can influence success tend to > cover a wide range (host hardware, Xen version, installed packages and > package versions, Dom0 distro and version, etc). > > > Hope these answers will help you identify the problem. > > ~CaseyI''m trying to use Xen for our testing server - in a "semi-production" / backend environment - where we produce software that claims to run on @30 different OS-es, and we don''t have 30 physical servers, I''m setting up one server to be able to boot each different supported OS with Xen, for testing purposes. Now at least I''ve got one working ... only 30 more to go ! Thank You & Regards, Jason
Reasonably Related Threads
- [PATCH 0 of 2] v2: memshare/xenpaging/xen-access fixes for xen-unstable
- [PATCH V3] libxl: write IO ABI for disk frontends
- [PATCH 0 of 2] Fix correctness race in xc_mem_paging_prep
- Xen 4.1 rc1 test report
- [PATCH 3 of 4] PoC: libxc+ocaml: add interfaces to send triggers to HVM guests