thr3ads.net - Xen users - help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ? [Feb 2013]

If this information is useful, please help other people find it:
Share via:

Jason Vas Dias

2013-Feb-21 17:29 UTC

help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Good day -

This is my first post to this list , and I''m new to Xen - any help on
this issue would be much appreciated .

I downloaded, built and installed xen-4.2.1 (hypervisor and tools) on
an x86_64 ArchLinux box
updated to latest software as of today.

I am trying to bring up a Linux guest from a remote iSCSI disk.

The iSCSI-initiator (open-iscsi)  logs in to the remote target OK and
makes it available as /dev/sde ,
on which is a bootable linux distribution . containing /boot/vmlinuz* etc.
I can mount this iSCSI disk rw OK on the domain_0 host - so presumably
I should be able to
boot up a guest domain from it ?

I modified the domain_0 host''s boot arguments to boot Xen OK :

        multiboot /boot/xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all
        module  /boot/vmlinuz-linux root=/dev/sda2 ro  debug
        module  /boot/initramfs-linux.img

and made an /etc/xen/${domain}.cfg file:
#
# xl.cfg(1) file for booting the iSCSI share:
#
name="$domain"
#vcpus=1
#uuid="ceb083a2-feaa-45b0-afe2-3bd44eb4d8c3"
memory=1024
on_poweroff="destroy"
on_reboot="destroy"
disk=["/dev/sde,raw,hda,rw"]
vif=["mac=00:09:3d:13:0d:57,model=tg3,bridge=xenbr0"]
kernel="/boot/vmlinuz-2.6.39-300.26.1.el6uek.x86_64"
ramdisk="/boot/initramfs-2.6.39-300.26.1.el6uek.x86_64.img"

Is the disk line OK ?  Anything else wrong with this file ?

But attempting to create the domain , as root, fails:
# xl -vvv create -c /etc/xen/${domain}.cfg
Parsing config from ${domain}.cfg
libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x119c890:
create: how=(nil) callback=(nil) poller=0x119c920
libxl: error: libxl_create.c:420:libxl__domain_make: domain creation fail
libxl: error: libxl_create.c:646:initiate_domain_create: cannot make domain: -3
libxl: error: libxl.c:1394:libxl__destroy_domid: non-existant domain -1
libxl: error: libxl.c:1358:domain_destroy_callback: unable to destroy
guest with domid 4294967295
libxl: error: libxl_create.c:1153:domcreate_destruction_cb: unable to
destroy domain 4294967295 following failed creation
libxl: debug: libxl_event.c:1499:libxl__ao_complete: ao 0x119c890:
complete, rc=-3
libxl: debug: libxl_create.c:1187:do_domain_create: ao 0x119c890:
inprogress: poller=0x119c920, flags=ic
libxl: debug: libxl_event.c:1471:libxl__ao__destroy: ao 0x119c890: destroy
xc: debug: hypercall buffer: total allocations:5 total releases:5
xc: debug: hypercall buffer: current allocations:0 maximum allocations:2
xc: debug: hypercall buffer: cache current size:2
xc: debug: hypercall buffer: cache hits:3 misses:2 toobig:0

Even though I''ve enabled all possible log messages in Xen with
''
loglvl=all guest_loglvl=all'' (or have I?)
I don''t see any log messages about this failed attempt to create a
domain at all , in dmesg output or
in the system log.

Tracing this command in gdb shows the hypervisor command is returning
-1, with errno set to 1 (EPERM) :

libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x622890:
create: how=(nil) callback=(nil) poller=0x622920

Breakpoint 2, xc_domain_create (xch=0x622240, ssidref=0,
handle=handle@entry=0x7fffffffe1b0
"\316\260\203\242\376\252E\260\257\342;\324N\264\330\303\031",
    flags=flags@entry=0, pdomid=pdomid@entry=0x7fffffffe28c) at xc_domain.c:33
33      {
(gdb) n
37          domctl.cmd = XEN_DOMCTL_createdomain;
(gdb)
38          domctl.domain = (domid_t)*pdomid;
(gdb)
39          domctl.u.createdomain.ssidref = ssidref;
(gdb)
40          domctl.u.createdomain.flags   = flags;
(gdb)
41          memcpy(domctl.u.createdomain.handle, handle,
sizeof(xen_domain_handle_t));
(gdb)
42          if ( (err = do_domctl(xch, &domctl)) != 0 )
(gdb) s
do_domctl (domctl=0x7fffffffe0a0, xch=0x622240) at xc_private.h:239
239         DECLARE_HYPERCALL_BOUNCE(domctl, sizeof(*domctl),
XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
(gdb) n
241         domctl->interface_version = XEN_DOMCTL_INTERFACE_VERSION;
(gdb)
243         if ( xc_hypercall_bounce_pre(xch, domctl) )
(gdb)
249         hypercall.op     = __HYPERVISOR_domctl;
(gdb)
250         hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(domctl);
(gdb)
252         if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
(gdb) s
do_xen_hypercall (xch=xch@entry=0x622240,
hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:246
246         return xch->ops->u.privcmd.hypercall(xch, xch->ops_handle,
hypercall);
(gdb) s
linux_privcmd_hypercall (xch=<optimized out>, h=9,
hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
116         return ioctl(fd, IOCTL_PRIVCMD_HYPERCALL, hypercall);
(gdb) fin
Run till exit from #0  linux_privcmd_hypercall (xch=<optimized out>,
h=9, hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
do_xen_hypercall (xch=xch@entry=0x622240,
hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
247     }
Value returned is $3 = -1
(gdb) fin
Run till exit from #0  do_xen_hypercall (xch=xch@entry=0x622240,
hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
0x00007ffff775b340 in do_domctl (domctl=0x7fffffffe0a0, xch=0x622240)
at xc_private.h:252
252         if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
Value returned is $4 = -1
(gdb) p errno
$5 = 1

Any ideas how to get this working,  or on where I could get better
diagnostics ?

I''m guessing this is caused by this code in 
arch/x86/platform_hypercall.c :

ret_t do_platform_op(XEN_GUEST_HANDLE(xen_platform_op_t) u_xenpf_op)
{
    ret_t ret = 0;
    struct xen_platform_op curop, *op = &curop;

    if ( !IS_PRIV(current->domain) )
        return -EPERM;

So how do I make the IS_PRIV macro return non-zero for my domain_0 ,
ie. set d->is_privileged /
make my domain0 into a "privileged" domain ? What is meant by
"privileged" in this context ?

There seems to be no documentation on this issue in any manual page
installed by xen''s ''make install'' ,
or anywhere else I could find.

Thanks in advance for any replies,
Regards, Jason

Jason Vas Dias

2013-Feb-21 18:53 UTC

head link

Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Aha! Finally I discovered ''xl dmesg'' and saw
     ''Xen does not allow DomU creation on this CPU for security
reasons.''

   o why not give an option to make Xen write to the standard kernel
log, so I could have found this message with ''dmesg'' ?
      It is far from obvious from the documentation that this the only
way of seeing Xen log messages .

It appears the dom0 CPU has AMD Errata 121 - see:
   http://lists.xen.org/archives/html/xen-announce/2012-06/msg00002.html

But since both my dom0 kernel (3.7.9-1-ARCH) and my domU kernel
(2.6.39-300.26.1.el6uek) apparently have the fix for this
errata ,  I added ''allow_unsafe'' to the Xen dom0 boot command
line,
and I''m now able to progress beyond the EPERM issue .
It would have been nice if Xen or xl could have emitted some log
message other than ''cannot make domain: -3'', particularly
when it was Xen itself that was refusing to allow the domain creation
to proceed .
It would also have been nice if Xen could have recognized that it was
running kernels with the fix for this issue, and NOT silently
disabled domain creation.

Regards,
Jason

On Thu, Feb 21, 2013 at 5:29 PM, Jason Vas Dias
<jason.vas.dias@gmail.com> wrote:> Good day -
>
> This is my first post to this list , and I''m new to Xen - any help
on
> this issue would be much appreciated .
>
> I downloaded, built and installed xen-4.2.1 (hypervisor and tools) on
> an x86_64 ArchLinux box
> updated to latest software as of today.
>
> I am trying to bring up a Linux guest from a remote iSCSI disk.
>
> The iSCSI-initiator (open-iscsi)  logs in to the remote target OK and
> makes it available as /dev/sde ,
> on which is a bootable linux distribution . containing /boot/vmlinuz* etc.
> I can mount this iSCSI disk rw OK on the domain_0 host - so presumably
> I should be able to
> boot up a guest domain from it ?
>
> I modified the domain_0 host''s boot arguments to boot Xen OK :
>
>         multiboot /boot/xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all
>         module  /boot/vmlinuz-linux root=/dev/sda2 ro  debug
>         module  /boot/initramfs-linux.img
>
> and made an /etc/xen/${domain}.cfg file:
> #
> # xl.cfg(1) file for booting the iSCSI share:
> #
> name="$domain"
> #vcpus=1
> #uuid="ceb083a2-feaa-45b0-afe2-3bd44eb4d8c3"
> memory=1024
> on_poweroff="destroy"
> on_reboot="destroy"
> disk=["/dev/sde,raw,hda,rw"]
> vif=["mac=00:09:3d:13:0d:57,model=tg3,bridge=xenbr0"]
> kernel="/boot/vmlinuz-2.6.39-300.26.1.el6uek.x86_64"
> ramdisk="/boot/initramfs-2.6.39-300.26.1.el6uek.x86_64.img"
>
> Is the disk line OK ?  Anything else wrong with this file ?
>
> But attempting to create the domain , as root, fails:
> # xl -vvv create -c /etc/xen/${domain}.cfg
> Parsing config from ${domain}.cfg
> libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x119c890:
> create: how=(nil) callback=(nil) poller=0x119c920
> libxl: error: libxl_create.c:420:libxl__domain_make: domain creation fail
> libxl: error: libxl_create.c:646:initiate_domain_create: cannot make
domain: -3
> libxl: error: libxl.c:1394:libxl__destroy_domid: non-existant domain -1
> libxl: error: libxl.c:1358:domain_destroy_callback: unable to destroy
> guest with domid 4294967295
> libxl: error: libxl_create.c:1153:domcreate_destruction_cb: unable to
> destroy domain 4294967295 following failed creation
> libxl: debug: libxl_event.c:1499:libxl__ao_complete: ao 0x119c890:
> complete, rc=-3
> libxl: debug: libxl_create.c:1187:do_domain_create: ao 0x119c890:
> inprogress: poller=0x119c920, flags=ic
> libxl: debug: libxl_event.c:1471:libxl__ao__destroy: ao 0x119c890: destroy
> xc: debug: hypercall buffer: total allocations:5 total releases:5
> xc: debug: hypercall buffer: current allocations:0 maximum allocations:2
> xc: debug: hypercall buffer: cache current size:2
> xc: debug: hypercall buffer: cache hits:3 misses:2 toobig:0
>
> Even though I''ve enabled all possible log messages in Xen with
''
> loglvl=all guest_loglvl=all'' (or have I?)
> I don''t see any log messages about this failed attempt to create a
> domain at all , in dmesg output or
> in the system log.
>
> Tracing this command in gdb shows the hypervisor command is returning
> -1, with errno set to 1 (EPERM) :
>
> libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x622890:
> create: how=(nil) callback=(nil) poller=0x622920
>
> Breakpoint 2, xc_domain_create (xch=0x622240, ssidref=0,
> handle=handle@entry=0x7fffffffe1b0
> "\316\260\203\242\376\252E\260\257\342;\324N\264\330\303\031",
>     flags=flags@entry=0, pdomid=pdomid@entry=0x7fffffffe28c) at
xc_domain.c:33
> 33      {
> (gdb) n
> 37          domctl.cmd = XEN_DOMCTL_createdomain;
> (gdb)
> 38          domctl.domain = (domid_t)*pdomid;
> (gdb)
> 39          domctl.u.createdomain.ssidref = ssidref;
> (gdb)
> 40          domctl.u.createdomain.flags   = flags;
> (gdb)
> 41          memcpy(domctl.u.createdomain.handle, handle,
> sizeof(xen_domain_handle_t));
> (gdb)
> 42          if ( (err = do_domctl(xch, &domctl)) != 0 )
> (gdb) s
> do_domctl (domctl=0x7fffffffe0a0, xch=0x622240) at xc_private.h:239
> 239         DECLARE_HYPERCALL_BOUNCE(domctl, sizeof(*domctl),
> XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
> (gdb) n
> 241         domctl->interface_version = XEN_DOMCTL_INTERFACE_VERSION;
> (gdb)
> 243         if ( xc_hypercall_bounce_pre(xch, domctl) )
> (gdb)
> 249         hypercall.op     = __HYPERVISOR_domctl;
> (gdb)
> 250         hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(domctl);
> (gdb)
> 252         if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
> (gdb) s
> do_xen_hypercall (xch=xch@entry=0x622240,
> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:246
> 246         return xch->ops->u.privcmd.hypercall(xch,
xch->ops_handle,
> hypercall);
> (gdb) s
> linux_privcmd_hypercall (xch=<optimized out>, h=9,
> hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
> 116         return ioctl(fd, IOCTL_PRIVCMD_HYPERCALL, hypercall);
> (gdb) fin
> Run till exit from #0  linux_privcmd_hypercall (xch=<optimized out>,
> h=9, hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
> do_xen_hypercall (xch=xch@entry=0x622240,
> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
> 247     }
> Value returned is $3 = -1
> (gdb) fin
> Run till exit from #0  do_xen_hypercall (xch=xch@entry=0x622240,
> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
> 0x00007ffff775b340 in do_domctl (domctl=0x7fffffffe0a0, xch=0x622240)
> at xc_private.h:252
> 252         if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
> Value returned is $4 = -1
> (gdb) p errno
> $5 = 1
>
> Any ideas how to get this working,  or on where I could get better
> diagnostics ?
>
> I''m guessing this is caused by this code in 
arch/x86/platform_hypercall.c :
>
> ret_t do_platform_op(XEN_GUEST_HANDLE(xen_platform_op_t) u_xenpf_op)
> {
>     ret_t ret = 0;
>     struct xen_platform_op curop, *op = &curop;
>
>     if ( !IS_PRIV(current->domain) )
>         return -EPERM;
>
> So how do I make the IS_PRIV macro return non-zero for my domain_0 ,
> ie. set d->is_privileged /
> make my domain0 into a "privileged" domain ? What is meant by
> "privileged" in this context ?
>
> There seems to be no documentation on this issue in any manual page
> installed by xen''s ''make install'' ,
> or anywhere else I could find.
>
> Thanks in advance for any replies,
> Regards, Jason

Jason Vas Dias

2013-Feb-21 19:24 UTC

head link

Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

I''m beginning to conclude that Xen is not of sufficient code quality
to be useful.

Firstly,  my ''xl create'' command returns a 0 exit status,
meaning
success, but yet no domain
is created or can be attached to or listed with ''xl list'' :

# xl create my_domain.cfg
Parsing config from my_domain.cfg
Daemon running with PID 1436
# echo $?
0
# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  1024     2     r-----      35.5

This appears to be due to this error message in ''xl dmesg''
output :

(XEN) Dom5 may compromise security on this CPU.
(XEN) traps.c:2584:d5 Domain attempted WRMSR 00000000c0010004 from
0x0000000000000000 to 0x000000000000abcd.

From reading the source, the traps.c message appears to indicate an
error condition, but you would not know this from the log message.

Anyone know how to avoid this unhandled WRMSR ?

Code that silently ignores and does not report error conditions does
not get my vote of confidence.

Can anyone convince me that Xen is of sufficient quality to be used ?

Regards,
Jason



On Thu, Feb 21, 2013 at 6:53 PM, Jason Vas Dias
<jason.vas.dias@gmail.com> wrote:> Aha! Finally I discovered ''xl dmesg'' and saw
>      ''Xen does not allow DomU creation on this CPU for security
reasons.''
>
>    o why not give an option to make Xen write to the standard kernel
> log, so I could have found this message with ''dmesg'' ?
>       It is far from obvious from the documentation that this the only
> way of seeing Xen log messages .
>
> It appears the dom0 CPU has AMD Errata 121 - see:
>    http://lists.xen.org/archives/html/xen-announce/2012-06/msg00002.html
>
> But since both my dom0 kernel (3.7.9-1-ARCH) and my domU kernel
> (2.6.39-300.26.1.el6uek) apparently have the fix for this
> errata ,  I added ''allow_unsafe'' to the Xen dom0 boot
command line,
> and I''m now able to progress beyond the EPERM issue .
> It would have been nice if Xen or xl could have emitted some log
> message other than ''cannot make domain: -3'', particularly
> when it was Xen itself that was refusing to allow the domain creation
> to proceed .
> It would also have been nice if Xen could have recognized that it was
> running kernels with the fix for this issue, and NOT silently
> disabled domain creation.
>
> Regards,
> Jason
>
> On Thu, Feb 21, 2013 at 5:29 PM, Jason Vas Dias
> <jason.vas.dias@gmail.com> wrote:
>> Good day -
>>
>> This is my first post to this list , and I''m new to Xen - any
help on
>> this issue would be much appreciated .
>>
>> I downloaded, built and installed xen-4.2.1 (hypervisor and tools) on
>> an x86_64 ArchLinux box
>> updated to latest software as of today.
>>
>> I am trying to bring up a Linux guest from a remote iSCSI disk.
>>
>> The iSCSI-initiator (open-iscsi)  logs in to the remote target OK and
>> makes it available as /dev/sde ,
>> on which is a bootable linux distribution . containing /boot/vmlinuz*
etc.
>> I can mount this iSCSI disk rw OK on the domain_0 host - so presumably
>> I should be able to
>> boot up a guest domain from it ?
>>
>> I modified the domain_0 host''s boot arguments to boot Xen OK :
>>
>>         multiboot /boot/xen.gz dom0_mem=1024M loglvl=all
guest_loglvl=all
>>         module  /boot/vmlinuz-linux root=/dev/sda2 ro  debug
>>         module  /boot/initramfs-linux.img
>>
>> and made an /etc/xen/${domain}.cfg file:
>> #
>> # xl.cfg(1) file for booting the iSCSI share:
>> #
>> name="$domain"
>> #vcpus=1
>> #uuid="ceb083a2-feaa-45b0-afe2-3bd44eb4d8c3"
>> memory=1024
>> on_poweroff="destroy"
>> on_reboot="destroy"
>> disk=["/dev/sde,raw,hda,rw"]
>> vif=["mac=00:09:3d:13:0d:57,model=tg3,bridge=xenbr0"]
>> kernel="/boot/vmlinuz-2.6.39-300.26.1.el6uek.x86_64"
>> ramdisk="/boot/initramfs-2.6.39-300.26.1.el6uek.x86_64.img"
>>
>> Is the disk line OK ?  Anything else wrong with this file ?
>>
>> But attempting to create the domain , as root, fails:
>> # xl -vvv create -c /etc/xen/${domain}.cfg
>> Parsing config from ${domain}.cfg
>> libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x119c890:
>> create: how=(nil) callback=(nil) poller=0x119c920
>> libxl: error: libxl_create.c:420:libxl__domain_make: domain creation
fail
>> libxl: error: libxl_create.c:646:initiate_domain_create: cannot make
domain: -3
>> libxl: error: libxl.c:1394:libxl__destroy_domid: non-existant domain -1
>> libxl: error: libxl.c:1358:domain_destroy_callback: unable to destroy
>> guest with domid 4294967295
>> libxl: error: libxl_create.c:1153:domcreate_destruction_cb: unable to
>> destroy domain 4294967295 following failed creation
>> libxl: debug: libxl_event.c:1499:libxl__ao_complete: ao 0x119c890:
>> complete, rc=-3
>> libxl: debug: libxl_create.c:1187:do_domain_create: ao 0x119c890:
>> inprogress: poller=0x119c920, flags=ic
>> libxl: debug: libxl_event.c:1471:libxl__ao__destroy: ao 0x119c890:
destroy
>> xc: debug: hypercall buffer: total allocations:5 total releases:5
>> xc: debug: hypercall buffer: current allocations:0 maximum
allocations:2
>> xc: debug: hypercall buffer: cache current size:2
>> xc: debug: hypercall buffer: cache hits:3 misses:2 toobig:0
>>
>> Even though I''ve enabled all possible log messages in Xen with
''
>> loglvl=all guest_loglvl=all'' (or have I?)
>> I don''t see any log messages about this failed attempt to
create a
>> domain at all , in dmesg output or
>> in the system log.
>>
>> Tracing this command in gdb shows the hypervisor command is returning
>> -1, with errno set to 1 (EPERM) :
>>
>> libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x622890:
>> create: how=(nil) callback=(nil) poller=0x622920
>>
>> Breakpoint 2, xc_domain_create (xch=0x622240, ssidref=0,
>> handle=handle@entry=0x7fffffffe1b0
>>
"\316\260\203\242\376\252E\260\257\342;\324N\264\330\303\031",
>>     flags=flags@entry=0, pdomid=pdomid@entry=0x7fffffffe28c) at
xc_domain.c:33
>> 33      {
>> (gdb) n
>> 37          domctl.cmd = XEN_DOMCTL_createdomain;
>> (gdb)
>> 38          domctl.domain = (domid_t)*pdomid;
>> (gdb)
>> 39          domctl.u.createdomain.ssidref = ssidref;
>> (gdb)
>> 40          domctl.u.createdomain.flags   = flags;
>> (gdb)
>> 41          memcpy(domctl.u.createdomain.handle, handle,
>> sizeof(xen_domain_handle_t));
>> (gdb)
>> 42          if ( (err = do_domctl(xch, &domctl)) != 0 )
>> (gdb) s
>> do_domctl (domctl=0x7fffffffe0a0, xch=0x622240) at xc_private.h:239
>> 239         DECLARE_HYPERCALL_BOUNCE(domctl, sizeof(*domctl),
>> XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
>> (gdb) n
>> 241         domctl->interface_version =
XEN_DOMCTL_INTERFACE_VERSION;
>> (gdb)
>> 243         if ( xc_hypercall_bounce_pre(xch, domctl) )
>> (gdb)
>> 249         hypercall.op     = __HYPERVISOR_domctl;
>> (gdb)
>> 250         hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(domctl);
>> (gdb)
>> 252         if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
>> (gdb) s
>> do_xen_hypercall (xch=xch@entry=0x622240,
>> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:246
>> 246         return xch->ops->u.privcmd.hypercall(xch,
xch->ops_handle,
>> hypercall);
>> (gdb) s
>> linux_privcmd_hypercall (xch=<optimized out>, h=9,
>> hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
>> 116         return ioctl(fd, IOCTL_PRIVCMD_HYPERCALL, hypercall);
>> (gdb) fin
>> Run till exit from #0  linux_privcmd_hypercall (xch=<optimized
out>,
>> h=9, hypercall=0x7fffffffe070) at xc_linux_osdep.c:116
>> do_xen_hypercall (xch=xch@entry=0x622240,
>> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
>> 247     }
>> Value returned is $3 = -1
>> (gdb) fin
>> Run till exit from #0  do_xen_hypercall (xch=xch@entry=0x622240,
>> hypercall=hypercall@entry=0x7fffffffe070) at xc_private.c:247
>> 0x00007ffff775b340 in do_domctl (domctl=0x7fffffffe0a0, xch=0x622240)
>> at xc_private.h:252
>> 252         if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
>> Value returned is $4 = -1
>> (gdb) p errno
>> $5 = 1
>>
>> Any ideas how to get this working,  or on where I could get better
>> diagnostics ?
>>
>> I''m guessing this is caused by this code in 
arch/x86/platform_hypercall.c :
>>
>> ret_t do_platform_op(XEN_GUEST_HANDLE(xen_platform_op_t) u_xenpf_op)
>> {
>>     ret_t ret = 0;
>>     struct xen_platform_op curop, *op = &curop;
>>
>>     if ( !IS_PRIV(current->domain) )
>>         return -EPERM;
>>
>> So how do I make the IS_PRIV macro return non-zero for my domain_0 ,
>> ie. set d->is_privileged /
>> make my domain0 into a "privileged" domain ? What is meant by
>> "privileged" in this context ?
>>
>> There seems to be no documentation on this issue in any manual page
>> installed by xen''s ''make install'' ,
>> or anywhere else I could find.
>>
>> Thanks in advance for any replies,
>> Regards, Jason

Casey DeLorme

2013-Feb-21 21:30 UTC

head link

Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Hello Jason,

Looking at your configuration, was the kernel built to be PV Compatible (
http://sanifool.wordpress.com/2013/01/09/booting-up-a-domu-on-xen-4-2/).

The disk line looks fine provided you wanted to pass the whole disk to your
PVM (including it''s partition table etc).  Also if you have it mounted
while trying to launch the PVM that may throw warnings or errors.

Is it possible you are missing the bootloader (
http://xenbits.xen.org/docs/4.2-testing/man/xl.cfg.5.html)?

I don''t believe it would affect your ability to launch the PVM, but the
I
am unfamiliar with your vif device model, and the mac address isn''t in
the
suggested Xen range (
http://xenbits.xen.org/docs/4.2-testing/misc/xl-network-configuration.html).

If you are looking for more detailed diagnostics, you might check that the
xen-debug kernel flag is set (and rebuild your ramdisk or kernel).

Logging must vary by distribution as I almost never see non-kernel software
log its errors to dmesg (/var/log/messages), but most will place logs into
the same directory and organize them by file or folder.  In Xen''s case
it
uses your toolstacks command xl dmesg (/var/logs/xen/).

---

On Thu, Feb 21, 2013 at 2:24 PM, Jason Vas Dias
<jason.vas.dias@gmail.com>wrote:
> I''m beginning to conclude that Xen is not of sufficient code
quality
> to be useful.
>
> Firstly,  my ''xl create'' command returns a 0 exit status,
meaning
> success, but yet no domain
> is created or can be attached to or listed with ''xl list''
:
>
> # xl create my_domain.cfg
> Parsing config from my_domain.cfg
> Daemon running with PID 1436
> # echo $?
> 0
> # xl list
> Name                                        ID   Mem VCPUs      State
> Time(s)
> Domain-0                                     0  1024     2     r-----
>  35.5
>
> This appears to be due to this error message in ''xl
dmesg'' output :
>
> (XEN) Dom5 may compromise security on this CPU.
> (XEN) traps.c:2584:d5 Domain attempted WRMSR 00000000c0010004 from
> 0x0000000000000000 to 0x000000000000abcd.
>
> From reading the source, the traps.c message appears to indicate an
> error condition, but you would not know this from the log message.
>
> Have you checked all of the log files in /var/logs/xen/?
Pretty sure that so long as the configuration itself doesn''t contain
syntactical errors and the command to qemu goes through that xl will assume
it did its job.

Is it possible xl create is launching the domains just fine (hence code 0),
and they are failing at a different stage (eg qemu during boot due to
configuration problems)?

Anyone know how to avoid this unhandled WRMSR ?>
> Code that silently ignores and does not report error conditions does
> not get my vote of confidence.
>
> Can anyone convince me that Xen is of sufficient quality to be used ?
>
> Plenty of well known companies are using Xen, but I don''t believe
they areusing the very latest stable release (4.2.1).  AFAIK 4.2.1 is not available
as a package in any stable operating systems yet.  Stable release of code
does not guarantee it will be compatible without plenty of configuration
based on your choice of Dom0.

However I am under the assumption that you are not using it for production
purposes given your choices (Xen 4.2.1 and ArchLinux don''t exactly
point to
concerns with stability or desirable ease of operation as a priority).  If
that is the case then as a personal user I can say that I have had great
success using Xen 4.2 on Debain Wheezy for several months now.  However I
have not dabbled in many of the areas you are (such as iSCSI, PVM''s and
programmatic domain control).

However, even with my limited experience, I can easily conclude that
because Xen can be used for such a wide variety of purposes it becomes that
much harder to ensure functionality as factors that can influence success
tend to cover a wide range (host hardware, Xen version, installed packages
and package versions, Dom0 distro and version, etc).

Hope these answers will help you identify the problem.

~Casey

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Jason Vas Dias

2013-Feb-22 19:12 UTC

head link

Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Hi Casey - many thanks for your informative response ! - I''ve now got
it working ,
with a number of caveats / gotchas, detailed below:

On Thu, Feb 21, 2013 at 9:30 PM, Casey DeLorme <cdelorme@gmail.com>
wrote:>
> Hello Jason,
>
> Looking at your configuration, was the kernel built to be PV Compatible
> (http://sanifool.wordpress.com/2013/01/09/booting-up-a-domu-on-xen-4-2/).
>Yes,  the configuration arguments of the latest ArchLinux kernel (
3.7.9-1-ARCH) check out for Xen .
> The disk line looks fine provided you wanted to pass the whole disk to your
> PVM (including it''s partition table etc).  Also if you have it
mounted while
> trying to launch the PVM that may throw warnings or errors.
>
> Is it possible you are missing the bootloader
> (http://xenbits.xen.org/docs/4.2-testing/man/xl.cfg.5.html)?
>
Yes, that was confusing - I first gave xl just
''kernel="/boot/vmlinuz..." , but then xl complained
it couldn''t find the kernel, so I mounted the iSCSI share and gave xl
the full path to the kernel ;  but then
I saw your post and specified the "bootloader="pygrub"''
setting, which
did not work until the fix
for this bug is built into libxl :
 Bug 1844 - xl coredumps whenever any "bootloader=" setting is used in
an xl.cfg file

Having fixed that, then I came up against a number of disk issues -
No setting for the disk ''vdev'' field seemed to have any effect
- ie.
sda , sda6, hda, sdc (the real device name) -
 the guest OS could not see that device.
Happily, with the dracut ''rdshell'' argument supported by the
guest OS,
I was able to get it into initramfs debug mode,
and saw the disk on /dev/xvdc  -  this appears to be the only usable
setting for the root= kernel argument - it is now
booting from this.  Perhaps this is just because I''m using the default
initramfs file and did not build one with the extra devices ?
> I don''t believe it would affect your ability to launch the PVM,
but the I am
> unfamiliar with your vif device model, and the mac address isn''t
in the
> suggested Xen range
>
(http://xenbits.xen.org/docs/4.2-testing/misc/xl-network-configuration.html).
>
Yes, I should have read that document more closely - the setting that works is
    vif=["bridge=eth0"]

Then, with the default ''peth0'' bridge created by default on
ArchLinux, I get :
# brctl show
bridge name     bridge id               STP enabled     interfaces
eth0            8000.00093d130d57       no              peth0

          vif22.0
>
> If you are looking for more detailed diagnostics, you might check that the
> xen-debug kernel flag is set (and rebuild your ramdisk or kernel).
>
>
> Logging must vary by distribution as I almost never see non-kernel software
> log its errors to dmesg (/var/log/messages), but most will place logs into
> the same directory and organize them by file or folder.  In Xen''s
case it
> uses your toolstacks command xl dmesg (/var/logs/xen/).
>
Yes, but my point was that xl should not silently fail because of the
errata-121 issue without
telling the user - perhaps, if xl finds domain creation failed, it
could automatically do its dmesg stuff
and dump the Xen log messages to stderr ?

Also, in many cases when domain creation failed because of the above
issues, xl returned a 0 exit status -
it really should be returning >0 in these cases.
> ---
>
>
> On Thu, Feb 21, 2013 at 2:24 PM, Jason Vas Dias
<jason.vas.dias@gmail.com>
> wrote:
>>
>> I''m beginning to conclude that Xen is not of sufficient code
quality
>> to be useful.
>>
>> Firstly,  my ''xl create'' command returns a 0 exit
status, meaning
>> success, but yet no domain
>> is created or can be attached to or listed with ''xl
list'' :
>>
>> # xl create my_domain.cfg
>> Parsing config from my_domain.cfg
>> Daemon running with PID 1436
>> # echo $?
>> 0
>> # xl list
>> Name                                        ID   Mem VCPUs      State
>> Time(s)
>> Domain-0                                     0  1024     2     r-----
>> 35.5
>>
>> This appears to be due to this error message in ''xl
dmesg'' output :
>>
>> (XEN) Dom5 may compromise security on this CPU.
>> (XEN) traps.c:2584:d5 Domain attempted WRMSR 00000000c0010004 from
>> 0x0000000000000000 to 0x000000000000abcd.
>>
>> From reading the source, the traps.c message appears to indicate an
>> error condition, but you would not know this from the log message.
>>
> Have you checked all of the log files in /var/logs/xen/?
>
> Pretty sure that so long as the configuration itself doesn''t
contain
> syntactical errors and the command to qemu goes through that xl will assume
> it did its job.
>
> Is it possible xl create is launching the domains just fine (hence code 0),
> and they are failing at a different stage (eg qemu during boot due to
> configuration problems)?
But then xl should not be returning ''0'' .
>
>> Anyone know how to avoid this unhandled WRMSR ?
>>
>> Code that silently ignores and does not report error conditions does
>> not get my vote of confidence.
>>
>> Can anyone convince me that Xen is of sufficient quality to be used ?
>>
> Plenty of well known companies are using Xen, but I don''t believe
they are
> using the very latest stable release (4.2.1).  AFAIK 4.2.1 is not available
> as a package in any stable operating systems yet.  Stable release of code
> does not guarantee it will be compatible without plenty of configuration
> based on your choice of Dom0.
>OK - I prefer to use latest available versions of everything so at
least I know all
known bugs are fixed, and I can help find new bugs.
> However I am under the assumption that you are not using it for production
> purposes given your choices (Xen 4.2.1 and ArchLinux don''t exactly
point to
> concerns with stability or desirable ease of operation as a priority).  If
> that is the case then as a personal user I can say that I have had great
> success using Xen 4.2 on Debain Wheezy for several months now.  However I
> have not dabbled in many of the areas you are (such as iSCSI,
PVM''s and
> programmatic domain control).
>
> However, even with my limited experience, I can easily conclude that
because
> Xen can be used for such a wide variety of purposes it becomes that much
> harder to ensure functionality as factors that can influence success tend
to
> cover a wide range (host hardware, Xen version, installed packages and
> package versions, Dom0 distro and version, etc).
>
>
> Hope these answers will help you identify the problem.
>
> ~Casey
I''m trying to use Xen for our testing server - in a
"semi-production"
/ backend environment -
where we produce software that claims to run on @30 different OS-es,
and we don''t have
30 physical servers, I''m setting up one server to be able to boot each
different supported OS
with Xen, for testing purposes.  Now at least I''ve got one working ...
only 30 more to go !

Thank You & Regards,
Jason

Maybe Matching Threads

Search for more maybe matching threads

Xen users - Feb 2013 - help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Re: help please - running a guest from an iSCSI disk ? getting more diagnostics than "cannot make domain: -3" ? how to make domain0 "privileged" ?

Maybe Matching Threads