Xen users - Oct 2012 - xen 4.2.0, xl dmesg permission denied

Hi!

Yesterday I''ve upgraded my box from Xen 4.1.2 to Xen 4.2.0, running a
Gentoo 64bit Dom0. Thus I built all the Xen tools and hypervisor using the
Gentoo package manager (configured with hvm, pygrub, qemu, screen, and without
flask, xsm, pae, xend, no custom cflags). With Xen 4.2.0 I can no longer view
the hypervisor dmesg:

dom0# xl dmesg
libxl: error: libxl.c:4327:libxl_xen_console_read_line: reading console ring
buffer: Permission denied

VMs wouldn''t start:
dom0# xl create config.xen
Parsing config from config.xen
failed to free memory for the domain

I wanted to see the hypervisor messages in order to look into my memory problem
and the dom0_mem:2G,max:2G boot option did not seem to have any effect:

dom0# head -n 2 /proc/meminfo 
MemTotal:        1697820 kB
MemFree:         1590232 kB

From /boot/grub2/grub.cfg:
menuentry ''Xen/Gentoo 3.6.2'' --class gnu-linux --class gnu
--class os {
        insmod raid
        insmod mdraid1x
        insmod gzio
        insmod part_msdos
        insmod ext2
        set root=''(mduuid/9df8572c68ce035ed264acd1adb42869)''
        search --no-floppy --fs-uuid --set=root
681460d3-25dc-4763-a069-9c7442228ad8
        echo    ''Xen wird geladen''
        multiboot /xen/xen.gz dom0_mem=2G,max:2G iommu=require,verbose
        echo    ''Dom0 wird geladen''
        module /xen/image-3.6.2-gentooamd placeholder dom0_mem=2G,max:2G
''pci=resource_alignment=00:12.0;00:12.1;00:12.2;01:07.0;01:07.1;01:07.2;04:00.0;04:00.1''
        echo    ''Done.''
}

The Dom0 kernel versions I tried are 3.2.11, 3.4.9, and 3.6.2, all with Gentoo
patchset. I spent a day searching for my permission issue on the net, looking
into the source code of libxl, reading the kernel config howto for xen, and
trying to figure out which permissions might be missing. I didn''t find
anything about permissions. When reverting back to Xen 4.1.2 I had an
intermediate state of still running under hypervisor 4.2, but having xen tools
4.1. In that state the "xl dmesg" succeeded. I conclude that there is
no permissions issue in the hypervisor at all, but I''m out of ideas.
Seems there is something wrong in the user land tools, but I couldn''t
figure it out.

In my desperation I tried the latest version from hg which identifies itself as
4.3-unstable. It failed, too.

Modules I loaded (via modprobe -v):
insmod
/lib/modules/3.6.2-gentooamd/kernel/drivers/xen/xen-pciback/xen-pciback.ko
hide=(00:12.0)(00:12.1)(00:12.2)(01:07.0)(01:07.1)(01:07.2)(04:00.0)(04:00.1)
passthrough=1
insmod
/lib/modules/3.6.2-gentooamd/kernel/drivers/block/xen-blkback/xen-blkback.ko
insmod /lib/modules/3.6.2-gentooamd/kernel/drivers/xen/xen-gntdev.ko
insmod /lib/modules/3.6.2-gentooamd/kernel/drivers/xen/xen-gntalloc.ko

What may be the cause of the "permission denied" message? Where shall
I look? Any pointers? I hope I''ve included enough information for you
to help me. Let me know if there is anything missing.

Frank

On Mon, 2012-10-22 at 20:10 +0100, Frank Birbacher
wrote:
> Hi!
> 
> Yesterday I''ve upgraded my box from Xen 4.1.2 to Xen 4.2.0,
running a Gentoo 64bit Dom0. Thus I built all the Xen tools and hypervisor using
the Gentoo package manager (configured with hvm, pygrub, qemu, screen, and
without flask, xsm, pae, xend, no custom cflags). With Xen 4.2.0 I can no longer
view the hypervisor dmesg:
> 
> dom0# xl dmesg
> libxl: error: libxl.c:4327:libxl_xen_console_read_line: reading console
ring buffer: Permission denied
The usual reason for this is that you have a mismatched pairing of
hypervisor / tools. Check you aren''t running the old tools (including
libraries) with the new hypervisor or vice versa.

One particular gotcha might be the libraries -- make sure the new ones
have been installed in the expected location and that xl is linking
against them at runtime (e.g. run with ldd).

Ian.

Hi Ian,

Am 23.10.2012 um 14:26 schrieb Ian Campbell:
> The usual reason for this is that you have a mismatched pairing of
> hypervisor / tools. Check you aren''t running the old tools
(including
> libraries) with the new hypervisor or vice versa.
Thanks so much, that has been the pointer I needed! I figured I''ve been
running an old hypervisor all the time because of replacing the wrong file in
/boot. Some time ago when switching from 4.1.1 to 4.1.2 I''d made a copy
of the hypervisor in /boot in case the new one wouldn''t run. And I was
still using that old hypervisor. I feel a little embarrassed.

Frank

On Wed, 2012-10-24 at 07:35 +0100, Frank Birbacher
wrote:
> Hi Ian,
> 
> Am 23.10.2012 um 14:26 schrieb Ian Campbell:
> > The usual reason for this is that you have a mismatched pairing of
> > hypervisor / tools. Check you aren''t running the old tools
(including
> > libraries) with the new hypervisor or vice versa.
> 
> Thanks so much, that has been the pointer I needed! I figured I''ve
> been running an old hypervisor all the time because of replacing the
> wrong file in /boot. Some time ago when switching from 4.1.1 to 4.1.2
> I''d made a copy of the hypervisor in /boot in case the new one
> wouldn''t run. And I was still using that old hypervisor. I feel a
> little embarrassed.
It''s easily done!

Ian.