Witold Baryluk
2012-Jan-31 11:32 UTC
[Pkg-xen-devel] Bug#658100: xl: hangs in qemu-dm when trying to run PV domU
Package: xen-utils-common
Version: 4.1.2-1
Severity: normal
Hi, I was doing some experiments today with XL TOOLSTACK,
and found few problems when trying to do PV domU boot.
I was trying to run kfreebsd9 or linux-3.2 kernel in qemu using PV i386,
and with simple configuration without any disks, nics or virtual framebuffer,
kernel boots (and then kernel itselfs correctly fails with information that root
cannot be mounted).
(ignoring spurious information/error "kernel is not a bzImage: Invalid
kernel",
and another - already reported - bug which was solved by simply 'mkdir
/var/lib/xen')
movax-dev:/etc/xen# cat kfreebsd9.cfg
name = "kfreebsd9"
#kernel = "/boot/vmlinuz-3.2.0-1-686-pae"
kernel = "/boot/kfreebsd-9.0-1-xen.gz"
memory = 256
movax-dev:/etc/xen# xl create kfreebsd9.cfg -c
Parsing config file kfreebsd9.cfg
xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel:
kernel is not a bzImage: Invalid kernel
Daemon running with PID 11583
WARNING: loader(8) metadata is missing!
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
#0 Sat Jan 7 14:06:17 UTC 2012 i386
Xen reported: 2793.110 MHz processor.
Timecounter "ixen" frequency 1953125 Hz quality 0
CPU: Intel(R) Pentium(R) 4 CPU 2.80GHz (2793.11-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf41 Family = f Model = 4 Stepping
= 1
Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x441d<SSE3,DTES64,MON,DS_CPL,CNXT-ID,xTPR>
AMD Features=0x100000<NX>
real memory = 268435456 (256 MB)
avail memory = 254488576 (242 MB)
[XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0)
[XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1)
xenstore0: <XenStore> on motherboard
[XEN] xen_rtc_probe: probing Hypervisor RTC clock
rtc0: <Xen Hypervisor Clock> on motherboard
[XEN] xen_rtc_attach: attaching Hypervisor RTC clock
xc0: <Xen Console> on motherboard
Event timer "ixen" quality 600
[XEN] hypervisor wallclock nudged; nudging TOD.
Timecounters tick every 10.000 msec
xenbusb_back0: <Xen Backend Devices> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
xenbusb_front0: <Xen Frontend Devices> on xenstore0
xenbusb_add_device: Device device/suspend/event-channel ignored. State 6
Timecounter "TSC" frequency 2793110000 Hz quality 800
Loader variables:
Manual root filesystem specification:
<fstype>:<device> [options]
Mount <device> using filesystem <fstype>
and with the specified (optional) option list.
eg. ufs:/dev/da0s1a
zfs:tank
cd9660:/dev/acd0 ro
(which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)
? List valid disk boot devices
. Yield 1 second (for background tasks)
<empty line> Abort manual input
mountroot>
panic: mountroot: unable to (re-)mount root.
cpuid = 0
KDB: enter: panic
[ thread pid 1 tid 100002 ]
Stopped at kdb_enter+0x38: movl $0,kdb_why
db> halt
movax-dev:/etc/xen#
Adding any disk or vfb stanzas to config, like:
vfb = [
"type=vnc,vncdisplay=1,vnclisten=0.0.0.0,vncpasswd=secret123456" ]
disk = [ ]
disk = [ "file:/home/FreeBSD-9.0-RELEASE-i386-dvd1.iso,sda,r" ]
disk = [ "file://home/kfreebsd9-disk1.img,sda,w" ]
(of course each disk = , once in given .cfg file)
makes it stop working.
With vfb, it looks qemu is started, but serial console is not functioning:
movax-dev:/etc/xen# xl create kfreebsd9.cfg -c
Parsing config file kfreebsd9.cfg
xc: error: panic: xc_dom_bzimageloader.c:556: xc_dom_probe_bzimage_kernel:
kernel is not a bzImage: Invalid kernel
Daemon running with PID 11761
xenconsole: Could not read tty from store: No such file or directory
movax-dev:/etc/xen#
(there is about 10 seconds delay between "Deamon running with PID
...", and "xenconsole: Could not read tty from store: ...")
Also "xl conole .." fails then:
movax-dev:/etc/xen# xl console 22
xenconsole: Could not read tty from store: No such file or directory
movax-dev:/etc/xen#
It looks that qemu-dm is running:
movax-dev:/etc/xen# ps aux | grep qemu
root 11759 0.1 0.9 18744 3000 ? Ssl 12:11 0:00 qemu-dm -d 22
-domain-name kfreebsd9 -vnc 0.0.0.0:1,password -vncunused -M xenpv
and in fact I am able to connect using VNC client, but screen/display is black.
Log file lists some problems with gnttab
movax-dev:/var/log/xen# cat qemu-dm-kfreebsd9.log
domid: 22
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
xen be core: xen be core: can't open gnttab device
can't open gnttab device
xs_read(): target get error. /local/domain/22/target.
movax-dev:/var/log/xen#
Exactly same happens, if instead of 'vfb', i add 'disk':
root 11939 0.3 0.7 17128 2520 ? Ssl 12:14 0:00 qemu-dm -d 23
-domain-name kfreebsd9 -nographic -M xenpv
movax-dev:/var/log/xen# cat qemu-dm-kfreebsd9.log
domid: 23
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
xen be core: xen be core: can't open gnttab device
can't open gnttab device
xen be core: xen be core: can't open gnttab device
can't open gnttab device
xs_read(): target get error. /local/domain/23/target.
(qemu) (qemu movax-dev:/var/log/xen#
I found NetBSD user/developer had very similar problem (when using xl as
TOOLSTACK,
just like me!) about year ago (Mar 2011).
http://mail-index.netbsd.org/port-xen/2011/03/31/msg006552.html
with some discussion, that NetBSD doesn't have gnttap implemented in kernel:
http://mail-index.netbsd.org/port-xen/2011/03/31/msg006558.html
http://mail-index.netbsd.org/port-xen/2011/03/31/msg006559.html
http://mail-index.netbsd.org/port-xen/2011/04/01/msg006575.html
but there is not resolution, or follow up on xen-devel.
But I was checking both kfreebsd and linux kernel (from unstable),
and results are same (Debian linux kernel just doesn't print information
about Invalid kernel)
# cat /var/log/xen/qemu-dm-debian-1.log
domid: 24
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
xen be core: xen be core: can't open gnttab device
can't open gnttab device
xs_read(): target get error. /local/domain/24/target.
#
movax-dev:/etc/xen# xl info
host : movax-dev
release : 3.2.0-1-686-pae
version : #1 SMP Tue Jan 24 06:09:30 UTC 2012
machine : i686
nr_cpus : 2
nr_nodes : 1
cores_per_socket : 1
threads_per_core : 2
cpu_mhz : 2793
hw_caps :
bfebfbff:00100000:00000000:00000180:0000441d:00000000:00000000:00000000
virt_caps :
total_memory : 1015
free_memory : 342
free_cpus : 0
xen_major : 4
xen_minor : 1
xen_extra : .2
xen_caps : xen-3.0-x86_32p
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xf5800000
xen_changeset : unavailable
xen_commandline : placeholder
cc_compiler : gcc version 4.6.2 (Debian 4.6.2-6)
cc_compile_by : waldi
cc_compile_domain : debian.org
cc_compile_date : Sun Dec 11 13:35:02 UTC 2011
xend_config_format : 4
movax-dev:/etc/xen# uname -a
Linux movax-dev 3.2.0-1-686-pae #1 SMP Tue Jan 24 06:09:30 UTC 2012 i686
GNU/Linux
movax-dev:/etc/xen# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Pentium(R) 4 CPU 2.80GHz
stepping : 1
microcode : 0x12
cpu MHz : 2793.110
cache size : 1024 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu de tsc msr pae cx8 apic cmov pat clflush acpi mmx fxsr sse sse2 ss
ht nx constant_tsc pni cid hypervisor
bogomips : 5586.22
clflush size : 64
cache_alignment : 128
address sizes : 36 bits physical, 32 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Pentium(R) 4 CPU 2.80GHz
stepping : 1
microcode : 0x12
cpu MHz : 2793.110
cache size : 1024 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
apicid : 1
initial apicid : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu de tsc msr pae cx8 apic cmov pat clflush acpi mmx fxsr sse sse2 ss
ht nx constant_tsc pni cid hypervisor
bogomips : 5586.22
clflush size : 64
cache_alignment : 128
address sizes : 36 bits physical, 32 bits virtual
power management:
movax-dev:/etc/xen# ls -l /boot/kfreebsd-9.0-1-xen.gz
/boot/vmlinuz-3.2.0-1-686-pae
-rw-r--r-- 1 root root 2209727 sty 7 15:06 /boot/kfreebsd-9.0-1-xen.gz
-rw-r--r-- 1 root root 2651200 sty 24 08:15 /boot/vmlinuz-3.2.0-1-686-pae
movax-dev:/etc/xen# dpkg -l | grep -- -image
ii gnumach-image-1-xen-486 2:1.3.99.dfsg.git20111010-1 The GNU
version of the Mach microkernel
ii gnumach-image-1.3.99-xen-486 2:1.3.99.dfsg.git20111010-1 The GNU
version of the Mach microkernel for Xen
ii kfreebsd-image-9-xen 9.0-1 kernel of
FreeBSD 9 image (meta-package)
ii kfreebsd-image-9.0-1-xen 9.0-1 kernel of
FreeBSD 9.0 image
ii libmng1 1.0.10-3
Multiple-image Network Graphics library
ii linux-image-3.2.0-1-686-pae 3.2.1-2 Linux 3.2
for modern PCs
ii linux-image-686-pae 3.2+42 Linux for
modern PCs (meta-package)
Regards,
Witek
-- System Information:
Debian Release: wheezy/sid
APT prefers unstable
APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: i386 (i686)
Kernel: Linux 3.2.0-1-686-pae (SMP w/2 CPU cores)
Locale: LANG=pl_PL.UTF-8, LC_CTYPE=pl_PL.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages xen-utils-common depends on:
ii gawk 1:3.1.8+dfsg-0.1
ii lsb-base 3.2-28.1
ii udev 175-3
ii xenstore-utils 4.1.2-2
xen-utils-common recommends no packages.
xen-utils-common suggests no packages.
-- Configuration Files:
/etc/default/xen changed:
TOOLSTACK=xl
-- no debconf information
Ian Campbell
2012-Jan-31 16:51 UTC
[Pkg-xen-devel] Bug#658100: Bug#658100: xl: hangs in qemu-dm when trying to run PV domU
On Tue, 2012-01-31 at 12:32 +0100, Witold Baryluk wrote:> I found NetBSD user/developer had very similar problem (when using xl as TOOLSTACK, > just like me!) about year ago (Mar 2011). > > http://mail-index.netbsd.org/port-xen/2011/03/31/msg006552.html > with some discussion, that NetBSD doesn't have gnttap implemented in kernel:I think that is correct and I'm afraid this will prevent the use of either vfb or qdisk (which is what file:// becomes in the absence of blktap, which freebsd does not have) disk backends. Userspace PV device backends simply cannot work without a gnttab device. There has been some work on xen-unstable (by Roger Pau Monet) to make file:// type disks use the in kernel vbd backend (which won't need vbd) but this is still a work in progress and won't be in 4.1. Hopefully this will be in in time for 4.2. As a workaround in the meantime you can use the FreeBSD equivalent of losetup (vnconfig?) to manually turn your file:/path/etc into a phy:/dev/vndN which will use the in kernel vbd backend. (I'm not a FreeBSD guy so I may have got some of the names above wrong) The vfb stuff cannot, AFAIK, ever work without a gnttab or equivalent and therefore without this kernel feature there is no chance of it working. I'm afraid that what's needed is for a FreeBSD kernel hacker to work on making this stuff work :-/ Ian. -- Ian Campbell Current Noise: Audioslave - Light My Way "Of course power tools and alcohol don't mix. Everyone knows power tools aren't soluble in alcohol..." -- Crazy Nigel
Debian Bug Tracking System
2012-Feb-03 09:54 UTC
[Pkg-xen-devel] Bug#658100: marked as done (xl: hangs in qemu-dm when trying to run PV domU)
Your message dated Fri, 03 Feb 2012 09:51:43 +0000 with message-id <1328262703.13189.64.camel at dagon.hellion.org.uk> and subject line Re: [Pkg-xen-devel] Bug#658100: xl: hangs in qemu-dm when trying to run PV domU has caused the Debian Bug report #658100, regarding xl: hangs in qemu-dm when trying to run PV domU to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact owner at bugs.debian.org immediately.) -- 658100: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=658100 Debian Bug Tracking System Contact owner at bugs.debian.org with problems -------------- next part -------------- An embedded message was scrubbed... From: Witold Baryluk <baryluk at smp.if.uj.edu.pl> Subject: xl: hangs in qemu-dm when trying to run PV domU Date: Tue, 31 Jan 2012 12:32:32 +0100 Size: 12293 URL: <http://lists.alioth.debian.org/pipermail/pkg-xen-devel/attachments/20120203/1dc43df4/attachment.mht> -------------- next part -------------- An embedded message was scrubbed... From: Ian Campbell <ijc at hellion.org.uk> Subject: Re: [Pkg-xen-devel] Bug#658100: xl: hangs in qemu-dm when trying to run PV domU Date: Fri, 03 Feb 2012 09:51:43 +0000 Size: 4672 URL: <http://lists.alioth.debian.org/pipermail/pkg-xen-devel/attachments/20120203/1dc43df4/attachment-0001.mht>
Possibly Parallel Threads
- Bug#658101: xen-utils-common: xl create fails because of missing /var/lib/xen
- Bug#658090: xen-utils-common: xl create fails due missing /var/lib/xen directory
- Re: FreeBSD PVHVM call for testing
- 9.0-RELEASE PV from scratch on XCP v1.1.0
- xen be core: xen be core: can''t open gnttab device