Hello all,
I have created a FreeBSD PV DomU image formatted with ZFS. I compiled
FreeBSD with KERNCONF=XEN for the kernel and the normal world and
distribution target. Then I transfered it to a Debian Dom0 with Xen 4.2.1.
I tried to boot the image befor I migrate it to LVM and I now get teh
following output:
Parsing config from freebsd-test.cfg
libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 12 device
model: spawn failed (rc=-3)
libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: No
such file or directory
Daemon running with PID 3451
WARNING: loader(8) metadata is missing!
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
APIC: Using the MPTable enumerator.
SMP: Added CPU 0 (BSP)
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-RELEASE #0: Wed Feb 20 14:16:03 CET 2013
root@build:/usr/obj/usr/src/sys/XEN i386
WARNING: WITNESS option enabled, expect reduced performance.
Xen reported: 3341.754 MHz processor.
Timecounter "ixen" frequency 1953125 Hz quality 0
CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz 686-class
CPU)
Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a
Stepping 5
Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT>
AMD Features=0x28100000<NX,RDTSCP,LM>
AMD Features2=0x1<LAHF>
Data TLB: 4 KB pages, 4-way set associative, 64 entries
1st-level data cache: 32 KB, 8-way set associative, 64 byte line size
L2 cache: 256 kbytes, 8-way associative, 64 bytes/line
real memory = 2147483648 (2048 MB)
Physical memory chunk(s):
0x0000000000a58000 - 0x000000007d995fff, 2096357376 bytes (511806 pages)
avail memory = 2092105728 (1995 MB)
INTR: Adding local APIC 0 as a target
ULE: setup cpu 0
[XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0)
[XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1)
Event-channel device installed.
io: <I/O>
random: <entropy source, Software, Yarrow>
mem: <memory>
Pentium Pro MTRR support enabled
null: <null device, zero device>
nfslock: pseudo-device
[XEN] xen_rtc_probe: probing Hypervisor RTC clock
rtc0: <Xen Hypervisor Clock> on motherboard
[XEN] xen_rtc_attach: attaching Hypervisor RTC clock
rtc0: registered as a time-of-day clock (resolution 1000000us, adjustment
0.500000000s)
xenstore0: <XenStore> on motherboard
Grant table initialized
xc0: <Xen Console> on motherboard
Device configuration finished.
procfs registered
Event timer "ixen" quality 600
Timecounters tick every 10.000 msec
lo0: bpf attached
xenbusb_front0: <Xen Frontend Devices> on xenstore0
xenbusb_add_device: Device device/suspend/event-channel ignored. State 6
xenbusb_back0: <Xen Backend Devices> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
[XEN] hypervisor wallclock nudged; nudging TOD.
run_interrupt_driven_hooks: still waiting after 60 seconds for
xenbus_free_evtchn
run_interrupt_driven_hooks: still waiting after 120 seconds for
xenbus_free_evtchn
run_interrupt_driven_hooks: still waiting after 180 seconds for
xenbus_free_evtchn
run_interrupt_driven_hooks: still waiting after 240 seconds for
xenbus_free_evtchn
run_interrupt_driven_hooks: still waiting after 300 seconds for
xenbus_free_evtchn
panic: run_interrupt_driven_config_hooks: waited too long
cpuid = 0
KDB: enter: panic
[ thread pid 0 tid 100000 ]
Stopped at kdb_enter+0x3a: movl $0,kdb_why
db>
My configuration looks like this:
kernel = ''/home/test/kernel/kernel''
extra = ''boot_verbose=1,kern.hz=100''
vcpus = ''1''
memory = ''512''
disk = [
''file:/home/test/disk.img,hda1,w'',
]
name = ''freebsd''
#vif = [ ''mac=00:50:56:00:2A:9F,bridge=xenbr0'' ]
on_poweroff = ''destroy''
on_reboot = ''restart''
on_crash = ''restart''
Can someone tell me what''s going wrong and how I could solve this? I
also
saw other boot parameters aspecially the
vfs.root.mountfrom but I don''t know what is needed to define here wehn
I have ZFS. I also have just a zpool and not a partition in the
classic way on the image.
So much questions. Would be great when someone coudl help me out.
Best Regards
_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
On 21/02/13 10:18, tech mailinglists wrote:> Hello all, > > I have created a FreeBSD PV DomU image formatted with ZFS. I compiled > FreeBSD with KERNCONF=XEN for the kernel and the normal world and > distribution target. Then I transfered it to a Debian Dom0 with Xen 4.2.1. > > I tried to boot the image befor I migrate it to LVM and I now get teh > following output: > > Parsing config from freebsd-test.cfg > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 12 > device model: spawn failed (rc=-3) > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: > No such file or directory > Daemon running with PID 3451 > WARNING: loader(8) metadata is missing! > GDB: no debug ports present > KDB: debugger backends: ddb > KDB: current backend: ddb > APIC: Using the MPTable enumerator. > SMP: Added CPU 0 (BSP) > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 9.1-RELEASE #0: Wed Feb 20 14:16:03 CET 2013 > root@build:/usr/obj/usr/src/sys/XEN i386 > WARNING: WITNESS option enabled, expect reduced performance. > Xen reported: 3341.754 MHz processor. > Timecounter "ixen" frequency 1953125 Hz quality 0 > CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz > 686-class CPU) > Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a > Stepping = 5 > > Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > > Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT> > AMD Features=0x28100000<NX,RDTSCP,LM> > AMD Features2=0x1<LAHF> > > Data TLB: 4 KB pages, 4-way set associative, 64 entries > 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size > L2 cache: 256 kbytes, 8-way associative, 64 bytes/line > real memory = 2147483648 (2048 MB) > Physical memory chunk(s): > 0x0000000000a58000 - 0x000000007d995fff, 2096357376 bytes (511806 pages) > avail memory = 2092105728 (1995 MB) > INTR: Adding local APIC 0 as a target > ULE: setup cpu 0 > [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) > [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) > Event-channel device installed. > io: <I/O> > random: <entropy source, Software, Yarrow> > mem: <memory> > Pentium Pro MTRR support enabled > null: <null device, zero device> > nfslock: pseudo-device > [XEN] xen_rtc_probe: probing Hypervisor RTC clock > rtc0: <Xen Hypervisor Clock> on motherboard > [XEN] xen_rtc_attach: attaching Hypervisor RTC clock > rtc0: registered as a time-of-day clock (resolution 1000000us, > adjustment 0.500000000s) > xenstore0: <XenStore> on motherboard > Grant table initialized > xc0: <Xen Console> on motherboard > Device configuration finished. > procfs registered > Event timer "ixen" quality 600 > Timecounters tick every 10.000 msec > lo0: bpf attached > xenbusb_front0: <Xen Frontend Devices> on xenstore0 > xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 > xenbusb_back0: <Xen Backend Devices> on xenstore0 > xctrl0: <Xen Control Device> on xenstore0 > [XEN] hypervisor wallclock nudged; nudging TOD. > run_interrupt_driven_hooks: still waiting after 60 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 120 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 180 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 240 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 300 seconds for > xenbus_free_evtchn > panic: run_interrupt_driven_config_hooks: waited too long > cpuid = 0 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at kdb_enter+0x3a: movl $0,kdb_why > db>Hello, I can tell you PVHVM (XENHVM) works fine with Xen 4.2.1 and -unstable (both 9.1 and HEAD), but I''m afraid I haven''t got time to test Xen PV (i386) yet. Can you check if this is a regression in the FreeBSD kernel or Xen itself? Regards, Roger.
Hello Roger,
I tried a further thing in the kernel configuration. I remove the option
KBD_INSTALL_CDEV and now I am no longer getting dropped to a minimal
console. The boot now looks like this:
Parsing config from /root/xen/freebsd-test.cfg
libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 20 device
model: spawn failed (rc=-3)
libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: No
such file or directory
Daemon running with PID 4383
WARNING: loader(8) metadata is missing!
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
APIC: Using the MPTable enumerator.
SMP: Added CPU 0 (BSP)
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-RELEASE #1 r247145M: Fri Feb 22 18:26:55 CET 2013
root@build:/usr/obj/usr/src/sys/XEN i386
Xen reported: 3341.754 MHz processor.
Timecounter "ixen" frequency 1953125 Hz quality 0
CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz 686-class
CPU)
Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a
Stepping 5
Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT>
AMD Features=0x28100000<NX,RDTSCP,LM>
AMD Features2=0x1<LAHF>
Data TLB: 4 KB pages, 4-way set associative, 64 entries
1st-level data cache: 32 KB, 8-way set associative, 64 byte line size
L2 cache: 256 kbytes, 8-way associative, 64 bytes/line
real memory = 536870912 (512 MB)
Physical memory chunk(s):
0x0000000000670000 - 0x000000001f628fff, 519802880 bytes (126905 pages)
avail memory = 516833280 (492 MB)
INTR: Adding local APIC 0 as a target
ULE: setup cpu 0
[XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0)
[XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1)
Event-channel device installed.
random: <entropy source, Software, Yarrow>
io: <I/O>
mem: <memory>
Pentium Pro MTRR support enabled
null: <null device, zero device>
nfslock: pseudo-device
[XEN] xen_rtc_probe: probing Hypervisor RTC clock
rtc0: <Xen Hypervisor Clock> on motherboard
[XEN] xen_rtc_attach: attaching Hypervisor RTC clock
rtc0: registered as a time-of-day clock (resolution 1000000us, adjustment
0.500000000s)
xenstore0: <XenStore> on motherboard
Grant table initialized
xc0: <Xen Console> on motherboard
Device configuration finished.
procfs registered
Event timer "ixen" quality 600
Timecounters tick every 10.000 msec
lo0: bpf attached
xenbusb_front0: <Xen Frontend Devices> on xenstore0
xenbusb_add_device: Device device/suspend/event-channel ignored. State 6
xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
xn0: bpf attached
xn0: Ethernet address: 00:16:3e:44:b1:b9
xenbusb_back0: <Xen Backend Devices> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
xn0: backend features: feature-sg feature-gso-tcp4
[XEN] hypervisor wallclock nudged; nudging TOD.
run_interrupt_driven_hooks: still waiting after 60 seconds for
xenbus_free_evtchn
[XEN] hypervisor wallclock nudged; nudging TOD.
run_interrupt_driven_hooks: still waiting after 120 seconds for
xenbus_free_evtchn
run_interrupt_driven_hooks: still waiting after 180 seconds for
xenbus_free_evtchn
run_interrupt_driven_hooks: still waiting after 240 seconds for
xenbus_free_evtchn
run_interrupt_driven_hooks: still waiting after 300 seconds for
xenbus_free_evtchn
[XEN] hypervisor wallclock nudged; nudging TOD.
[XEN] hypervisor wallclock nudged; nudging TOD.
[XEN] hypervisor wallclock nudged; nudging TOD.
I will test HVM and PVHVM but in the past I had no problems with this. It
would be greate to have pure PV mode. The only thing which confuses me is
the libxl related error on the domain creation:
Parsing config from /root/xen/freebsd-test.cfg
libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 20 device
model: spawn failed (rc=-3)
libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: No
such file or directory
I know that in a few cases PV guests have QEmu process but I can not say
while this failes. Eventually there is something not compiled in, into Xen
or the toolstack?
I saw that you have made a talk on FOSDEM about BSD and PVH. I downloaded
the slide PDF but I am not able to open the document. It says that the
document is corrupted. Am I doing a mistake? Or is there a problem with the
document on the FOSDEM page.
Best Regards
2013/2/22 Roger Pau Monné <roger.pau@citrix.com>
> On 21/02/13 10:18, tech mailinglists wrote:
> > Hello all,
> >
> > I have created a FreeBSD PV DomU image formatted with ZFS. I compiled
> > FreeBSD with KERNCONF=XEN for the kernel and the normal world and
> > distribution target. Then I transfered it to a Debian Dom0 with Xen
> 4.2.1.
> >
> > I tried to boot the image befor I migrate it to LVM and I now get teh
> > following output:
> >
> > Parsing config from freebsd-test.cfg
> > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 12
> > device model: spawn failed (rc=-3)
> > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error:
> > No such file or directory
> > Daemon running with PID 3451
> > WARNING: loader(8) metadata is missing!
> > GDB: no debug ports present
> > KDB: debugger backends: ddb
> > KDB: current backend: ddb
> > APIC: Using the MPTable enumerator.
> > SMP: Added CPU 0 (BSP)
> > Copyright (c) 1992-2012 The FreeBSD Project.
> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
1994
> > The Regents of the University of California. All rights
reserved.
> > FreeBSD is a registered trademark of The FreeBSD Foundation.
> > FreeBSD 9.1-RELEASE #0: Wed Feb 20 14:16:03 CET 2013
> > root@build:/usr/obj/usr/src/sys/XEN i386
> > WARNING: WITNESS option enabled, expect reduced performance.
> > Xen reported: 3341.754 MHz processor.
> > Timecounter "ixen" frequency 1953125 Hz quality 0
> > CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz
> > 686-class CPU)
> > Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model =
1a
> > Stepping = 5
> >
> >
>
Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
> >
> >
>
Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT>
> > AMD Features=0x28100000<NX,RDTSCP,LM>
> > AMD Features2=0x1<LAHF>
> >
> > Data TLB: 4 KB pages, 4-way set associative, 64 entries
> > 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size
> > L2 cache: 256 kbytes, 8-way associative, 64 bytes/line
> > real memory = 2147483648 (2048 MB)
> > Physical memory chunk(s):
> > 0x0000000000a58000 - 0x000000007d995fff, 2096357376 bytes (511806
pages)
> > avail memory = 2092105728 (1995 MB)
> > INTR: Adding local APIC 0 as a target
> > ULE: setup cpu 0
> > [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0)
> > [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1)
> > Event-channel device installed.
> > io: <I/O>
> > random: <entropy source, Software, Yarrow>
> > mem: <memory>
> > Pentium Pro MTRR support enabled
> > null: <null device, zero device>
> > nfslock: pseudo-device
> > [XEN] xen_rtc_probe: probing Hypervisor RTC clock
> > rtc0: <Xen Hypervisor Clock> on motherboard
> > [XEN] xen_rtc_attach: attaching Hypervisor RTC clock
> > rtc0: registered as a time-of-day clock (resolution 1000000us,
> > adjustment 0.500000000s)
> > xenstore0: <XenStore> on motherboard
> > Grant table initialized
> > xc0: <Xen Console> on motherboard
> > Device configuration finished.
> > procfs registered
> > Event timer "ixen" quality 600
> > Timecounters tick every 10.000 msec
> > lo0: bpf attached
> > xenbusb_front0: <Xen Frontend Devices> on xenstore0
> > xenbusb_add_device: Device device/suspend/event-channel ignored. State
6
> > xenbusb_back0: <Xen Backend Devices> on xenstore0
> > xctrl0: <Xen Control Device> on xenstore0
> > [XEN] hypervisor wallclock nudged; nudging TOD.
> > run_interrupt_driven_hooks: still waiting after 60 seconds for
> > xenbus_free_evtchn
> > run_interrupt_driven_hooks: still waiting after 120 seconds for
> > xenbus_free_evtchn
> > run_interrupt_driven_hooks: still waiting after 180 seconds for
> > xenbus_free_evtchn
> > run_interrupt_driven_hooks: still waiting after 240 seconds for
> > xenbus_free_evtchn
> > run_interrupt_driven_hooks: still waiting after 300 seconds for
> > xenbus_free_evtchn
> > panic: run_interrupt_driven_config_hooks: waited too long
> > cpuid = 0
> > KDB: enter: panic
> > [ thread pid 0 tid 100000 ]
> > Stopped at kdb_enter+0x3a: movl $0,kdb_why
> > db>
>
> Hello,
>
> I can tell you PVHVM (XENHVM) works fine with Xen 4.2.1 and -unstable
> (both 9.1 and HEAD), but I''m afraid I haven''t got time to
test Xen PV
> (i386) yet.
>
> Can you check if this is a regression in the FreeBSD kernel or Xen itself?
>
> Regards, Roger.
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
On 22/02/13 19:30, tech mailinglists wrote:> Hello Roger, > > > I tried a further thing in the kernel configuration. I remove the option > KBD_INSTALL_CDEV and now I am no longer getting dropped to a minimal > console. The boot now looks like this: > > Parsing config from /root/xen/freebsd-test.cfg > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 20 > device model: spawn failed (rc=-3) > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: > No such file or directory > Daemon running with PID 4383 > WARNING: loader(8) metadata is missing! > GDB: no debug ports present > KDB: debugger backends: ddb > KDB: current backend: ddb > APIC: Using the MPTable enumerator. > SMP: Added CPU 0 (BSP) > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 9.1-RELEASE #1 r247145M: Fri Feb 22 18:26:55 CET 2013 > root@build:/usr/obj/usr/src/sys/XEN i386 > Xen reported: 3341.754 MHz processor. > Timecounter "ixen" frequency 1953125 Hz quality 0 > CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz > 686-class CPU) > Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a > Stepping = 5 > > Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > > Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT> > AMD Features=0x28100000<NX,RDTSCP,LM> > AMD Features2=0x1<LAHF> > > Data TLB: 4 KB pages, 4-way set associative, 64 entries > 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size > L2 cache: 256 kbytes, 8-way associative, 64 bytes/line > real memory = 536870912 (512 MB) > Physical memory chunk(s): > 0x0000000000670000 - 0x000000001f628fff, 519802880 bytes (126905 pages) > avail memory = 516833280 (492 MB) > INTR: Adding local APIC 0 as a target > ULE: setup cpu 0 > [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) > [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) > Event-channel device installed. > random: <entropy source, Software, Yarrow> > io: <I/O> > mem: <memory> > Pentium Pro MTRR support enabled > null: <null device, zero device> > nfslock: pseudo-device > [XEN] xen_rtc_probe: probing Hypervisor RTC clock > rtc0: <Xen Hypervisor Clock> on motherboard > [XEN] xen_rtc_attach: attaching Hypervisor RTC clock > rtc0: registered as a time-of-day clock (resolution 1000000us, > adjustment 0.500000000s) > xenstore0: <XenStore> on motherboard > Grant table initialized > xc0: <Xen Console> on motherboard > Device configuration finished. > procfs registered > Event timer "ixen" quality 600 > Timecounters tick every 10.000 msec > lo0: bpf attached > xenbusb_front0: <Xen Frontend Devices> on xenstore0 > xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 > xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0 > xn0: bpf attached > xn0: Ethernet address: 00:16:3e:44:b1:b9 > xenbusb_back0: <Xen Backend Devices> on xenstore0 > xctrl0: <Xen Control Device> on xenstore0 > xn0: backend features: feature-sg feature-gso-tcp4 > [XEN] hypervisor wallclock nudged; nudging TOD. > run_interrupt_driven_hooks: still waiting after 60 seconds for > xenbus_free_evtchn > [XEN] hypervisor wallclock nudged; nudging TOD. > run_interrupt_driven_hooks: still waiting after 120 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 180 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 240 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 300 seconds for > xenbus_free_evtchn > [XEN] hypervisor wallclock nudged; nudging TOD. > [XEN] hypervisor wallclock nudged; nudging TOD. > [XEN] hypervisor wallclock nudged; nudging TOD. > > I will test HVM and PVHVM but in the past I had no problems with this. > It would be greate to have pure PV mode. The only thing which confuses > me is the libxl related error on the domain creation: > > Parsing config from /root/xen/freebsd-test.cfg > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 20 > device model: spawn failed (rc=-3) > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: > No such file or directory > > I know that in a few cases PV guests have QEmu process but I can not say > while this failes. Eventually there is something not compiled in, into > Xen or the toolstack?PV guests use Qemu to provide a block backend for raw file images, so if Qemu fails to start your PV guest won''t we able to access the hard drive, that could be the reason why the FreeBSD PV guest is unable to boot. Could you check Qemu log in /var/log/xen/qemu-dm-freebsd.log?> > I saw that you have made a talk on FOSDEM about BSD and PVH. I > downloaded the slide PDF but I am not able to open the document. It says > that the document is corrupted. Am I doing a mistake? Or is there a > problem with the document on the FOSDEM page.I''ve just checked and the slides look OK to me, I''m able to open them without problems. SHA1 sum: 0b0070149e0b4cf0b5557230793acccd1291ccde new_xen_paravirt_mode.pdf
I was able to fix the Qemu problem. I simply had to install a dependency which was needed by Qemu. The problem doesn''t appeared earlier because the Linux PV guests seem to need no qemu. I think it''s related to the disk notation which is used there (xvda) so kernel xen block driver is used and in the FreeBSD case (hda) so the IDE emulation of qemu is used. I am now have trouble to boot the guest with PV-GRUB. So I have a ZFS disk and I searched on the web and found this on GitHub: https://github.com/aszeszo/pv-grub It''s PV-GRUB with ZFS and it can read the disk but I can not boot. The first thing is how can I cross compile a single component in the Xen build system to get a 32bit variant and not a 64bit variant as yet? And can I load FreeBSD with GRUB actually? I saw that Chainloading would be needed so I would have to boot the loader or zfsloader with GRUB but this failes with the following error: xc: error: panic: xc_dom_core.c:587: xc_dom_find_loader: no loader found: Invalid kernel xc_dom_parse_image returned -1 close(3) Error 9: Unknown boot failure So would be great when there can be found a solution for this problem. Best Regards 2013/2/23 Roger Pau Monné <roger.pau@citrix.com>> On 22/02/13 19:30, tech mailinglists wrote: > > Hello Roger, > > > > > > I tried a further thing in the kernel configuration. I remove the option > > KBD_INSTALL_CDEV and now I am no longer getting dropped to a minimal > > console. The boot now looks like this: > > > > Parsing config from /root/xen/freebsd-test.cfg > > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 20 > > device model: spawn failed (rc=-3) > > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: > > No such file or directory > > Daemon running with PID 4383 > > WARNING: loader(8) metadata is missing! > > GDB: no debug ports present > > KDB: debugger backends: ddb > > KDB: current backend: ddb > > APIC: Using the MPTable enumerator. > > SMP: Added CPU 0 (BSP) > > Copyright (c) 1992-2012 The FreeBSD Project. > > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > > The Regents of the University of California. All rights reserved. > > FreeBSD is a registered trademark of The FreeBSD Foundation. > > FreeBSD 9.1-RELEASE #1 r247145M: Fri Feb 22 18:26:55 CET 2013 > > root@build:/usr/obj/usr/src/sys/XEN i386 > > Xen reported: 3341.754 MHz processor. > > Timecounter "ixen" frequency 1953125 Hz quality 0 > > CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz > > 686-class CPU) > > Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a > > Stepping = 5 > > > > > Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > > > > > Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT> > > AMD Features=0x28100000<NX,RDTSCP,LM> > > AMD Features2=0x1<LAHF> > > > > Data TLB: 4 KB pages, 4-way set associative, 64 entries > > 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size > > L2 cache: 256 kbytes, 8-way associative, 64 bytes/line > > real memory = 536870912 (512 MB) > > Physical memory chunk(s): > > 0x0000000000670000 - 0x000000001f628fff, 519802880 bytes (126905 pages) > > avail memory = 516833280 (492 MB) > > INTR: Adding local APIC 0 as a target > > ULE: setup cpu 0 > > [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) > > [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) > > Event-channel device installed. > > random: <entropy source, Software, Yarrow> > > io: <I/O> > > mem: <memory> > > Pentium Pro MTRR support enabled > > null: <null device, zero device> > > nfslock: pseudo-device > > [XEN] xen_rtc_probe: probing Hypervisor RTC clock > > rtc0: <Xen Hypervisor Clock> on motherboard > > [XEN] xen_rtc_attach: attaching Hypervisor RTC clock > > rtc0: registered as a time-of-day clock (resolution 1000000us, > > adjustment 0.500000000s) > > xenstore0: <XenStore> on motherboard > > Grant table initialized > > xc0: <Xen Console> on motherboard > > Device configuration finished. > > procfs registered > > Event timer "ixen" quality 600 > > Timecounters tick every 10.000 msec > > lo0: bpf attached > > xenbusb_front0: <Xen Frontend Devices> on xenstore0 > > xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 > > xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0 > > xn0: bpf attached > > xn0: Ethernet address: 00:16:3e:44:b1:b9 > > xenbusb_back0: <Xen Backend Devices> on xenstore0 > > xctrl0: <Xen Control Device> on xenstore0 > > xn0: backend features: feature-sg feature-gso-tcp4 > > [XEN] hypervisor wallclock nudged; nudging TOD. > > run_interrupt_driven_hooks: still waiting after 60 seconds for > > xenbus_free_evtchn > > [XEN] hypervisor wallclock nudged; nudging TOD. > > run_interrupt_driven_hooks: still waiting after 120 seconds for > > xenbus_free_evtchn > > run_interrupt_driven_hooks: still waiting after 180 seconds for > > xenbus_free_evtchn > > run_interrupt_driven_hooks: still waiting after 240 seconds for > > xenbus_free_evtchn > > run_interrupt_driven_hooks: still waiting after 300 seconds for > > xenbus_free_evtchn > > [XEN] hypervisor wallclock nudged; nudging TOD. > > [XEN] hypervisor wallclock nudged; nudging TOD. > > [XEN] hypervisor wallclock nudged; nudging TOD. > > > > I will test HVM and PVHVM but in the past I had no problems with this. > > It would be greate to have pure PV mode. The only thing which confuses > > me is the libxl related error on the domain creation: > > > > Parsing config from /root/xen/freebsd-test.cfg > > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 20 > > device model: spawn failed (rc=-3) > > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: > > No such file or directory > > > > I know that in a few cases PV guests have QEmu process but I can not say > > while this failes. Eventually there is something not compiled in, into > > Xen or the toolstack? > > PV guests use Qemu to provide a block backend for raw file images, so if > Qemu fails to start your PV guest won''t we able to access the hard > drive, that could be the reason why the FreeBSD PV guest is unable to > boot. Could you check Qemu log in /var/log/xen/qemu-dm-freebsd.log? > > > > > I saw that you have made a talk on FOSDEM about BSD and PVH. I > > downloaded the slide PDF but I am not able to open the document. It says > > that the document is corrupted. Am I doing a mistake? Or is there a > > problem with the document on the FOSDEM page. > > I''ve just checked and the slides look OK to me, I''m able to open them > without problems. SHA1 sum: > > 0b0070149e0b4cf0b5557230793acccd1291ccde new_xen_paravirt_mode.pdf > >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Am 23.02.2013 09:13, schrieb Roger Pau Monné:> On 22/02/13 19:30, tech mailinglists wrote: >> Hello Roger, >> >> >> I tried a further thing in the kernel configuration. I remove the option >> KBD_INSTALL_CDEV and now I am no longer getting dropped to a minimal >> console. The boot now looks like this: >> >> Parsing config from /root/xen/freebsd-test.cfg >> libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 20 >> device model: spawn failed (rc=-3) >> libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: >> No such file or directory >> Daemon running with PID 4383 >> WARNING: loader(8) metadata is missing! >> GDB: no debug ports present >> KDB: debugger backends: ddb >> KDB: current backend: ddb >> APIC: Using the MPTable enumerator. >> SMP: Added CPU 0 (BSP) >> Copyright (c) 1992-2012 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >> The Regents of the University of California. All rights reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 9.1-RELEASE #1 r247145M: Fri Feb 22 18:26:55 CET 2013 >> root@build:/usr/obj/usr/src/sys/XEN i386 >> Xen reported: 3341.754 MHz processor. >> Timecounter "ixen" frequency 1953125 Hz quality 0 >> CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz >> 686-class CPU) >> Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a >> Stepping = 5 >> >> Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> >> >> Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT> >> AMD Features=0x28100000<NX,RDTSCP,LM> >> AMD Features2=0x1<LAHF> >> >> Data TLB: 4 KB pages, 4-way set associative, 64 entries >> 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size >> L2 cache: 256 kbytes, 8-way associative, 64 bytes/line >> real memory = 536870912 (512 MB) >> Physical memory chunk(s): >> 0x0000000000670000 - 0x000000001f628fff, 519802880 bytes (126905 pages) >> avail memory = 516833280 (492 MB) >> INTR: Adding local APIC 0 as a target >> ULE: setup cpu 0 >> [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) >> [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) >> Event-channel device installed. >> random: <entropy source, Software, Yarrow> >> io: <I/O> >> mem: <memory> >> Pentium Pro MTRR support enabled >> null: <null device, zero device> >> nfslock: pseudo-device >> [XEN] xen_rtc_probe: probing Hypervisor RTC clock >> rtc0: <Xen Hypervisor Clock> on motherboard >> [XEN] xen_rtc_attach: attaching Hypervisor RTC clock >> rtc0: registered as a time-of-day clock (resolution 1000000us, >> adjustment 0.500000000s) >> xenstore0: <XenStore> on motherboard >> Grant table initialized >> xc0: <Xen Console> on motherboard >> Device configuration finished. >> procfs registered >> Event timer "ixen" quality 600 >> Timecounters tick every 10.000 msec >> lo0: bpf attached >> xenbusb_front0: <Xen Frontend Devices> on xenstore0 >> xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 >> xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0 >> xn0: bpf attached >> xn0: Ethernet address: 00:16:3e:44:b1:b9 >> xenbusb_back0: <Xen Backend Devices> on xenstore0 >> xctrl0: <Xen Control Device> on xenstore0 >> xn0: backend features: feature-sg feature-gso-tcp4 >> [XEN] hypervisor wallclock nudged; nudging TOD. >> run_interrupt_driven_hooks: still waiting after 60 seconds for >> xenbus_free_evtchn >> [XEN] hypervisor wallclock nudged; nudging TOD. >> run_interrupt_driven_hooks: still waiting after 120 seconds for >> xenbus_free_evtchn >> run_interrupt_driven_hooks: still waiting after 180 seconds for >> xenbus_free_evtchn >> run_interrupt_driven_hooks: still waiting after 240 seconds for >> xenbus_free_evtchn >> run_interrupt_driven_hooks: still waiting after 300 seconds for >> xenbus_free_evtchn >> [XEN] hypervisor wallclock nudged; nudging TOD. >> [XEN] hypervisor wallclock nudged; nudging TOD. >> [XEN] hypervisor wallclock nudged; nudging TOD. >> >> I will test HVM and PVHVM but in the past I had no problems with this. >> It would be greate to have pure PV mode. The only thing which confuses >> me is the libxl related error on the domain creation: >> >> Parsing config from /root/xen/freebsd-test.cfg >> libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 20 >> device model: spawn failed (rc=-3) >> libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: >> No such file or directory >> >> I know that in a few cases PV guests have QEmu process but I can not say >> while this failes. Eventually there is something not compiled in, into >> Xen or the toolstack? > PV guests use Qemu to provide a block backend for raw file images, so if > Qemu fails to start your PV guest won''t we able to access the hard > drive, that could be the reason why the FreeBSD PV guest is unable to > boot. Could you check Qemu log in /var/log/xen/qemu-dm-freebsd.log? > >> I saw that you have made a talk on FOSDEM about BSD and PVH. I >> downloaded the slide PDF but I am not able to open the document. It says >> that the document is corrupted. Am I doing a mistake? Or is there a >> problem with the document on the FOSDEM page. > I''ve just checked and the slides look OK to me, I''m able to open them > without problems. SHA1 sum: > > 0b0070149e0b4cf0b5557230793acccd1291ccde new_xen_paravirt_mode.pdf >I was able to fix the Qemu related problem, it was simply a missing dependency. After I installed this I could start the guest to the point that it can''t find the root partition (ZFS). I thought that I have to do it with PV-GRUB but the normal PV-GRUB don''t support ZFS so I searched and found this on github: https://github.com/aszeszo/pv-grub I was able to cross compile this to x86_32 and so it can read the ZFS disk. But when I try to boot the kernel from /boot/kernel/kernel I get into the mountroot prompt. And when I try chainloading (I read that this would be the only way) I get this error: xc: error: panic: xc_dom_core.c:587: xc_dom_find_loader: no loader found: Invalid kernel xc_dom_parse_image returned -1 close(3) Error 9: Unknown boot failure So is there a way to parse a kernel parameter to help the kernel to find the ZFS volume or can a loader so like (/boot/loader, /boot/zfsloader) be loaded with PV-GRUB? Would be great when someone could help me I think I am not far away to have a working PV FreeBSD DomU with a filesystem which is support by FreeBSD and Linux so this would be very great. Best Regards
On 21/02/13 10:18, tech mailinglists wrote:> Hello all, > > I have created a FreeBSD PV DomU image formatted with ZFS. I compiled > FreeBSD with KERNCONF=XEN for the kernel and the normal world and > distribution target. Then I transfered it to a Debian Dom0 with Xen 4.2.1. > > I tried to boot the image befor I migrate it to LVM and I now get teh > following output: > > Parsing config from freebsd-test.cfg > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 12 > device model: spawn failed (rc=-3) > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: > No such file or directory > Daemon running with PID 3451 > WARNING: loader(8) metadata is missing! > GDB: no debug ports present > KDB: debugger backends: ddb > KDB: current backend: ddb > APIC: Using the MPTable enumerator. > SMP: Added CPU 0 (BSP) > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 9.1-RELEASE #0: Wed Feb 20 14:16:03 CET 2013 > root@build:/usr/obj/usr/src/sys/XEN i386 > WARNING: WITNESS option enabled, expect reduced performance. > Xen reported: 3341.754 MHz processor. > Timecounter "ixen" frequency 1953125 Hz quality 0 > CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz > 686-class CPU) > Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a > Stepping = 5 > > Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > > Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT> > AMD Features=0x28100000<NX,RDTSCP,LM> > AMD Features2=0x1<LAHF> > > Data TLB: 4 KB pages, 4-way set associative, 64 entries > 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size > L2 cache: 256 kbytes, 8-way associative, 64 bytes/line > real memory = 2147483648 (2048 MB) > Physical memory chunk(s): > 0x0000000000a58000 - 0x000000007d995fff, 2096357376 bytes (511806 pages) > avail memory = 2092105728 (1995 MB) > INTR: Adding local APIC 0 as a target > ULE: setup cpu 0 > [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) > [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) > Event-channel device installed. > io: <I/O> > random: <entropy source, Software, Yarrow> > mem: <memory> > Pentium Pro MTRR support enabled > null: <null device, zero device> > nfslock: pseudo-device > [XEN] xen_rtc_probe: probing Hypervisor RTC clock > rtc0: <Xen Hypervisor Clock> on motherboard > [XEN] xen_rtc_attach: attaching Hypervisor RTC clock > rtc0: registered as a time-of-day clock (resolution 1000000us, > adjustment 0.500000000s) > xenstore0: <XenStore> on motherboard > Grant table initialized > xc0: <Xen Console> on motherboard > Device configuration finished. > procfs registered > Event timer "ixen" quality 600 > Timecounters tick every 10.000 msec > lo0: bpf attached > xenbusb_front0: <Xen Frontend Devices> on xenstore0 > xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 > xenbusb_back0: <Xen Backend Devices> on xenstore0 > xctrl0: <Xen Control Device> on xenstore0 > [XEN] hypervisor wallclock nudged; nudging TOD. > run_interrupt_driven_hooks: still waiting after 60 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 120 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 180 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 240 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 300 seconds for > xenbus_free_evtchn > panic: run_interrupt_driven_config_hooks: waited too long > cpuid = 0 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at kdb_enter+0x3a: movl $0,kdb_why > db>Hello, I''ve been trying to get a FreeBSD PV guest, I''ve installed FreeBSD i386 HEAD and then tried to recompile the kernel using KERNCONF=XEN, but Xen refuses to load the resulting kernel: root@loki:~# xl -vvv create -c freebsd32pv.cfg Parsing config from freebsd32pv.cfg libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x22e5a20: create: how=(nil) callback=(nil) poller=0x22e5a80 libxl: debug: libxl_device.c:229:libxl__device_disk_set_backend: Disk vdev=hda spec.backend=unknown libxl: debug: libxl_device.c:265:libxl__device_disk_set_backend: Disk vdev=hda, using backend phy libxl: debug: libxl_create.c:677:initiate_domain_create: running bootloader libxl: debug: libxl_bootloader.c:327:libxl__bootloader_run: no bootloader configured, using user supplied kernel libxl: debug: libxl_event.c:561:libxl__ev_xswatch_deregister: watch w=0x22e60c0: deregister unregistered libxl: debug: libxl_numa.c:435:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=8, nr_vcpus=15, free_memkb=4353 libxl: detail: libxl_dom.c:192:numa_place_domain: NUMA placement candidate with 1 nodes, 8 cpus and 4353 KB free selected domainbuilder: detail: xc_dom_allocate: cmdline="boot_verbose=1,vfs.root.mountfrom=ufs:/dev/ad0s1a,kern.hz=100", features="(null)" libxl: debug: libxl_dom.c:380:libxl__build_pv: pv kernel mapped 0 path /root/kernel.freebsd domainbuilder: detail: xc_dom_kernel_file: filename="/root/kernel.freebsd" domainbuilder: detail: xc_dom_malloc_filemap : 5235 kB domainbuilder: detail: xc_dom_boot_xen_init: ver 4.2, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe OK xc: detail: elf_parse_binary: phdr: paddr=0xc0000000 memsz=0x439eb8 xc: detail: elf_parse_binary: phdr: paddr=0xc043a000 memsz=0x23a578 xc: detail: elf_parse_binary: memory: 0xc0000000 -> 0xc0674578 xc: detail: elf_xen_parse: __xen_guest: "LOADER=generic,GUEST_OS=freebsd,GUEST_VER=7.0,XEN_VER=xen-3.0,BSD_SYMTAB,VIRT_BASE=0xc0000000" xc: detail: elf_xen_parse_guest_info: LOADER="generic" xc: detail: elf_xen_parse_guest_info: GUEST_OS="freebsd" xc: detail: elf_xen_parse_guest_info: GUEST_VER="7.0" xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0" xc: detail: elf_xen_parse_guest_info: BSD_SYMTAB="" xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0xc0000000" xc: detail: elf_xen_addr_calc_check: ELF_PADDR_OFFSET unset, using 0xc0000000 xc: detail: elf_xen_addr_calc_check: addresses: xc: detail: virt_base = 0xc0000000 xc: detail: elf_paddr_offset = 0xc0000000 xc: detail: virt_offset = 0x0 xc: detail: virt_kstart = 0xc0000000 xc: detail: virt_kend = 0xc07439d0 xc: detail: virt_entry = 0xc0046000 xc: detail: p2m_base = 0xffffffffffffffff domainbuilder: detail: xc_dom_load_elf_symtab/parse: bsd_symtab_start=c07439d0, kernel.end=0xc07439d0 -- symtab=0xc07439d4, maxaddr=0xc0743e40 domainbuilder: detail: xc_dom_load_elf_symtab: h=25 symtab, size=0x4a310, maxaddr=0xc078e150 domainbuilder: detail: xc_dom_load_elf_symtab: h=26 strtab, size=0x69500, maxaddr=0xc07f7650 domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_32: 0xc0000000 -> 0xc07f7650 domainbuilder: detail: xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k each domainbuilder: detail: xc_dom_mem_init: 0x20000 pages domainbuilder: detail: xc_dom_boot_mem_init: called domainbuilder: detail: xc_dom_malloc : 1024 kB domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_alloc_segment: kernel : 0xc0000000 -> 0xc07f8000 (pfn 0x0 + 0x7f8 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x0+0x7f8 at 0x7f88fc22c000 xc: detail: elf_load_binary: phdr 2 at 0x0x7f88fc22c000 -> 0x0x7f88fc665eb8 xc: detail: elf_load_binary: phdr 3 at 0x0x7f88fc666000 -> 0x0x7f88fc694c9c xc: detail: elf_load_bsdsyms: shdr 4 at 0x0x7f88fca4bd84 -> 0x0x7f88fc8a09e8 xc: detail: elf_load_bsdsyms: shdr 24 at 0x0x7f88fce8cec4 -> 0x0x7f88fc8bc0a4 xc: detail: elf_load_bsdsyms: shdr 25 at 0x0x7f88fce8d418 -> 0x0x7f88fc8bc1c0 xc: detail: elf_load_bsdsyms: shdr 26 at 0x0x7f88fced7728 -> 0x0x7f88fc9064d0 domainbuilder: detail: xc_dom_load_elf_symtab/load: bsd_symtab_start=c07439d0, kernel.end=0xc07f8000 -- symtab=0xc07439d4, maxaddr=0xc0743e40 domainbuilder: detail: xc_dom_load_elf_symtab: h=25 symtab, size=0x4a310, maxaddr=0xc078e150 domainbuilder: detail: xc_dom_load_elf_symtab: h=26 strtab, size=0x69500, maxaddr=0xc07f7650 domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0xc07f8000 -> 0xc0878000 (pfn 0x7f8 + 0x80 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x7f8+0x80 at 0x7f88fec35000 domainbuilder: detail: xc_dom_alloc_page : start info : 0xc0878000 (pfn 0x878) domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xc0879000 (pfn 0x879) domainbuilder: detail: xc_dom_alloc_page : console : 0xc087a000 (pfn 0x87a) domainbuilder: detail: nr_page_tables: 0x00000000ffffffff/32: 0x0000000000000000 -> 0x00000000ffffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x00000000003fffff/22: 0x00000000c0000000 -> 0x00000000c0bfffff, 3 table(s) domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xc087b000 -> 0xc087f000 (pfn 0x87b + 0x4 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x87b+0x4 at 0x7f88fedbe000 domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xc087f000 (pfn 0x87f) domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xc0880000 domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0xc0c00000 domainbuilder: detail: xc_dom_boot_image: called domainbuilder: detail: arch_setup_bootearly: doing nothing domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32 domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64 xc: error: panic: xc_dom_boot.c:127: xc_dom_compat_check: guest type xen-3.0-x86_32 not supported by xen kernel, sorry: Invalid kernel It refuses to boot because it detects the kernel doesn''t support PAE. I''ve hacked sys/i386/xen/locore.s in order to add the PAE flag: .ascii "LOADER=generic,GUEST_OS=freebsd,GUEST_VER=7.0,XEN_VER=xen-3.0,BSD_SYMTAB,VIRT_BASE=0xc0000000,PAE=yes[extended-cr3]" I know there''s an elfnote that should tell Xen if kernel support PAE or not, but it seems to be ignored: ELFNOTE(Xen, XEN_ELFNOTE_PAE_MODE, .asciz, "yes") Anyway, I''ve recompiled the kernel with the PAE flag hack, but then when trying to load the resulting kernel it crashes very early: (XEN) d12:v0: unhandled page fault (ec=0002) (XEN) Pagetable walk from 0000000000000100: (XEN) L4[0x000] = 0000000141710027 000000000000087b (XEN) L3[0x000] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S (XEN) Domain 12 (vcpu#0) crashed on cpu#7: (XEN) ----[ Xen-4.2.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 7 (XEN) RIP: e019:[<00000000c00452a0>] (XEN) RFLAGS: 0000000000000246 EM: 1 CONTEXT: pv guest (XEN) rax: 0000000000000100 rbx: 0000000000000000 rcx: 0000000000000000 (XEN) rdx: 00000000c0878000 rsi: 0000000000000000 rdi: 0000000000000000 (XEN) rbp: 00000000c043e12c rsp: 00000000c043e07c r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000013fb58000 cr2: 0000000000000100 (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 (XEN) Guest stack trace from esp=c043e07c: (XEN) 00000002 c00452a0 0001e019 00010046 c03adc1f 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 c0878000 00000000 c0884000 00000000 c0046015 c0878000 c043e138 (XEN) 0000e021 00000040 00000020 c0468d0c 01800000 00000001 c014ffe0 c043e2c4 (XEN) 01800000 0fffffff c01500b0 c043e2c4 01b00000 01000000 c016ef90 c043e2d4 (XEN) 01b00000 01000000 c016f0b0 c0647994 c0647a3c 00000000 ffffffff c0000001 (XEN) c0469810 00000000 c03db795 00000000 c03ecf36 00000000 00000000 c03db81e (XEN) c0469810 00000000 ffffffff 80040006 c0469804 00000000 c03db7f3 c0172990 (XEN) c03db706 00000000 00000000 c03db7fa c0469810 00000000 ffffffff 80040006 (XEN) c043e2c0 00000000 c03db786 c0172990 c03db706 00000000 00000000 c03db7c7 (XEN) c0469810 00000000 ffffffff 80040002 c04697fc 00000000 c03db79d c0172990 (XEN) c03de14d 00000000 00000000 c03db7a8 02380000 0fffffff c004a6b0 00000000 (XEN) c0469810 00000000 ffffffff c0000006 00000000 00000000 c03db643 c004a520 (XEN) c03db706 00000000 00000000 c03db709 c0469810 00000000 ffffffff 80000003 (XEN) 00000000 00000000 c03fb8f6 c004a470 c0433acf 00000000 00000000 c03db6d7 (XEN) 0000c000 00500000 00000000 3454f8e9 c03db791 00000000 c0647994 c03db832 (XEN) 00000000 c0436966 c004baa0 00000000 00000000 00000000 00000000 c0408707 Is Xen i386 PV broken? Or I''m being incredebly stupid and missing something obvious?
On 03/26/13 02:31, Roger Pau Monné wrote:> Is Xen i386 PV broken?Not completely broken, but it''s certainly not in a good state. I believe it''s broken with SMP, for example -- if the "crashed on cpu#7" in your output means cpu#7 from the guest, it would certainly explain things. HVM is the way to go with FreeBSD/Xen. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
On 26/03/13 10:38, Colin Percival wrote:> On 03/26/13 02:31, Roger Pau Monné wrote: >> Is Xen i386 PV broken? > > Not completely broken, but it''s certainly not in a good state. I believe > it''s broken with SMP, for example -- if the "crashed on cpu#7" in your > output means cpu#7 from the guest, it would certainly explain things.My guest only has one vcpu (vcpu#0): Domain 12 (vcpu#0) crashed on cpu#7 That is running on physical CPU 7> HVM is the way to go with FreeBSD/Xen.Yes, I''m already working on that, and got vector callbacks working on both i386 and amd64 HVM guests, thanks to Justin T. Gibbs patch. Now I was trying to boot a PV guest to see how much breakage this change introduced to PV, but I''m not able to make it work, even without my patches. I''ve replied to this xen-users thread because the author seem to have a working FreeBSD DomU PV guest, and I was wondering how he did it. From my POV it seems like PV guests hasn''t been working for a long time, since Xen 3.3 dropped support for non-PAE guests, and the FreeBSD kernel is detected as non-PAE.
On 03/26/13 03:10, Roger Pau Monné wrote:> On 26/03/13 10:38, Colin Percival wrote: >> On 03/26/13 02:31, Roger Pau Monné wrote: >>> Is Xen i386 PV broken? >> >> Not completely broken, but it''s certainly not in a good state. I believe >> it''s broken with SMP, for example -- if the "crashed on cpu#7" in your >> output means cpu#7 from the guest, it would certainly explain things. > > My guest only has one vcpu (vcpu#0):Ok, I wasn''t sure how to parse that output.>> HVM is the way to go with FreeBSD/Xen. > > Yes, I''m already working on that, and got vector callbacks working on > both i386 and amd64 HVM guests, thanks to Justin T. Gibbs patch. Now I > was trying to boot a PV guest to see how much breakage this change > introduced to PV, but I''m not able to make it work, even without my patches. > > I''ve replied to this xen-users thread because the author seem to have a > working FreeBSD DomU PV guest, and I was wondering how he did it. From > my POV it seems like PV guests hasn''t been working for a long time, > since Xen 3.3 dropped support for non-PAE guests, and the FreeBSD kernel > is detected as non-PAE.I had FreeBSD 8.2-RELEASE and a 9.0-CURRENT @ January 2011 running with PV in EC2 (http://www.daemonology.net/freebsd-on-ec2/, look for "t1.micro instances only") and that used PAE. But it''s entirely likely that something got broken in the past two years and nobody noticed because nobody ever uses PV... -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
On 26/03/13 11:14, Colin Percival wrote:> On 03/26/13 03:10, Roger Pau Monné wrote: >> On 26/03/13 10:38, Colin Percival wrote: >>> On 03/26/13 02:31, Roger Pau Monné wrote: >>>> Is Xen i386 PV broken? >>> >>> Not completely broken, but it''s certainly not in a good state. I believe >>> it''s broken with SMP, for example -- if the "crashed on cpu#7" in your >>> output means cpu#7 from the guest, it would certainly explain things. >> >> My guest only has one vcpu (vcpu#0): > > Ok, I wasn''t sure how to parse that output. > >>> HVM is the way to go with FreeBSD/Xen. >> >> Yes, I''m already working on that, and got vector callbacks working on >> both i386 and amd64 HVM guests, thanks to Justin T. Gibbs patch. Now I >> was trying to boot a PV guest to see how much breakage this change >> introduced to PV, but I''m not able to make it work, even without my patches. >> >> I''ve replied to this xen-users thread because the author seem to have a >> working FreeBSD DomU PV guest, and I was wondering how he did it. From >> my POV it seems like PV guests hasn''t been working for a long time, >> since Xen 3.3 dropped support for non-PAE guests, and the FreeBSD kernel >> is detected as non-PAE. > > I had FreeBSD 8.2-RELEASE and a 9.0-CURRENT @ January 2011 running with PV > in EC2 (http://www.daemonology.net/freebsd-on-ec2/, look for "t1.micro > instances only") and that used PAE. But it''s entirely likely that something > got broken in the past two years and nobody noticed because nobody ever uses > PV...I''ve checked and 9.1 is working OK (at least boots and seems to be functional). I''ve also found the reason why HEAD doesn''t work, and it''s probably caused by the switch to clang, which seems to have problems with the ELFNOTE macro, here is a patch that makes i386 PV boot again: --- diff --git a/sys/i386/include/asmacros.h b/sys/i386/include/asmacros.h index c1c3f64..474dfe1 100644 --- a/sys/i386/include/asmacros.h +++ b/sys/i386/include/asmacros.h @@ -211,7 +211,7 @@ #ifdef __STDC__ #define ELFNOTE(name, type, desctype, descdata...) \ -.pushsection .note.name ; \ +.pushsection .note.name,"",@note ; \ .align 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4f - 3f /* descsz */ ; \ @@ -223,7 +223,7 @@ .popsection #else /* !__STDC__, i.e. -traditional */ #define ELFNOTE(name, type, desctype, descdata) \ -.pushsection .note.name ; \ +.pushsection .note.name,"",@note ; \ .align 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4f - 3f /* descsz */ ; \ --- But there''s even more problems... rtc0: [XEN] xen_rtc_gettime rtc0: [XEN] xen_rtc_gettime: wallclock 1364233979 sec; 999999989 nsec rtc0: [XEN] xen_rtc_gettime: uptime 76890 sec; 166126554 nsec rtc0: [XEN] xen_rtc_gettime: TOD 1364310870 sec; 166126543 nsec start_init: trying /sbin/init pid 26 (sh), uid 0: exited on signal 4 panic: removing pages from non-current pmap cpuid = 0 KDB: enter: panic [ thread pid 26 tid 100037 ] Stopped at kdb_enter+0x3d: movl $0,kdb_why db> bt Tracing pid 26 tid 100037 td 0xc25e4900 kdb_enter(c03f8ad8,c03f8ad8,c04321de,ccf7699c,c04321de,...) at kdb_enter+0x3d/frame 0xccf76930 kassert_panic(c047c478,100,c04321de,ccf7699c,ccf7699c,...) at kassert_panic+0x232/frame 0xccf7696c kassert_panic(c04321de,c0431504,e0a,7bc,c044fe94,...) at kassert_panic+0xea/frame 0xccf76990 pmap_remove_pages(c2427638,c064bd74,8,c2427638,c248c8b8,...) at pmap_remove_pages+0xb3/frame 0xccf76a00 vmspace_exit(c25e4900,0,c03f2588,140,4,...) at vmspace_exit+0xb1/frame 0xccf76a28 exit1(c25e4900,4,1a,c248cacc,0,...) at exit1+0x654/frame 0xccf76a88 sigexit(c25e4900,4,c03f8b70,b0a,c0153a4f,...) at sigexit+0xc7f/frame 0xccf76c20 postsig(4,0,c03fee92,10d,0,...) at postsig+0x3b2/frame 0xccf76cdc ast(ccf76d18) at ast+0x388/frame 0xccf76d0c vm86_biosret() at vm86_biosret+0x9c/frame 0xbf7fcba8 --------------050003050001070201070206 Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0; name="0001-xen-fix-ELFNOTE-macro.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-xen-fix-ELFNOTE-macro.patch" From ef73078b37f283401159119538b8ac85bbd512d6 Mon Sep 17 00:00:00 2001 From: Roger Pau Monne <roger.pau@citrix.com> Date: Tue, 26 Mar 2013 12:39:58 +0100 Subject: [PATCH] xen: fix ELFNOTE macro Add an empty flag and the corresponding type of section to pushsection. --- sys/i386/include/asmacros.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/i386/include/asmacros.h b/sys/i386/include/asmacros.h index c1c3f64..474dfe1 100644 --- a/sys/i386/include/asmacros.h +++ b/sys/i386/include/asmacros.h @@ -211,7 +211,7 @@ #ifdef __STDC__ #define ELFNOTE(name, type, desctype, descdata...) \ -.pushsection .note.name ; \ +.pushsection .note.name,"",@note ; \ .align 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4f - 3f /* descsz */ ; \ @@ -223,7 +223,7 @@ .popsection #else /* !__STDC__, i.e. -traditional */ #define ELFNOTE(name, type, desctype, descdata) \ -.pushsection .note.name ; \ +.pushsection .note.name,"",@note ; \ .align 4 ; \ .long 2f - 1f /* namesz */ ; \ .long 4f - 3f /* descsz */ ; \ -- 1.7.7.5 (Apple Git-26) --------------050003050001070201070206 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users --------------050003050001070201070206--