Someone suggested that the following query would have been more properly addressed to xen-devel than xen-users. The short version is that as soon as I turn on paging (CR0|=PG|PE), I loose the ability to do an fxsave, ltr, and several other things in my HVM client. It would appear that Xen is making an assumption about the paging environment that my OS is not satisfying. ----- Forwarded by Aaron Larson/ddci on 11/02/2010 12:55 PM ----- Aaron Larson/ddci wrote on 10/28/2010 03:57:59 PM:> I am attempting to get a 32-bit protected mode OS running under Xen as > an HVM guest, but it is hanging on the execution of fxsave (and > fnsave) and ltr (load task register) instructions. The OS is known to > run on a variety of X86 machines from 386 through Core2 DOU, and also > VMware workstation and userland qemu. I saw some discussion of Xen > fxsave issue, most notably > http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00874.html > but my I''m not trying to step, just execute so my interpretation is my > problem is different (although I certainly could be wrong). I didn''t > see any issues with fnsave or ltr. > > I''m using Xen under OpenSuse 11.1 (Xen 3.3.1), although I''ve also > tried OpenSuse 11.3 (Xen 4.0.0) with identical results. > > Any assistance would be greatly appreciated. > > The code snippet for the fxsave case: > > diagMsg << "\nSaving FP context @" << (DWORD) > (&initialFPcontext) << "..."; > // Ensure buffer is writeable. > ((BYTE*)(&initialFPcontext))[0]=0; > ((BYTE*)(&initialFPcontext))[511]=0; > // Initialze the FPU. > diagMsg << "fninit..."; > asm volatile ("fninit"); > > if (simdContextSavingSupported()) > { > diagMsg << "fxSave..."; > asm volatile ("fxsave (%0)"::"r"(&initialFPcontext)); > diagMsg << "done\n"; > ... > > The results are: > > Saving FP context @C082D470...fninit...fxSave... > > An objdump -d of the binary yields: > > c0811ffb: 0f ae 00 fxsave (%eax) > > "xm list" shows the VM continuing to get CPU cycles: > > # xm list > Name ID Mem VCPUs State Time(s) > Domain-0 0 3855 2 r----- 367.4 > myos 59 16 1 r----- 17.7 > > GDB attached to the running VM shows that the processor is executing > the fxsave instruction and eax is as expected: > > (gdb) p/x $eip > $1 = 0xc0811ffb > (gdb) p/x $eax > $2 = 0xc082d470 > > The /var/log/xen/qemu-dm-myos.log: > domid: 59 > qemu: the number of cpus is 1 > Watching /local/domain/0/device-model/59/logdirty/next-active > Watching /local/domain/0/device-model/59/command > xs_read(): vncpasswd get error. /vm/9ec94cd1-20c5-07da-a1d4- > fdd500b8aef6/vncpasswd. > qemu_map_cache_init nr_buckets = 4000 size 196608 > /usr/src/packages/BUILD/xen-3.3.1-testing/tools/ioemu- > dir/hw/xen_blktap.c:714: Init blktap pipes > shared page at pfn ffe > buffered io page at pfn ffc > Time offset set 0 > Register xen platform. > Done register platform. > medium change watch on `hdc'' (index: 0): /home/alarson/pci-scanner.iso > I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0 > > The fxsave instruction lists the following faults, and I''ve checked > them all. > > Protected Mode faults: > #GP(0) illegal memory operand effective address in segment > register. > #SS(0) illegal address in SS segment > #PF(fault-code) > > The instruction does not have a segment prefix, and the > preceding code ensures the buffer is writable. > > #NM if CR0[ts] or CR0[em]=1 (bits 3 or 2). > > The CR0 is C0000033 > > #UD If CPUID.01H:EDX.FXSR[bit 24] = 0. > > CPUID.01H is EAX=0001067A EBX=00020800 ECX=80082201 EDX=0789FBBF > > EDX[24] is 1 > > #UD If the LOCK prefix is used > > Instruction has no prefixes. > > #AC for alignment > > FP context address is 0xC082D470 (16 byte aligned) > > cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 23 > model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz > stepping : 10 > cpu MHz : 2992.582 > cache size : 6144 KB > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 13 > wp : yes > flags : fpu de tsc msr pae cx8 apic mtrr cmov pat clflush > acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 > bogomips : 5987.06 > clflush size : 64 > power management: > > processor : 1 > [same as for processor 0] > > An xm list -l yields the following for the domain: > (domain > (domid 58) > (bootloader ) > (on_crash preserve) > (uuid ca61601b-4728-d39c-2a4c-4fe0ced6954e) > (bootloader_args ) > (vcpus 1) > (name myos) > (cpus (())) > (on_reboot restart) > (on_poweroff destroy) > (maxmem 16) > (memory 16) > (shadow_memory 2) > (features ) > (on_xend_start ignore) > (on_xend_stop ignore) > (start_time 1288284595.39) > (cpu_time 319.894000271) > (online_vcpus 1) > (image > (hvm > (kernel ) > (args ''VNC_VIEWER=127.0.0.2:5514 '') > (timer_mode 0) > (vnc 1) > (nographic 0) > (hpet 0) > (rtc_timeoffset 0) > (loader /usr/lib/xen/boot/hvmloader) > (pci ()) > (guest_os_type default) > (apic 1) > (sdl 0) > (extid 0) > (hap 0) > (monitor 0) > (device_model /usr/lib/xen/bin/qemu-dm) > (boot d) > (display localhost:10.0) > (pae 0) > (stdvga 0) > (usb 0) > (xauthority /root/.Xauthority) > (isa 0) > (acpi 0) > (vncunused 1) > (localtime 0) > (notes (SUSPEND_CANCEL 1)) > ) > ) > (status 1) > (state --p---) > (store_mfn 4093) > (device > (console > (protocol vt100) > (location 3) > (uuid 9b2b7ffe-8520-8a0b-6a0a-8864bcc3c8c4) > ) > ) > (device > (vbd > (uuid 65beb1fb-b726-a262-8837-6a99d24a45e3) > (bootable 1) > (dev hdc:cdrom) > (uname file:/home/alarson/pci-scanner.iso) > (mode r) > (backend 0) > (bootable 1) > (VDI ) > ) > ) > (device > (vfb > (vncunused 1) > (type vnc) > (uuid 7cb2e9d4-8376-8f6b-7851-6bf7c4058914) > (location 127.0.0.1:5900) > ) > ) > )This message is intended only for the use of the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the sender of this E-Mail by return E-Mail or by telephone. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Since the guest is allegedly continually running, you might try the ''d'' debug key (hit d on Xen serial console) and get execution backtrace for every physical CPU, and see what the hypervisor is spending its time doing for your guest on that instruction. Or add tracing to Xen to see where time is going. Eg. Could be endlessly trying to emulate fxsave in arch/x86/x86_emulate/x86_emulate.c. Or could be taking endless faults via arch/x86/hvm/vmx/vmx.c:vmx_fpu_dirty_intercept(). In short, you need to do some more spadework. -- Keir On 02/11/2010 18:13, "alarson@ddci.com" <alarson@ddci.com> wrote:> Someone suggested that the following query would have been more properly > addressed to xen-devel than xen-users. > > The short version is that as soon as I turn on paging (CR0|=PG|PE), I > loose the ability to do an fxsave, ltr, and several other things in my HVM > client. > > It would appear that Xen is making an assumption about the paging > environment that my OS is not satisfying. > > ----- Forwarded by Aaron Larson/ddci on 11/02/2010 12:55 PM ----- > > Aaron Larson/ddci wrote on 10/28/2010 03:57:59 PM: > >> I am attempting to get a 32-bit protected mode OS running under Xen as >> an HVM guest, but it is hanging on the execution of fxsave (and >> fnsave) and ltr (load task register) instructions. The OS is known to >> run on a variety of X86 machines from 386 through Core2 DOU, and also >> VMware workstation and userland qemu. I saw some discussion of Xen >> fxsave issue, most notably >> http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00874.html >> but my I''m not trying to step, just execute so my interpretation is my >> problem is different (although I certainly could be wrong). I didn''t >> see any issues with fnsave or ltr. >> >> I''m using Xen under OpenSuse 11.1 (Xen 3.3.1), although I''ve also >> tried OpenSuse 11.3 (Xen 4.0.0) with identical results. >> >> Any assistance would be greatly appreciated. >> >> The code snippet for the fxsave case: >> >> diagMsg << "\nSaving FP context @" << (DWORD) >> (&initialFPcontext) << "..."; >> // Ensure buffer is writeable. >> ((BYTE*)(&initialFPcontext))[0]=0; >> ((BYTE*)(&initialFPcontext))[511]=0; >> // Initialze the FPU. >> diagMsg << "fninit..."; >> asm volatile ("fninit"); >> >> if (simdContextSavingSupported()) >> { >> diagMsg << "fxSave..."; >> asm volatile ("fxsave (%0)"::"r"(&initialFPcontext)); >> diagMsg << "done\n"; >> ... >> >> The results are: >> >> Saving FP context @C082D470...fninit...fxSave... >> >> An objdump -d of the binary yields: >> >> c0811ffb: 0f ae 00 fxsave (%eax) >> >> "xm list" shows the VM continuing to get CPU cycles: >> >> # xm list >> Name ID Mem VCPUs State Time(s) >> Domain-0 0 3855 2 r----- 367.4 >> myos 59 16 1 r----- 17.7 >> >> GDB attached to the running VM shows that the processor is executing >> the fxsave instruction and eax is as expected: >> >> (gdb) p/x $eip >> $1 = 0xc0811ffb >> (gdb) p/x $eax >> $2 = 0xc082d470 >> >> The /var/log/xen/qemu-dm-myos.log: >> domid: 59 >> qemu: the number of cpus is 1 >> Watching /local/domain/0/device-model/59/logdirty/next-active >> Watching /local/domain/0/device-model/59/command >> xs_read(): vncpasswd get error. /vm/9ec94cd1-20c5-07da-a1d4- >> fdd500b8aef6/vncpasswd. >> qemu_map_cache_init nr_buckets = 4000 size 196608 >> /usr/src/packages/BUILD/xen-3.3.1-testing/tools/ioemu- >> dir/hw/xen_blktap.c:714: Init blktap pipes >> shared page at pfn ffe >> buffered io page at pfn ffc >> Time offset set 0 >> Register xen platform. >> Done register platform. >> medium change watch on `hdc'' (index: 0): /home/alarson/pci-scanner.iso >> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0 >> >> The fxsave instruction lists the following faults, and I''ve checked >> them all. >> >> Protected Mode faults: >> #GP(0) illegal memory operand effective address in segment >> register. >> #SS(0) illegal address in SS segment >> #PF(fault-code) >> >> The instruction does not have a segment prefix, and the >> preceding code ensures the buffer is writable. >> >> #NM if CR0[ts] or CR0[em]=1 (bits 3 or 2). >> >> The CR0 is C0000033 >> >> #UD If CPUID.01H:EDX.FXSR[bit 24] = 0. >> >> CPUID.01H is EAX=0001067A EBX=00020800 ECX=80082201 EDX=0789FBBF >> >> EDX[24] is 1 >> >> #UD If the LOCK prefix is used >> >> Instruction has no prefixes. >> >> #AC for alignment >> >> FP context address is 0xC082D470 (16 byte aligned) >> >> cat /proc/cpuinfo >> processor : 0 >> vendor_id : GenuineIntel >> cpu family : 6 >> model : 23 >> model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz >> stepping : 10 >> cpu MHz : 2992.582 >> cache size : 6144 KB >> fdiv_bug : no >> hlt_bug : no >> f00f_bug : no >> coma_bug : no >> fpu : yes >> fpu_exception : yes >> cpuid level : 13 >> wp : yes >> flags : fpu de tsc msr pae cx8 apic mtrr cmov pat clflush >> acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1 >> bogomips : 5987.06 >> clflush size : 64 >> power management: >> >> processor : 1 >> [same as for processor 0] >> >> An xm list -l yields the following for the domain: >> (domain >> (domid 58) >> (bootloader ) >> (on_crash preserve) >> (uuid ca61601b-4728-d39c-2a4c-4fe0ced6954e) >> (bootloader_args ) >> (vcpus 1) >> (name myos) >> (cpus (())) >> (on_reboot restart) >> (on_poweroff destroy) >> (maxmem 16) >> (memory 16) >> (shadow_memory 2) >> (features ) >> (on_xend_start ignore) >> (on_xend_stop ignore) >> (start_time 1288284595.39) >> (cpu_time 319.894000271) >> (online_vcpus 1) >> (image >> (hvm >> (kernel ) >> (args ''VNC_VIEWER=127.0.0.2:5514 '') >> (timer_mode 0) >> (vnc 1) >> (nographic 0) >> (hpet 0) >> (rtc_timeoffset 0) >> (loader /usr/lib/xen/boot/hvmloader) >> (pci ()) >> (guest_os_type default) >> (apic 1) >> (sdl 0) >> (extid 0) >> (hap 0) >> (monitor 0) >> (device_model /usr/lib/xen/bin/qemu-dm) >> (boot d) >> (display localhost:10.0) >> (pae 0) >> (stdvga 0) >> (usb 0) >> (xauthority /root/.Xauthority) >> (isa 0) >> (acpi 0) >> (vncunused 1) >> (localtime 0) >> (notes (SUSPEND_CANCEL 1)) >> ) >> ) >> (status 1) >> (state --p---) >> (store_mfn 4093) >> (device >> (console >> (protocol vt100) >> (location 3) >> (uuid 9b2b7ffe-8520-8a0b-6a0a-8864bcc3c8c4) >> ) >> ) >> (device >> (vbd >> (uuid 65beb1fb-b726-a262-8837-6a99d24a45e3) >> (bootable 1) >> (dev hdc:cdrom) >> (uname file:/home/alarson/pci-scanner.iso) >> (mode r) >> (backend 0) >> (bootable 1) >> (VDI ) >> ) >> ) >> (device >> (vfb >> (vncunused 1) >> (type vnc) >> (uuid 7cb2e9d4-8376-8f6b-7851-6bf7c4058914) >> (location 127.0.0.1:5900) >> ) >> ) >> ) > > > This message is intended only for the use of the individual or entity to which > it is addressed. If the reader of this message is not the intended recipient, > or the employee or agent responsible for delivering the message to the > intended recipient, you are hereby notified that any dissemination, > distribution or copying of this message is strictly prohibited. If you have > received this communication in error, please notify us immediately by replying > to the sender of this E-Mail by return E-Mail or by telephone. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
alarson@ddci.com
2010-Nov-03 01:10 UTC
Re: [Xen-devel] fxsave, fnsave, ltr hang for guest OS.
Keir Fraser <keir.xen@gmail.com> wrote on 11/02/2010 03:16:38 PM:> Since the guest is allegedly continually running, you might try the ''d'' > debug key (hit d on Xen serial console) and get execution backtrace for > every physical CPU, and see what the hypervisor is spending its timedoing> for your guest on that instruction. Or add tracing to Xen to see wheretime> is going. Eg. Could be endlessly trying to emulate fxsave in > arch/x86/x86_emulate/x86_emulate.c. Or could be taking endless faultsvia> arch/x86/hvm/vmx/vmx.c:vmx_fpu_dirty_intercept(). > > In short, you need to do some more spadework.Thanks Keir, I was headed in that general direction, but I was hoping to get lucky. I setup the serial cable and did a number of backtraces. I''m hoping that the backtrace for the host is correlated with the client backtrace, but I''m suspicious. Some of the backtraces I got list my client (e.g., on CPU1) but don''t list a CPU1 host state. In all cases my clint is doing what I expect (attempting to execute an fnsave instruction). I slurped the relevant host stacktraces from the console debug log, and I don''t see a pattern (again hoping to get lucky). I''ll put away my private OS shovel, and see if I can start fashioning a Xen shovel. If you see something obvious below that looks suspicious, let me know, otherwise I''ll see what my Xen shovel digs up. Thanks again for your help. Host CPU traces from the CPU''s associated with my OS client. (XEN) Xen call trace: (XEN) [<ffff82c4801ee984>] __mfn_valid+0x44/0x50 (XEN) [<ffff82c4801d513f>] sh_page_fault__guest_2+0x71f/0x1880 (XEN) [<ffff82c4801480ea>] apic_timer_interrupt+0x2a/0x30 (XEN) [<ffff82c4801a7bbb>] hvm_maybe_deassert_evtchn_irq+0x1b/0x60 (XEN) [<ffff82c4801bc973>] vmx_vmexit_handler+0xe33/0x1bb0 (XEN) [<ffff82c480119b2d>] _csched_cpu_pick+0xfd/0x350 (XEN) [<ffff82c4801afbd9>] vpic_update_int_output+0x29/0x80 (XEN) [<ffff82c4801a7b93>] hvm_isa_irq_deassert+0x93/0xa0 (XEN) [<ffff82c4801b0a79>] pt_update_irq+0x159/0x1f0 (XEN) [<ffff82c4801518bd>] vcpu_kick+0x1d/0x80 (XEN) [<ffff82c4801a7af5>] hvm_isa_irq_assert+0x95/0xa0 (XEN) [<ffff82c4801ae2a0>] vlapic_accept_pic_intr+0x20/0x30 (XEN) [<ffff82c4801a8184>] hvm_vcpu_has_pending_irq+0x24/0x60 (XEN) [<ffff82c4801b5cd8>] vmx_intr_assist+0x1d8/0x1f0 (XEN) [<ffff82c4801b5923>] vmx_asm_do_vmentry+0x0/0xdd (XEN) Xen call trace: (XEN) [<ffff82c4801a7a98>] hvm_isa_irq_assert+0x38/0xa0 (XEN) [<ffff82c4801b5b46>] vmx_intr_assist+0x46/0x1f0 (XEN) [<ffff82c4801b5928>] vmx_asm_do_vmentry+0x5/0xdd (XEN) Xen call trace: (XEN) [<ffff82c4801bbc05>] vmx_vmexit_handler+0xc5/0x1bb0 (XEN) [<ffff82c480119b2d>] _csched_cpu_pick+0xfd/0x350 (XEN) [<ffff82c4801afbd9>] vpic_update_int_output+0x29/0x80 (XEN) [<ffff82c4801a7b93>] hvm_isa_irq_deassert+0x93/0xa0 (XEN) [<ffff82c4801b0a79>] pt_update_irq+0x159/0x1f0 (XEN) [<ffff82c4801518bd>] vcpu_kick+0x1d/0x80 (XEN) [<ffff82c4801a7af5>] hvm_isa_irq_assert+0x95/0xa0 (XEN) [<ffff82c4801ae2a0>] vlapic_accept_pic_intr+0x20/0x30 (XEN) [<ffff82c4801a8184>] hvm_vcpu_has_pending_irq+0x24/0x60 (XEN) [<ffff82c4801b5cd8>] vmx_intr_assist+0x1d8/0x1f0 (XEN) [<ffff82c4801b5923>] vmx_asm_do_vmentry+0x0/0xdd (XEN) Xen call trace: (XEN) [<ffff82c4801bbc05>] vmx_vmexit_handler+0xc5/0x1bb0 (XEN) [<ffff82c480119b2d>] _csched_cpu_pick+0xfd/0x350 (XEN) [<ffff82c4801afbd9>] vpic_update_int_output+0x29/0x80 (XEN) [<ffff82c4801a7b93>] hvm_isa_irq_deassert+0x93/0xa0 (XEN) [<ffff82c4801b0a79>] pt_update_irq+0x159/0x1f0 (XEN) [<ffff82c4801518bd>] vcpu_kick+0x1d/0x80 (XEN) [<ffff82c4801a7af5>] hvm_isa_irq_assert+0x95/0xa0 (XEN) [<ffff82c4801ae2a0>] vlapic_accept_pic_intr+0x20/0x30 (XEN) [<ffff82c4801a8184>] hvm_vcpu_has_pending_irq+0x24/0x60 (XEN) [<ffff82c4801b5cd8>] vmx_intr_assist+0x1d8/0x1f0 (XEN) [<ffff82c4801b5923>] vmx_asm_do_vmentry+0x0/0xdd (XEN) Xen call trace: (XEN) [<ffff82c48017bd95>] x86_emulate+0x2e5/0x100b0 (XEN) [<ffff82c480136500>] serial_putc+0x100/0x130 (XEN) [<ffff82c480136166>] __serial_putc+0x86/0x190 (XEN) [<ffff82c4801480ea>] apic_timer_interrupt+0x2a/0x30 (XEN) [<ffff82c480149743>] smp_apic_timer_interrupt+0x53/0xa0 (XEN) [<ffff82c480149743>] smp_apic_timer_interrupt+0x53/0xa0 (XEN) [<ffff82c4801d2daa>] sh_gva_to_gfn__guest_2+0x10a/0x170 (XEN) [<ffff82c4801a051b>] __hvm_copy+0x33b/0x3e0 (XEN) [<ffff82c4801d5c19>] sh_page_fault__guest_2+0x11f9/0x1880 (XEN) [<ffff82c480149743>] smp_apic_timer_interrupt+0x53/0xa0 (XEN) [<ffff82c4801480ea>] apic_timer_interrupt+0x2a/0x30 (XEN) [<ffff82c4801bc973>] vmx_vmexit_handler+0xe33/0x1bb0 (XEN) [<ffff82c480119b2d>] _csched_cpu_pick+0xfd/0x350 (XEN) [<ffff82c4801b0893>] pt_restore_timer+0x23/0xb0 (XEN) [<ffff82c4801afbd9>] vpic_update_int_output+0x29/0x80 (XEN) [<ffff82c4801a7b93>] hvm_isa_irq_deassert+0x93/0xa0 (XEN) [<ffff82c4801b0a79>] pt_update_irq+0x159/0x1f0 (XEN) [<ffff82c4801518bd>] vcpu_kick+0x1d/0x80 (XEN) [<ffff82c4801a7af5>] hvm_isa_irq_assert+0x95/0xa0 (XEN) [<ffff82c4801ae2a0>] vlapic_accept_pic_intr+0x20/0x30 (XEN) [<ffff82c4801a8184>] hvm_vcpu_has_pending_irq+0x24/0x60 (XEN) [<ffff82c4801b5cd8>] vmx_intr_assist+0x1d8/0x1f0 (XEN) [<ffff82c4801b5923>] vmx_asm_do_vmentry+0x0/0xdd Full log: My client can be identified by "cr3: 0000000000a3b000" (XEN) ''d'' pressed -> dumping registers (XEN) (XEN) *** Dumping CPU1 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: e008:[<ffff82c4801ee984>] __mfn_valid+0x44/0x50 (XEN) RFLAGS: 0000000000000286 CONTEXT: hypervisor (XEN) rax: 0000000000000001 rbx: ffff8301273e0000 rcx: 0000000000000000 (XEN) rdx: 0000000000000000 rsi: ffff818000001818 rdi: 000000000011fe04 (XEN) rbp: ffff8300cfa4a000 rsp: ffff830128f6fb20 r8: 00000000001223ed (XEN) r9: 0000000000000000 r10: 0000000000303b90 r11: 00000000001223e8 (XEN) r12: ffff830128f6ff28 r13: 0000000000000006 r14: 0000000000000303 (XEN) r15: ffff830128f6ff28 cr0: 0000000080050033 cr4: 00000000000426f0 (XEN) cr3: 0000000127230000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff830128f6fb20: (XEN) ffff82c4801d513f 00000000c082243e 0000000000000000 0000000000000001 (XEN) 0000000000000000 0000000100000003 ffff8300cfa4bab8 00000000001223ed (XEN) 000000000011ef03 0000000000303b90 ffff8301273e0e30 0000000000000303 (XEN) ffff82c480266de0 ffff830128f6fca8 0000000000000060 ffff830128f6ff28 (XEN) ffff830128f6ff28 ffff818000001818 ffff830128f6fce4 0000000000000000 (XEN) ffffffffffffffff 0000000000000000 000000008046fe60 0000000000000c0c (XEN) ffff830128f6fcc8 ffff830128f6ff28 0000002000000020 0000000100000000 (XEN) 8dc0821d676836dd b5e850ffffff5885 f68910c483ffff37 1fe58955f689feeb (XEN) 00000000c081745c 0c9b000800000003 00000000ffffffff 0c93001000000000 (XEN) 00000000ffffffff ffff82c400000000 ffff830128f6ff28 ffff8300cfa4a000 (XEN) ffff82c4801480ea 000000000000000e 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff8300cfa4a000 ffff830128f6ff28 0000000000000000 (XEN) 0000000000000000 0000000000303b90 0030306500a04c67 000000000011fe3b (XEN) 000000000011fe04 00000000c082243e 0000000000303b90 000000011ef03025 (XEN) 0000000100000000 ffff82c4801a7bbb ffff830128f6ff28 ffff8300cfa4a000 (XEN) 0000000000303b90 0000000000000000 0000000000000000 000000000000000e (XEN) ffff82c4801bc973 0000000000000000 0000000000000001 ffff830128f6fdf8 (XEN) ffff830128f6fd98 ffff830128f6fdd8 ffff82c480119b2d ffff82c400000001 (XEN) ffff82c480266ee0 ffff82c480266f00 ffff82c480266f00 ffff82c480266080 (XEN) 00ff82c4801480ea ffff82c480266ee0 ffff82c480266080 0000000000000001 (XEN) Xen call trace: (XEN) [<ffff82c4801ee984>] __mfn_valid+0x44/0x50 (XEN) [<ffff82c4801d513f>] sh_page_fault__guest_2+0x71f/0x1880 (XEN) [<ffff82c4801480ea>] apic_timer_interrupt+0x2a/0x30 (XEN) [<ffff82c4801a7bbb>] hvm_maybe_deassert_evtchn_irq+0x1b/0x60 (XEN) [<ffff82c4801bc973>] vmx_vmexit_handler+0xe33/0x1bb0 (XEN) [<ffff82c480119b2d>] _csched_cpu_pick+0xfd/0x350 (XEN) [<ffff82c4801afbd9>] vpic_update_int_output+0x29/0x80 (XEN) [<ffff82c4801a7b93>] hvm_isa_irq_deassert+0x93/0xa0 (XEN) [<ffff82c4801b0a79>] pt_update_irq+0x159/0x1f0 (XEN) [<ffff82c4801518bd>] vcpu_kick+0x1d/0x80 (XEN) [<ffff82c4801a7af5>] hvm_isa_irq_assert+0x95/0xa0 (XEN) [<ffff82c4801ae2a0>] vlapic_accept_pic_intr+0x20/0x30 (XEN) [<ffff82c4801a8184>] hvm_vcpu_has_pending_irq+0x24/0x60 (XEN) [<ffff82c4801b5cd8>] vmx_intr_assist+0x1d8/0x1f0 (XEN) [<ffff82c4801b5923>] vmx_asm_do_vmentry+0x0/0xdd (XEN) (XEN) *** Dumping CPU1 guest state (d1:v0): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: 0008:[<00000000c081745c>] (XEN) RFLAGS: 0000000000010046 CONTEXT: hvm guest (XEN) rax: 0000000000303d9c rbx: 00000000c0821d53 rcx: 00000000000b8b98 (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: 0000000000303d9c (XEN) rbp: 0000000000303e44 rsp: 0000000000303904 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000011 cr4: 0000000000000200 (XEN) cr3: 0000000000a3b000 cr2: 0000000000000000 (XEN) ds: 0010 es: 0010 fs: 0010 gs: 0010 ss: 0010 cs: 0008 (XEN) (XEN) *** Dumping CPU0 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff82c48011faa2>] _spin_lock+0x12/0x20 (XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor (XEN) rax: 0000000000000000 rbx: ffff8301273e0000 rcx: 0000000000000000 (XEN) rdx: 0000000000000000 rsi: 0000000000000000 rdi: ffff8301273e0e30 (XEN) rbp: 0000000000000000 rsp: ffff82c48046fde0 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: ffff82c480204330 r11: 0000000000000282 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: ffff8800c903b218 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff82c48046fde0: (XEN) ffff82c4801d2142 0000000000000002 ffff82c48014fdbc 0000000000000000 (XEN) ffff82c4802660a8 ffff82c480121598 ffff8301273e0e30 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 000000000000140e (XEN) ffffffffffffffea 0000000000000006 00007ffffa3015f0 ffff830128f40000 (XEN) ffffffff802e2770 ffff8800ee7d47c0 ffff82c4801a4a4c 0000000001c9c380 (XEN) 000002515ca4324d ffff830128f60330 ffff8300cfdfa000 0000000000000001 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000206 ffffffff800073d0 0000000000000100 00007f71258806b7 (XEN) ffff8301273e0000 0000000000000206 ffff8300cfdfa000 ffffffff80003440 (XEN) 0000000000305000 0000000000000000 ffffffff802e2770 ffff8800ee7d47c0 (XEN) ffff82c4801ee169 ffff8800ee7d47c0 ffffffff802e2770 0000000000000000 (XEN) 0000000000305000 ffffffff80003440 ffff8800ce3c7e38 0000000000000282 (XEN) 0000000000b00b60 0000000000000000 0000000000000001 0000000000000022 (XEN) ffffffff8000344a 0000000000000001 00007ffffa3015f0 0000000000000006 (XEN) 0000010000000000 ffffffff8000344a 000000000000e033 0000000000000282 (XEN) ffff8800ce3c7db0 000000000000e02b 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff8300cfdfa000 (XEN) Xen call trace: (XEN) [<ffff82c48011faa2>] _spin_lock+0x12/0x20 (XEN) [<ffff82c4801d2142>] shadow_track_dirty_vram+0xd2/0x600 (XEN) [<ffff82c48014fdbc>] context_switch+0x18c/0xec0 (XEN) [<ffff82c480121598>] add_entry+0x38/0xa0 (XEN) [<ffff82c4801a4a4c>] do_hvm_op+0x8dc/0x11b0 (XEN) [<ffff82c4801ee169>] syscall_enter+0xa9/0xae (XEN) (XEN) *** Dumping CPU0 guest state (d0:v1): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e033:[<ffffffff8000344a>] (XEN) RFLAGS: 0000000000000282 EM: 0 CONTEXT: pv guest (XEN) rax: 0000000000000022 rbx: ffff8800ce3c7e38 rcx: ffffffff8000344a (XEN) rdx: 0000000000000001 rsi: 00007ffffa3015f0 rdi: 0000000000000006 (XEN) rbp: ffffffff80003440 rsp: ffff8800ce3c7db0 r8: 0000000000000001 (XEN) r9: 0000000000000000 r10: 0000000000b00b60 r11: 0000000000000282 (XEN) r12: 0000000000305000 r13: 0000000000000000 r14: ffffffff802e2770 (XEN) r15: ffff8800ee7d47c0 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: 00007f9c5e443000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033 (XEN) Guest stack trace from rsp=ffff8800ce3c7db0: (XEN) 0000000000000246 0000000000000000 ffffffff802e288d ffff8800ce3e8ca0 (XEN) ffff8800cc3ebf00 ffff880002024c80 ffff8800cc3ebf00 ffff8800ce3e8c60 (XEN) ffffffff800f3eb3 ffff8800ce3c7ed8 25602560ce3c7ed8 0000000000000022 (XEN) 0000000000000006 00007ffffa3015f0 0000000000000001 0000000000b00b60 (XEN) 0000000000000001 ffff8800ce3c7e38 ffff8800ce3c7e38 ffff8800cc3ebf00 (XEN) ffff8800010db140 ffff8800cc3386c0 0000000000305000 0000000000000000 (XEN) ffffffff802e2770 00007ffffa3015c0 ffffffff8016d134 ffff8800ce3c7f30 (XEN) ffff8800010db1b8 00007ffffa301000 ffff8800cc3386c0 0000000000305000 (XEN) 00007ffffa3015c0 00007ffffa3015c0 0000000000000011 0000000000000001 (XEN) ffffffff80123ac9 ffff8800cc3386c0 0000000000000005 0000000000305000 (XEN) ffffffff80124060 0000000000000001 ffff8800ce3c7f30 00007ffffa302000 (XEN) 0000000000000011 ffff8800cc3386c0 0000000000000005 0000000000305000 (XEN) ffffffff80124460 00007ffffa301000 0000000000001000 00007ffffa301000 (XEN) 00000001e0744840 0000000000ae4448 0000000000000005 0000000000000000 (XEN) 00007ffffa3015f0 0000000000ae4448 ffffffff80007438 0000000000000246 (XEN) 00007ffffa301740 0000000000000000 0000000000000000 0000000000000010 (XEN) ffffffff800073d0 00007ffffa3015c0 0000000000305000 0000000000000005 (XEN) 0000000000000010 00007f712587be87 000000000000e033 0000000000000246 (XEN) 00007ffffa3015b8 000000000000e02b (XEN) (XEN) ''d'' pressed -> dumping registers (XEN) (XEN) *** Dumping CPU1 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: e008:[<ffff82c4801a7a98>] hvm_isa_irq_assert+0x38/0xa0 (XEN) RFLAGS: 0000000000000202 CONTEXT: hypervisor (XEN) rax: 0000000000000000 rbx: ffff8301273e0000 rcx: 0000000000000001 (XEN) rdx: 0000000000000001 rsi: 0000000000000000 rdi: ffff8301273e09a8 (XEN) rbp: 0000000000000000 rsp: ffff830128f6fe98 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000002 r11: 0000000000000000 (XEN) r12: ffff8301273e09a8 r13: 0000000000000002 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080050033 cr4: 00000000000426f0 (XEN) cr3: 0000000127230000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff830128f6fe98: (XEN) 0000000000303e44 ffff8300cfa4a000 0000000000303e44 0000000000000000 (XEN) 0000000000004016 ffff82c4801b5b46 ffffffffffffffff ffff830128f6ff00 (XEN) ffff82c480483280 0000000000000000 0000000000000000 ffff8300cfa4a000 (XEN) 0000000000303e44 ffff8300cfa4a000 0000000000303e44 0000000000000000 (XEN) 0000000000000000 ffff82c4801b5928 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000303e44 00000000c0821d53 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000303d9c 00000000000b8b98 00000000c082243e 0000000000303b90 (XEN) 0000000000303d9c 000000f100000003 00000000c081745c 0000000000000000 (XEN) 0000000000010046 0000000000303904 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000001 (XEN) ffff8300cfa4a000 (XEN) Xen call trace: (XEN) [<ffff82c4801a7a98>] hvm_isa_irq_assert+0x38/0xa0 (XEN) [<ffff82c4801b5b46>] vmx_intr_assist+0x46/0x1f0 (XEN) [<ffff82c4801b5928>] vmx_asm_do_vmentry+0x5/0xdd (XEN) (XEN) *** Dumping CPU1 guest state (d1:v0): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: 0008:[<00000000c081745c>] (XEN) RFLAGS: 0000000000010046 CONTEXT: hvm guest (XEN) rax: 0000000000303d9c rbx: 00000000c0821d53 rcx: 00000000000b8b98 (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: 0000000000303d9c (XEN) rbp: 0000000000303e44 rsp: 0000000000303904 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000011 cr4: 0000000000000200 (XEN) cr3: 0000000000a3b000 cr2: 0000000000000000 (XEN) ds: 0010 es: 0010 fs: 0010 gs: 0010 ss: 0010 cs: 0008 (XEN) (XEN) *** Dumping CPU0 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor (XEN) rax: 0000000000000003 rbx: ffff830128f26c20 rcx: ffff82c480483280 (XEN) rdx: 0000000000000808 rsi: 0000000000000000 rdi: ffff830128f26c80 (XEN) rbp: ffff830128f26c80 rsp: ffff82c48046fe78 r8: ffff830128f60350 (XEN) r9: 000002534ef40b7f r10: 0000000000000000 r11: ffff82c4804ad5c0 (XEN) r12: 00000000002c1327 r13: ffff82c48046fe18 r14: ffff8300cfdfc000 (XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: ffff8800c903b218 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff82c48046fe78: (XEN) ffff82c48018f1e2 ffff82c4802660a8 ffff8300cfdfc000 ffffffffffffffff (XEN) ffff830128f60330 ffff82c48012181a ffff8300cfdfc000 0000000000000000 (XEN) 0000000000000000 ffff82c480483280 00001aa200002221 ffff82c48046ff28 (XEN) ffff82c48046ff28 ffff82c480266c00 ffff82c48046ff28 ffff82c48046fe18 (XEN) ffff8300cfdfc000 0000000000000002 ffff82c48014d73e ffff8300cfdfc000 (XEN) ffff8300cfc66000 00000000ffffffff 0000000000000000 0000000000000000 (XEN) ffffffffffffffff ffffffff806f5fd8 ffffffff80751100 0000000000000000 (XEN) 0000000000000246 0000000000000001 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffffffff800033aa 0000000000000000 0000000000000000 (XEN) 0000000000000001 0000010000000000 ffffffff800033aa 000000000000e033 (XEN) 0000000000000246 ffffffff806f5f58 000000000000e02b 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff8300cfdfc000 (XEN) Xen call trace: (XEN) [<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) [<ffff82c48018f1e2>] acpi_processor_idle+0x2d2/0x6d0 (XEN) [<ffff82c48012181a>] timer_softirq_action+0x21a/0x360 (XEN) [<ffff82c48014d73e>] idle_loop+0x2e/0x90 (XEN) (XEN) No guest context (CPU0 is idle). (XEN) (XEN) ''d'' pressed -> dumping registers (XEN) *** Dumping CPU1 guest state (d1:v0): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: 0008:[<00000000c081745c>] (XEN) RFLAGS: 0000000000010046 CONTEXT: hvm guest (XEN) rax: 0000000000303d9c rbx: 00000000c0821d53 rcx: 00000000000b8b98 (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: 0000000000303d9c (XEN) rbp: 0000000000303e44 rsp: 0000000000303904 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000011 cr4: 0000000000000200 (XEN) cr3: 0000000000a3b000 cr2: 0000000000000000 (XEN) ds: 0010 es: 0010 fs: 0010 gs: 0010 ss: 0010 cs: 0008 (XEN) (XEN) *** Dumping CPU0 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor (XEN) rax: 0000000000000003 rbx: ffff830128f26c20 rcx: ffff82c480483280 (XEN) rdx: 0000000000000808 rsi: 0000000000000000 rdi: ffff830128f26c80 (XEN) rbp: ffff830128f26c80 rsp: ffff82c48046fe78 r8: ffff830128f60348 (XEN) r9: 00000254bcdb0417 r10: 0000000000000000 r11: ffff8300cfdfa060 (XEN) r12: 00000000009fcd27 r13: ffff82c48046fe18 r14: ffff8300cfdfc000 (XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: ffff8800c903b218 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff82c48046fe78: (XEN) ffff82c48018f1e2 ffff82c4802660a8 ffff8300cfdfc000 ffffffffffffffff (XEN) ffff830128f60330 ffff82c48012181a ffff8300cfdfc000 0000000000000000 (XEN) 0000000000000000 ffff82c480483280 0000042a00001ecf ffff82c48046ff28 (XEN) ffff82c48046ff28 ffff82c480266c00 ffff82c48046ff28 ffff82c48046fe18 (XEN) ffff8300cfdfc000 0000000000000002 ffff82c48014d73e ffff8300cfdfc000 (XEN) ffff8300cfc66000 00000000ffffffff 0000000000000000 0000000000000000 (XEN) ffffffffffffffff ffffffff806f5fd8 ffffffff80751100 0000000000000000 (XEN) 0000000000000246 0000000000000001 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffffffff800033aa 0000000000000000 0000000000000000 (XEN) 0000000000000001 0000010000000000 ffffffff800033aa 000000000000e033 (XEN) 0000000000000246 ffffffff806f5f58 000000000000e02b 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff8300cfdfc000 (XEN) Xen call trace: (XEN) [<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) [<ffff82c48018f1e2>] acpi_processor_idle+0x2d2/0x6d0 (XEN) [<ffff82c48012181a>] timer_softirq_action+0x21a/0x360 (XEN) [<ffff82c48014d73e>] idle_loop+0x2e/0x90 (XEN) (XEN) No guest context (CPU0 is idle). (XEN) (XEN) ''d'' pressed -> dumping registers (XEN) *** Dumping CPU1 guest state (d1:v0): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: 0008:[<00000000c081745c>] (XEN) RFLAGS: 0000000000010046 CONTEXT: hvm guest (XEN) rax: 0000000000303d9c rbx: 00000000c0821d53 rcx: 00000000000b8b98 (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: 0000000000303d9c (XEN) rbp: 0000000000303e44 rsp: 0000000000303904 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000011 cr4: 0000000000000200 (XEN) cr3: 0000000000a3b000 cr2: 0000000000000000 (XEN) ds: 0010 es: 0010 fs: 0010 gs: 0010 ss: 0010 cs: 0008 (XEN) (XEN) *** Dumping CPU0 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor (XEN) rax: 0000000000000003 rbx: ffff830128f26c20 rcx: ffff82c480483280 (XEN) rdx: 0000000000000808 rsi: 0000000000000000 rdi: ffff830128f26c80 (XEN) rbp: ffff830128f26c80 rsp: ffff82c48046fe78 r8: ffff830128f60348 (XEN) r9: 00000256997a35e6 r10: 0000000000000000 r11: ffff82c4804b20e0 (XEN) r12: 000000000053e293 r13: ffff82c48046fe18 r14: ffff8300cfdfc000 (XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: ffff8800c9093168 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff82c48046fe78: (XEN) ffff82c48018f1e2 ffff82c4802660a8 ffff8300cfdfc000 ffffffffffffffff (XEN) ffff830128f60330 ffff82c48012181a ffff8300cfdfc000 0000000000000000 (XEN) 0000000000000000 ffff82c480483280 00001b8000002547 ffff82c48046ff28 (XEN) ffff82c48046ff28 ffff82c480266c00 ffff82c48046ff28 ffff82c48046fe18 (XEN) ffff8300cfdfc000 0000000000000002 ffff82c48014d73e ffff8300cfdfc000 (XEN) ffff8300cfc66000 00000000ffffffff 0000000000000000 0000000000000000 (XEN) ffffffffffffffff ffffffff806f5fd8 ffffffff80751100 0000000000000000 (XEN) 0000000000000246 0000000000000001 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffffffff800033aa 0000000000000000 0000000000000000 (XEN) 0000000000000001 0000010000000000 ffffffff800033aa 000000000000e033 (XEN) 0000000000000246 ffffffff806f5f58 000000000000e02b 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff8300cfdfc000 (XEN) Xen call trace: (XEN) [<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) [<ffff82c48018f1e2>] acpi_processor_idle+0x2d2/0x6d0 (XEN) [<ffff82c48012181a>] timer_softirq_action+0x21a/0x360 (XEN) [<ffff82c48014d73e>] idle_loop+0x2e/0x90 (XEN) (XEN) No guest context (CPU0 is idle). (XEN) (XEN) ''d'' pressed -> dumping registers (XEN) (XEN) *** Dumping CPU1 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: e008:[<ffff82c4801bbc05>] vmx_vmexit_handler+0xc5/0x1bb0 (XEN) RFLAGS: 0000000000000287 CONTEXT: hypervisor (XEN) rax: 0000000000004404 rbx: ffff830128f6ff28 rcx: 0000000080000b0e (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: ffff830128f6ff28 (XEN) rbp: ffff8300cfa4a000 rsp: ffff830128f6fd28 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 000000000000000e cr0: 0000000080050033 cr4: 00000000000426f0 (XEN) cr3: 0000000127230000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff830128f6fd28: (XEN) 0000000000000004 0000000000000001 ffff830128f6fdf8 ffff830128f6fd98 (XEN) ffff830128f6fdd8 ffff82c480119b2d ffff830100000001 ffff82c480266ee0 (XEN) ffff82c480266f00 ffff82c480266f00 ffff82c480266080 00ff82c4801480ea (XEN) ffff82c480266ee0 ffff82c480266080 0000000000000001 0000000080000b0e (XEN) 0000000000000000 0000000000000000 0000000000000002 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000003 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff82c4801afbd9 ffffffffffff8000 ffff8301273e0000 (XEN) ffff8301273e09a8 ffff82c4801a7b93 ffff83012bc78748 ffff8300cfa4a000 (XEN) 0000000000000000 0000000000000001 ffff8300cfa4a000 ffff82c4801b0a79 (XEN) 0000000000000002 ffff82c4801518bd 0000000000000000 ffff8301273e0000 (XEN) 0000000000000000 ffff82c4801a7af5 ffff82c4801ae2a0 ffff82c4801a8184 (XEN) ffff8300cfa4a000 ffff8300cfa4a000 0000000000000001 ffff82c4801b5cd8 (XEN) ffffffffffffffff ffff830128f6ff00 ffff82c480483280 0000000000000000 (XEN) 0000000000000000 ffff8300cfa4a000 0000000000303e44 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff82c4801b5923 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000303e44 00000000c0821d53 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000303d9c 00000000000b8b98 (XEN) 00000000c082243e 0000000000303b90 0000000000303d9c 000000f100000003 (XEN) Xen call trace: (XEN) [<ffff82c4801bbc05>] vmx_vmexit_handler+0xc5/0x1bb0 (XEN) [<ffff82c480119b2d>] _csched_cpu_pick+0xfd/0x350 (XEN) [<ffff82c4801afbd9>] vpic_update_int_output+0x29/0x80 (XEN) [<ffff82c4801a7b93>] hvm_isa_irq_deassert+0x93/0xa0 (XEN) [<ffff82c4801b0a79>] pt_update_irq+0x159/0x1f0 (XEN) [<ffff82c4801518bd>] vcpu_kick+0x1d/0x80 (XEN) [<ffff82c4801a7af5>] hvm_isa_irq_assert+0x95/0xa0 (XEN) [<ffff82c4801ae2a0>] vlapic_accept_pic_intr+0x20/0x30 (XEN) [<ffff82c4801a8184>] hvm_vcpu_has_pending_irq+0x24/0x60 (XEN) [<ffff82c4801b5cd8>] vmx_intr_assist+0x1d8/0x1f0 (XEN) [<ffff82c4801b5923>] vmx_asm_do_vmentry+0x0/0xdd (XEN) (XEN) *** Dumping CPU1 guest state (d1:v0): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: 0008:[<00000000c081745c>] (XEN) RFLAGS: 0000000000010046 CONTEXT: hvm guest (XEN) rax: 0000000000303d9c rbx: 00000000c0821d53 rcx: 00000000000b8b98 (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: 0000000000303d9c (XEN) rbp: 0000000000303e44 rsp: 0000000000303904 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000011 cr4: 0000000000000200 (XEN) cr3: 0000000000a3b000 cr2: 0000000000000000 (XEN) ds: 0010 es: 0010 fs: 0010 gs: 0010 ss: 0010 cs: 0008 (XEN) (XEN) *** Dumping CPU0 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor (XEN) rax: 0000000000000003 rbx: ffff830128f26c20 rcx: ffff82c480483280 (XEN) rdx: 0000000000000808 rsi: 0000000000000000 rdi: ffff830128f26c80 (XEN) rbp: ffff830128f26c80 rsp: ffff82c48046fe78 r8: ffff830128f60348 (XEN) r9: 00000257e2bb7c5e r10: 0000000000000000 r11: ffff8300cfdfa060 (XEN) r12: 000000000081459b r13: ffff82c48046fe18 r14: ffff8300cfdfc000 (XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: ffff8800c903b218 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff82c48046fe78: (XEN) ffff82c48018f1e2 ffff82c4802660a8 ffff8300cfdfc000 ffffffffffffffff (XEN) ffff830128f60330 ffff82c48012181a ffff8300cfdfc000 0000000000000000 (XEN) 0000000000000000 ffff82c480483280 0000239f000029ff ffff82c48046ff28 (XEN) ffff82c48046ff28 ffff82c480266c00 ffff82c48046ff28 ffff82c48046fe18 (XEN) ffff8300cfdfc000 0000000000000002 ffff82c48014d73e ffff8300cfdfc000 (XEN) ffff8300cfc66000 00000000ffffffff 0000000000000000 0000000000000000 (XEN) ffffffffffffffff ffffffff806f5fd8 ffffffff80751100 0000000000000000 (XEN) 0000000000000246 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffffffff800033aa 0000000000000000 0000000000000000 (XEN) 0000000000000001 0000010000000000 ffffffff800033aa 000000000000e033 (XEN) 0000000000000246 ffffffff806f5f58 000000000000e02b 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff8300cfdfc000 (XEN) Xen call trace: (XEN) [<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) [<ffff82c48018f1e2>] acpi_processor_idle+0x2d2/0x6d0 (XEN) [<ffff82c48012181a>] timer_softirq_action+0x21a/0x360 (XEN) [<ffff82c48014d73e>] idle_loop+0x2e/0x90 (XEN) (XEN) No guest context (CPU0 is idle). (XEN) (XEN) ''d'' pressed -> dumping registers (XEN) (XEN) *** Dumping CPU1 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: e008:[<ffff82c4801bbc05>] vmx_vmexit_handler+0xc5/0x1bb0 (XEN) RFLAGS: 0000000000000287 CONTEXT: hypervisor (XEN) rax: 0000000000004404 rbx: ffff830128f6ff28 rcx: 0000000080000b0e (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: ffff830128f6ff28 (XEN) rbp: ffff8300cfa4a000 rsp: ffff830128f6fd28 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 000000000000000e cr0: 0000000080050033 cr4: 00000000000426f0 (XEN) cr3: 0000000127230000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff830128f6fd28: (XEN) 0000000000000004 0000000000000001 ffff830128f6fdf8 ffff830128f6fd98 (XEN) ffff830128f6fdd8 ffff82c480119b2d ffff830100000001 ffff82c480266ee0 (XEN) ffff82c480266f00 ffff82c480266f00 ffff82c480266080 0000000000000000 (XEN) ffff82c480266ee0 ffff82c480266080 0000000000000001 0000000080000b0e (XEN) 0000000000000000 0000000000000000 0000000000000002 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000003 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff82c4801afbd9 ffffffffffff8000 ffff8301273e0000 (XEN) ffff8301273e09a8 ffff82c4801a7b93 ffff83012bc78748 ffff8300cfa4a000 (XEN) 0000000000000000 0000000000000001 ffff8300cfa4a000 ffff82c4801b0a79 (XEN) 0000000000000002 ffff82c4801518bd 0000000000000000 ffff8301273e0000 (XEN) 0000000000000000 ffff82c4801a7af5 ffff82c4801ae2a0 ffff82c4801a8184 (XEN) ffff8300cfa4a000 ffff8300cfa4a000 0000000000000001 ffff82c4801b5cd8 (XEN) ffffffffffffffff ffff830128f6ff00 ffff82c480483280 0000000000000000 (XEN) 0000000000000000 ffff8300cfa4a000 0000000000303e44 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff82c4801b5923 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000303e44 00000000c0821d53 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000303d9c 00000000000b8b98 (XEN) 00000000c082243e 0000000000303b90 0000000000303d9c 000000f100000003 (XEN) Xen call trace: (XEN) [<ffff82c4801bbc05>] vmx_vmexit_handler+0xc5/0x1bb0 (XEN) [<ffff82c480119b2d>] _csched_cpu_pick+0xfd/0x350 (XEN) [<ffff82c4801afbd9>] vpic_update_int_output+0x29/0x80 (XEN) [<ffff82c4801a7b93>] hvm_isa_irq_deassert+0x93/0xa0 (XEN) [<ffff82c4801b0a79>] pt_update_irq+0x159/0x1f0 (XEN) [<ffff82c4801518bd>] vcpu_kick+0x1d/0x80 (XEN) [<ffff82c4801a7af5>] hvm_isa_irq_assert+0x95/0xa0 (XEN) [<ffff82c4801ae2a0>] vlapic_accept_pic_intr+0x20/0x30 (XEN) [<ffff82c4801a8184>] hvm_vcpu_has_pending_irq+0x24/0x60 (XEN) [<ffff82c4801b5cd8>] vmx_intr_assist+0x1d8/0x1f0 (XEN) [<ffff82c4801b5923>] vmx_asm_do_vmentry+0x0/0xdd (XEN) (XEN) *** Dumping CPU1 guest state (d1:v0): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: 0008:[<00000000c081745c>] (XEN) RFLAGS: 0000000000010046 CONTEXT: hvm guest (XEN) rax: 0000000000303d9c rbx: 00000000c0821d53 rcx: 00000000000b8b98 (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: 0000000000303d9c (XEN) rbp: 0000000000303e44 rsp: 0000000000303904 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000011 cr4: 0000000000000200 (XEN) cr3: 0000000000a3b000 cr2: 0000000000000000 (XEN) ds: 0010 es: 0010 fs: 0010 gs: 0010 ss: 0010 cs: 0008 (XEN) (XEN) *** Dumping CPU0 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor (XEN) rax: 0000000000000003 rbx: ffff830128f26c20 rcx: ffff82c480483280 (XEN) rdx: 0000000000000808 rsi: 0000000000000000 rdi: ffff830128f26c80 (XEN) rbp: ffff830128f26c80 rsp: ffff82c48046fe78 r8: ffff830128f60348 (XEN) r9: 0000025bb52c7d0d r10: 0000000000000000 r11: ffff82c4804b20e0 (XEN) r12: 000000000002c315 r13: ffff82c48046fe18 r14: ffff8300cfdfc000 (XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: ffff8800c903b218 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff82c48046fe78: (XEN) ffff82c48018f1e2 ffff82c4802660a8 ffff8300cfdfc000 ffffffffffffffff (XEN) ffff830128f60330 ffff82c48012181a ffff8300cfdfc000 0000000000000000 (XEN) 0000000000000000 ffff82c480483280 00001d1b0000269b ffff82c48046ff28 (XEN) ffff82c48046ff28 ffff82c480266c00 ffff82c48046ff28 ffff82c48046fe18 (XEN) ffff8300cfdfc000 0000000000000002 ffff82c48014d73e ffff8300cfdfc000 (XEN) ffff8300cfc66000 00000000ffffffff 0000000000000000 0000000000000000 (XEN) ffffffffffffffff ffffffff806f5fd8 ffffffff80751100 0000000000000000 (XEN) 0000000000000246 0000000000000001 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffffffff800033aa 0000000000000000 0000000000000000 (XEN) 0000000000000001 0000010000000000 ffffffff800033aa 000000000000e033 (XEN) 0000000000000246 ffffffff806f5f58 000000000000e02b 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff8300cfdfc000 (XEN) Xen call trace: (XEN) [<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) [<ffff82c48018f1e2>] acpi_processor_idle+0x2d2/0x6d0 (XEN) [<ffff82c48012181a>] timer_softirq_action+0x21a/0x360 (XEN) [<ffff82c48014d73e>] idle_loop+0x2e/0x90 (XEN) (XEN) No guest context (CPU0 is idle). (XEN) (XEN) ''d'' pressed -> dumping registers (XEN) *** Dumping CPU1 guest state (d1:v0): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: 0008:[<00000000c081745c>] (XEN) RFLAGS: 0000000000010046 CONTEXT: hvm guest (XEN) rax: 0000000000303d9c rbx: 00000000c0821d53 rcx: 00000000000b8b98 (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: 0000000000303d9c (XEN) rbp: 0000000000303e44 rsp: 0000000000303904 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000011 cr4: 0000000000000200 (XEN) cr3: 0000000000a3b000 cr2: 0000000000000000 (XEN) ds: 0010 es: 0010 fs: 0010 gs: 0010 ss: 0010 cs: 0008 (XEN) (XEN) *** Dumping CPU0 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor (XEN) rax: 0000000000000003 rbx: ffff830128f26c20 rcx: ffff82c480483280 (XEN) rdx: 0000000000000808 rsi: 0000000000000000 rdi: ffff830128f26c80 (XEN) rbp: ffff830128f26c80 rsp: ffff82c48046fe78 r8: ffff830128f60348 (XEN) r9: 0000025dd29780f3 r10: 0000000000000000 r11: ffff8300cfdfa060 (XEN) r12: 0000000000f1ee84 r13: ffff82c48046fe18 r14: ffff8300cfdfc000 (XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: ffff8800c903b218 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff82c48046fe78: (XEN) ffff82c48018f1e2 ffff830128f60330 ffff82c48012181a ffffffffffffffff (XEN) ffff830128f60330 ffff82c48012181a ffff82c480483280 0000000000000000 (XEN) 0000000000000000 ffff82c480483280 00000669000006f0 0000000000000002 (XEN) ffff82c48046ff28 ffff82c480266c00 ffff82c48046ff28 ffff82c48046fe18 (XEN) ffff8300cfdfc000 0000000000000002 ffff82c48014d73e ffff8300cfdfc000 (XEN) ffff8300cfc66000 00000000ffffffff 0000000000000000 0000000000000000 (XEN) ffffffffffffffff ffffffff806f5fd8 ffffffff80751100 0000000000000000 (XEN) 0000000000000246 0000000000000001 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffffffff800033aa 0000000000000000 0000000000000000 (XEN) 0000000000000001 0000010000000000 ffffffff800033aa 000000000000e033 (XEN) 0000000000000246 ffffffff806f5f58 000000000000e02b 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff8300cfdfc000 (XEN) Xen call trace: (XEN) [<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) [<ffff82c48018f1e2>] acpi_processor_idle+0x2d2/0x6d0 (XEN) [<ffff82c48012181a>] timer_softirq_action+0x21a/0x360 (XEN) [<ffff82c48012181a>] timer_softirq_action+0x21a/0x360 (XEN) [<ffff82c48014d73e>] idle_loop+0x2e/0x90 (XEN) (XEN) No guest context (CPU0 is idle). (XEN) (XEN) ''d'' pressed -> dumping registers (XEN) (XEN) *** Dumping CPU1 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: e008:[<ffff82c48017bd95>] x86_emulate+0x2e5/0x100b0 (XEN) RFLAGS: 0000000000000282 CONTEXT: hypervisor (XEN) rax: 00000000000000dd rbx: 0000000000000000 rcx: 0000000000000000 (XEN) rdx: 00000000000000c8 rsi: ffff830128f6fc01 rdi: ffff830128f6fad1 (XEN) rbp: ffff830128f6fbe8 rsp: ffff830128f6f788 r8: ffff830128f6fbe8 (XEN) r9: 0000000000000000 r10: 0000000000000001 r11: 00000000001223e8 (XEN) r12: 00000000000000dd r13: 0000000000000004 r14: ffff82c480230d80 (XEN) r15: 0000000000000000 cr0: 0000000080050033 cr4: 00000000000426f0 (XEN) cr3: 0000000127230000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff830128f6f788: (XEN) ffff82c48022aae0 0000000000000000 ffff82c400000000 ffff82c480136500 (XEN) 00000008802667c0 0000000000000086 ffff82c400000004 ffff82c4ffffffff (XEN) 000000c800000096 ffff830128f6f848 ffff830128f6f958 0000000400000002 (XEN) ffff830128f6fad0 000000008048bb7f 000000c800000000 0000000000303b90 (XEN) 00ff82c48022aae0 0000000100000000 ffff82c48022ab36 00000000801365c0 (XEN) 3830613362303106 ffff82c480136166 ffff82c48022aa06 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000303e44 00000000c0821d53 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000303d9c 00000000000b8b98 (XEN) 00000000c082243e 0000000000303b90 0000000000303d9c 000000f100000003 (XEN) 00000000c081745d 0000000000000000 0000000000010046 0000000000303904 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff8300cfa4a000 00000000000c0817 ffff82c4801480ea (XEN) 0000000000000018 00000000c081745c ffff830128f6faa4 ffff8300cfa4bab8 (XEN) 00000000000c0817 ffff8300cfa4a000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff82c480149743 ffff830100000003 ffff814000000028 (XEN) 0000000000000001 ffff82c480149743 ffff830128f6fae4 ffff830128f6fca8 (XEN) 0000000000000001 ffff82c4801d2daa 0000000000000a1e 000000011fe3b000 (XEN) 0000000000000a04 ffff830128f6ff28 0000000000000001 0000000000a1e000 (XEN) Xen call trace: (XEN) [<ffff82c48017bd95>] x86_emulate+0x2e5/0x100b0 (XEN) [<ffff82c480136500>] serial_putc+0x100/0x130 (XEN) [<ffff82c480136166>] __serial_putc+0x86/0x190 (XEN) [<ffff82c4801480ea>] apic_timer_interrupt+0x2a/0x30 (XEN) [<ffff82c480149743>] smp_apic_timer_interrupt+0x53/0xa0 (XEN) [<ffff82c480149743>] smp_apic_timer_interrupt+0x53/0xa0 (XEN) [<ffff82c4801d2daa>] sh_gva_to_gfn__guest_2+0x10a/0x170 (XEN) [<ffff82c4801a051b>] __hvm_copy+0x33b/0x3e0 (XEN) [<ffff82c4801d5c19>] sh_page_fault__guest_2+0x11f9/0x1880 (XEN) [<ffff82c480149743>] smp_apic_timer_interrupt+0x53/0xa0 (XEN) [<ffff82c4801480ea>] apic_timer_interrupt+0x2a/0x30 (XEN) [<ffff82c4801bc973>] vmx_vmexit_handler+0xe33/0x1bb0 (XEN) [<ffff82c480119b2d>] _csched_cpu_pick+0xfd/0x350 (XEN) [<ffff82c4801b0893>] pt_restore_timer+0x23/0xb0 (XEN) [<ffff82c4801afbd9>] vpic_update_int_output+0x29/0x80 (XEN) [<ffff82c4801a7b93>] hvm_isa_irq_deassert+0x93/0xa0 (XEN) [<ffff82c4801b0a79>] pt_update_irq+0x159/0x1f0 (XEN) [<ffff82c4801518bd>] vcpu_kick+0x1d/0x80 (XEN) [<ffff82c4801a7af5>] hvm_isa_irq_assert+0x95/0xa0 (XEN) [<ffff82c4801ae2a0>] vlapic_accept_pic_intr+0x20/0x30 (XEN) [<ffff82c4801a8184>] hvm_vcpu_has_pending_irq+0x24/0x60 (XEN) [<ffff82c4801b5cd8>] vmx_intr_assist+0x1d8/0x1f0 (XEN) [<ffff82c4801b5923>] vmx_asm_do_vmentry+0x0/0xdd (XEN) (XEN) *** Dumping CPU1 guest state (d1:v0): *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 1 (XEN) RIP: 0008:[<00000000c081745c>] (XEN) RFLAGS: 0000000000010046 CONTEXT: hvm guest (XEN) rax: 0000000000303d9c rbx: 00000000c0821d53 rcx: 00000000000b8b98 (XEN) rdx: 00000000c082243e rsi: 0000000000303b90 rdi: 0000000000303d9c (XEN) rbp: 0000000000303e44 rsp: 0000000000303904 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000011 cr4: 0000000000000200 (XEN) cr3: 0000000000a3b000 cr2: 0000000000000000 (XEN) ds: 0010 es: 0010 fs: 0010 gs: 0010 ss: 0010 cs: 0008 (XEN) (XEN) *** Dumping CPU0 host state: *** (XEN) ----[ Xen-4.0.0_21091_06-0.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e008:[<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor (XEN) rax: 0000000000000003 rbx: ffff830128f26c20 rcx: ffff82c480483280 (XEN) rdx: 0000000000000808 rsi: 0000000000000000 rdi: ffff830128f26c80 (XEN) rbp: ffff830128f26c80 rsp: ffff82c48046fe78 r8: ffff830128f60348 (XEN) r9: 0000026435d7983e r10: 0000000000000000 r11: ffff8300cfdfa060 (XEN) r12: 0000000000cccdfb r13: ffff82c48046fe18 r14: ffff8300cfdfc000 (XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000002c9dd000 cr2: ffff8800c903b218 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008 (XEN) Xen stack trace from rsp=ffff82c48046fe78: (XEN) ffff82c48018f1e2 ffff82c4802660a8 ffff8300cfdfc000 ffffffffffffffff (XEN) ffff830128f60330 ffff82c48012181a ffff8300cfdfc000 0000000000000000 (XEN) 0000000000000000 ffff82c480483280 000025660000288a ffff82c48046ff28 (XEN) ffff82c48046ff28 ffff82c480266c00 ffff82c48046ff28 ffff82c48046fe18 (XEN) ffff8300cfdfc000 0000000000000002 ffff82c48014d73e ffff8300cfdfc000 (XEN) ffff8300cfc66000 00000000ffffffff 0000000000000000 0000000000000000 (XEN) ffffffffffffffff ffffffff806f5fd8 ffffffff80751100 0000000000000000 (XEN) 0000000000000246 0000000000000001 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffffffff800033aa 0000000000000000 0000000000000000 (XEN) 0000000000000001 0000010000000000 ffffffff800033aa 000000000000e033 (XEN) 0000000000000246 ffffffff806f5f58 000000000000e02b 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) ffff8300cfdfc000 (XEN) Xen call trace: (XEN) [<ffff82c48018ec2a>] acpi_idle_do_entry+0x2a/0x60 (XEN) [<ffff82c48018f1e2>] acpi_processor_idle+0x2d2/0x6d0 (XEN) [<ffff82c48012181a>] timer_softirq_action+0x21a/0x360 (XEN) [<ffff82c48014d73e>] idle_loop+0x2e/0x90 (XEN) (XEN) No guest context (CPU0 is idle). (XEN) This message is intended only for the use of the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the sender of this E-Mail by return E-Mail or by telephone. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 03/11/2010 01:10, "alarson@ddci.com" <alarson@ddci.com> wrote:> Thanks Keir, I was headed in that general direction, but I was hoping to > get lucky. > > I setup the serial cable and did a number of backtraces. I''m hoping that > the backtrace for the host is correlated with the client backtrace, but > I''m suspicious. Some of the backtraces I got list my client (e.g., on > CPU1) but don''t list a CPU1 host state. In all cases my clint is doing > what I expect (attempting to execute an fnsave instruction). > > I slurped the relevant host stacktraces from the console debug log, and I > don''t see a pattern (again hoping to get lucky). I''ll put away my private > OS shovel, and see if I can start fashioning a Xen shovel. If you see > something obvious below that looks suspicious, let me know, otherwise I''ll > see what my Xen shovel digs up. Thanks again for your help. > > Host CPU traces from the CPU''s associated with my OS client.Do a ''debug=y make'' build of Xen and your backtraces will be precise as wthe build will include frame pointers. Looking at these backtraces, they mostly seem to be in vmx_intr_assist() doing interrupt delivery work (e.g., hvm_isa_irq_assert()). Perhaps you are stuck in some kind of interrupt delivery loop? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
alarson@ddci.com
2010-Nov-04 16:32 UTC
Re: [Xen-devel] fxsave, fnsave, ltr hang for guest OS.
History: fnsave in my 32-bit protected mode OS ''hangs'' when paging is turned on. Keir Fraser <keir.xen@gmail.com> wrote on 11/03/2010 01:48:53 AM:> Do a ''debug=y make'' build of Xen and your backtraces will be precise aswthe> build will include frame pointers. Looking at these backtraces, theymostly> seem to be in vmx_intr_assist() doing interrupt delivery work (e.g., > hvm_isa_irq_assert()). Perhaps you are stuck in some kind of interrupt > delivery loop?It turns out that OpenSuse provides a debug xen kernel so I was able to use that yesterday, but I wasn''t making much progress until it occurred to me that by pressing ''d'', I''m sampling the stack of xen, but I don''t know for sure which client the xen stack represents. When I restricted the two clients (dom0 and my os) to exclusively their own CPU, then, if you exclude the first trace below, a pattern seems to emerge, and it would seem that I should start with sh_page_fault(). I think I can see a path forward, but I figured I''d post hoping to get lucky again... (XEN) Xen call trace: (XEN) [<ffff82c480120ecf>] _spin_is_locked+0x0/0x20 (XEN) [<ffff82c4801b1df4>] vpic_get_highest_priority_irq+0x52/0xc7 (XEN) [<ffff82c4801b1eaa>] vpic_update_int_output+0x41/0x9d (XEN) [<ffff82c4801b27b9>] vpic_irq_negative_edge+0xab/0xc0 (XEN) [<ffff82c4801a9bea>] hvm_isa_irq_deassert+0x8a/0xa7 (XEN) [<ffff82c4801b2ee0>] pt_update_irq+0x1b9/0x1f2 (XEN) [<ffff82c4801b8296>] vmx_intr_assist+0x6f/0x229 (XEN) Xen call trace: (XEN) [<ffff82c4801bb952>] vmx_get_segment_register+0x182/0x3a5 (XEN) [<ffff82c4801ced63>] hvm_get_seg_reg+0x43/0x4d (XEN) [<ffff82c4801cf11f>] shadow_init_emulation+0xa9/0x144 (XEN) [<ffff82c4801dfaac>] sh_page_fault__guest_2+0x18c1/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801d29bd>] sh_remove_write_access+0xcce/0xce0 (XEN) [<ffff82c4801dfef8>] sh_page_fault__guest_2+0x1d0d/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801faacd>] __mfn_valid+0x3e/0x58 (XEN) [<ffff82c4801d80b2>] l1e_propagate_from_guest+0xd2/0x644 (XEN) [<ffff82c4801df0d1>] sh_page_fault__guest_2+0xee6/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801faad1>] __mfn_valid+0x42/0x58 (XEN) [<ffff82c4801df0f7>] sh_page_fault__guest_2+0xf0c/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801faa90>] __mfn_valid+0x1/0x58 (XEN) [<ffff82c4801de6e8>] sh_page_fault__guest_2+0x4fd/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c48017f5e0>] x86_emulate+0x2c/0xefc8 (XEN) [<ffff82c4801dfab7>] sh_page_fault__guest_2+0x18cc/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801caae8>] set_ad_bits+0x0/0x34 (XEN) [<ffff82c4801de6e8>] sh_page_fault__guest_2+0x4fd/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801be932>] vmx_vmexit_handler+0x3fb/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801d769a>] sh_gva_to_gfn__guest_2+0x8/0x168 (XEN) [<ffff82c4801a20ef>] __hvm_copy+0xe7/0x3ef (XEN) [<ffff82c4801a4fd9>] hvm_fetch_from_guest_virt_nofault+0x46/0x48 (XEN) [<ffff82c4801cf196>] shadow_init_emulation+0x120/0x144 (XEN) [<ffff82c4801dfaac>] sh_page_fault__guest_2+0x18c1/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801be932>] vmx_vmexit_handler+0x3fb/0x1ab8 (XEN) Xen call trace: (XEN) [<ffff82c4801ced20>] hvm_get_seg_reg+0x0/0x4d (XEN) [<ffff82c4801dfaac>] sh_page_fault__guest_2+0x18c1/0x1d45 (XEN) [<ffff82c4801bed2c>] vmx_vmexit_handler+0x7f5/0x1ab8 This message is intended only for the use of the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the sender of this E-Mail by return E-Mail or by telephone. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 04/11/2010 16:32, "alarson@ddci.com" <alarson@ddci.com> wrote:> It turns out that OpenSuse provides a debug xen kernel so I was able > to use that yesterday, but I wasn''t making much progress until it > occurred to me that by pressing ''d'', I''m sampling the stack of xen, > but I don''t know for sure which client the xen stack represents. When > I restricted the two clients (dom0 and my os) to exclusively their own > CPU, then, if you exclude the first trace below, a pattern seems to > emerge, and it would seem that I should start with sh_page_fault().Hm, well maybe. sh_page_fault() is the entry point to one of the most complex pieces of the hypervisor, so you''re likely to find it more of a sticky tar pit than a source of salvation. It may be involved, but you might want to wade in armed with some more debug info first so that your search has some greater focus. I would dispense with sampling the ''d'' key -- clearly a bunch of stuff is happening -- and instead go for printk() tracing in the vmexit handler vmx_vmexit_handler(). Perhaps enable this tracing only when the saved guest EIP happens to be the address that you know your ''hanging'' instruction resides at, so that the logging console only gets noisy when the bad situation occurs. From that function you can log useful things like: what is the vmexit for? Faulting linear address (if a page fault)? Etc etc. See if there is a (looping) pattern to the vmexit reasons and see thereby if you can work out the overall livelock loop you are experiencing (this assumes your problem is a livelock loop of some kind, which seems quite possible from what you''ve seen so far). -- Keir> I think I can see a path forward, but I figured I''d post hoping to get > lucky again..._______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
alarson@ddci.com
2010-Nov-12 21:27 UTC
Re: [Xen-devel] fxsave, fnsave, ltr hang for guest OS.
History: fnsave in my 32-bit protected mode OS ''hangs'' when paging is turned on. Keir Fraser <keir.xen@gmail.com> wrote on 11/04/2010 11:50:52 AM:> On 04/11/2010 16:32, "alarson@ddci.com" <alarson@ddci.com> wrote: >AL> ... When I restricted the two clients (dom0 and my os) to AL> exclusively their own CPU, then, if you exclude the first trace AL> below, a pattern seems to emerge, and it would seem that I should AL> start with sh_page_fault(). KF> Hm, well maybe. sh_page_fault() is the entry point to one of the KF> most complex pieces of the hypervisor, so you''re likely to find it KF> more of a sticky tar pit than a source of salvation. It may be KF> involved, but you might want to wade in armed with some more debug KF> info first so that your search has some greater focus. KF> I would dispense with sampling the ''d'' key -- clearly a bunch of KF> stuff is happening -- and instead go for printk() tracing in the KF> vmexit handler vmx_vmexit_handler(). ... You advice was most helpful once again. However, I''m now even more confused than I was before. I downloaded the OpenSuse source RPM for xen (xen-4.0.0_21091_06-0.1.1.src.rpm) and added printk debugging statements as suggested. The following are the outputs I eventually found most useful. All expressions denote existing source code variables: (XEN) sh_page_fault va=303b90, regs->error_code=3 (XEN) x86_emulate: b=dd, modrm=31, modrm_reg=6 (XEN) sh_page_fault called x86_emulate va=303b90,result=1 (XEN) vmx_vmexit_handler reason=0 (XEN) vmx_vmexit_handler intr_info=80000b0e, vector=e The key line is: (XEN) x86_emulate: b=dd, modrm=31, modrm_reg=6 The instruction I''m executing is: c0817476: dd 31 fnsave (%ecx) Opcode 0xDD and the modrm_reg (0x6) matches. The source for x86_emulate() opcode 0xdd has no case for modrm_reg=0x6, so x86_emulate() returns X86EMUL_UNHANDLEABLE (i.e., 1). What I don''t understand is why fnsave works before I turn on paging and doesn''t afterwords. Am I just looking at an unimplemented feature, or is there something more nefarious going on? I haven''t looked at why load task register (ltr) and accesses to the APIC behave similarly. Does Xen assume fnsave, ltr, etc. happen with paging turned off? This message is intended only for the use of the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the sender of this E-Mail by return E-Mail or by telephone. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 12/11/2010 21:27, "alarson@ddci.com" <alarson@ddci.com> wrote:> You advice was most helpful once again. However, I''m now even more > confused than I was before. I downloaded the OpenSuse source RPM for > xen (xen-4.0.0_21091_06-0.1.1.src.rpm) and added printk debugging > statements as suggested. The following are the outputs I eventually > found most useful. All expressions denote existing source code > variables: > > (XEN) sh_page_fault va=303b90, regs->error_code=3 > (XEN) x86_emulate: b=dd, modrm=31, modrm_reg=6So you took a vmexit on a page fault while trying to execute fnsave. The shadow fault handler then tries to emulate the faulting instruction (fnsave) presumably because it thinks the instruction is attempting to write to a page that is read-only in the shadow page tables (generally that would be because Xen thinks the instruction is trying to write to a page-table page).> (XEN) sh_page_fault called x86_emulate va=303b90,result=1Emulation failed, because the emulator does not support fnsave. The shadow fault handler will take this as a hint that the instrction is not writing to a page table page, and hence flushes some shadow page table state which is possibly out of date. This makes the previously read-lonely page writable, and we expect that on return to guest context the CPU will execute the previously-faulting instruction without problem. Since you end up in a loop not progressing past the fnsave instruction, it seems quite likely that you have a bug and are writing to a pagetable page. In fact a pagetable page that maps something that is needed to execute the fnsave instruction. You need that page to both be writable (so that fnsave can write its data) and read-only (because it is a pagetable page that maps something that is used by the fnsave instruction) and so I''m guessing you end up in an endless loop with that page flipping between being read-only and read-write in the shadow page table. Hope that makes sense. :-)> I haven''t looked at why load task register (ltr) and > accesses to the APIC behave similarly. Does Xen assume > fnsave, ltr, etc. happen with paging turned off?In general those instructions would not even vmexit. The CPU would just execute them directly. The reason you are having problems when you turn paging on is probably because of some guest bug where data structures written yo by FNSAVE, LTR, etc are overlapping with your page tables. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
alarson@ddci.com
2010-Nov-15 22:22 UTC
Re: [Xen-devel] fxsave, fnsave, ltr hang for guest OS.
Keir Fraser <keir.xen@gmail.com> wrote on 11/12/2010 04:34:05 PM: KF> On 12/11/2010 21:27, "alarson@ddci.com" <alarson@ddci.com> wrote: ... AL> (XEN) sh_page_fault va=303b90, regs->error_code=3 AL> (XEN) x86_emulate: b=dd, modrm=31, modrm_reg=6 ... KF> Since you end up in a loop not progressing past the fnsave KF> instruction, it seems quite likely that you have a bug and are KF> writing to a pagetable page. In fact a pagetable page that maps KF> something that is needed to execute the fnsave instruction. You KF> need that page to both be writable (so that fnsave can write its KF> data) and read-only (because it is a pagetable page that maps KF> something that is used by the fnsave instruction) and so I''m KF> guessing you end up in an endless loop with that page flipping KF> between being read-only and read-write in the shadow page table. KF> Hope that makes sense. :-) I understand what you are saying, but I''m confident that''s not what''s happening. Just to be sure, I modified one of our analysis programs to list all the page tables and the PDT and confirmed that the page at virtual address 303b90 doesn''t map to any of them. In fact I verified that none of the present pages maps to any PT or the PDT. If not, it seems like I need to find out who is calling sh_page_fault, and that leads me to sh_paging_mode which seems even more of a sticky tar pit than sh_page_fault. I think I should be looking at emulate_gva_to_mfn() and propagate_page_fault(), but I haven''t spent enough time with the code to be sure. If you have other ideas, I''d love to hear them. This message is intended only for the use of the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the sender of this E-Mail by return E-Mail or by telephone. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, At 22:22 +0000 on 15 Nov (1289859751), alarson@ddci.com wrote:> I understand what you are saying, but I''m confident that''s not what''s > happening. Just to be sure, I modified one of our analysis programs > to list all the page tables and the PDT and confirmed that the page at > virtual address 303b90 doesn''t map to any of them. In fact I verified > that none of the present pages maps to any PT or the PDT. > > If not, it seems like I need to find out who is calling sh_page_fault, > and that leads me to sh_paging_mode which seems even more of a sticky > tar pit than sh_page_fault. I think I should be looking at > emulate_gva_to_mfn() and propagate_page_fault(), but I haven''t spent > enough time with the code to be sure. If you have other ideas, I''d > love to hear them.I think the most useful thing, since you know the virtual address, is to instrument the sh_page_fault() function to see which path it takes, and in particular why it returns what it does. Adding printks guarded by "if ( va == 0x303b90 )" should be OK. It would also be interesting to instrument guest_walk_pages() for just that address, say by adding a print_gw() call at the end. Cheers, Tim. -- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, Xen Platform Team Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
alarson@ddci.com
2010-Nov-19 17:32 UTC
Re: [Xen-devel] fxsave, fnsave, ltr hang for guest OS.
> Keir Fraser <keir.xen@gmail.com> wrote on 11/12/2010 04:34:05 PM: > KF> On 12/11/2010 21:27, "alarson@ddci.com" <alarson@ddci.com> wrote: > ...AL> (XEN) sh_page_fault va=303b90, regs->error_code=3 AL> (XEN) x86_emulate: b=dd, modrm=31, modrm_reg=6> ...KF> Since you end up in a loop not progressing past the fnsave KF> instruction, it seems quite likely that you have a bug and are KF> writing to a pagetable page. In fact a pagetable page that maps KF> something that is needed to execute the fnsave instruction. You KF> need that page to both be writable (so that fnsave can write its KF> data) and read-only (because it is a pagetable page that maps KF> something that is used by the fnsave instruction) and so I''m KF> guessing you end up in an endless loop with that page flipping KF> between being read-only and read-write in the shadow page table. KF> Hope that makes sense. :-) AL> I understand what you are saying, but I''m confident that''s not what''s AL> happening. Just to be sure, I modified one of our analysis programs AL> to list all the page tables and the PDT and confirmed that the page at AL> virtual address 303b90 doesn''t map to any of them. In fact I verified AL> that none of the present pages maps to any PT or the PDT. Just to report final resolution. The problem turned out to be that the destination of the fnsave was to a page specifying read only access and the OS was depending on CR0.WP=0 to permit write access in kernel mode. Xen doesn''t allow this configuration unless using HAP (Hardware Assisted Paging). Many thanks to Keir for his invaluable assistance with this. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 19/11/2010 17:32, "alarson@ddci.com" <alarson@ddci.com> wrote:> Just to report final resolution. The problem turned out to be that > the destination of the fnsave was to a page specifying read only > access and the OS was depending on CR0.WP=0 to permit write access in > kernel mode. Xen doesn''t allow this configuration unless using HAP > (Hardware Assisted Paging).To be precise, we have a go at emulating the instruction, but our emulator is incomplete. Running a whole OS in CR0.WP=0 mode is pretty much doomed if running on Xen without HAP. I think there is some supported OS that does do some small amount of stuff with CR0.WP=0, and we are complete enough to work in that case. -- Keir> Many thanks to Keir for his invaluable assistance with this._______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
At 18:01 +0000 on 19 Nov (1290189698), Keir Fraser wrote:> On 19/11/2010 17:32, "alarson@ddci.com" <alarson@ddci.com> wrote: > > > Just to report final resolution. The problem turned out to be that > > the destination of the fnsave was to a page specifying read only > > access and the OS was depending on CR0.WP=0 to permit write access in > > kernel mode. Xen doesn''t allow this configuration unless using HAP > > (Hardware Assisted Paging). > > To be precise, we have a go at emulating the instruction, but our emulator > is incomplete. Running a whole OS in CR0.WP=0 mode is pretty much doomed if > running on Xen without HAP. I think there is some supported OS that does do > some small amount of stuff with CR0.WP=0, and we are complete enough to work > in that case.To be precise, the shadow code/emulator can run various flavours of Windows when anti-virus software briefly turns off CR0.WP in order to overwrite protected kernel structures (IDT &c). That only involves straightforward memory writes in an OS that otherwise obeys its own pagetable restrictions. Cheers, Tim. -- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, Xen Platform Team Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel