Christopher S. Aker
2008-Jul-03 14:38 UTC
2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
Xen: 3.2.1-rc5 64bit
Dom0: 2.6.18.8 (at cs 524) 32-pae
DomU: 2.6.26-rc8 32-pae
root at newark13:~# xm create -f /linodes/xencaker/xen.conf -c
Using config file "/linodes/xencaker/xen.conf".
Started domain xencaker
root at newark13:~#
root at newark13:~# xm dmesg
...
(XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on VCPU
0 [ec=0000]
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 332 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.2.1-rc5 x86_64 debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e019:[<00000000c0103b55>]
(XEN) RFLAGS: 0000000000000282 CONTEXT: guest
(XEN) rax: 00000000ffffffea rbx: 00000000deadbeef rcx: 00000000deadbeef
(XEN) rdx: 00000000deadbeef rsi: 00000000deadbeef rdi: 00000000c06c1000
(XEN) rbp: 00000000c06c0000 rsp: 00000000c0667f08 r8: 0000000000000000
(XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000
(XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000
(XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026b0
(XEN) cr3: 00000002d351b000 cr2: 0000000000000000
(XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019
(XEN) Guest stack trace from esp=c0667f08:
(XEN) c0103b55 0001e019 00010082 00000001 006c0000 c0756d50 c06c1000
c065c580
(XEN) c067e4dc c0667f44 c0678ac9 c0667f38 00100000 c0756d50 00000000
00000000
(XEN) 00040800 00010676 c0667fb8 c06a3b00 c065c680 c065c580 c0676452
00000000
(XEN) 00000000 00000000 c065c580 00000000 c0667fb0 c0667fa4 c0667fac
1f898975
(XEN) 40000000 c0667fdc f5800000 00000007 40000000 c0667fdc f5800000
00000007
(XEN) c066e771 c0537060 f5800000 00000007 c012500f 40000000 c06753a2
00000000
(XEN) c069c5c0 c069c5dc c069c698 c069c6b0 00000000 1f898975 00080201
00040800
(XEN) 00010676 f5800000 00000000 c0752000 00000000 00000000
-Chris
Jeremy Fitzhardinge
2008-Jul-03 15:44 UTC
2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
Christopher S. Aker wrote:> Xen: 3.2.1-rc5 64bit > Dom0: 2.6.18.8 (at cs 524) 32-pae > DomU: 2.6.26-rc8 32-pae > > root at newark13:~# xm create -f /linodes/xencaker/xen.conf -c > Using config file "/linodes/xencaker/xen.conf". > Started domain xencakerIs this consistent or sporadic? Does it depend on how much guest memory you give it? What was the last kernel version which worked?> root at newark13:~# > > root at newark13:~# xm dmesg > ...Were there any other errors/warnings printed here?> (XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on > VCPU 0 [ec=0000] > (XEN) domain_crash_sync called from entry.S > (XEN) Domain 332 (vcpu#0) crashed on cpu#0: > (XEN) ----[ Xen-3.2.1-rc5 x86_64 debug=y Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: e019:[<00000000c0103b55>]Could you decode this to see which function this corresponds to?> (XEN) RFLAGS: 0000000000000282 CONTEXT: guest > (XEN) rax: 00000000ffffffea rbx: 00000000deadbeef rcx: > 00000000deadbeef > (XEN) rdx: 00000000deadbeef rsi: 00000000deadbeef rdi: > 00000000c06c1000 > (XEN) rbp: 00000000c06c0000 rsp: 00000000c0667f08 r8: > 0000000000000000 > (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: > 0000000000000000 > (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: > 0000000000000000 > (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: > 00000000000026b0 > (XEN) cr3: 00000002d351b000 cr2: 0000000000000000 > (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 > (XEN) Guest stack trace from esp=c0667f08: > (XEN) c0103b55 0001e019 00010082 00000001 006c0000 c0756d50 c06c1000 > c065c580 > (XEN) c067e4dc c0667f44 c0678ac9 c0667f38 00100000 c0756d50 00000000 > 00000000 > (XEN) 00040800 00010676 c0667fb8 c06a3b00 c065c680 c065c580 c0676452 > 00000000 > (XEN) 00000000 00000000 c065c580 00000000 c0667fb0 c0667fa4 c0667fac > 1f898975 > (XEN) 40000000 c0667fdc f5800000 00000007 40000000 c0667fdc f5800000 > 00000007 > (XEN) c066e771 c0537060 f5800000 00000007 c012500f 40000000 c06753a2 > 00000000 > (XEN) c069c5c0 c069c5dc c069c698 c069c6b0 00000000 1f898975 00080201 > 00040800 > (XEN) 00010676 f5800000 00000000 c0752000 00000000 00000000 > > -ChrisThanks, J
Jeremy Fitzhardinge
2008-Dec-19 21:26 UTC
Re: [Xen-devel] Re: 2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
Guillaume Rousse wrote:> Jeremy Fitzhardinge a écrit : >>> FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it >>> down further if that''d be useful to you. >>> >> >> Yes, please. > Was there any final conclusion there ? Because I''m currently trying a > 2.6.27.5 with the same kind of issue:I don''t know of any outstanding bugs which cause this kind of symptom. What''s happening on the Linux side when the domain crashes this way? J> (XEN) mm.c:1482:d31 Bad L1 flags 800000 > (XEN) traps.c:437:d31 Unhandled invalid opcode fault/trap [#6] on VCPU > 0 [ec=0000] > (XEN) domain_crash_sync called from entry.S > (XEN) Domain 31 (vcpu#0) crashed on cpu#2: > (XEN) ----[ Xen-3.3.0 x86_64 debug=n Not tainted ]---- > (XEN) CPU: 2 > (XEN) RIP: e033:[<ffffffff8020b3ed>] > (XEN) RFLAGS: 0000000000000282 EM: 1 CONTEXT: pv guest > (XEN) rax: 00000000ffffffea rbx: 000000000000077f rcx: > 0000000000000054 > (XEN) rdx: 0000000000000000 rsi: 8000000331df5061 rdi: > ffffffff8077f000 > (XEN) rbp: ffffffff806e1f88 rsp: ffffffff806e1f28 r8: > 0000000000000010 > (XEN) r9: 0000000000000010 r10: ffffffff806e1e88 r11: > ffffffff80827f60 > (XEN) r12: ffffffff8077f000 r13: 8000000000000161 r14: > 8000000331df5061 > (XEN) r15: 0000000000000800 cr0: 000000008005003b cr4: > 00000000000026b0 > (XEN) cr3: 000000033c35c000 cr2: 0000000000000000 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033 > (XEN) Guest stack trace from rsp=ffffffff806e1f28: > (XEN) 0000000000000054 ffffffff80827f60 ffffffff8020b3ed > 000000010000e030 > (XEN) 0000000000010082 ffffffff806e1f68 000000000000e02b > ffffffff8020b3e9 > (XEN) 0000000000000200 000000000000000c 000000000000000c > 0000000000001800 > (XEN) ffffffff806e1ff8 ffffffff806f99cf 0000000000000000 > 0000000000001000 > (XEN) 0000000000001800 ffffffff80e48000 0000000000020000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 > ffffffff80d45000 > (XEN) ffffffff80d46000 ffffffff80d47000 ffffffff80d48000 > ffffffff80d49000 > (XEN) ffffffff80d4a000 ffffffff80d4b000 ffffffff80d4c000 > ffffffff80d4d000 > (XEN) ffffffff80d4e000 ffffffff80d4f000 ffffffff80d50000 > ffffffff80d51000 > (XEN) ffffffff80d52000 ffffffff80d53000 ffffffff80d54000 > ffffffff80d55000 > (XEN) ffffffff80d56000 ffffffff80d57000 ffffffff80d58000 > ffffffff80d59000 > (XEN) ffffffff80d5a000 ffffffff80d5b000 ffffffff80d5c000 > ffffffff80d5d000 > (XEN) ffffffff80d5e000 ffffffff80d5f000 ffffffff80d60000 > ffffffff80d61000 > (XEN) ffffffff80d62000 ffffffff80d63000 ffffffff80d64000 > ffffffff80d65000 > (XEN) ffffffff80d66000 ffffffff80d67000 ffffffff80d68000 > ffffffff80d69000 > (XEN) ffffffff80d6a000 ffffffff80d6b000 ffffffff80d6c000 > ffffffff80d6d000 > (XEN) ffffffff80d6e000 ffffffff80d6f000 ffffffff80d70000 > ffffffff80d71000 > (XEN) ffffffff80d72000 ffffffff80d73000 ffffffff80d74000 > ffffffff80d75000 > (XEN) ffffffff80d76000 ffffffff80d77000 ffffffff80d78000 > ffffffff80d79000_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Guillaume Rousse
2008-Dec-20 13:37 UTC
Re: [Xen-devel] Re: 2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
Jeremy Fitzhardinge a écrit :> Guillaume Rousse wrote: >> Jeremy Fitzhardinge a écrit : >>>> FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it >>>> down further if that''d be useful to you. >>>> >>> >>> Yes, please. >> Was there any final conclusion there ? Because I''m currently trying a >> 2.6.27.5 with the same kind of issue: > > I don''t know of any outstanding bugs which cause this kind of symptom. > What''s happening on the Linux side when the domain crashes this way?Immediate crash, without further information. I''ve opened a but report on mandriva bugzilla: https://qa.mandriva.com/show_bug.cgi?id=46470 And I''ve yet to try with vanilla linus kernel. -- Guillaume Rousse Moyens Informatiques - INRIA Futurs Tel: 01 69 35 69 62 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-20 19:43 UTC
Re: [Xen-devel] Re: 2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
Guillaume Rousse wrote:> Jeremy Fitzhardinge a écrit : >> Guillaume Rousse wrote: >>> Jeremy Fitzhardinge a écrit : >>>>> FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it >>>>> down further if that''d be useful to you. >>>>> >>>> >>>> Yes, please. >>> Was there any final conclusion there ? Because I''m currently trying >>> a 2.6.27.5 with the same kind of issue: >> >> I don''t know of any outstanding bugs which cause this kind of >> symptom. What''s happening on the Linux side when the domain crashes >> this way? > Immediate crash, without further information. > > I''ve opened a but report on mandriva bugzilla: > https://qa.mandriva.com/show_bug.cgi?id=46470Does earlyprintk=xen reveal anything more? What does the rip (ffffffff8020b3ed) map to in the kernel?> And I''ve yet to try with vanilla linus kernel.Please do. If possible, build a linus kernel with the mandriva config. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Guillaume Rousse
2008-Dec-23 14:17 UTC
Re: [Xen-devel] Re: 2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
Jeremy Fitzhardinge a écrit :> Guillaume Rousse wrote: >> Jeremy Fitzhardinge a écrit : >>> Guillaume Rousse wrote: >>>> Jeremy Fitzhardinge a écrit : >>>>>> FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it >>>>>> down further if that''d be useful to you. >>>>>> >>>>> >>>>> Yes, please. >>>> Was there any final conclusion there ? Because I''m currently trying >>>> a 2.6.27.5 with the same kind of issue: >>> >>> I don''t know of any outstanding bugs which cause this kind of >>> symptom. What''s happening on the Linux side when the domain crashes >>> this way? >> Immediate crash, without further information. >> >> I''ve opened a but report on mandriva bugzilla: >> https://qa.mandriva.com/show_bug.cgi?id=46470 > > Does earlyprintk=xen reveal anything more? What does the rip > (ffffffff8020b3ed) map to in the kernel?I guess earlyprintk=xen is a dom0 option, because passing it to domU didn''t change anything. The given adress map to set_page_prot, according to the System.map file. BTW, it only happens with ISCSI-stored hosts, whereas it works OK with LVM-stored hosts (tough on slightly different hardware). On other hosts, the guest kernel boots OK, and then fails to mount underlying FS (but that''s a different issue). I updated upstream report with details. -- Guillaume Rousse Service des Moyens Informatiques INRIA Saclay - Ile de France Tel: 01 69 35 69 62 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2008-Dec-31 21:45 UTC
Re: [Xen-devel] Re: 2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
Guillaume Rousse wrote:> Jeremy Fitzhardinge a écrit : >> Guillaume Rousse wrote: >>> Jeremy Fitzhardinge a écrit : >>>> Guillaume Rousse wrote: >>>>> Jeremy Fitzhardinge a écrit : >>>>>>> FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow >>>>>>> it down further if that''d be useful to you. >>>>>>> >>>>>> >>>>>> Yes, please. >>>>> Was there any final conclusion there ? Because I''m currently >>>>> trying a 2.6.27.5 with the same kind of issue: >>>> >>>> I don''t know of any outstanding bugs which cause this kind of >>>> symptom. What''s happening on the Linux side when the domain >>>> crashes this way? >>> Immediate crash, without further information. >>> >>> I''ve opened a but report on mandriva bugzilla: >>> https://qa.mandriva.com/show_bug.cgi?id=46470 >> >> Does earlyprintk=xen reveal anything more? What does the rip >> (ffffffff8020b3ed) map to in the kernel? > I guess earlyprintk=xen is a dom0 option, because passing it to domU > didn''t change anything.It will be most helpful if you''re running a debug=y build of Xen, so the domU''s early console output appears on the Xen console.> The given adress map to set_page_prot, according to the System.map file.That''s interesting.> BTW, it only happens with ISCSI-stored hosts, whereas it works OK with > LVM-stored hosts (tough on slightly different hardware).Uh, that''s very mysterious. The underlying storage shouldn''t have any effect on whether the kernel can boot - it''s either loaded or not.> On other hosts, the guest kernel boots OK, and then fails to mount > underlying FS (but that''s a different issue). I updated upstream > report with details.I''m on vacation at the moment, but I''ll give this a closer look when I return. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel