Xen devel - Aug 2012 - E5606 with no HVM; Assertion 'i == 1' failed at p2m-ept.c:524

Hi Xen developers and enthousiasts,

Recently I have been having a lot of problems with HVM hosts on a dual
Xeon E5606 box. Problem is that HVM hosts don''t work on it at all and
crash the server.

Today I got the logging to work over an IPMI console so then I compiled
a fresh xen-unstable. Apart from the message I posted in the subject
the call trace reads as follows:

(XEN) Xen call trace:
(XEN)    [<ffff82c4801e1be9>] ept_get_entry+0x13a/0x28f
(XEN)    [<ffff82c4801da274>] __get_gfn_type_access+0x175/0x256
(XEN)    [<ffff82c4801b5a55>] hvm_hap_nested_page_fault+0x133/0x422
(XEN)    [<ffff82c4801d3c8e>] vmx_vmexit_handler+0x136b/0x1614

The entire log from boot to crash can be viewed at the following link:
http://pastebin.com/5wcH7GWR

Here''s the kernel''s config
http://pastebin.com/E51S61Qk

And although I''m not sure if knowing BIOS settings is useful it
doesn''t
hurt to share the many settings it has. So I captured them and posted
the images:
http://imgur.com/a/wGBVS/all#0

I hope you can help me with getting this fixed. It''s been a nightmare
despite the lack of sleep :) but then again I also learned much along
the way. I triple checked the consistency of the above posts to make
sure they are all from the same build. The problem has been occuring
for a fairly long time now but I had to make sure there was no issue on
my side. I''m still not sure but I''m also fresh out of ideas,
so thanks
up front for your help or suggestions.

As always please CC me when you reply to the list, I am not receiving
xen-devel emails because most are way outside my comfort zone...

-- 
Thank you,
Mark van Dijk.               ,--------------------------------
----------------------------''        Wed Aug 15 20:38 UTC 2012
Today is Boomtime, the 8th day of Bureaucracy in the YOLD 3178

>>> On 15.08.12 at 22:39, Mark van Dijk
<lists+xen@internecto.net> wrote:
> (XEN) Xen call trace:
> (XEN)    [<ffff82c4801e1be9>] ept_get_entry+0x13a/0x28f
> (XEN)    [<ffff82c4801da274>] __get_gfn_type_access+0x175/0x256
> (XEN)    [<ffff82c4801b5a55>] hvm_hap_nested_page_fault+0x133/0x422
> (XEN)    [<ffff82c4801d3c8e>] vmx_vmexit_handler+0x136b/0x1614
> 
> The entire log from boot to crash can be viewed at the following link:
> http://pastebin.com/5wcH7GWR 
Unfortunately quite a few of the registers contain values that
could sensibly be "i". Could you either disassemble the
instructions around the place yourself to find out which one it
is, or make the xen-syms file corresponding to this run available?

I''m suspecting "i" to actually be 2, and the code not having
got
updated when 1Gb page support got added to PoD. You could
hence alternatively also try the attached debugging patch,
which - if my guess is right - may at once fix your problem.

However, because of this outright claiming that HVM doesn''t
work seems a little harsh - did you try running your guests
without the use of PoD (i.e. with memory == maxmem)?

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Thu, 16 Aug 2012 09:11:50 +0100
"Jan Beulich" <JBeulich@suse.com> wrote:
> >>> On 15.08.12 at 22:39, Mark van Dijk
<lists+xen@internecto.net>
> >>> wrote:
> > (XEN) Xen call trace:
> > (XEN)    [<ffff82c4801e1be9>] ept_get_entry+0x13a/0x28f
> > (XEN)    [<ffff82c4801da274>] __get_gfn_type_access+0x175/0x256
> > (XEN)    [<ffff82c4801b5a55>]
hvm_hap_nested_page_fault+0x133/0x422
> > (XEN)    [<ffff82c4801d3c8e>] vmx_vmexit_handler+0x136b/0x1614
> > 
> > The entire log from boot to crash can be viewed at the following
> > link: http://pastebin.com/5wcH7GWR 
> 
> Unfortunately quite a few of the registers contain values that
> could sensibly be "i". Could you either disassemble the
> instructions around the place yourself to find out which one it
> is, or make the xen-syms file corresponding to this run available?
Hi Jan, I''m not capable to disassemble anything because my experience
with that is zero. So it''s probably easier to give you the xen-syms
file, I posted it to a pastebin with a little detour:

curl http://sprunge.us/cGeL | openssl enc -a -d | \
 xz -d > xen-syms-4.2.0-rc3-pre

md5sum aa27f5aeea45f72cab88848e5996080e
> 
> I''m suspecting "i" to actually be 2, and the code not
having got
> updated when 1Gb page support got added to PoD. You could
> hence alternatively also try the attached debugging patch,
> which - if my guess is right - may at once fix your problem.
Sure, I''ll give that a go and let you know, thanks.
> However, because of this outright claiming that HVM doesn''t
> work seems a little harsh - did you try running your guests
> without the use of PoD (i.e. with memory == maxmem)?
If it sounded like I meant to complain that HVM is broken then I
apologise, that''s not what I meant. I meant it didn''t work on
my system
and I had no idea that this relates to maxmem. :)

I''ll admit that I didn''t try the hvm without the maxmem
setting but the
VM has settings ''memory = 1024; maxmem = 4096''.

Mark

>>> On 16.08.12 at 10:48, "Mark van Dijk"
<lists+xen@internecto.net> wrote:
> On Thu, 16 Aug 2012 09:11:50 +0100
> "Jan Beulich" <JBeulich@suse.com> wrote:
> 
>> >>> On 15.08.12 at 22:39, Mark van Dijk
<lists+xen@internecto.net>
>> >>> wrote:
>> > (XEN) Xen call trace:
>> > (XEN)    [<ffff82c4801e1be9>] ept_get_entry+0x13a/0x28f
>> > (XEN)    [<ffff82c4801da274>]
__get_gfn_type_access+0x175/0x256
>> > (XEN)    [<ffff82c4801b5a55>]
hvm_hap_nested_page_fault+0x133/0x422
>> > (XEN)    [<ffff82c4801d3c8e>]
vmx_vmexit_handler+0x136b/0x1614
>> > 
>> > The entire log from boot to crash can be viewed at the following
>> > link: http://pastebin.com/5wcH7GWR 
>> 
>> Unfortunately quite a few of the registers contain values that
>> could sensibly be "i". Could you either disassemble the
>> instructions around the place yourself to find out which one it
>> is, or make the xen-syms file corresponding to this run available?
> 
> Hi Jan, I''m not capable to disassemble anything because my
experience
> with that is zero. So it''s probably easier to give you the
xen-syms
> file, I posted it to a pastebin with a little detour:
> 
> curl http://sprunge.us/cGeL | openssl enc -a -d | \
>  xz -d > xen-syms-4.2.0-rc3-pre
Thanks. That confirms that "i" (in r12d) is indeed 2. So there are
good chances that the patch I sent earlier will help. Please let us
know.

Jan

On Thu, 16 Aug 2012 12:18:33 +0100
"Jan Beulich" <JBeulich@suse.com> wrote:
> >>> On 16.08.12 at 10:48, "Mark van Dijk"
<lists+xen@internecto.net>
> >>> wrote:
> > On Thu, 16 Aug 2012 09:11:50 +0100
> > "Jan Beulich" <JBeulich@suse.com> wrote:
> > 
> >> >>> On 15.08.12 at 22:39, Mark van Dijk
<lists+xen@internecto.net>
> >> >>> wrote:
> >> > (XEN) Xen call trace:
> >> > (XEN)    [<ffff82c4801e1be9>] ept_get_entry+0x13a/0x28f
> >> > (XEN)    [<ffff82c4801da274>]
__get_gfn_type_access+0x175/0x256
> >> > (XEN)    [<ffff82c4801b5a55>]
> >> > hvm_hap_nested_page_fault+0x133/0x422 (XEN)
> >> > [<ffff82c4801d3c8e>] vmx_vmexit_handler+0x136b/0x1614
> >> > 
> >> > The entire log from boot to crash can be viewed at the
following
> >> > link: http://pastebin.com/5wcH7GWR 
> >> 
> >> Unfortunately quite a few of the registers contain values that
> >> could sensibly be "i". Could you either disassemble the
> >> instructions around the place yourself to find out which one it
> >> is, or make the xen-syms file corresponding to this run available?
> > 
> > Hi Jan, I''m not capable to disassemble anything because my
> > experience with that is zero. So it''s probably easier to give
you
> > the xen-syms file, I posted it to a pastebin with a little detour:
> > 
> > curl http://sprunge.us/cGeL | openssl enc -a -d | \
> >  xz -d > xen-syms-4.2.0-rc3-pre
> 
> Thanks. That confirms that "i" (in r12d) is indeed 2. So there
are
> good chances that the patch I sent earlier will help. Please let us
> know.
Good news: the VM boots. Thanks Jan!

My console reads:

...
(XEN) irq.c:270: Dom1 PCI link 3 changed 5 -> 0
(XEN) PoD[bfc00] level=2
(XEN) PoD[7fc00] level=2

So that''s one more confirmation. :)

Mark

-- 
Stay in touch,
Mark van Dijk.               ,---------------------------------
----------------------------''         Thu Aug 16 11:57 UTC 2012
Today is Pungenday, the 9th day of Bureaucracy in the YOLD 3178