thr3ads.net - Xen devel - PTE Write access is not set [Dec 2011]

If this information is useful, please help other people find it:
Share via:

Liwei

2011-Dec-14 08:05 UTC

PTE Write access is not set

Hello list,
    Using the 3.1.5 kernel and 4.1.3-rc1-pre xen (Latest ChangeSet:
Tue Dec 06 10:53:12 2011 +0000 23199:d9f8316a8291), these crashes
occur after about a day or so of running a HVM:
=================8<================(XEN) realmode.c:115:d9 Failed to emulate
insn.
(XEN) realmode.c:165:d9 Real-mode emulation failed @ e40a:00005f6c: d9
33 01 fc 30 34
(XEN) domain_crash called from realmode.c:166
(XEN) Domain 9 (vcpu#0) crashed on cpu#6:
(XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    6
(XEN) RIP:    e40a:[<0000000000005f6c>]
(XEN) RFLAGS: 0000000000010002   CONTEXT: hvm guest
(XEN) rax: 000000000000e7e5   rbx: 0000000000000000   rcx: 00000000000d0000
(XEN) rdx: 00000000000001f7   rsi: 0000000000000000   rdi: 0000000000000dba
(XEN) rbp: 0000000000000182   rsp: 0000000000000db0   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
(XEN) cr3: 0000000000000000   cr2: 0000000000000000
(XEN) ds: 9fc0   es: 9e00   fs: 0000   gs: 0000   ss: 9e00   cs: e40a
=================8<================(XEN) realmode.c:115:d5 Failed to emulate
insn.
(XEN) realmode.c:165:d5 Real-mode emulation failed @ c5d5:000242bc: d9
33 01 fc 30 34
(XEN) domain_crash called from realmode.c:166
(XEN) Domain 5 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    c5d5:[<00000000000242bc>]
(XEN) RFLAGS: 0000000000010013   CONTEXT: hvm guest
(XEN) rax: 000000000000001b   rbx: 0000000000000702   rcx: 0000000000010ba7
(XEN) rdx: 0000000000008fd5   rsi: 0000000000001333   rdi: 000000000000900e
(XEN) rbp: 000000000000703e   rsp: 000000000000c5cf   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
(XEN) cr3: 0000000000000000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: c5d5
(XEN) [VT-D]iommu.c:856: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:831: DMAR:[DMA Write] Request device [0d:00.1]
fault addr d8964f44000, iommu reg = ffff82c3fff57000
(XEN) DMAR:[fault reason 05h] PTE Write access is not set
(XEN) print_vtd_entries: iommu = ffff83045ce565c0 bdf = d:0.1 gmfn = d8964f44
(XEN)     root_entry = ffff8304519d1000
(XEN)     root_entry[d] = 4127af001
(XEN)     context = ffff8304127af000
(XEN)     context[1] = 2_414aca001
(XEN)     l4 = ffff830414aca000
(XEN)     l4_index = 1b
(XEN)     l4[1b] = 0
(XEN)     l4[1b] not present
=================8<================
    If I try to restart the crashed VM, the same crash always occurs
again during POST (the last thing I see on the VNC console is the BIOS
boot, before the connection terminates). A dom0 reboot is necessary to
get things to work again.
    0d:00.1 is part of an Intel 82575GB quad port network adapter. The
adapter contains three ports, 0d:00.0, 0d:00.1, 0e:00.0, 0e:00.1. Each
of them is assigned to a VM. So far the crashes have occurred to VMs
using 0d:00.1, 0e:00.0 and 0e:00.1. (Pretty sure a crash involving
0d:00.0 will come soon).
    Is this a potential bug, or (more likely?) problematic hardware?

The relevant PCI subtree looks like this:
=================8<================ \-[0000:00]-+-00.0
             +-03.0-[01]--+-00.0
             |            \-00.1
             +-05.0-[02-0e]----00.0-[03-0e]--+-00.0-[04]----00.0

+-02.0-[05-0a]----00.0-[06-0a]--+-02.0-[07]----00.0
                                             |
      +-03.0-[08]----00.0
                                             |
      +-04.0-[09]----00.0
                                             |
      \-05.0-[0a]----00.0

\-03.0-[0b-0e]----00.0-[0c-0e]--+-02.0-[0d]--+-00.0

      |            \-00.1

      \-04.0-[0e]--+-00.0

                   \-00.1
=================8<================
Thanks!

Ian Campbell

2011-Dec-14 09:36 UTC

head link

Re: PTE Write access is not set

On Wed, 2011-12-14 at 08:05 +0000, Liwei wrote:> Hello list,
>     Using the 3.1.5 kernel and 4.1.3-rc1-pre xen (Latest ChangeSet:
> Tue Dec 06 10:53:12 2011 +0000 23199:d9f8316a8291), these crashes
> occur after about a day or so of running a HVM:
> =================8<================> (XEN) realmode.c:115:d9 Failed
to emulate insn.
> (XEN) realmode.c:165:d9 Real-mode emulation failed @ e40a:00005f6c: d9
> 33 01 fc 30 34
I think this is FNSTENV/FSTENV which doesn''t seem to be emulated by
Xen :-( xen/arch/x86/x86_emulate/x86_emulate.c has:
                 /* case 6: fstenv - TODO */

Ian.
> (XEN) domain_crash called from realmode.c:166
> (XEN) Domain 9 (vcpu#0) crashed on cpu#6:
> (XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
> (XEN) CPU:    6
> (XEN) RIP:    e40a:[<0000000000005f6c>]
> (XEN) RFLAGS: 0000000000010002   CONTEXT: hvm guest
> (XEN) rax: 000000000000e7e5   rbx: 0000000000000000   rcx: 00000000000d0000
> (XEN) rdx: 00000000000001f7   rsi: 0000000000000000   rdi: 0000000000000dba
> (XEN) rbp: 0000000000000182   rsp: 0000000000000db0   r8:  0000000000000000
> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
> (XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
> (XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
> (XEN) cr3: 0000000000000000   cr2: 0000000000000000
> (XEN) ds: 9fc0   es: 9e00   fs: 0000   gs: 0000   ss: 9e00   cs: e40a
> =================8<================> (XEN) realmode.c:115:d5 Failed
to emulate insn.
> (XEN) realmode.c:165:d5 Real-mode emulation failed @ c5d5:000242bc: d9
> 33 01 fc 30 34
> (XEN) domain_crash called from realmode.c:166
> (XEN) Domain 5 (vcpu#0) crashed on cpu#0:
> (XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
> (XEN) CPU:    0
> (XEN) RIP:    c5d5:[<00000000000242bc>]
> (XEN) RFLAGS: 0000000000010013   CONTEXT: hvm guest
> (XEN) rax: 000000000000001b   rbx: 0000000000000702   rcx: 0000000000010ba7
> (XEN) rdx: 0000000000008fd5   rsi: 0000000000001333   rdi: 000000000000900e
> (XEN) rbp: 000000000000703e   rsp: 000000000000c5cf   r8:  0000000000000000
> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
> (XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
> (XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
> (XEN) cr3: 0000000000000000   cr2: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: c5d5
> (XEN) [VT-D]iommu.c:856: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]iommu.c:831: DMAR:[DMA Write] Request device [0d:00.1]
> fault addr d8964f44000, iommu reg = ffff82c3fff57000
> (XEN) DMAR:[fault reason 05h] PTE Write access is not set
> (XEN) print_vtd_entries: iommu = ffff83045ce565c0 bdf = d:0.1 gmfn =
d8964f44
> (XEN)     root_entry = ffff8304519d1000
> (XEN)     root_entry[d] = 4127af001
> (XEN)     context = ffff8304127af000
> (XEN)     context[1] = 2_414aca001
> (XEN)     l4 = ffff830414aca000
> (XEN)     l4_index = 1b
> (XEN)     l4[1b] = 0
> (XEN)     l4[1b] not present
> =================8<================> 
>     If I try to restart the crashed VM, the same crash always occurs
> again during POST (the last thing I see on the VNC console is the BIOS
> boot, before the connection terminates). A dom0 reboot is necessary to
> get things to work again.
>     0d:00.1 is part of an Intel 82575GB quad port network adapter. The
> adapter contains three ports, 0d:00.0, 0d:00.1, 0e:00.0, 0e:00.1. Each
> of them is assigned to a VM. So far the crashes have occurred to VMs
> using 0d:00.1, 0e:00.0 and 0e:00.1. (Pretty sure a crash involving
> 0d:00.0 will come soon).
>     Is this a potential bug, or (more likely?) problematic hardware?
> 
> The relevant PCI subtree looks like this:
> =================8<================>  \-[0000:00]-+-00.0
>              +-03.0-[01]--+-00.0
>              |            \-00.1
>              +-05.0-[02-0e]----00.0-[03-0e]--+-00.0-[04]----00.0
> 
> +-02.0-[05-0a]----00.0-[06-0a]--+-02.0-[07]----00.0
>                                              |
>       +-03.0-[08]----00.0
>                                              |
>       +-04.0-[09]----00.0
>                                              |
>       \-05.0-[0a]----00.0
> 
> \-03.0-[0b-0e]----00.0-[0c-0e]--+-02.0-[0d]--+-00.0
> 
>       |            \-00.1
> 
>       \-04.0-[0e]--+-00.0
> 
>                    \-00.1
> =================8<================> 
> Thanks!
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Keir Fraser

2011-Dec-14 09:41 UTC

head link

Re: PTE Write access is not set

On 14/12/2011 09:36, "Ian Campbell" <Ian.Campbell@citrix.com>
wrote:
> On Wed, 2011-12-14 at 08:05 +0000, Liwei wrote:
>> Hello list,
>>     Using the 3.1.5 kernel and 4.1.3-rc1-pre xen (Latest ChangeSet:
>> Tue Dec 06 10:53:12 2011 +0000 23199:d9f8316a8291), these crashes
>> occur after about a day or so of running a HVM:
>> =================8<================>> (XEN) realmode.c:115:d9
Failed to emulate insn.
>> (XEN) realmode.c:165:d9 Real-mode emulation failed @ e40a:00005f6c: d9
>> 33 01 fc 30 34
> 
> I think this is FNSTENV/FSTENV which doesn''t seem to be emulated
by
> Xen :-( xen/arch/x86/x86_emulate/x86_emulate.c has:
>                  /* case 6: fstenv - TODO */
I wonder why a long-running HVM guest would be in real mode executing FP
code. That seems a bit fishy.

 -- Keir

Jan Beulich

2011-Dec-14 09:57 UTC

head link

Re: PTE Write access is not set

>>> On 14.12.11 at 09:05, Liwei <xieliwei@gmail.com> wrote:
>     Using the 3.1.5 kernel and 4.1.3-rc1-pre xen (Latest ChangeSet:
The kernel version is the HVM guest one or the host one, because ...
> Tue Dec 06 10:53:12 2011 +0000 23199:d9f8316a8291), these crashes
> occur after about a day or so of running a HVM:
> =================8<================> (XEN) realmode.c:115:d9 Failed
to emulate insn.
> (XEN) realmode.c:165:d9 Real-mode emulation failed @ e40a:00005f6c: d9
> 33 01 fc 30 34
... if this is Linux I''m having a rather hard time seeing why it would
enter realmode at normal run time (and then even use floating point
instructions there). This would be expected at boot time and at most
during guest-internal suspend/resume, but I''m sure you would have
mentioned if the guest was in the process of doing the latter (and
you excluded the former).

I''m rather suspecting the VM to have gone astray earlier.
> (XEN) domain_crash called from realmode.c:166
> (XEN) Domain 9 (vcpu#0) crashed on cpu#6:
> (XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
> (XEN) CPU:    6
> (XEN) RIP:    e40a:[<0000000000005f6c>]
> (XEN) RFLAGS: 0000000000010002   CONTEXT: hvm guest
> (XEN) rax: 000000000000e7e5   rbx: 0000000000000000   rcx: 00000000000d0000
> (XEN) rdx: 00000000000001f7   rsi: 0000000000000000   rdi: 0000000000000dba
> (XEN) rbp: 0000000000000182   rsp: 0000000000000db0   r8:  0000000000000000
> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
> (XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
> (XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
> (XEN) cr3: 0000000000000000   cr2: 0000000000000000
> (XEN) ds: 9fc0   es: 9e00   fs: 0000   gs: 0000   ss: 9e00   cs: e40a
> =================8<================> (XEN) realmode.c:115:d5 Failed
to emulate insn.
> (XEN) realmode.c:165:d5 Real-mode emulation failed @ c5d5:000242bc: d9
> 33 01 fc 30 34
> (XEN) domain_crash called from realmode.c:166
> (XEN) Domain 5 (vcpu#0) crashed on cpu#0:
> (XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
> (XEN) CPU:    0
> (XEN) RIP:    c5d5:[<00000000000242bc>]
> (XEN) RFLAGS: 0000000000010013   CONTEXT: hvm guest
> (XEN) rax: 000000000000001b   rbx: 0000000000000702   rcx: 0000000000010ba7
> (XEN) rdx: 0000000000008fd5   rsi: 0000000000001333   rdi: 000000000000900e
> (XEN) rbp: 000000000000703e   rsp: 000000000000c5cf   r8:  0000000000000000
> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
> (XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
> (XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
> (XEN) cr3: 0000000000000000   cr2: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: c5d5
Further, you present two instances of a VM death, ...
> (XEN) [VT-D]iommu.c:856: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]iommu.c:831: DMAR:[DMA Write] Request device [0d:00.1]
> fault addr d8964f44000, iommu reg = ffff82c3fff57000
> (XEN) DMAR:[fault reason 05h] PTE Write access is not set
> (XEN) print_vtd_entries: iommu = ffff83045ce565c0 bdf = d:0.1 gmfn = 
> d8964f44
> (XEN)     root_entry = ffff8304519d1000
> (XEN)     root_entry[d] = 4127af001
> (XEN)     context = ffff8304127af000
> (XEN)     context[1] = 2_414aca001
> (XEN)     l4 = ffff830414aca000
> (XEN)     l4_index = 1b
> (XEN)     l4[1b] = 0
> (XEN)     l4[1b] not present
... but only one instance of an IOMMU fault. Hence it''s not even clear
they are connected. It also can''t be the result of an immediate guest
restart, as the second VM that''s dying has a lower numbered domain
ID than the first one.

Further, the IOMMU fault is not about the lack of write access, but the
requested mapping simply is not present at all. Which is because the gmfn
looks rather bogus - I doubt you''re running a guest with nearly 14Tb
of memory (Xen doesn''t even support that much) or with an extremely
spares physical address space (you would have to have arranged this
yourself, no VMs get created this way afaik).

Please provide consistent and complete information.

Jan

Ian Campbell

2011-Dec-14 10:01 UTC

head link

Re: PTE Write access is not set

On Wed, 2011-12-14 at 09:41 +0000, Keir Fraser wrote:> On 14/12/2011 09:36, "Ian Campbell"
<Ian.Campbell@citrix.com> wrote:
> 
> > On Wed, 2011-12-14 at 08:05 +0000, Liwei wrote:
> >> Hello list,
> >>     Using the 3.1.5 kernel and 4.1.3-rc1-pre xen (Latest
ChangeSet:
> >> Tue Dec 06 10:53:12 2011 +0000 23199:d9f8316a8291), these crashes
> >> occur after about a day or so of running a HVM:
> >> =================8<================> >> (XEN)
realmode.c:115:d9 Failed to emulate insn.
> >> (XEN) realmode.c:165:d9 Real-mode emulation failed @
e40a:00005f6c: d9
> >> 33 01 fc 30 34
> > 
> > I think this is FNSTENV/FSTENV which doesn''t seem to be
emulated by
> > Xen :-( xen/arch/x86/x86_emulate/x86_emulate.c has:
> >                  /* case 6: fstenv - TODO */
> 
> I wonder why a long-running HVM guest would be in real mode executing FP
> code. That seems a bit fishy.
I assumed it was some driver calling into a BIOS ROM or something
horrific like that. 

RFLAGS was 0x0000000000010002 (RF and the MB1 bit), no VM86 etc in
there.

CS is e40a -- Is that ring 2? surely not... I don''t see anything in
Linux which uses that segment (it''s not an easy grep though).

So, yes, fishy.

Ian.
> 
>  -- Keir
> 
>

Tim Deegan

2011-Dec-14 12:58 UTC

head link

Re: PTE Write access is not set

At 09:57 +0000 on 14 Dec (1323856663), Jan Beulich
wrote:> > (XEN) realmode.c:115:d9 Failed to emulate insn.
> > (XEN) realmode.c:165:d9 Real-mode emulation failed @ e40a:00005f6c: d9
> > 33 01 fc 30 34
> 
> ... if this is Linux I''m having a rather hard time seeing why it
would
> enter realmode at normal run time (and then even use floating point
> instructions there). This would be expected at boot time and at most
> during guest-internal suspend/resume, but I''m sure you would have
> mentioned if the guest was in the process of doing the latter (and
> you excluded the former).
> 
> I''m rather suspecting the VM to have gone astray earlier.
> 
> > (XEN) domain_crash called from realmode.c:166
> > (XEN) Domain 9 (vcpu#0) crashed on cpu#6:
> > (XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
> > (XEN) CPU:    6
> > (XEN) RIP:    e40a:[<0000000000005f6c>]
> > (XEN) RFLAGS: 0000000000010002   CONTEXT: hvm guest
> > (XEN) rax: 000000000000e7e5   rbx: 0000000000000000   rcx:
00000000000d0000
> > (XEN) rdx: 00000000000001f7   rsi: 0000000000000000   rdi:
0000000000000dba
> > (XEN) rbp: 0000000000000182   rsp: 0000000000000db0   r8: 
0000000000000000
> > (XEN) r9:  0000000000000000   r10: 0000000000000000   r11:
0000000000000000
> > (XEN) r12: 0000000000000000   r13: 0000000000000000   r14:
0000000000000000
> > (XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4:
0000000000000000
> > (XEN) cr3: 0000000000000000   cr2: 0000000000000000
> > (XEN) ds: 9fc0   es: 9e00   fs: 0000   gs: 0000   ss: 9e00   cs: e40a
I''d say the guest has recently rebooted - even if it was going back
down to real mode for some reason, I doubt that cr2 and cr3 would be all
zeros.

Tim.

Liwei

2011-Dec-14 14:31 UTC

head link

Re: PTE Write access is not set

Hello all,
    My apologies for the unclear email. Here are the events in
chronological order:

======First instance of VM Crash=====Guest is openSuSE 11.4 on
2.6.37.6-0.5-desktop
No symptoms in guest logs, logging just stopped.
Looking at the serial logs, it appears that the guest just vapourised
(unless the "First re-creation of guest" below is actually the first
crash, though Jan and Tim mentioned that that crash couldn''t have
occurred during run-time).
This VM has one part of the Intel 82575GB (0e:00.1) pass-throughed.

++++First re-creation of guest++++
xl create /etc/xen/vm-test4.cfg

----Xen log----
(XEN) HVM7: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM7: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk ( 160 GBytes)
(XEN) realmode.c:115:d7 Failed to emulate insn.
(XEN) realmode.c:165:d7 Real-mode emulation failed @ a8c0:00005a8e: ff
fa 00 15 99 2c
(XEN) domain_crash called from realmode.c:166
(XEN) Domain 7 (vcpu#0) crashed on cpu#3:
(XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    3
(XEN) RIP:    a8c0:[<0000000000005a8e>]
(XEN) RFLAGS: 0000000000010006   CONTEXT: hvm guest
(XEN) rax: 0000000000000120   rbx: 0000000000000000   rcx: 00000000000dffff
(XEN) rdx: 00000000000001f7   rsi: 000000000000ffff   rdi: 000000000000fffe
(XEN) rbp: 0000000000000db5   rsp: 000000000000eff8   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
(XEN) cr3: 0000000000000000   cr2: 0000000000000000
(XEN) ds: 9fc0   es: 9e00   fs: 0000   gs: 0000   ss: 9e00   cs: a8c0

++++Second re-creation of guest++++
xl create /etc/xen/vm-test4.cfg

(XEN) HVM8: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM8: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk ( 160 GBytes)
(XEN) realmode.c:115:d8 Failed to emulate insn.
(XEN) realmode.c:165:d8 Real-mode emulation failed @ a8c0:0000fcb5: 72
5c 1b 80 c4 82
(XEN) domain_crash called from realmode.c:166
(XEN) Domain 8 (vcpu#0) crashed on cpu#5:
(XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    5
(XEN) RIP:    a8c0:[<000000000000fcb5>]
(XEN) RFLAGS: 0000000000010002   CONTEXT: hvm guest
(XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx: 00000000000d0000
(XEN) rdx: 00000000000001f7   rsi: 0000000000000000   rdi: 0000000000000dba
(XEN) rbp: 0000000000000db8   rsp: 0000000000000d38   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
(XEN) cr3: 0000000000000000   cr2: 0000000000000000
(XEN) ds: 9fc0   es: 9e00   fs: 0000   gs: 0000   ss: 9e00   cs: a8c0

[Did the same thing a few more times with similar results and gave up,
rebooted dom0 and all was well]


[About 20 hours later]
======Second instance of VM Crash=====Guest is pfSense 2.1 on FreeBSD
8.1-RELEASE-p6
Unfortunately, I didn''t get the chance to extract the guest logs as
they''re purged on boot.
Similarly, looking at the serial logs, it appears that the guest just
vapourised (unless the "First re-creation of guest" below is actually
the first crash, though Jan and Tim mentioned that that crash couldn''t
have occurred during run-time).
This VM has two parts of the Intel 82575GB (0d:00.0 and 0d:00.1) pass-throughed.

++++First re-creation of guest++++
xl create /etc/xen/vm-pfsense.cfg

(XEN) HVM5: Booting from Hard Disk...
(XEN) HVM5: Booting from 0000:7c00
(XEN) realmode.c:115:d5 Failed to emulate insn.
(XEN) realmode.c:165:d5 Real-mode emulation failed @ c5d5:000242bc: d9
33 01 fc 30 34
(XEN) domain_crash called from realmode.c:166
(XEN) Domain 5 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    c5d5:[<00000000000242bc>]
(XEN) RFLAGS: 0000000000010013   CONTEXT: hvm guest
(XEN) rax: 000000000000001b   rbx: 0000000000000702   rcx: 0000000000010ba7
(XEN) rdx: 0000000000008fd5   rsi: 0000000000001333   rdi: 000000000000900e
(XEN) rbp: 000000000000703e   rsp: 000000000000c5cf   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
(XEN) cr3: 0000000000000000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: c5d5
(XEN) [VT-D]iommu.c:856: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:831: DMAR:[DMA Write] Request device [0d:00.1]
fault addr d8964f44000, iommu reg = ffff82c3fff57000
(XEN) DMAR:[fault reason 05h] PTE Write access is not set
(XEN) print_vtd_entries: iommu = ffff83045ce565c0 bdf = d:0.1 gmfn = d8964f44
(XEN)     root_entry = ffff8304519d1000
(XEN)     root_entry[d] = 4127af001
(XEN)     context = ffff8304127af000
(XEN)     context[1] = 2_414aca001
(XEN)     l4 = ffff830414aca000
(XEN)     l4_index = 1b
(XEN)     l4[1b] = 0
(XEN)     l4[1b] not present

++++Second re-creation of guest++++
xl create /etc/xen/vm-pfsense.cfg

(XEN) HVM6: Options: apmbios pcibios eltorito PMM
(XEN) HVM6:
(XEN) realmode.c:115:d6 Failed to emulate insn.
(XEN) realmode.c:165:d6 Real-mode emulation failed @ a8c0:00005a83: 72
5c 1b 80 c4 82
(XEN) domain_crash called from realmode.c:166
(XEN) Domain 6 (vcpu#0) crashed on cpu#6:
(XEN) ----[ Xen-4.1.3-rc1-pre  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    6
(XEN) RIP:    a8c0:[<0000000000005a83>]
(XEN) RFLAGS: 0000000000010086   CONTEXT: hvm guest
(XEN) rax: 0000000000000000   rbx: 0000000000000db4   rcx: 00000000000f0027
(XEN) rdx: 0000000000000004   rsi: 0000000000000006   rdi: 0000000000000db2
(XEN) rbp: 0000000000000dc8   rsp: 000000000000df33   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
(XEN) cr3: 0000000000000000   cr2: 0000000000000000
(XEN) ds: 9fc0   es: c000   fs: 0000   gs: 0000   ss: 9e00   cs: a8c0
(XEN) [VT-D]iommu.c:856: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:831: DMAR:[DMA Write] Request device [0d:00.1]
fault addr 1a16ee8000000, iommu reg = ffff82c3fff57000
(XEN) DMAR:[fault reason 04h] Access beyond MGAW
(XEN) print_vtd_entries: iommu = ffff83045ce565c0 bdf = d:0.1 gmfn = 1a16ee8000
(XEN)     root_entry = ffff8304519d1000
(XEN)     root_entry[d] = 4127af001
(XEN)     context = ffff8304127af000
(XEN)     context[1] = 102_2159b1001
(XEN)     l4 = ffff8302159b1000
(XEN)     l4_index = 142
(XEN)     l4[142] = 0
(XEN)     l4[142] not present

[Did the same thing a few more times with similar results and gave up,
rebooted dom0 and all was well]

Typical log of a booting openSuSE guest:
(XEN) HVM2: HVM Loader
(XEN) HVM2: Detected Xen v4.1.3-rc1-pre
(XEN) HVM2: CPU speed is 2930 MHz
(XEN) HVM2: Xenbus rings @0xfeffc000, event channel 9
(XEN) irq.c:264: Dom2 PCI link 0 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:264: Dom2 PCI link 1 changed 0 -> 10
(XEN) HVM2: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:264: Dom2 PCI link 2 changed 0 -> 11
(XEN) HVM2: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:264: Dom2 PCI link 3 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 3 routed to IRQ5
(XEN) HVM2: pci dev 01:2 INTD->IRQ5
(XEN) HVM2: pci dev 01:3 INTA->IRQ10
(XEN) HVM2: pci dev 03:0 INTA->IRQ5
(XEN) HVM2: pci dev 04:0 INTA->IRQ5
(XEN) HVM2: pci dev 05:0 INTB->IRQ11
(XEN) HVM2: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM2: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM2: pci dev 05:0 bar 14 size 00200000: f3000000
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) HVM2: pci dev 05:0 bar 10 size 00020000: f3200000
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) HVM2: pci dev 05:0 bar 1c size 00004000: f3220000
(XEN) HVM2: pci dev 02:0 bar 14 size 00001000: f3224000
(XEN) HVM2: pci dev 04:0 bar 10 size 00000400: 0000c001
(XEN) HVM2: pci dev 03:0 bar 10 size 00000100: 0000c401
(XEN) HVM2: pci dev 04:0 bar 14 size 00000100: 0000c501
(XEN) HVM2: pci dev 01:2 bar 20 size 00000020: 0000c601
(XEN) HVM2: pci dev 05:0 bar 18 size 00000020: 0000c621
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c620 f_mport=a880 np=20
(XEN) HVM2: pci dev 01:1 bar 20 size 00000010: 0000c641
(XEN) HVM2: Multiprocessor initialisation:
(XEN) HVM2:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2:  - CPU1 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2:  - CPU2 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2:  - CPU3 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2:  - CPU4 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2:  - CPU5 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2:  - CPU6 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2:  - CPU7 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM2: Writing SMBIOS tables ...
(XEN) HVM2: Loading ROMBIOS ...
(XEN) HVM2: 12604 bytes of ROMBIOS high-memory extensions:
(XEN) HVM2:   Relocating to 0xfc000000-0xfc00313c ... done
(XEN) HVM2: Creating MP tables ...
(XEN) HVM2: Loading Cirrus VGABIOS ...
(XEN) HVM2: Loading ACPI ...
(XEN) HVM2:  - Lo data: 000ea020-000ea04f
(XEN) HVM2:  - Hi data: fc003400-fc01355f
(XEN) HVM2: vm86 TSS at fc013800
(XEN) HVM2: BIOS map:
(XEN) HVM2:  c0000-c8fff: VGA BIOS
(XEN) HVM2:  eb000-eb26b: SMBIOS tables
(XEN) HVM2:  f0000-fffff: Main BIOS
(XEN) HVM2: E820 table:
(XEN) HVM2:  [00]: 00000000:00000000 - 00000000:0009e000: RAM
(XEN) HVM2:  [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED
(XEN) HVM2:  [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED
(XEN) HVM2:  HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM2:  [03]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM2:  [04]: 00000000:00100000 - 00000000:7f800000: RAM
(XEN) HVM2:  HOLE: 00000000:7f800000 - 00000000:fc000000
(XEN) HVM2:  [05]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM2: Invoking ROMBIOS ...
(XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d2 entering stdvga and caching modes
(XEN) HVM2: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM2: Bochs BIOS - build: 06/23/99
(XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM2: Options: apmbios pcibios eltorito PMM
(XEN) HVM2:
(XEN) HVM2: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM2: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk ( 160 GBytes)
(XEN) HVM2: IDE time out
(XEN) HVM2:
(XEN) HVM2:
(XEN) HVM2:
(XEN) HVM2: Press F12 for boot menu.
(XEN) HVM2:
(XEN) HVM2: Booting from Hard Disk...
(XEN) HVM2: Booting from 0000:7c00
(XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=81
(XEN) HVM2: int13_harddisk: function 08, unmapped device for ELDL=81
(XEN) HVM2: *** int 15h function AX=00c0, BX=0000 not yet supported!
(XEN) stdvga.c:151:d2 leaving stdvga
(XEN) stdvga.c:147:d2 entering stdvga and caching modes
(XEN) HVM2: *** int 15h function AX=ec00, BX=0002 not yet supported!
(XEN) HVM2: KBD: unsupported int 16h function 03
(XEN) HVM2: *** int 15h function AX=e980, BX=0000 not yet supported!
(XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=81
(XEN) HVM2: int13_harddisk: function 02, unmapped device for ELDL=81
(XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=82
(XEN) HVM2: int13_harddisk: function 02, unmapped device for ELDL=82
(XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=83
(XEN) HVM2: int13_harddisk: function 02, unmapped device for ELDL=83
(XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=84
(XEN) HVM2: int13_harddisk: function 02, unmapped device for ELDL=84
(XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=85
(XEN) HVM2: int13_harddisk: function 02, unmapped device for ELDL=85
(XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=86
(XEN) HVM2: int13_harddisk: function 02, unmapped device for ELDL=86
(XEN) HVM2: int13_harddisk: function 41, unmapped device for ELDL=87
(XEN) HVM2: int13_harddisk: function 02, unmapped device for ELDL=87
(XEN) HVM2: int13_harddisk: function 41, ELDL out of range 88
(XEN) HVM2: int13_harddisk: function 02, ELDL out of range 88
(XEN) HVM2: int13_harddisk: function 41, ELDL out of range 89
(XEN) HVM2: int13_harddisk: function 02, ELDL out of range 89
(XEN) HVM2: int13_harddisk: function 41, ELDL out of range 8a
(XEN) HVM2: int13_harddisk: function 02, ELDL out of range 8a
(XEN) HVM2: int13_harddisk: function 41, ELDL out of range 8b
(XEN) HVM2: int13_harddisk: function 02, ELDL out of range 8b
(XEN) HVM2: int13_harddisk: function 41, ELDL out of range 8c
(XEN) HVM2: int13_harddisk: function 02, ELDL out of range 8c
(XEN) HVM2: int13_harddisk: function 41, ELDL out of range 8d
(XEN) HVM2: int13_harddisk: function 02, ELDL out of range 8d
(XEN) HVM2: int13_harddisk: function 41, ELDL out of range 8e
(XEN) HVM2: int13_harddisk: function 02, ELDL out of range 8e
(XEN) HVM2: int13_harddisk: function 41, ELDL out of range 8f
(XEN) HVM2: int13_harddisk: function 02, ELDL out of range 8f
(XEN) stdvga.c:151:d2 leaving stdvga
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fb620 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=fb400 nr_mfns=200
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c620 f_mport=a880 np=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3220 mfn=fb644 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3220 mfn=fb644 nr_mfns=1
(XEN) irq.c:264: Dom2 PCI link 0 changed 5 -> 0
(XEN) irq.c:264: Dom2 PCI link 1 changed 10 -> 0
(XEN) irq.c:264: Dom2 PCI link 2 changed 11 -> 0
(XEN) irq.c:264: Dom2 PCI link 3 changed 5 -> 0
(XEN) irq.c:330: Dom2 callback via changed to PCI INTx Dev 0x03 IntA
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) msi.c:867: MSI is already in use on device 0e:00.1
(XEN) physdev.c:170: dom3: 16:-1 already mapped to 16
(XEN) physdev.c:170: dom3: 19:-1 already mapped to 20

Typical log of a booting pfSense guest:
(XEN) HVM1: HVM Loader
(XEN) HVM1: Detected Xen v4.1.3-rc1-pre
(XEN) HVM1: CPU speed is 2930 MHz
(XEN) HVM1: Xenbus rings @0xfeffc000, event channel 9
(XEN) irq.c:264: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:264: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:264: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:264: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: pci dev 04:0 INTA->IRQ5
(XEN) HVM1: pci dev 05:0 INTB->IRQ11
(XEN) HVM1: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM1: pci dev 04:0 bar 14 size 00200000: f3000000
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) HVM1: pci dev 05:0 bar 14 size 00200000: f3200000
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) HVM1: pci dev 04:0 bar 10 size 00020000: f3400000
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) HVM1: pci dev 05:0 bar 10 size 00020000: f3420000
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) HVM1: pci dev 04:0 bar 1c size 00004000: f3440000
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) HVM1: pci dev 05:0 bar 1c size 00004000: f3444000
(XEN) HVM1: pci dev 02:0 bar 14 size 00001000: f3448000
(XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) HVM1: pci dev 04:0 bar 18 size 00000020: 0000c101
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c100 f_mport=9c00 np=20
(XEN) HVM1: pci dev 05:0 bar 18 size 00000020: 0000c121
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c120 f_mport=9880 np=20
(XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c141
(XEN) HVM1: Multiprocessor initialisation:
(XEN) HVM1:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1:  - CPU1 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1:  - CPU2 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1:  - CPU3 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1:  - CPU4 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1:  - CPU5 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1:  - CPU6 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1:  - CPU7 ... 36-bit phys ... fixed MTRRs ... var MTRRs
[2/8] ... done.
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 12604 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1:   Relocating to 0xfc000000-0xfc00313c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading Cirrus VGABIOS ...
(XEN) HVM1: Loading ACPI ...
(XEN) HVM1:  - Lo data: 000ea020-000ea04f
(XEN) HVM1:  - Hi data: fc003400-fc01355f
(XEN) HVM1: vm86 TSS at fc013800
(XEN) HVM1: BIOS map:
(XEN) HVM1:  c0000-c8fff: VGA BIOS
(XEN) HVM1:  eb000-eb26b: SMBIOS tables
(XEN) HVM1:  f0000-fffff: Main BIOS
(XEN) HVM1: E820 table:
(XEN) HVM1:  [00]: 00000000:00000000 - 00000000:0009e000: RAM
(XEN) HVM1:  [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED
(XEN) HVM1:  [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED
(XEN) HVM1:  HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM1:  [03]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM1:  [04]: 00000000:00100000 - 00000000:3f800000: RAM
(XEN) HVM1:  HOLE: 00000000:3f800000 - 00000000:fc000000
(XEN) HVM1:  [05]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d1 entering stdvga and caching modes
(XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM1: Bochs BIOS - build: 06/23/99
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM1: Options: apmbios pcibios eltorito PMM
(XEN) HVM1:
(XEN) HVM1: ata0 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
(XEN) HVM1: ata0-1: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM1: ata0  slave: QEMU HARDDISK ATA-7 Hard-Disk (16384 MBytes)
(XEN) HVM1:
(XEN) HVM1:
(XEN) HVM1:
(XEN) HVM1: Press F12 for boot menu.
(XEN) HVM1:
(XEN) HVM1: Booting from Hard Disk...
(XEN) HVM1: Booting from 0000:7c00
(XEN) irq.c:264: Dom1 PCI link 0 changed 5 -> 0
(XEN) irq.c:264: Dom1 PCI link 1 changed 10 -> 0
(XEN) irq.c:264: Dom1 PCI link 2 changed 11 -> 0
(XEN) irq.c:264: Dom1 PCI link 3 changed 5 -> 0
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c100 f_mport=9c00 np=20
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c100 f_mport=9c00 np=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3400 mfn=fa900 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=faa00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3440 mfn=fa940 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3440 mfn=fa940 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:1062:d0 ioport_map:remove f_gport=c120 f_mport=9880 np=20
(XEN) domctl.c:1038:d0 ioport_map:add f_gport=c120 f_mport=9880 np=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3420 mfn=fa920 nr_mfns=20
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3200 mfn=fac00 nr_mfns=200
(XEN) domctl.c:982:d0 memory_map:add: gfn=f3444 mfn=fa944 nr_mfns=4
(XEN) domctl.c:992:d0 memory_map:remove: gfn=f3444 mfn=fa944 nr_mfns=1
(XEN) msi.c:867: MSI is already in use on device 0d:00.0
(XEN) msi.c:867: MSI is already in use on device 0d:00.0
(XEN) msi.c:867: MSI is already in use on device 0d:00.0
(XEN) msi.c:867: MSI is already in use on device 0d:00.0
(XEN) msi.c:867: MSI is already in use on device 0d:00.0
(XEN) msi.c:867: MSI is already in use on device 0d:00.1
(XEN) msi.c:867: MSI is already in use on device 0d:00.1
(XEN) msi.c:867: MSI is already in use on device 0d:00.1
(XEN) msi.c:867: MSI is already in use on device 0d:00.1
(XEN) msi.c:867: MSI is already in use on device 0d:00.1

Thanks!

Jan Beulich

2011-Dec-14 14:52 UTC

head link

Re: PTE Write access is not set

>>> On 14.12.11 at 15:31, Liwei <xieliwei@gmail.com> wrote:
> ======First instance of VM Crash=====> Guest is openSuSE 11.4 on
2.6.37.6-0.5-desktop
> No symptoms in guest logs, logging just stopped.
> Looking at the serial logs, it appears that the guest just vapourised
> (unless the "First re-creation of guest" below is actually the
first
> crash, though Jan and Tim mentioned that that crash couldn''t have
> occurred during run-time).
> This VM has one part of the Intel 82575GB (0e:00.1) pass-throughed.
> 
> ++++First re-creation of guest++++
> xl create /etc/xen/vm-test4.cfg
> 
> ----Xen log----
> (XEN) HVM7: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM7: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk ( 160 GBytes)
> (XEN) realmode.c:115:d7 Failed to emulate insn.
> (XEN) realmode.c:165:d7 Real-mode emulation failed @ a8c0:00005a8e: ff
> fa 00 15 99 2c
> (XEN) domain_crash called from realmode.c:166
> 
> ++++Second re-creation of guest++++
> xl create /etc/xen/vm-test4.cfg
> 
> (XEN) HVM8: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM8: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk ( 160 GBytes)
> (XEN) realmode.c:115:d8 Failed to emulate insn.
> (XEN) realmode.c:165:d8 Real-mode emulation failed @ a8c0:0000fcb5: 72
> 5c 1b 80 c4 82
> (XEN) domain_crash called from realmode.c:166
> 
> [Did the same thing a few more times with similar results and gave up,
> rebooted dom0 and all was well]
So this suggests that the virtual firmware has a problem coming back
up with the passed through device still active (and the IOMMU may in
fact be shielding the rest of the system from bad effects of this).

This by itself is a problem, and the sudden death of the guest is
another (particularly if indeed there''s no trace of that event in any
logs). Debugging this may be difficult for anyone who can''t
reproduce it, so we''ll have to rely on you providing more insight.
Enable more debugging/verbosity in the hypervisor and try again is
probably all I can suggest at this point.

Jan

Liwei

2011-Dec-14 14:59 UTC

head link

Re: PTE Write access is not set

On 14 December 2011 22:52, Jan Beulich <JBeulich@suse.com>
wrote:>
> So this suggests that the virtual firmware has a problem coming back
> up with the passed through device still active (and the IOMMU may in
> fact be shielding the rest of the system from bad effects of this).
>
> This by itself is a problem, and the sudden death of the guest is
> another (particularly if indeed there''s no trace of that event in
any
> logs). Debugging this may be difficult for anyone who can''t
> reproduce it, so we''ll have to rely on you providing more insight.
> Enable more debugging/verbosity in the hypervisor and try again is
> probably all I can suggest at this point.
Currently I have earlyprintk=xen, loglvl=all and guest_loglvl=all.
Other than adding loglevel=10 and debug initcall_debug, what other
options and/or procedures do you suggest?
>
> Jan
>

Jan Beulich

2011-Dec-14 15:16 UTC

head link

Re: PTE Write access is not set

>>> On 14.12.11 at 15:59, Liwei <xieliwei@gmail.com> wrote:
> On 14 December 2011 22:52, Jan Beulich <JBeulich@suse.com> wrote:
>>
>> So this suggests that the virtual firmware has a problem coming back
>> up with the passed through device still active (and the IOMMU may in
>> fact be shielding the rest of the system from bad effects of this).
>>
>> This by itself is a problem, and the sudden death of the guest is
>> another (particularly if indeed there''s no trace of that event
in any
>> logs). Debugging this may be difficult for anyone who can''t
>> reproduce it, so we''ll have to rely on you providing more
insight.
>> Enable more debugging/verbosity in the hypervisor and try again is
>> probably all I can suggest at this point.
> 
> Currently I have earlyprintk=xen, loglvl=all and guest_loglvl=all.
"earlyprintk=xen" is completely irrelevant here. For the others, as
I see them commonly misplaced, I''ll take the risk of embarrassing you
by asking whether they are on the Xen command line (and not the
kernel one).
> Other than adding loglevel=10 and debug initcall_debug, what other
> options and/or procedures do you suggest?
"hvm_debug=" comes to mind (grep for the various DBG_LEVEL_*
defines and uses to see which one might provide greatest benefit,
and be prepared to go through a lot of data afterwards, though
presumably just the tail is going to be interesting if you make the
guest not immediately restart after death).

Jan

Liwei

2011-Dec-14 15:57 UTC

head link

Re: PTE Write access is not set

On 14 December 2011 23:16, Jan Beulich <JBeulich@suse.com>
wrote:>
> "earlyprintk=xen" is completely irrelevant here. For the others,
as
> I see them commonly misplaced, I''ll take the risk of embarrassing
you
> by asking whether they are on the Xen command line (and not the
> kernel one).
>
>
> "hvm_debug=" comes to mind (grep for the various DBG_LEVEL_*
> defines and uses to see which one might provide greatest benefit,
> and be prepared to go through a lot of data afterwards, though
> presumably just the tail is going to be interesting if you make the
> guest not immediately restart after death).
>
> Jan
>
Would this be okay? I enabled all levels except for the VLAPIC related ones.

GRUB_CMDLINE_LINUX_DEFAULT="quiet console=tty0 console=hvc0
earlyprintk=xen nomodeset xen-pciback.permissive
xen-pciback.hide=[snip] pci=resource_alignment=[snip] loglevel=10
debug initcall_debug hvm_debug=3647"
GRUB_CMDLINE_XEN="dom0_mem=1024M loglvl=all guest_loglvl=all
console_to_ring com1=115200,8n1,0xdc00,0 console=com1"

I''ll report back as soon as the crash occurs again.

Xen devel - Dec 2011 - PTE Write access is not set

PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set

Re: PTE Write access is not set