Xen devel - Aug 2005 - [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

I remember some people complained that x86_64 xenlinux failed to boot on
the systems with processor(s) that does not NX/XD. The patch should fix
the problem as far as I tested. 

Signed-off-by: Jun Nakajima <jun.nakajima@intel.com>

Jun
---
Intel Open Source Technology Center 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Nakajima, Jun wrote:
> I remember some people complained that x86_64 xenlinux failed to boot on
> the systems with processor(s) that does not NX/XD. The patch should fix
> the problem as far as I tested.
Should there be a warning if NX/XD is unsupported? If it isn''t because
of
a BIOS option, the user probably wants to know in order to turn on the option.

-- 
David Hopwood <david.nospam.hopwood@blueyonder.co.uk>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This is identical to the patch Scott Parish sent awhile back. Jun you
said that this was not a good way of going about fixing this (actually
it was kind of toward another problem ...but I think we were also
trying to get x86-64 xen to just boot at the time). I guess this is
the best way then?

On 8/17/05, Nakajima, Jun <jun.nakajima@intel.com>
wrote:
> I remember some people complained that x86_64 xenlinux failed to boot on
> the systems with processor(s) that does not NX/XD. The patch should fix
> the problem as far as I tested.
> 
> Signed-off-by: Jun Nakajima <jun.nakajima@intel.com>
> 
> Jun
> ---
> Intel Open Source Technology Center
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 
> 
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jerone Young wrote:
> This is identical to the patch Scott Parish sent awhile back. Jun you
> said that this was not a good way of going about fixing this (actually
> it was kind of toward another problem ...but I think we were also
> trying to get x86-64 xen to just boot at the time). I guess this is
> the best way then?
As far as I remember, it was not root caused at that time, then I was
not sure it was the right fix... 

The problem is that __PAGE_KERNEL is used by several places, such as
ioremap_change_attr
xen_contig_memory
blkback.c:  dispatch_probe

And __PAGE_KERNEL on native x86_64 Linux has always NX on:
#define __PAGE_KERNEL \
        (_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED |
_PAGE_NX)

So, actually we have two options to fix this:
1. Change __PAGE_KERNEL => _KERNPG_TABLE
2. The above (i.e. & __supported_pte_mask)

I don''t have particular preference, but the latter one''s
change is
smaller, and it can fix other places (that are using the attrributes
derived from __PAGE_KERNEL). But the former one is cleaner to me. If
people agree, I can make such a patch.

Jun
---
Intel Open Source Technology Center

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 Aug 2005, at 05:29, Nakajima, Jun wrote:
> I don''t have particular preference, but the latter one''s
change is
> smaller, and it can fix other places (that are using the attrributes
> derived from __PAGE_KERNEL). But the former one is cleaner to me. If
> people agree, I can make such a patch.
How does native x86/64 Linux avoid this problem? It doesn''t mask 
__PAGE_KERNEL with __supported_pte_mask, so I''m reluctant to take this 
patch. Seems to me there is some other underlying difference between us 
and native that is not best solved by diverging even further.

Unless native doesn;t boot on these failing boxes either?

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> On 18 Aug 2005, at 05:29, Nakajima, Jun wrote:
> 
>> I don''t have particular preference, but the latter
one''s change is
>> smaller, and it can fix other places (that are using the attrributes
>> derived from __PAGE_KERNEL). But the former one is cleaner to me. If
>> people agree, I can make such a patch.
> 
> How does native x86/64 Linux avoid this problem? It doesn''t mask
> __PAGE_KERNEL with __supported_pte_mask, so I''m reluctant to take
this
> patch. Seems to me there is some other underlying difference between
> us and native that is not best solved by diverging even further.
> 
> Unless native doesn;t boot on these failing boxes either?
> 
>   -- Keir
One obvious reason is: xen-specific drivers are using __PAGE_KERNEL (and
they don''t exist in the native Linux). If xen_contig_memory, for
example, does not work, it won''t boot... Fixing xen_contig_memory did
not fix the booting problem as long as I remember? Since then we fixed
some attribute problems in other xen-specific drivers when we brough up
domU, so I''ll take a look.

xen_contig_memory
...
    /* 3. Map the new extent in place of old pages. */
    for (i = 0; i < (1<<order); i++) {
        BUG_ON(HYPERVISOR_update_va_mapping(
            vstart + (i*PAGE_SIZE),
            __pte_ma(((mfn+i)<<PAGE_SHIFT)|__PAGE_KERNEL), 0));
        xen_machphys_update(mfn+i, (__pa(vstart)>>PAGE_SHIFT)+i);
        phys_to_machine_mapping[(__pa(vstart)>>PAGE_SHIFT)+i] = mfn+i;
    }


blkback.c:  dispatch_probe
static void dispatch_probe(blkif_t *blkif, blkif_request_t *req)
{
 ...
#ifdef CONFIG_XEN_BLKDEV_TAP_BE
    if ( HYPERVISOR_update_va_mapping_otherdomain(
        MMAP_VADDR(pending_idx, 0),
        (pte_t) { (req->frame_and_sects[0] & PAGE_MASK) | __PAGE_KERNEL
},
        0, (blkif->is_blktap ? ID_TO_DOM(req->id) : blkif->domid) ) )

        goto out;
#else
    if ( HYPERVISOR_update_va_mapping_otherdomain(
        MMAP_VADDR(pending_idx, 0),
        (pte_t) { (req->frame_and_sects[0] & PAGE_MASK) | __PAGE_KERNEL
},
        0, blkif->domid) )

        goto out;
#endif
#endif /* endif CONFIG_XEN_BLKDEV_GRANT */

Jun
---
Intel Open Source Technology Center

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 Aug 2005, at 15:15, Nakajima, Jun wrote:
>
> One obvious reason is: xen-specific drivers are using __PAGE_KERNEL 
> (and
> they don''t exist in the native Linux). If xen_contig_memory, for
> example, does not work, it won''t boot... Fixing xen_contig_memory
did
> not fix the booting problem as long as I remember?
Yeah, I see what you mean. I''ve just applied a patch that cleans up the
few remaining bad spots where we weren''t using pfn_pte() or 
pfn_pte_ma() constructors. I can now boot okay even with NX/XD disabled 
in Xen.

By the way -- from what I''ve seen of this NX/XD discussion so far, it 
sounds like even if CPUID says that NX/XD is supported, and you set the 
EFER.NXE bit, the BIOS can still cause this to be ignored! Is this true 
and, if so, what will happen if you try to use NX/XD anyway? i.e., will 
the bit be ignored, or will its use cause a page fault?

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 2005-08-18 at 16:09 +0100, Keir Fraser wrote:
> On 18 Aug 2005, at 15:15, Nakajima, Jun wrote:
> 
> >
> > One obvious reason is: xen-specific drivers are using __PAGE_KERNEL 
> > (and
> > they don''t exist in the native Linux). If xen_contig_memory,
for
> > example, does not work, it won''t boot... Fixing
xen_contig_memory did
> > not fix the booting problem as long as I remember?
> 
> Yeah, I see what you mean. I''ve just applied a patch that cleans
up the
> few remaining bad spots where we weren''t using pfn_pte() or 
> pfn_pte_ma() constructors. I can now boot okay even with NX/XD disabled 
> in Xen.
> 
> By the way -- from what I''ve seen of this NX/XD discussion so far,
it
> sounds like even if CPUID says that NX/XD is supported, and you set the 
> EFER.NXE bit, the BIOS can still cause this to be ignored!
If I remember correctly (will create a patch shortly to make 100% sure).
If the option is off in the BIOS than NX bit will not show up as a CPU
feature flag. And of course once it''s on in the bios then it does.
>  Is this true 
> and, if so, what will happen if you try to use NX/XD anyway? i.e., will 
> the bit be ignored, or will its use cause a page fault?
> 
>   -- Keir
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
-- 
Jerone Young
IBM Linux Technology Center
jyoung5@us.ibm.com
512-838-1157 (T/L: 678-1157)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> 
> By the way -- from what I''ve seen of this NX/XD discussion so far,
it
> sounds like even if CPUID says that NX/XD is supported, and you set
> the EFER.NXE bit, the BIOS can still cause this to be ignored! Is this
> true and, if so, what will happen if you try to use NX/XD anyway?
> i.e., will the bit be ignored, or will its use cause a page fault?
I don''t think that can happen. When NX is disabled or not there it
doesn''t appear in CPUID.

-Andi

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 2005-08-18 at 17:51 +0200, Andi Kleen wrote:
> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> > 
> > By the way -- from what I''ve seen of this NX/XD discussion so
far, it
> > sounds like even if CPUID says that NX/XD is supported, and you set
> > the EFER.NXE bit, the BIOS can still cause this to be ignored! Is this
> > true and, if so, what will happen if you try to use NX/XD anyway?
> > i.e., will the bit be ignored, or will its use cause a page fault?
> 
> I don''t think that can happen. When NX is disabled or not there it
> doesn''t appear in CPUID.
Ah but it does:

With NX bit cut "on" in bios:
/proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      :                   Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 1
cpu MHz         : 800.028
cache size      : 1024 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr   <--- NX found here
bogomips        : 6396.31
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 48 bits virtual
power management:

With NX bit cut "off" in bios:
cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      :                   Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 1
cpu MHz         : 3200.280
cache size      : 1024 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
lm pni monitor ds_cpl cid cmpxchg16b
bogomips        : 6324.22
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 48 bits virtual
power management:

Notice no nx in the flags when NX is turned off in the bios. 



> 
> -Andi
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
-- 
Jerone Young
IBM Linux Technology Center
jyoung5@us.ibm.com
512-838-1157 (T/L: 678-1157)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18 Aug 2005, at 16:39, Jerone Young wrote:
> If I remember correctly (will create a patch shortly to make 100% 
> sure).
> If the option is off in the BIOS than NX bit will not show up as a CPU
> feature flag. And of course once it''s on in the bios then it does.
Good to hear. Case closed on nx/xd then...

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Aug 18, 2005 at 11:05:42AM -0500, Jerone Young
wrote:
> On Thu, 2005-08-18 at 17:51 +0200, Andi Kleen wrote:
> > Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> > > 
> > > By the way -- from what I''ve seen of this NX/XD
discussion so far, it
> > > sounds like even if CPUID says that NX/XD is supported, and you
set
> > > the EFER.NXE bit, the BIOS can still cause this to be ignored! Is
this
> > > true and, if so, what will happen if you try to use NX/XD anyway?
> > > i.e., will the bit be ignored, or will its use cause a page
fault?
> > 
> > I don''t think that can happen. When NX is disabled or not
there it
> > doesn''t appear in CPUID.
> 
> Ah but it does:
Your example confirms what I said.

-Andi

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 2005-08-18 at 18:18 +0200, Andi Kleen wrote:
> On Thu, Aug 18, 2005 at 11:05:42AM -0500, Jerone Young wrote:
> > On Thu, 2005-08-18 at 17:51 +0200, Andi Kleen wrote:
> > > Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> > > > 
> > > > By the way -- from what I''ve seen of this NX/XD
discussion so far, it
> > > > sounds like even if CPUID says that NX/XD is supported, and
you set
> > > > the EFER.NXE bit, the BIOS can still cause this to be
ignored! Is this
> > > > true and, if so, what will happen if you try to use NX/XD
anyway?
> > > > i.e., will the bit be ignored, or will its use cause a page
fault?
> > > 
> > > I don''t think that can happen. When NX is disabled or
not there it
> > > doesn''t appear in CPUID.
> > 
> > Ah but it does:
hehe... I didn''t realize I left that statement in there...I just wanted
to post to the list to confirm with an example. Don''t worry Andi ...
your still the x86-64 god in my book!
> 
> Your example confirms what I said.
> 
> -Andi
> 
-- 
Jerone Young
IBM Linux Technology Center
jyoung5@us.ibm.com
512-838-1157 (T/L: 678-1157)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel