thr3ads.net - Xen devel - [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M. [Dec 2007]

If this information is useful, please help other people find it:
Share via:

Kimball Murray

2007-Dec-07 15:43 UTC

[Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

One of the hypervisor patches applied after the 3.1.0 build (15044.patch) can
cause BIOS-reserved memory regions between 1M and 16M to be overwritten.

Originally, memory in this area was added to the boot allocator only if it
was not reserved, but the 15044 patch removed that code and added this:

+    init_boot_pages(1<<20, 16<<20); /* Initial seed: 15MB */

Stratus is one vendor whose BIOS e820 tables define a 256K reseverved area
starting at 15M (0xf00000).  Here are the e820 tables for that platform:

(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009a000 (usable)
(XEN)  000000000009a800 - 00000000000a0000 (reserved)
(XEN)  00000000000cc000 - 00000000000d0000 (reserved)
(XEN)  00000000000dc000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000000f00000 (usable)
(XEN)  0000000000f00000 - 0000000000f40000 (reserved)
(XEN)  0000000000f40000 - 000000007fef0000 (usable)
(XEN)  000000007fef0000 - 000000007fefa000 (ACPI data)
(XEN)  000000007fefa000 - 000000007ff00000 (ACPI NVS)
(XEN)  000000007ff00000 - 0000000080000000 (reserved)
(XEN)  00000000e0000000 - 00000000f0000000 (reserved)
(XEN)  00000000fec80000 - 00000000fec90000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000180000000 (usable)

It turns out that the reserved area at 15M was used by the system''s SMI
handler
as a register restore area.  Because the hypervisor was stepping on this memory,
the system MTRR settings would change from write-back to write-through after
any SMI was triggered.  Other, more spectacular, failures could result on other
platforms.

The attached patch re-adds code that pays attention to the e820 tables
when seeding the boot allocator with low-memory pages.


Signed-off-by: Kimball Murray (kimball.murray@gmail.com)


---------------------------- snip -----------------------
diff -Naur xen/arch/x86/setup.c ../kernel-2.6.18-55bb/xen/arch/x86/setup.c
--- xen/arch/x86/setup.c	2007-10-23 15:23:10.000000000 -0400
+++ ../kernel-2.6.18-55bb/xen/arch/x86/setup.c	2007-11-30 16:04:21.000000000
-0500
@@ -441,6 +441,39 @@
     return p;
 }
 
+/*
+ * Add some low memory pages (from "lower" to "upper") to
the boot
+ * allocator, but be careful not to add any reserved areas (pay
+ * attention to e820 map).
+ */
+static void seed_boot_allocator(unsigned long lower, unsigned long upper)
+{
+    int i;
+
+    for ( i = 0; i < e820_raw_nr; i++ )
+    {
+        uint64_t start, end;
+
+        if (e820_raw[i].type != E820_RAM)
+            continue;
+
+        start = e820_raw[i].addr;
+        if (start >= upper)
+            continue;
+        if (start < lower)
+            start = lower;
+
+        end = e820_raw[i].addr + e820_raw[i].size;
+        if (end <= lower)
+            continue;
+
+        if (end > upper)
+            end = upper;
+
+        init_boot_pages(start, end);
+    }
+}
+
 void __init __start_xen(unsigned long mbi_p)
 {
     char *memmap_type = NULL;
@@ -810,9 +843,10 @@
     xenheap_phys_end += xen_phys_start;
     reserve_in_boot_e820(xen_phys_start,
                          xen_phys_start + (opt_xenheap_megabytes<<20));
-    init_boot_pages(1<<20, 16<<20); /* Initial seed: 15MB */
+    /* Initial seed, lower 1M - 16M, but pay attention to e820 tables */
+    seed_boot_allocator(1 << 20, 16 << 20);
 #else
-    init_boot_pages(xenheap_phys_end, 16<<20); /* Initial seed: 4MB */
+    seed_boot_allocator(xenheap_phys_end, 16<<20); /* Initial seed: 4MB
*/
 #endif
 
     if ( kexec_crash_area.size != 0 )

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2007-Dec-11 09:07 UTC

head link

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

Hopefully this is fixed by c/s 15546 in xen-3.1-testing.hg. It would be
great if you could confirm this fixes the issue.

 Thanks,
 Keir


On 7/12/07 15:43, "Kimball Murray" <kimball.murray@gmail.com>
wrote:
> One of the hypervisor patches applied after the 3.1.0 build (15044.patch)
can
> cause BIOS-reserved memory regions between 1M and 16M to be overwritten.
> 
> Originally, memory in this area was added to the boot allocator only if it
> was not reserved, but the 15044 patch removed that code and added this:
> 
> +    init_boot_pages(1<<20, 16<<20); /* Initial seed: 15MB */
> 
> Stratus is one vendor whose BIOS e820 tables define a 256K reseverved area
> starting at 15M (0xf00000).  Here are the e820 tables for that platform:
> 
> (XEN) Xen-e820 RAM map:
> (XEN)  0000000000000000 - 000000000009a000 (usable)
> (XEN)  000000000009a800 - 00000000000a0000 (reserved)
> (XEN)  00000000000cc000 - 00000000000d0000 (reserved)
> (XEN)  00000000000dc000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 0000000000f00000 (usable)
> (XEN)  0000000000f00000 - 0000000000f40000 (reserved)
> (XEN)  0000000000f40000 - 000000007fef0000 (usable)
> (XEN)  000000007fef0000 - 000000007fefa000 (ACPI data)
> (XEN)  000000007fefa000 - 000000007ff00000 (ACPI NVS)
> (XEN)  000000007ff00000 - 0000000080000000 (reserved)
> (XEN)  00000000e0000000 - 00000000f0000000 (reserved)
> (XEN)  00000000fec80000 - 00000000fec90000 (reserved)
> (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> (XEN)  00000000ff000000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 0000000180000000 (usable)
> 
> It turns out that the reserved area at 15M was used by the
system''s SMI
> handler
> as a register restore area.  Because the hypervisor was stepping on this
> memory,
> the system MTRR settings would change from write-back to write-through
after
> any SMI was triggered.  Other, more spectacular, failures could result on
> other
> platforms.
> 
> The attached patch re-adds code that pays attention to the e820 tables
> when seeding the boot allocator with low-memory pages.
> 
> 
> Signed-off-by: Kimball Murray (kimball.murray@gmail.com)
> 
> 
> ---------------------------- snip -----------------------
> diff -Naur xen/arch/x86/setup.c ../kernel-2.6.18-55bb/xen/arch/x86/setup.c
> --- xen/arch/x86/setup.c 2007-10-23 15:23:10.000000000 -0400
> +++ ../kernel-2.6.18-55bb/xen/arch/x86/setup.c 2007-11-30
16:04:21.000000000
> -0500
> @@ -441,6 +441,39 @@
>      return p;
>  }
>  
> +/*
> + * Add some low memory pages (from "lower" to "upper")
to the boot
> + * allocator, but be careful not to add any reserved areas (pay
> + * attention to e820 map).
> + */
> +static void seed_boot_allocator(unsigned long lower, unsigned long upper)
> +{
> +    int i;
> +
> +    for ( i = 0; i < e820_raw_nr; i++ )
> +    {
> +        uint64_t start, end;
> +
> +        if (e820_raw[i].type != E820_RAM)
> +            continue;
> +
> +        start = e820_raw[i].addr;
> +        if (start >= upper)
> +            continue;
> +        if (start < lower)
> +            start = lower;
> +
> +        end = e820_raw[i].addr + e820_raw[i].size;
> +        if (end <= lower)
> +            continue;
> +
> +        if (end > upper)
> +            end = upper;
> +
> +        init_boot_pages(start, end);
> +    }
> +}
> +
>  void __init __start_xen(unsigned long mbi_p)
>  {
>      char *memmap_type = NULL;
> @@ -810,9 +843,10 @@
>      xenheap_phys_end += xen_phys_start;
>      reserve_in_boot_e820(xen_phys_start,
>                           xen_phys_start +
(opt_xenheap_megabytes<<20));
> -    init_boot_pages(1<<20, 16<<20); /* Initial seed: 15MB */
> +    /* Initial seed, lower 1M - 16M, but pay attention to e820 tables */
> +    seed_boot_allocator(1 << 20, 16 << 20);
>  #else
> -    init_boot_pages(xenheap_phys_end, 16<<20); /* Initial seed: 4MB
*/
> +    seed_boot_allocator(xenheap_phys_end, 16<<20); /* Initial seed:
4MB */
>  #endif
>  
>      if ( kexec_crash_area.size != 0 )
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Kimball Murray

2007-Dec-14 20:35 UTC

head link

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

Unfortunately, it does not appear to fix the issue.  I think I found why.

First I added some debug to init_boot_pages to determine when and
where the caller was adding a region that stepped on reserved BIOS
memory at 15M.  The debug output showed this:

(XEN) init_boot_pages-KAM: called from ffff828c801b7657 with (100000,
1000000) <- BAD!
(XEN) init_boot_pages-KAM: called from ffff828c801b7657 with (f40000, 1000000)
(XEN) init_boot_pages-KAM: called from ffff828c801b7690 with (1000000, 7e195000)
(XEN) init_boot_pages-KAM: called from ffff828c801b7690 with
(7fe00000, 7fef0000)
(XEN) init_boot_pages-KAM: called from ffff828c801b7690 with
(100000000, 180000000)

It''s the very first call to init_boot_pages(), in __start_xen(), that
tramples the 15M BIOS hole.  And I see this code in __start_xen:

        /* Need to create mappings above 16MB. */
        map_s = max_t(uint64_t, s, 16<<20);
        map_e = e;
#if defined(CONFIG_X86_32) /* mappings are truncated on x86_32 */
        map_e = min_t(uint64_t, map_e, BOOTSTRAP_DIRECTMAP_END);
#endif

        /* Pass mapped memory to allocator /before/ creating new mappings. */
        if ( s < map_s )
            init_boot_pages(s, map_s);

So the first time through the loop, s == 1M and e == 15M.  But map_s
gets set to 16M and then we slip into the if statement and call
init_boot_pages with (1M, 16M), crushing the BIOS hole.



On 12/11/07, Keir Fraser <Keir.Fraser@cl.cam.ac.uk>
wrote:> Hopefully this is fixed by c/s 15546 in xen-3.1-testing.hg. It would be
> great if you could confirm this fixes the issue.
>
>  Thanks,
>  Keir
>
>
> On 7/12/07 15:43, "Kimball Murray"
<kimball.murray@gmail.com> wrote:
>
> > One of the hypervisor patches applied after the 3.1.0 build
(15044.patch) can
> > cause BIOS-reserved memory regions between 1M and 16M to be
overwritten.
> >
> > Originally, memory in this area was added to the boot allocator only
if it
> > was not reserved, but the 15044 patch removed that code and added
this:
> >
> > +    init_boot_pages(1<<20, 16<<20); /* Initial seed: 15MB
*/
> >
> > Stratus is one vendor whose BIOS e820 tables define a 256K reseverved
area
> > starting at 15M (0xf00000).  Here are the e820 tables for that
platform:
> >
> > (XEN) Xen-e820 RAM map:
> > (XEN)  0000000000000000 - 000000000009a000 (usable)
> > (XEN)  000000000009a800 - 00000000000a0000 (reserved)
> > (XEN)  00000000000cc000 - 00000000000d0000 (reserved)
> > (XEN)  00000000000dc000 - 0000000000100000 (reserved)
> > (XEN)  0000000000100000 - 0000000000f00000 (usable)
> > (XEN)  0000000000f00000 - 0000000000f40000 (reserved)
> > (XEN)  0000000000f40000 - 000000007fef0000 (usable)
> > (XEN)  000000007fef0000 - 000000007fefa000 (ACPI data)
> > (XEN)  000000007fefa000 - 000000007ff00000 (ACPI NVS)
> > (XEN)  000000007ff00000 - 0000000080000000 (reserved)
> > (XEN)  00000000e0000000 - 00000000f0000000 (reserved)
> > (XEN)  00000000fec80000 - 00000000fec90000 (reserved)
> > (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> > (XEN)  00000000ff000000 - 0000000100000000 (reserved)
> > (XEN)  0000000100000000 - 0000000180000000 (usable)
> >
> > It turns out that the reserved area at 15M was used by the
system''s SMI
> > handler
> > as a register restore area.  Because the hypervisor was stepping on
this
> > memory,
> > the system MTRR settings would change from write-back to write-through
after
> > any SMI was triggered.  Other, more spectacular, failures could result
on
> > other
> > platforms.
> >
> > The attached patch re-adds code that pays attention to the e820 tables
> > when seeding the boot allocator with low-memory pages.
> >
> >
> > Signed-off-by: Kimball Murray (kimball.murray@gmail.com)
> >
> >
> > ---------------------------- snip -----------------------
> > diff -Naur xen/arch/x86/setup.c
../kernel-2.6.18-55bb/xen/arch/x86/setup.c
> > --- xen/arch/x86/setup.c 2007-10-23 15:23:10.000000000 -0400
> > +++ ../kernel-2.6.18-55bb/xen/arch/x86/setup.c 2007-11-30
16:04:21.000000000
> > -0500
> > @@ -441,6 +441,39 @@
> >      return p;
> >  }
> >
> > +/*
> > + * Add some low memory pages (from "lower" to
"upper") to the boot
> > + * allocator, but be careful not to add any reserved areas (pay
> > + * attention to e820 map).
> > + */
> > +static void seed_boot_allocator(unsigned long lower, unsigned long
upper)
> > +{
> > +    int i;
> > +
> > +    for ( i = 0; i < e820_raw_nr; i++ )
> > +    {
> > +        uint64_t start, end;
> > +
> > +        if (e820_raw[i].type != E820_RAM)
> > +            continue;
> > +
> > +        start = e820_raw[i].addr;
> > +        if (start >= upper)
> > +            continue;
> > +        if (start < lower)
> > +            start = lower;
> > +
> > +        end = e820_raw[i].addr + e820_raw[i].size;
> > +        if (end <= lower)
> > +            continue;
> > +
> > +        if (end > upper)
> > +            end = upper;
> > +
> > +        init_boot_pages(start, end);
> > +    }
> > +}
> > +
> >  void __init __start_xen(unsigned long mbi_p)
> >  {
> >      char *memmap_type = NULL;
> > @@ -810,9 +843,10 @@
> >      xenheap_phys_end += xen_phys_start;
> >      reserve_in_boot_e820(xen_phys_start,
> >                           xen_phys_start +
(opt_xenheap_megabytes<<20));
> > -    init_boot_pages(1<<20, 16<<20); /* Initial seed: 15MB
*/
> > +    /* Initial seed, lower 1M - 16M, but pay attention to e820 tables
*/
> > +    seed_boot_allocator(1 << 20, 16 << 20);
> >  #else
> > -    init_boot_pages(xenheap_phys_end, 16<<20); /* Initial seed:
4MB */
> > +    seed_boot_allocator(xenheap_phys_end, 16<<20); /* Initial
seed: 4MB */
> >  #endif
> >
> >      if ( kexec_crash_area.size != 0 )
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2007-Dec-14 23:48 UTC

head link

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

On 14/12/07 20:35, "Kimball Murray" <kimball.murray@gmail.com>
wrote:
> So the first time through the loop, s == 1M and e == 15M.  But map_s
> gets set to 16M and then we slip into the if statement and call
> init_boot_pages with (1M, 16M), crushing the BIOS hole.
Thanks for the analysis. I''ll sort out a fix and get back to you.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2007-Dec-15 18:30 UTC

head link

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

On 14/12/07 23:48, "Keir Fraser" <Keir.Fraser@cl.cam.ac.uk>
wrote:
>> So the first time through the loop, s == 1M and e == 15M.  But map_s
>> gets set to 16M and then we slip into the if statement and call
>> init_boot_pages with (1M, 16M), crushing the BIOS hole.
> 
> Thanks for the analysis. I''ll sort out a fix and get back to you.
Please try c/s 16624 in the staging tree
(http://xenbits.xensource.com/staging/xen-unstable.hg).

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Kimball Murray

2007-Dec-15 22:46 UTC

head link

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

Works fine, thanks!


On Dec 15, 2007 1:30 PM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk>
wrote:> On 14/12/07 23:48, "Keir Fraser" <Keir.Fraser@cl.cam.ac.uk>
wrote:
>
> >> So the first time through the loop, s == 1M and e == 15M.  But
map_s
> >> gets set to 16M and then we slip into the if statement and call
> >> init_boot_pages with (1M, 16M), crushing the BIOS hole.
> >
> > Thanks for the analysis. I''ll sort out a fix and get back to
you.
>
> Please try c/s 16624 in the staging tree
> (http://xenbits.xensource.com/staging/xen-unstable.hg).
>
>  -- Keir
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Dec 2007 - Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

[Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.

Re: [Xen-devel] Xen 3.1.2 can corrupt BIOS resevered memory between 1M and 16M.