Xen devel - Aug 2010 - [PATCH 0 of 4] [RFC] x86 shadow: get rid of the need for contiguous memory

This series of patches removes the need for shadow pagetable memory to
be allocated in 4-page contiguous blocks, by reusing the page_info 
list header for yet one more thing. 

It fixes a long-standing issue where on a fairly full machine which has
seen a lot of ballooning HVM domain creation can fail because the 
remaining memory is too fragmented to use for shadows.

Posting as an RFC for now because I haven''t had a chance to do any
heavy testing (compile tests under 32-bit WinXP seem fine though) 
and I''m away for the next week.  I hope to fold in any feedback 
and commit this change the week after that.

Cheers,

Tim.

5 files changed, 332 insertions(+), 301 deletions(-)
xen/arch/x86/mm/shadow/common.c  |  345 +++++++++++++-------------------------
xen/arch/x86/mm/shadow/multi.c   |  143 +++++++++------
xen/arch/x86/mm/shadow/private.h |  117 +++++++++++-
xen/include/asm-x86/domain.h     |    3 
xen/include/asm-x86/mm.h         |   25 +-

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

x86 shadow: for multi-page shadows, explicitly track the first page
(where the refcounts are) and check that none of the routines
that do refcounting ever see the second, third or fourth page.
This is just stating and enforcing an existing implicit requirement.

Signed-off-by: Tim Deegan <Tim.Deegan@citrix.com>


4 files changed, 47 insertions(+), 14 deletions(-)
xen/arch/x86/mm/shadow/common.c  |    8 +++++-
xen/arch/x86/mm/shadow/multi.c   |   48 ++++++++++++++++++++++++++++----------
xen/arch/x86/mm/shadow/private.h |    2 +
xen/include/asm-x86/mm.h         |    3 +-




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

x86 shadow: explicitly link the pages of multipage shadows
together using their list headers.  Update the users of the
pinned-shadows list to expect l2_32 shadows to have four entries
in the list, which must be kept together during updates.

Signed-off-by: Tim Deegan <Tim.Deegan@citrix.com>


4 files changed, 137 insertions(+), 34 deletions(-)
xen/arch/x86/mm/shadow/common.c  |   18 +++--
xen/arch/x86/mm/shadow/multi.c   |   23 +++----
xen/arch/x86/mm/shadow/private.h |  115 +++++++++++++++++++++++++++++++++++---
xen/include/asm-x86/mm.h         |   15 +++-




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

x86 shadow: remove the assumption that multipage shadows are contiguous
and move from page to page using the linked list instead.

Signed-off-by: Tim Deegan <Tim.Deegan@citrix.com>


2 files changed, 43 insertions(+), 32 deletions(-)
xen/arch/x86/mm/shadow/common.c |    4 +-
xen/arch/x86/mm/shadow/multi.c  |   71 ++++++++++++++++++++++-----------------




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

x86 shadow: allocate all shadow memory in single pages
now that multi-page shadows need not be contiguous.

Signed-off-by: Tim Deegan <Tim.Deegan@citrix.com>


4 files changed, 105 insertions(+), 221 deletions(-)
xen/arch/x86/mm/shadow/common.c |  315 ++++++++++++---------------------------
xen/arch/x86/mm/shadow/multi.c  |    1 
xen/include/asm-x86/domain.h    |    3 
xen/include/asm-x86/mm.h        |    7 




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

ooohh!!  thank you thank you

/me wonders how many other order>0 allocations are left now
that will break a running or newly-launching domain when the
allocation fails due to fragmentation, and if it will be easy
to track them down and shoot them
> -----Original Message-----
> From: Tim Deegan [mailto:Tim.Deegan@citrix.com]
> Sent: Friday, August 20, 2010 9:58 AM
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] [PATCH 0 of 4] [RFC] x86 shadow: get rid of the
> need for contiguous memory
> 
> 
> This series of patches removes the need for shadow pagetable memory to
> be allocated in 4-page contiguous blocks, by reusing the page_info
> list header for yet one more thing.
> 
> It fixes a long-standing issue where on a fairly full machine which has
> seen a lot of ballooning HVM domain creation can fail because the
> remaining memory is too fragmented to use for shadows.
> 
> Posting as an RFC for now because I haven''t had a chance to do any
> heavy testing (compile tests under 32-bit WinXP seem fine though)
> and I''m away for the next week.  I hope to fold in any feedback
> and commit this change the week after that.
> 
> Cheers,
> 
> Tim.
> 
> 5 files changed, 332 insertions(+), 301 deletions(-)
> xen/arch/x86/mm/shadow/common.c  |  345 +++++++++++++------------------
> -------
> xen/arch/x86/mm/shadow/multi.c   |  143 +++++++++------
> xen/arch/x86/mm/shadow/private.h |  117 +++++++++++-
> xen/include/asm-x86/domain.h     |    3
> xen/include/asm-x86/mm.h         |   25 +-
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hack up the vcpu dand domain structures and handle only being able to
allocate a single-page hypercall compat tranlsation page, and that might be
pretty much it.

 -- Keir

On 20/08/2010 17:29, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:
> ooohh!!  thank you thank you
> 
> /me wonders how many other order>0 allocations are left now
> that will break a running or newly-launching domain when the
> allocation fails due to fragmentation, and if it will be easy
> to track them down and shoot them
> 
>> -----Original Message-----
>> From: Tim Deegan [mailto:Tim.Deegan@citrix.com]
>> Sent: Friday, August 20, 2010 9:58 AM
>> To: xen-devel@lists.xensource.com
>> Subject: [Xen-devel] [PATCH 0 of 4] [RFC] x86 shadow: get rid of the
>> need for contiguous memory
>> 
>> 
>> This series of patches removes the need for shadow pagetable memory to
>> be allocated in 4-page contiguous blocks, by reusing the page_info
>> list header for yet one more thing.
>> 
>> It fixes a long-standing issue where on a fairly full machine which has
>> seen a lot of ballooning HVM domain creation can fail because the
>> remaining memory is too fragmented to use for shadows.
>> 
>> Posting as an RFC for now because I haven''t had a chance to do
any
>> heavy testing (compile tests under 32-bit WinXP seem fine though)
>> and I''m away for the next week.  I hope to fold in any
feedback
>> and commit this change the week after that.
>> 
>> Cheers,
>> 
>> Tim.
>> 
>> 5 files changed, 332 insertions(+), 301 deletions(-)
>> xen/arch/x86/mm/shadow/common.c  |  345 +++++++++++++------------------
>> -------
>> xen/arch/x86/mm/shadow/multi.c   |  143 +++++++++------
>> xen/arch/x86/mm/shadow/private.h |  117 +++++++++++-
>> xen/include/asm-x86/domain.h     |    3
>> xen/include/asm-x86/mm.h         |   25 +-
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel