Robert Phillips
2006-Jun-22 14:10 UTC
Re: [Fwd: Re: [Xen-devel] [RFC] XI Shadow Page Table Mechanism]
I am replying for Ben. -- rsp On 6/22/06, Ben Thomas <bthomas@virtualiron.com> wrote:> > > > -------- Original Message -------- > Subject: Re: [Xen-devel] [RFC] XI Shadow Page Table Mechanism > Date: Wed, 21 Jun 2006 15:31:59 -0500 > From: Anthony Liguori <aliguori@us.ibm.com > > To: Ben Thomas <bthomas@virtualiron.com> > CC: xen-devel@lists.xensource.com > References: < 44998B2C.2000109@virtualiron.com> > > Ben Thomas wrote: > > This post contains the design document for what is currently known > > as the "XI Shadow Mechanism". This is a design for shadow > > page table code for fully virtualized HVM domains running on a 64-bit > > Xen hypervisor. > > > > This work was undertaken to address a number of goals. These are > > enumerated in the document and include: > > > > - ability to run fully virtualized 32, 32PAE and 64 bit guest > > domains concurrently on a 64-bit hypervisor > > This isn''t supported currently? Since an HVM must go through 16 bit, 32 > bit, and 64 bit mode to boot up, how can we start more than one guest at > a time currently if this doesn''t already work?Ed Smith''s daily test results show there have been problems with SMP. We haven''t diagnosed these problems but, from reading the pre-XI shadow code, it''s not clear how it copes with multiple VCPUs in the same domain running in different modes.> - support live migration of fully virtualized domains > > Sweet. What about the current shadow page code limited this?Nothing intrinsic. Just a SMOP.> - provide good performance and robustness > > > > This design has been implemented and is currently being tested. > > It has been supporting the variety of memory models as noted above, > > and using widely used Windows and Linux distributions (SuSe, > > RedHat and others). At a point in the near future, a patch > > will be available. > > > > This design center is the x86-64 architecture. It is not our > > intent to completely replace all shadow page management, and > > we''ve attempted to limit the scope of change. > > > > A preliminary version of this design concept has undergone > > brief review with some members of the Xen community. We hope > > that this is of value to the Xen community and welcome your > > feedback and comments. > > Can''t really comment as it''s not quotable. Some questions that > immediately come to mind are: > > - how do you deal with large pages within the hypervisor? do you > coalesce or just hope there is contiguous pages available?We just hope that contiguous pages are available. However we allocate pages for the guest in large extents to maximize this likelihood. In practice this is very effective for guests created soon after boot time. There may be fragmentation problems later. - what is the performance benefit in saving the shadow pages for each> domain? there''s clearly a memory trade-off here so understanding the > performance gain seems important.The performance benefit appears to be substantial but we have not done a thorough study yet. - OOM can be dealt with in the existing code by just invalidating> existing mappings to free up pages. what advantages do your approach > have to this? (i realize we don''t do this today but in theory, we could).Ultimately XI deals with OOM by tearing down cached shadow pages, just as you say. But it uses LRU to pick the victims. Interesting stuff. I''m eager to see the code.> > Regards, > > Anthony Liguori > > > Thanks, > > -b > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > -- > ------------------------------------------------------------------------ > Ben Thomas Virtual Iron Software > bthomas@virtualiron.com Tower 1, Floor 2 > 978-849-1214 900 Chelmsford Street > Lowell, MA 01851 >------------------------------------------------------------------------ Robert S. Phillips Virtual Iron Software rsp.vi.xen@virtualiron.com Tower 1, Floor 2 978-849-1220 900 Chelmsford Street Lowell, MA 01851 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Anthony Liguori
2006-Jun-22 14:14 UTC
Re: [Fwd: Re: [Xen-devel] [RFC] XI Shadow Page Table Mechanism]
Robert Phillips wrote:> > This isn''t supported currently? Since an HVM must go through 16 > bit, 32 > bit, and 64 bit mode to boot up, how can we start more than one > guest at > a time currently if this doesn''t already work? > > > Ed Smith''s daily test results show there have been problems with SMP. > We haven''t diagnosed these problems but, from reading the pre-XI > shadow code, it''s not clear how it copes with multiple VCPUs in the > same domain running in different modes.There''s only a short period of time when this would be happening right? During boot up?> - how do you deal with large pages within the hypervisor? do you > coalesce or just hope there is contiguous pages available? > > > We just hope that contiguous pages are available. However we allocate > pages for the guest in large extents to maximize this likelihood. In > practice this is very effective for guests created soon after boot > time. There may be fragmentation problems later.Any ideas about how to deal with this long term? Large page support would also be useful for PV domains. There was a lot of people at the last summit that were interested in this... Thanks for the responses, Anthony Liguori> - what is the performance benefit in saving the shadow pages for each > domain? there''s clearly a memory trade-off here so understanding the > performance gain seems important. > > > The performance benefit appears to be substantial but we have not done > a thorough study yet. > > - OOM can be dealt with in the existing code by just invalidating > existing mappings to free up pages. what advantages do your approach > have to this? (i realize we don''t do this today but in theory, we > could). > > > Ultimately XI deals with OOM by tearing down cached shadow pages, > just as you say. But it uses LRU to pick the victims. > > Interesting stuff. I''m eager to see the code. > > Regards, > > Anthony Liguori > > > Thanks, > > -b > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com <mailto:Xen-devel@lists.xensource.com> > > http://lists.xensource.com/xen-devel > > > -- > ------------------------------------------------------------------------ > Ben Thomas Virtual Iron > Software > bthomas@virtualiron.com > <mailto:bthomas@virtualiron.com> Tower > 1, Floor 2 > 978-849-1214 900 Chelmsford > Street > Lowell, MA 01851 > > ------------------------------------------------------------------------ > Robert S. Phillips Virtual Iron Software > rsp.vi.xen@virtualiron.com > <mailto:rsp.vi.xen@virtualiron.com> Tower 1, Floor 2 > 978-849-1220 900 Chelmsford Street > Lowell, MA 01851_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel