On IA32 host or IA32 PAE host, at present, generally, we can''t create an HVM guest with more than 2G memory, because generally it''s almost impossible for Qemu to find a large enough and consecutive virtual address space to map an HVM guest''s whole physical address space. The attached patch fixes this issue using dynamic mapping based on little blocks of memory. -- Dexuan Cui Signed-off-by: Jun Nakajima <jun.nakajima@intel.com> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan wrote:> On IA32 host or IA32 PAE host, at present, generally, we can''t create an > HVM guest with more than 2G memory, because generally it''s almost > impossible for Qemu to find a large enough and consecutive virtual > address space to map an HVM guest''s whole physical address space. > The attached patch fixes this issue using dynamic mapping based on > little blocks of memory.I have a feeling this patch is going to require quite a lot more work for V2E. Have you guys considered implementing this functionality for upstream QEMU? The Xen specific portions should be small and upstream QEMU has the same problem. Regards, Anthony Liguori> -- Dexuan Cui > > Signed-off-by: Jun Nakajima <jun.nakajima@intel.com> > Signed-off-by: Dexuan Cui <dexuan.cui@intel.com> > > > ------------------------------------------------------------------------ > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Cui, Dexuan wrote:> On IA32 host or IA32 PAE host, at present, generally, we can''t create an > HVM guest with more than 2G memory, because generally it''s almost > impossible for Qemu to find a large enough and consecutive virtual > address space to map an HVM guest''s whole physical address space. > The attached patch fixes this issue using dynamic mapping based on > little blocks of memory. >(Sorry if this comes through twice). Have you considered doing something similar for mainline QEMU? The reason I ask is that V2E pulls in all of the dynamic translation code. My initial reaction is that doing map cache will require a significant amount change to the dynamic translation bits since we can no longer make the assumption that memory can be accessed directly. I don''t fully have my head around it yet, but this may involve lots of nastiness with keeping track of which TB''s reference what memory and invalidating those TBs when map cache references are invalidated. The QEMU TLB may simplify some of this but I''m not entirely sure. Have you given this any thought? Thanks, Anthony Liguori> -- Dexuan Cui > > Signed-off-by: Jun Nakajima <jun.nakajima@intel.com> > Signed-off-by: Dexuan Cui <dexuan.cui@intel.com> > > > ------------------------------------------------------------------------ > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Have you considered doing something similar for mainline QEMU? The > reason I ask is that V2E pulls in all of the dynamic translation code. > My initial reaction is that doing map cache will require a significant > amount change to the dynamic translation bits since we can no longer > make the assumption that memory can be accessed directly. I don''tfully> have my head around it yet, but this may involve lots of nastinesswith> keeping track of which TB''s reference what memory and invalidatingthose> TBs when map cache references are invalidated. The QEMU TLB may > simplify some of this but I''m not entirely sure. > > Have you given this any thought?Being able to invalidate (sections of) qemu''s mappings (at least asynchronously) is essential to allow the balloon driver to work for HVM guests. V2E is going to have to bite the bullet on this one. Of course, in a 64b environment the map cache can be direct mapped, but you still need the ability to do invalidates. BTW: I''m comfortable if V2E only works on 64b. Sooner or latter there''s going to be some new feature which isn''t supported on Yonah... Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt wrote:>> Have you considered doing something similar for mainline QEMU? The >> reason I ask is that V2E pulls in all of the dynamic translation code. >> My initial reaction is that doing map cache will require a significant >> amount change to the dynamic translation bits since we can no longer >> make the assumption that memory can be accessed directly. I don''t >> > fully > >> have my head around it yet, but this may involve lots of nastiness >> > with > >> keeping track of which TB''s reference what memory and invalidating >> > those > >> TBs when map cache references are invalidated. The QEMU TLB may >> simplify some of this but I''m not entirely sure. >> >> Have you given this any thought? >> > > Being able to invalidate (sections of) qemu''s mappings (at least > asynchronously) is essential to allow the balloon driver to work for HVM > guests.To be able to change portions of the physical memory mapping right? You don''t strictly need a map cache for this (you can simply remap portions of the address space). You really only need the map cache for > 2GB guests (which admittedly could be a ballooned down guest that started out > 2GB).> V2E is going to have to bite the bullet on this one. > > Of course, in a 64b environment the map cache can be direct mapped, but > you still need the ability to do invalidates. BTW: I''m comfortable if > V2E only works on 64b.We may be able to work around this using one of QEMU''s TLB. If the map cache goes in for 3.0.4, then we can look at just supporting 64 bit for 3.0.5 and fixing 32 bit post 3.0.5 (if that''s necessary). Sound like a reasonable plan? Regards, Anthony Liguori> Sooner or latter there''s going to be some new > feature which isn''t supported on Yonah... > > Ian >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > Being able to invalidate (sections of) qemu''s mappings (at least > > asynchronously) is essential to allow the balloon driver to work forHVM> > guests. > > To be able to change portions of the physical memory mapping right?You> don''t strictly need a map cache for this (you can simply remapportions> of the address space).Yes (where remap includes ''unmap'')> You really only need the map cache for > 2GB > guests (which admittedly could be a ballooned down guest that started > out > 2GB).One could argue that for large memory guests having a mapping of all of guest memory is ''wasteful'' anyhow, not that page tables take up that much space. It''s probably good hygiene to only map what you need.> > V2E is going to have to bite the bullet on this one. > > > > Of course, in a 64b environment the map cache can be direct mapped,but> > you still need the ability to do invalidates. BTW: I''m comfortableif> > V2E only works on 64b. > > We may be able to work around this using one of QEMU''s TLB. If themap> cache goes in for 3.0.4, then we can look at just supporting 64 bitfor> 3.0.5 and fixing 32 bit post 3.0.5 (if that''s necessary). Sound likea> reasonable plan?If necessary, however I still like the mapcache approach. I think running a 32b dom0 on a 64b hypervisor is actually going to be a pretty common way of running things (likely gives best performance). Ian> Regards, > > Anthony Liguori > > > Sooner or latter there''s going to be some new > > feature which isn''t supported on Yonah... > > > > Ian > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Keir, There is a minor mistake in the patch I posted previously. The "1UL" in the following line should be "1ULL": #define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT Sorry for that. Btw, it seems the patch has not been checked in. Could you give some commetns? Thanks! -- Dexuan -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Cui, Dexuan Sent: 2006年12月5日 1:34 To: xen-devel@lists.xensource.com Subject: [Xen-devel] [Patch] Qemu map cache On IA32 host or IA32 PAE host, at present, generally, we can''t create an HVM guest with more than 2G memory, because generally it''s almost impossible for Qemu to find a large enough and consecutive virtual address space to map an HVM guest''s whole physical address space. The attached patch fixes this issue using dynamic mapping based on little blocks of memory. -- Dexuan Cui Signed-off-by: Jun Nakajima <jun.nakajima@intel.com> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 7/12/06 02:08, "Cui, Dexuan" <dexuan.cui@intel.com> wrote:> There is a minor mistake in the patch I posted previously. > The "1UL" in the following line should be "1ULL": > #define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT > > Sorry for that. > > Btw, it seems the patch has not been checked in. Could you give some commetns?I''m reworking it a bit. It''ll go in this morning. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel