Fan, Huaxiang
2011-Jul-21 08:28 UTC
[Xen-devel] How to request contiguous memory for a domu
Hi gurus, I used xen 3.4.2. I noticed that, every time I reboot my box, the dom0 occupy all available physical memory (except those used by xen hypervisor), then /etc/init.d/xendomains try to boot domu specified in /etc/xen/auto, then dom0 balloon out to free a portion of memory (of course the resulting memory should be above dom0_min_mem) to meet the memory requirement of incoming domu. My question is, is there a guarantee that the domu get contiguous physical memory? Or how can I guarantee that? (My memory capacity is big enough for meeting all domus and dom0 memory requirements.) What about xen 4.1.0? Thanks in advance. HUAXIANG FAN Software Engineer II WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD. ph: +8610.5884.4327 fax: +8610.5884.4727 www.websense.cn<http://www.websense.cn> Websense TRITON(tm) For Essential Information Protection(tm) Web Security<http://www.websense.com/content/Regional/SCH/WebSecurityOverview.aspx> | Data Security<http://www.websense.com/content/Regional/SCH/DataSecurity.aspx> | Email Security<http://www.websense.com/content/Regional/SCH/MessagingSecurity.aspx> Protected by Websense Hosted Email Security -- www.websense.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Jul-21 08:45 UTC
Re: [Xen-devel] How to request contiguous memory for a domu
On Thu, 2011-07-21 at 09:28 +0100, Fan, Huaxiang wrote:> Hi gurus, > > > > I used xen 3.4.2. I noticed that, every time I reboot my box, the dom0 > occupy all available physical memory (except those used by xen > hypervisor), then /etc/init.d/xendomains try to boot domu specified > in /etc/xen/auto, then dom0 balloon out to free a portion of memory > (of course the resulting memory should be above dom0_min_mem) to meet > the memory requirement of incoming domu. > > > > My question is, is there a guarantee that the domu get contiguous > physical memory?No> Or how can I guarantee that?A guest can try to exchange memory for contiguous memory using XENMEM_exchange but there are limits on how bug a single contiguous region can be requested that way (I think it''s around order 9 or so). Why do you want domU memory to be contiguous? There is no general requirement on dom0 or domU for this to be the case and the p2m/m2p mechanisms exist to allow non-contiguous machine memory. In general guests only need to arrange for contiguous memory for specific DMA buffers to specific physical devices which require it. Please explain what you are actually trying to achieve so we can best advise you.> (My memory capacity is big enough for meeting all domus and dom0 > memory requirements.) > > > > What about xen 4.1.0? > > > > Thanks in advance. > > HUAXIANG FAN > Software Engineer II > > WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD. > ph: +8610.5884.4327 > fax: +8610.5884.4727 > www.websense.cn > > Websense TRITON™ > For Essential Information Protection™ > Web Security | Data Security | Email Security > > > > > > > Protected by Websense Hosted Email Security — www.websense.com >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Fan, Huaxiang
2011-Jul-21 08:57 UTC
[Xen-users] RE: [Xen-devel] How to request contiguous memory for a domu
Hi Ian, Thanks for your quick reply. We are running cpu/memory intensive application/services on a domu, we are worry about non-contiguous memory allocation might affect the performance. We are migrating the application/services from a single CPU socket (4 cores, i.e. 8 ht cores) hardware platform to a dual CPU socket (8 cores, i.e., 16 ht cores) hardware platform. And we saw performance degradation. As you know each physical and a half memory constitute a node. We also worry about the domu memory allocation across nodes boundary. Thanks, Huaxiang -----Original Message----- From: Ian Campbell [mailto:Ian.Campbell@citrix.com] Sent: Thursday, July 21, 2011 4:45 PM To: Fan, Huaxiang Cc: xen-users@lists.xensource.com; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] How to request contiguous memory for a domu On Thu, 2011-07-21 at 09:28 +0100, Fan, Huaxiang wrote:> Hi gurus, > > > > I used xen 3.4.2. I noticed that, every time I reboot my box, the dom0 > occupy all available physical memory (except those used by xen > hypervisor), then /etc/init.d/xendomains try to boot domu specified > in /etc/xen/auto, then dom0 balloon out to free a portion of memory > (of course the resulting memory should be above dom0_min_mem) to meet > the memory requirement of incoming domu. > > > > My question is, is there a guarantee that the domu get contiguous > physical memory?No> Or how can I guarantee that?A guest can try to exchange memory for contiguous memory using XENMEM_exchange but there are limits on how bug a single contiguous region can be requested that way (I think it's around order 9 or so). Why do you want domU memory to be contiguous? There is no general requirement on dom0 or domU for this to be the case and the p2m/m2p mechanisms exist to allow non-contiguous machine memory. In general guests only need to arrange for contiguous memory for specific DMA buffers to specific physical devices which require it. Please explain what you are actually trying to achieve so we can best advise you.> (My memory capacity is big enough for meeting all domus and dom0 > memory requirements.) > > > > What about xen 4.1.0? > > > > Thanks in advance. > > HUAXIANG FAN > Software Engineer II > > WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD. > ph: +8610.5884.4327 > fax: +8610.5884.4727 > www.websense.cn > > Websense TRITON™ > For Essential Information Protection™ > Web Security | Data Security | Email Security > > > > > > > Protected by Websense Hosted Email Security — www.websense.com >To report this as spam, please forward to spam@websense.com. Thank you. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ian Campbell
2011-Jul-21 09:09 UTC
RE: [Xen-devel] How to request contiguous memory for a domu
Please read http://wiki.xensource.com/xenwiki/AskingXenDevelQuestions and do not top post and do not cross post between xen-users and xen-devel. On Thu, 2011-07-21 at 09:57 +0100, Fan, Huaxiang wrote:> Hi Ian, > > Thanks for your quick reply. > > We are running cpu/memory intensive application/services on a domu, we > are worry about non-contiguous memory allocation might affect the > performance. > > We are migrating the application/services from a single CPU socket (4 > cores, i.e. 8 ht cores) hardware platform to a dual CPU socket (8 > cores, i.e., 16 ht cores) hardware platform. And we saw performance > degradation. As you know each physical and a half memory constitute a > node. We also worry about the domu memory allocation across nodes > boundary.Does turning on Xen''s NUMA support help at all? And then possibly pinning each guests VCPUs to a particular physical node. IIRC xl has a cpupool command which sets up a cpupool per node which might make this a bit easier. I must admit though that I''m not too familiar with Xen NUMA capabilities, hopefully someone else on the list can advise further. Ian.> > Thanks, > Huaxiang > > -----Original Message----- > From: Ian Campbell [mailto:Ian.Campbell@citrix.com] > Sent: Thursday, July 21, 2011 4:45 PM > To: Fan, Huaxiang > Cc: xen-users@lists.xensource.com; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] How to request contiguous memory for a domu > > On Thu, 2011-07-21 at 09:28 +0100, Fan, Huaxiang wrote: > > Hi gurus, > > > > > > > > I used xen 3.4.2. I noticed that, every time I reboot my box, the dom0 > > occupy all available physical memory (except those used by xen > > hypervisor), then /etc/init.d/xendomains try to boot domu specified > > in /etc/xen/auto, then dom0 balloon out to free a portion of memory > > (of course the resulting memory should be above dom0_min_mem) to meet > > the memory requirement of incoming domu. > > > > > > > > My question is, is there a guarantee that the domu get contiguous > > physical memory? > > No > > > Or how can I guarantee that? > > A guest can try to exchange memory for contiguous memory using > XENMEM_exchange but there are limits on how bug a single contiguous > region can be requested that way (I think it''s around order 9 or so). > > Why do you want domU memory to be contiguous? There is no general > requirement on dom0 or domU for this to be the case and the p2m/m2p > mechanisms exist to allow non-contiguous machine memory. In general > guests only need to arrange for contiguous memory for specific DMA > buffers to specific physical devices which require it. > > Please explain what you are actually trying to achieve so we can best > advise you. > > > (My memory capacity is big enough for meeting all domus and dom0 > > memory requirements.) > > > > > > > > What about xen 4.1.0? > > > > > > > > Thanks in advance. > > > > HUAXIANG FAN > > Software Engineer II > > > > WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD. > > ph: +8610.5884.4327 > > fax: +8610.5884.4727 > > www.websense.cn > > > > Websense TRITON™ > > For Essential Information Protection™ > > Web Security | Data Security | Email Security > > > > > > > > > > > > > > Protected by Websense Hosted Email Security — www.websense.com > > > > > > > To report this as spam, please forward to spam@websense.com. Thank you._______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel