I''m trying to understand how Xen''s memory balloon driver works. IIRC VMware''s is "dynamic" so if my host has, say, 2 GB memory to spare then I can still have 3 guests with 1GB of memory assigned each and so long as all 3 aren''t actually *using* all the memory then VMware will dynamically shuffle the hosts memory around between guests (actually I think VMware even preserves host swap space for guest RAM so theoretically they could all could be using 1GB).>From my understanding with Xen, this is also possible for the most partbut it has to be done manually with ''xm mem-set''. Is this true? The reason I am asking is because I have a few domUs that pretty much run idle except for certain parts of the day. For example, my backup server is on a domU. When it''s idle it''s only using about 33MB of it''s assigned ram, but once a day when backups are run it uses up to 233MB. When it''s idle I''d like it so the unused RAM is available to other guests. Do I have to manually do a ''xm mem-set'' and guesstimate how much memory is going to be used for each guest at certain times of the day or is there a way that Xen can do this dynamically (a la VMware)? _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 21 Feb 2007 at 11:11, Marduk wrote: [...]> I''m trying to understand how Xen''s memory balloon driver works.[...] That reminds me of: Should XEN maintain multiple memory zones for NUMA, and then extend the memory interface to request memory from a specific memory zone? I think recent Linux kernels can even migrate memory pages from one zone to another (probably if a process has a sub-optimal page allocated, are more suitable page can be assigned instead). Regards, Ulrich _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > Ulrich Windl > Sent: 22 February 2007 08:31 > To: xen-users@lists.xensource.com > Subject: Re: [Xen-users] Xen and distributing memory > > On 21 Feb 2007 at 11:11, Marduk wrote: > > [...] > > I''m trying to understand how Xen''s memory balloon driver works. > [...] > > That reminds me of: Should XEN maintain multiple memory zones > for NUMA, and then > extend the memory interface to request memory from a specific > memory zone? I think > recent Linux kernels can even migrate memory pages from one > zone to another > (probably if a process has a sub-optimal page allocated, are > more suitable page > can be assigned instead).At present Xen doesn''t support NUMA, but there''s been work (patches have been submitted but not accepted) to support different memory regions (zones or whatever one might call them) for different locations in the machine. The first step of this would be to give limited set of processors "local memory" when assigning domain-memory. A step further would be to extend the information to give the guest-OS information abuout which socket has which areas of memory, so that the kernel can have a good idea of where to allocate memory. Of course, this doesn''t solve all problemns. Someone (Emmanuel Ackaroy(sp?) I think) gave the following example: one process starts, allocating a large chunk of memory, fills this memory with data and spawns another 3 processes, doing some heavy calculation work on the previously allocated memory. Without direct interfaces to allocate memory to a particular processor, it''s not possible for the OS to allocate that memory correctly... :-( Of course, a further complication is if we have 2 VCPU''s but 4 physically viable CPU''s in the system. Which CPU''s memory do we allocate to which processor? -- Mats> > Regards, > Ulrich > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 26 Feb 2007 at 13:38, Petersson, Mats wrote:> > > > -----Original Message----- > > From: xen-users-bounces@lists.xensource.com > > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > > Ulrich Windl > > Sent: 22 February 2007 08:31 > > To: xen-users@lists.xensource.com > > Subject: Re: [Xen-users] Xen and distributing memory > > > > On 21 Feb 2007 at 11:11, Marduk wrote: > > > > [...] > > > I''m trying to understand how Xen''s memory balloon driver works. > > [...] > > > > That reminds me of: Should XEN maintain multiple memory zones > > for NUMA, and then > > extend the memory interface to request memory from a specific > > memory zone? I think > > recent Linux kernels can even migrate memory pages from one > > zone to another > > (probably if a process has a sub-optimal page allocated, are > > more suitable page > > can be assigned instead). > > At present Xen doesn''t support NUMA, but there''s been work (patches have > been submitted but not accepted) to support different memory regions > (zones or whatever one might call them) for different locations in the > machine. > > The first step of this would be to give limited set of processors "local > memory" when assigning domain-memory. > > A step further would be to extend the information to give the guest-OS > information abuout which socket has which areas of memory, so that the > kernel can have a good idea of where to allocate memory. Of course, this > doesn''t solve all problemns. Someone (Emmanuel Ackaroy(sp?) I think) > gave the following example: one process starts, allocating a large chunk > of memory, fills this memory with data and spawns another 3 processes, > doing some heavy calculation work on the previously allocated memory. > Without direct interfaces to allocate memory to a particular processor, > it''s not possible for the OS to allocate that memory correctly... :-(The strategy would be to allocate the memory for the CPU requesting it. Then the children would default to run on the same CPU because of the shared memory. The master process would be fine with heavy computation. The question is which memory would the children get? If they get memory from another zone, the scheduling preference would be to use a CPU more local to that memory than the current one. That''s what I understood.> > Of course, a further complication is if we have 2 VCPU''s but 4 > physically viable CPU''s in the system. Which CPU''s memory do we allocate > to which processor?Assuming that every virtual CPU is physical in the end, CPU-local RAM should be allocated. That requires that the kernel do know a bit more about the hardware in Dom0. More than now... Regards, Ulrich _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> I''m trying to understand how Xen''s memory balloon driver works. IIRC > VMware''s is "dynamic" so if my host has, say, 2 GB memory to spare then > I can still have 3 guests with 1GB of memory assigned each and so long > as all 3 aren''t actually *using* all the memory then VMware will > dynamically shuffle the hosts memory around between guests (actually I > think VMware even preserves host swap space for guest RAM so > theoretically they could all could be using 1GB).Yep.> >From my understanding with Xen, this is also possible for the most part > > but it has to be done manually with ''xm mem-set''. Is this true?Yep. VMware has an additional feature over Xen in that it''s able to identify common memory contents within and across virtual machines and then share the underlying page frames transparently. This looks the same to the virtual machines but frees up memory when multiple processes / operating systems have the same data in memory.> The reason I am asking is because I have a few domUs that pretty much > run idle except for certain parts of the day. For example, my backup > server is on a domU. When it''s idle it''s only using about 33MB of it''s > assigned ram, but once a day when backups are run it uses up to 233MB. > When it''s idle I''d like it so the unused RAM is available to other > guests. Do I have to manually do a ''xm mem-set'' and guesstimate how > much memory is going to be used for each guest at certain times of the > day or is there a way that Xen can do this dynamically (a la VMware)?It can''t be done automatically at the moment. A nice feature would be to figure out a way of passing load metrics from the guests to dom0 (or deriving them from Xen directly) and then perform the ballooning automatically based on these metrics and an administrator-configured policy. Maybe this will happen at some point... Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Mark, You made that very clear to me. Thanks for the explanation. I imagine some of those VMware features will be adopted by Xen as well. -m _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users