Xen users - Feb 2013 - "dynamic memory allocation" for windows guests?

Hello,

In our office we would like to do something like "memory
overcommitment" or rather "dynamic allocation" for a bunch of
windows domUs (server 2003 & 2008).

We thought this should be a simple feature, however we did not find any fitting
solution.
We use Xen 4.2.1 with xl toolstack.

The idea is to use ballooning to dynamically (and automatically) change physical
memory available to the domUs according to their memory needs.
There was something called "xenballoond" for linux guests before
"tmem" was integrated, and this seems to be exactly what we're
looking for, except that we need it for windows domUs.

Is there really no such thing for windows guests, or are we just too stupid to
find it?
We would be happy about any hints.

If there really is none, then, is there a reason for that?

It seems like a simple thing:
- start a process on each windows guest to monitor memory usage
- if there is spare memory -> tell the hypervisor to lower the mem-target
- if memory is used up, decide upon CPU usage and mem history:
  . idle for some time -> try to free buffers by allocating and freeing
blocks
  . loaded -> do nothing or increase mem-target, according to pagefile usage
  . recently increased -> increase mem-target
- If memory is needed quickly, the pagefile will be used temporarily.
- In a trivial implementation, a simple script on dom0 could receive
  the mem-target requests and do the actual "xl mem-set" calls
  (according to some balancing algorithm).

Or is this approach just too naive?

In our scenario (virtual desktops and testservers) there are always idle guests
wasting memory and not really using it. I suppose there must be a way to
automatically (if just slowly) shift memory to those in need, and thus reducing
overall memory requirements.

yours,
- peter.

-- 
Peter Gansterer

PARADIGMA Unternehmensberatung GmbH
Mariahilferstraße 47/1/3
A-1060 Wien
Tel: 0043-(0)1-585 49 72
http://www.paradigma.net

Firmenbuchnummer: FN 134564 p
Rechtsform: GmbH
Firmenbuchgericht: Handelsgericht Wien

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Hi again,

Are there really no hints about this topic?

It would also be useful to definitely know there is no solution for what we are
looking for - or even input as to why this wouldn't work.
Otherwise we'd think about trying it on our own.

- peda.


On 2013-02-07 18:37:16.481416 Peter Gansterer wrote:
> Hello,
> 
> In our office we would like to do something like "memory
overcommitment"
> or rather "dynamic allocation" for a bunch of windows domUs
(server 2003 & 2008).
> 
> We thought this should be a simple feature, however we did not find any
> fitting solution.
> We use Xen 4.2.1 with xl toolstack.
> 
> The idea is to use ballooning to dynamically (and automatically) change
> physical memory available to the domUs according to their memory needs.
> There was something called "xenballoond" for linux guests before
"tmem"
> was integrated, and this seems to be exactly what we're looking for,
> except that we need it for windows domUs.
> 
> Is there really no such thing for windows guests, or are we just too stupid
to find it?
> We would be happy about any hints.
> 
> If there really is none, then, is there a reason for that?
> 
> It seems like a simple thing:
> - start a process on each windows guest to monitor memory usage
> - if there is spare memory -> tell the hypervisor to lower the
mem-target
> - if memory is used up, decide upon CPU usage and mem history:
>   . idle for some time -> try to free buffers by allocating and freeing
blocks
>   . loaded -> do nothing or increase mem-target, according to pagefile
usage
>   . recently increased -> increase mem-target
> - If memory is needed quickly, the pagefile will be used temporarily.
> - In a trivial implementation, a simple script on dom0 could receive
>   the mem-target requests and do the actual "xl mem-set" calls
>   (according to some balancing algorithm).
> 
> Or is this approach just too naive?
> 
> In our scenario (virtual desktops and testservers) there are always idle
> guests wasting memory and not really using it. I suppose there must be a
> way to automatically (if just slowly) shift memory to those in need, and
> thus reducing overall memory requirements.
> 
> yours,
> - peter.
> 
> -- 
> Peter Gansterer
> 
> PARADIGMA Unternehmensberatung GmbH
> Mariahilferstraße 47/1/3
> A-1060 Wien
> Tel: 0043-(0)1-585 49 72
> http://www.paradigma.net
> 
> Firmenbuchnummer: FN 134564 p
> Rechtsform: GmbH
> Firmenbuchgericht: Handelsgericht Wien
_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

On 10/02/13 11:20, Peter Gansterer wrote:
> Hi again,
>
> Are there really no hints about this topic?
>
> It would also be useful to definitely know there is no solution for what we
are looking for - or even input as to why this wouldn''t work.
> Otherwise we''d think about trying it on our own.
You should look into the Xen Cloud Platform (XCP) [1], which has a 
feature called Dynamic Memory Control, or memory ballooning [2]. It 
requires you to install special PV drivers in your Windows guest. One of 
these drivers is a ballooning daemon, which allocates a range of memory 
from the guest OS, and then gives that memory back to Xen, which gives 
the physical memory backed by that memory region to other guests. See 
[2], "How do balloon drivers work and what is a memory balloon" for
more
details.

Mike

[1] http://wiki.xen.org/wiki/XCP_Overview
[2] http://wiki.xen.org/wiki/Dynamic_Memory_Control

On 2013-02-10 13:13:51 Mike McClurg wrote:
> On 10/02/13 11:20, Peter Gansterer wrote:
> > Hi again,
> >
> > Are there really no hints about this topic?
> >
> > It would also be useful to definitely know there is no solution
> > for what we are looking for - or even input as to why this
wouldn''t work.
> > Otherwise we''d think about trying it on our own.
> 
> You should look into the Xen Cloud Platform (XCP) [1], which has a 
> feature called Dynamic Memory Control, or memory ballooning [2]. It 
> requires you to install special PV drivers in your Windows guest. One of 
> these drivers is a ballooning daemon, which allocates a range of memory 
> from the guest OS, and then gives that memory back to Xen, which gives 
> the physical memory backed by that memory region to other guests. See 
> [2], "How do balloon drivers work and what is a memory balloon"
for more
> details.
> 
> Mike
> 
> [1] http://wiki.xen.org/wiki/XCP_Overview
> [2] http://wiki.xen.org/wiki/Dynamic_Memory_Control
Thx for the hint.
I had a look at the docs; but from there it seems this is just an automatic way
to distribute memory targets when starting or stopping guests. As long as guests
don''t change, memory config stays static.
There is no hint, that memory targets are chosen according to the guests actual
requirements.

I don''t like the idea of seting up an XCP system just to confirm that
this is not what we''re looking for.

Maybe there is an XCP user on the list, who can tell me if DMC dynamically
changes memory targets according to requirements in the domU (even if no
machines are started or stopped)?

- peter.

On 10/02/13 13:44, Peter Gansterer wrote:
> On 2013-02-10 13:13:51 Mike McClurg wrote:
>> On 10/02/13 11:20, Peter Gansterer wrote:
>>> Hi again,
>>>
>>> Are there really no hints about this topic?
>>>
>>> It would also be useful to definitely know there is no solution
>>> for what we are looking for - or even input as to why this
wouldn''t work.
>>> Otherwise we''d think about trying it on our own.
>>
>> You should look into the Xen Cloud Platform (XCP) [1], which has a
>> feature called Dynamic Memory Control, or memory ballooning [2]. It
>> requires you to install special PV drivers in your Windows guest. One
of
>> these drivers is a ballooning daemon, which allocates a range of memory
>> from the guest OS, and then gives that memory back to Xen, which gives
>> the physical memory backed by that memory region to other guests. See
>> [2], "How do balloon drivers work and what is a memory
balloon" for more
>> details.
>>
>> Mike
>>
>> [1] http://wiki.xen.org/wiki/XCP_Overview
>> [2] http://wiki.xen.org/wiki/Dynamic_Memory_Control
>
> Thx for the hint.
> I had a look at the docs; but from there it seems this is just an automatic
way to distribute memory targets when starting or stopping guests. As long as
guests don''t change, memory config stays static.
> There is no hint, that memory targets are chosen according to the guests
actual requirements.
>
> I don''t like the idea of seting up an XCP system just to confirm
that this is not what we''re looking for.
>
> Maybe there is an XCP user on the list, who can tell me if DMC dynamically
changes memory targets according to requirements in the domU (even if no
machines are started or stopped)?
DMC allows you to change the memory targets of the guest while the guest 
is running, not just before the guest boots. It allows you to 
"overprovision" your system, so that the maximum amount of memory 
allocated to guests is greater than the total amount of physical memory 
on the system.

It does not dynamically change memory targets based on memory pressure 
in the guest (as mentioned in the DMC FAQ I referenced above). You''ll 
probably have to roll your own custom solution for this.

If you would like to demo XCP without committing a physical box, you can 
run XCP inside a VM (it works on VirtualBox, Xen, and VMWare Player). 
Unfortunately, you can''t run HVM guests inside a virtual XCP like this,
only PV guests, so you won''t be able to boot Windows guests inside your
nested XCP system.

Mike

On 2013-02-10 14:00:32 Mike McClurg wrote:
> [...]
>
> DMC allows you to change the memory targets of the guest while the guest 
> is running, not just before the guest boots. It allows you to 
> "overprovision" your system, so that the maximum amount of memory
> allocated to guests is greater than the total amount of physical memory 
> on the system.
> 
> It does not dynamically change memory targets based on memory pressure 
> in the guest (as mentioned in the DMC FAQ I referenced above).
You''ll
> probably have to roll your own custom solution for this.
> 
> If you would like to demo XCP without committing a physical box, you can 
> run XCP inside a VM (it works on VirtualBox, Xen, and VMWare Player). 
> Unfortunately, you can''t run HVM guests inside a virtual XCP like
this,
> only PV guests, so you won''t be able to boot Windows guests inside
your
> nested XCP system.
All right, thx for confirming.
So it''s not exactly what we want to do.

Am I the only one who thinks it would be useful to change memory targets
according to actual guest requirements, or is my naive approach just not
feasible in practice? I''d rather refrain from creating our own solution
if it already failed before...

- peter.

Peter,
There are a couple of options.  First, I believe recent versions of Xen
will let you set a maximum allocation and then a lower initial
allocation (XCP at least lets you do this) and you can then allocate
memory between the initial and maximum allocation.  Unfortunately, due
to limitations with Windows virtualization, what actually happens is
that Xen tells Windows that is has memory at the maximum level but marks
the difference between the current allocation and the maximum allocation
as used memory.  As you increase allocation up from the current
allocation to your maximum allocation it frees up the memory in that
range.  This requires a driver/daemon inside of Windows to accomplish
the dynamic allocation - I think GPLPV drivers have support, and I know
the XCP/XenServer drivers include support.

Overcommitment is something entirely different and, while supported in
the open source versions of the Xen hypervisor (look up xenpaging), is
not at this time supported in XCP or commercial XenServer (to my
knowledge).  Overcommitment is the idea of actually allocating more
memory to guests than you have available on your physical machines, with
the risk of running into a situation where you cannot fulfill the
required commitment level to all of the guests.  Overcommitment is a
hypervisor/dom0 feature and shouldn''t require any additional drivers
inside the guest to accomplish.

-Nick
>>> On 2013/02/10 at 04:20, "Peter Gansterer"
<peter.gansterer@paradigma.net>
wrote: 
> Hi again,
> 
> Are there really no hints about this topic?
> 
> It would also be useful to definitely know there is no solution for
what we 
> are looking for - or even input as to why this wouldn''t work.
> Otherwise we''d think about trying it on our own.
> 
> - peda.
> 
> 
> On 2013-02-07 18:37:16.481416 Peter Gansterer wrote:
>> Hello,
>> 
>> In our office we would like to do something like "memory
overcommitment"
>> or rather "dynamic allocation" for a bunch of windows domUs
(server
2003 & 
> 2008).
>> 
>> We thought this should be a simple feature, however we did not find
any
>> fitting solution.
>> We use Xen 4.2.1 with xl toolstack.
>> 
>> The idea is to use ballooning to dynamically (and automatically)
change
>> physical memory available to the domUs according to their memory
needs.
>> There was something called "xenballoond" for linux guests
before
"tmem"
>> was integrated, and this seems to be exactly what we''re
looking
for,
>> except that we need it for windows domUs.
>> 
>> Is there really no such thing for windows guests, or are we just too
stupid 
> to find it?
>> We would be happy about any hints.
>> 
>> If there really is none, then, is there a reason for that?
>> 
>> It seems like a simple thing:
>> - start a process on each windows guest to monitor memory usage
>> - if there is spare memory -> tell the hypervisor to lower the
mem-target
>> - if memory is used up, decide upon CPU usage and mem history:
>>   . idle for some time -> try to free buffers by allocating and
freeing 
> blocks
>>   . loaded -> do nothing or increase mem-target, according to
pagefile usage
>>   . recently increased -> increase mem-target
>> - If memory is needed quickly, the pagefile will be used
temporarily.
>> - In a trivial implementation, a simple script on dom0 could
receive
>>   the mem-target requests and do the actual "xl mem-set"
calls
>>   (according to some balancing algorithm).
>> 
>> Or is this approach just too naive?
>> 
>> In our scenario (virtual desktops and testservers) there are always
idle
>> guests wasting memory and not really using it. I suppose there must
be a
>> way to automatically (if just slowly) shift memory to those in need,
and
>> thus reducing overall memory requirements.
>> 
>> yours,
>> - peter.
>> 
>> -- 
>> Peter Gansterer
>> 
>> PARADIGMA Unternehmensberatung GmbH
>> Mariahilferstraße 47/1/3
>> A-1060 Wien
>> Tel: 0043-(0)1-585 49 72
>> http://www.paradigma.net
>> 
>> Firmenbuchnummer: FN 134564 p
>> Rechtsform: GmbH
>> Firmenbuchgericht: Handelsgericht Wien


--------
This e-mail may contain confidential and privileged material for the sole use of
the intended recipient.  If this email is not intended for you, or you are not
responsible for the delivery of this message to the intended recipient, please
note that this message may contain SEAKR Engineering (SEAKR)
Privileged/Proprietary Information.  In such a case, you are strictly prohibited
from downloading, photocopying, distributing or otherwise using this message,
its contents or attachments in any way.  If you have received this message in
error, please notify us immediately by replying to this e-mail and delete the
message from your mailbox.  Information contained in this message that does not
relate to the business of SEAKR is neither endorsed by nor attributable to
SEAKR.

On 2013-02-10 16:28:47 Nick Couchman wrote:
> Peter,
> There are a couple of options.  First, I believe recent versions of Xen
> will let you set a maximum allocation and then a lower initial
> allocation (XCP at least lets you do this) and you can then allocate
> memory between the initial and maximum allocation.  Unfortunately, due
> to limitations with Windows virtualization, what actually happens is
> that Xen tells Windows that is has memory at the maximum level but marks
> the difference between the current allocation and the maximum allocation
> as used memory.  As you increase allocation up from the current
> allocation to your maximum allocation it frees up the memory in that
> range.  This requires a driver/daemon inside of Windows to accomplish
> the dynamic allocation - I think GPLPV drivers have support, and I know
> the XCP/XenServer drivers include support.
I am well aware of "ballooning" as a means to change the amount of
memory allocated to a running guest. But you still have to manually set that
memory target...
> Overcommitment is something entirely different and, while supported in
> the open source versions of the Xen hypervisor (look up xenpaging), is
> not at this time supported in XCP or commercial XenServer (to my
> knowledge).  Overcommitment is the idea of actually allocating more
> memory to guests than you have available on your physical machines, with
> the risk of running into a situation where you cannot fulfill the
> required commitment level to all of the guests.  Overcommitment is a
> hypervisor/dom0 feature and shouldn''t require any additional
drivers
> inside the guest to accomplish.
Now THAT is a good hint!
I wasn''t aware of "xenpaging".
We will definitely give it a try, although there is a big fat Warning right on
top of "xenpaging.txt" :-/

Thanks a lot!
- peter.