Xen devel - May 2008 - [PATCH] balloon: selfballooning and post memory info via xenbus

OK, here's the promised patch.  The overall objective of the
patch is to enable limited memory load-balancing capabilities
as a step toward allowing limited memory overcommit.  With
this and some other minor hackery, I was able to run as
many as 15 lightly loaded 512MB domains on a 2GB system
(yes, veerrrryyy slooowwwlly).

Review/comments appreciated.

With this patch, balloon.c communicates (limited) useful
memory usage information via xenbus.  It also implements
"selfballooning" which applies the memory information
locally to immediately adjust the balloon, giving up memory
when it is not needed and asking for it back when it is needed,
implementing a first-come-first-served system-wide ballooning
"policy".  When a domain asks for memory but none is available,
it must use its own configured swap disk, resulting in
(potentially severe) performance degradation.  Naturally,
it is not recommended to turn on selfballooning in a domain
that has no swap disk or if performance is more important
than increasing the number of VMs runnable on a physical machine.

A key assumption is that the Linux variable vm_committed_space
is a reasonable first approximation of memory needed by a domain.
This approximation will probably improve over time, but is
a good start for now.  The variable is bound on the lower end
by the recently submitted minimum_target() algorithm patch;
thus O-O-M conditions should not occur.

The code is a bit complicated in a couple of places because of
race conditions involving xenstored startup relative to
turning on selfballooning locally.  Because the key variable
(vm_committed_space) is not exported by Linux, I implemented
a horrible hack which still allows the code to work in a
module, however I fully expect that this part of the patch
will not be accepted (which will limit the functionality to
pvm domains only... probably OK for now).

Existing balloon functionality which is unchanged:
- Set target for VM from domain0
- Set target inside VM by writing to /proc/xen/balloon
Existing balloon info on xenbus which is unchanged:
- /local/domain/X/memory/target
To turn on selfballooning:
- Inside a VM: "echo 1 > /proc/xen/balloon"
- From domain0: "xenstore-write /local/domain/X/memory/selfballoon 1"
To turn off selfballooning:
- Inside a VM: "echo 0 > /proc/xen/balloon"
- From domain0: "xenstore-write /local/domain/X/memory/selfballoon 0"
New balloon info now on xenbus:
- /local/domain/X/memory/selfballoon [0 or 1]
- /local/domain/X/memory/actual [kB] *
- /local/domain/X/memory/minimum [kB] *
- /local/domain/X/memory/selftarget [kB] * (only valid if
  selfballoon==1)
 * writeable only by balloon driver in X when either
   selfballooning is first enabled, or target is changed
   by domain0

Thanks,
Dan

==================================Thanks... for the memory
I really could use more / My throughput's on the floor
The balloon is flat / My swap disk's fat / I've OOM's in store
Overcommitted so much
(with apologies to the late great Bob Hope)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Awfully quiet out there... comments please?

Here's a second version.  This one adds hysteresis, both
"up" and "down".

This smooths changes in balloon size to better accomodate
spiky or sine-wave-ish memory loads.

It can also be used as a rudimentary QoS device to modify
the "eagerness" of some domains in attempting to increase
or decrease memory.

By setting downhysteresis (when
selfballooning) to a value of n between 2 and 99,
memory will be reduced by (current - selftarget)/n
every second whenever current > selftarget.  By
setting uphysteresis (when selfballooning) to a value
of n between 2 and 99, memory will be increased by
(current - selftarget)/n every second whenever
current < selftarget.  Both default off.  It is
expected that downhysterisis will often be a largish
value and uphysteresis will often be a smallish value,
so as to respond rapidly to increases in memory usage
and more gradually to decreases in memory usage, which
is as desired.

Dan
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of Dan
> Magenheimer
> Sent: Monday, May 05, 2008 4:45 PM
> To: Xen-Devel (E-mail)
> Subject: [Xen-devel] [PATCH] balloon: selfballooning and post memory
> info via xenbus
> 
> 
> OK, here's the promised patch.  The overall objective of the
> patch is to enable limited memory load-balancing capabilities
> as a step toward allowing limited memory overcommit.  With
> this and some other minor hackery, I was able to run as
> many as 15 lightly loaded 512MB domains on a 2GB system
> (yes, veerrrryyy slooowwwlly).
> 
> Review/comments appreciated.
> 
> With this patch, balloon.c communicates (limited) useful
> memory usage information via xenbus.  It also implements
> "selfballooning" which applies the memory information
> locally to immediately adjust the balloon, giving up memory
> when it is not needed and asking for it back when it is needed,
> implementing a first-come-first-served system-wide ballooning
> "policy".  When a domain asks for memory but none is available,
> it must use its own configured swap disk, resulting in
> (potentially severe) performance degradation.  Naturally,
> it is not recommended to turn on selfballooning in a domain
> that has no swap disk or if performance is more important
> than increasing the number of VMs runnable on a physical machine.
> 
> A key assumption is that the Linux variable vm_committed_space
> is a reasonable first approximation of memory needed by a domain.
> This approximation will probably improve over time, but is
> a good start for now.  The variable is bound on the lower end
> by the recently submitted minimum_target() algorithm patch;
> thus O-O-M conditions should not occur.
> 
> The code is a bit complicated in a couple of places because of
> race conditions involving xenstored startup relative to
> turning on selfballooning locally.  Because the key variable
> (vm_committed_space) is not exported by Linux, I implemented
> a horrible hack which still allows the code to work in a
> module, however I fully expect that this part of the patch
> will not be accepted (which will limit the functionality to
> pvm domains only... probably OK for now).
> 
> Existing balloon functionality which is unchanged:
> - Set target for VM from domain0
> - Set target inside VM by writing to /proc/xen/balloon
> Existing balloon info on xenbus which is unchanged:
> - /local/domain/X/memory/target
> To turn on selfballooning:
> - Inside a VM: "echo 1 > /proc/xen/balloon"
> - From domain0: "xenstore-write /local/domain/X/memory/selfballoon
1"
> To turn off selfballooning:
> - Inside a VM: "echo 0 > /proc/xen/balloon"
> - From domain0: "xenstore-write /local/domain/X/memory/selfballoon
0"
> New balloon info now on xenbus:
> - /local/domain/X/memory/selfballoon [0 or 1]
> - /local/domain/X/memory/actual [kB] *
> - /local/domain/X/memory/minimum [kB] *
> - /local/domain/X/memory/selftarget [kB] * (only valid if
>   selfballoon==1)
>  * writeable only by balloon driver in X when either
>    selfballooning is first enabled, or target is changed
>    by domain0
> 
> Thanks,
> Dan
> 
> ==================================> Thanks... for the memory
> I really could use more / My throughput's on the floor
> The balloon is flat / My swap disk's fat / I've OOM's in store
> Overcommitted so much
> (with apologies to the late great Bob Hope)
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel