thr3ads.net - Xen devel - RE: [Xen-devel] [PATCH] balloon: selfballooning and post memory info via xenbus, [May 2008]

If this information is useful, please help other people find it:
Share via:

viets@work.de

2008-May-16 09:35 UTC

RE: [Xen-devel] [PATCH] balloon: selfballooning and post memory info via xenbus,

Hello,

thanks for the patch, I was waiting for this feature.

I''ve tried this patch and I''ve seen that if I malloc a great
size of memory in time, this fails, but if I malloc a small size first and then
resize it slowly, it works.

this highly suffisticated (:p) program I use to test the ballooning:

#include <stdio.h>
#include <unistd.h>

int main () {
  void *v;
  int i;
  for(i=40; i < 50; ++i) {
    v = malloc((i*32*1024*1024));
    printf("%i\n", i);
    if ( v != NULL) {
      system("cat /proc/xen/balloon");
      sleep(1);
      free(v);
    }
  }
}

same effect I''ve got if I change the blocksize in a dd: 

works: dd if=zero of=/test.img count=1 bs=32M
doesn''t work: dd if=zero of=/test.img count=1 bs=256M

Don''t know whether this is the right test for this...

greetings
Torben Viets


Dan Magenheimer wrote> OK, here''s the promised patch.  The overall objective of the
> patch is to enable limited memory load-balancing capabilities
> as a step toward allowing limited memory overcommit.  With
> this and some other minor hackery, I was able to run as
> many as 15 lightly loaded 512MB domains on a 2GB system
> (yes, veerrrryyy slooowwwlly).
>
> Review/comments appreciated.
>
> With this patch, balloon.c communicates (limited) useful
> memory usage information via xenbus.  It also implements
> "selfballooning" which applies the memory information
> locally to immediately adjust the balloon, giving up memory
> when it is not needed and asking for it back when it is needed,
> implementing a first-come-first-served system-wide ballooning
> "policy".  When a domain asks for memory but none is available,
> it must use its own configured swap disk, resulting in
> (potentially severe) performance degradation.  Naturally,
> it is not recommended to turn on selfballooning in a domain
> that has no swap disk or if performance is more important
> than increasing the number of VMs runnable on a physical machine.
> 
> A key assumption is that the Linux variable vm_committed_space
> is a reasonable first approximation of memory needed by a domain.
> This approximation will probably improve over time, but is
> a good start for now.  The variable is bound on the lower end
> by the recently submitted minimum_target() algorithm patch;
> thus O-O-M conditions should not occur.
> 
> The code is a bit complicated in a couple of places because of
> race conditions involving xenstored startup relative to
> turning on selfballooning locally.  Because the key variable
> (vm_committed_space) is not exported by Linux, I implemented
> a horrible hack which still allows the code to work in a
> module, however I fully expect that this part of the patch
> will not be accepted (which will limit the functionality to
> pvm domains only... probably OK for now).
> 
> Existing balloon functionality which is unchanged:
> - Set target for VM from domain0
> - Set target inside VM by writing to /proc/xen/balloon
> Existing balloon info on xenbus which is unchanged:
> - /local/domain/X/memory/target
> To turn on selfballooning:
> - Inside a VM: "echo 1 > /proc/xen/balloon"
> - From domain0: "xenstore-write /local/domain/X/memory/selfballoon
1"
> To turn off selfballooning:
> - Inside a VM: "echo 0 > /proc/xen/balloon"
> - From domain0: "xenstore-write /local/domain/X/memory/selfballoon
0"
> New balloon info now on xenbus:
> - /local/domain/X/memory/selfballoon [0 or 1]
> - /local/domain/X/memory/actual [kB] *
> - /local/domain/X/memory/minimum [kB] *
> - /local/domain/X/memory/selftarget [kB] * (only valid if
>   selfballoon==1)
>  * writeable only by balloon driver in X when either
>    selfballooning is first enabled, or target is changed
>    by domain0
> 
> Thanks,
> Dan
> 
> ==================================> Thanks... for the memory
> I really could use more / My throughput''s on the floor
> The balloon is flat / My swap disk''s fat / I''ve
OOM''s in store
> Overcommitted so much
> (with apologies to the late great Bob Hope)
-- 
Torben Viets                              w               Viets@work.de
n@work Internet Informationssysteme GmbH  o      http://support.work.de
Wandalenweg 5                             r   Tel.: +49 40 23 88 09 - 0
D-20097 Hamburg                           k   Fax:  +49 40 23 88 09 -29 
HR B 61 668  - Amtsgericht Hamburg 	      	      Gf Jan Diegelmann 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Dan Magenheimer

2008-May-16 15:19 UTC

head link

RE: [Xen-devel] [PATCH] balloon: selfballooning and post memory info via xenbus,

> thanks for the patch, I was waiting for this feature.
Thanks very much for the testing and feedback!  Could you
comment on what you plan to use it for?  (Keir hasn''t accepted
it yet, so I am looking for user support ;-)

First question: Do you have a swap (virtual) disk configured and,
if so, how big is it?  (Use "swapon -s" and the size shows in KB.)
Selfballooning shouldn''t be run in a domain with no swap disk.
Also, how big is your "memory=" in your vm.cfg file?

I''m not able to reproduce your dd failure at all, even with
bs=2047M (dd doesn''t permit larger values for bs).
Your program (I called it "mallocmem") does eventually fail for
me but not until i==88.  However, I have a 2GB swap disk configured.

I think both tests are really measuring the total virtual memory
space configured, e.g. the sum of physical memory (minus kernel
overhead) and configured swap space.  I think you will find that
both will fail similarly with ballooning off and even on a physical
system, just at different points in virtual memory usage.
Indeed, by adding additional output to mallocmem, I can see that
it fails exactly when it attempts to malloc memory larger than
the CommitLimit value in /proc/meminfo.  I expect the same is
true for the dd test.

Note that CommitLimit DOES go down when memory is ballooned-out
from a guest.  So your test does point out to me that I should
include a warning in the documentation not only that a swap disk
should be configured, but also that the swap disk should be
configured larger for a guest if selfballooning will be turned on.

Thanks,
Dan
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of 
> viets@work.de
> Sent: Friday, May 16, 2008 3:36 AM
> To: xen-devel@lists.xensource.com
> Subject: RE: [Xen-devel] [PATCH] balloon: selfballooning and 
> post memory
> info via xenbus,
> 
> 
> Hello,
> 
> thanks for the patch, I was waiting for this feature.
> 
> I''ve tried this patch and I''ve seen that if I malloc a
great
> size of memory in time, this fails, but if I malloc a small 
> size first and then resize it slowly, it works.
> 
> this highly suffisticated (:p) program I use to test the ballooning:
> 
> #include <stdio.h>
> #include <unistd.h>
> 
> int main () {
>   void *v;
>   int i;
>   for(i=40; i < 50; ++i) {
>     v = malloc((i*32*1024*1024));
>     printf("%i\n", i);
>     if ( v != NULL) {
>       system("cat /proc/xen/balloon");
>       sleep(1);
>       free(v);
>     }
>   }
> }
> 
> same effect I''ve got if I change the blocksize in a dd:
> 
> works: dd if=zero of=/test.img count=1 bs=32M
> doesn''t work: dd if=zero of=/test.img count=1 bs=256M
> 
> Don''t know whether this is the right test for this...
> 
> greetings
> Torben Viets
> 
> 
> Dan Magenheimer wrote
> > OK, here''s the promised patch.  The overall objective of the
> > patch is to enable limited memory load-balancing capabilities
> > as a step toward allowing limited memory overcommit.  With
> > this and some other minor hackery, I was able to run as
> > many as 15 lightly loaded 512MB domains on a 2GB system
> > (yes, veerrrryyy slooowwwlly).
> >
> > Review/comments appreciated.
> >
> > With this patch, balloon.c communicates (limited) useful
> > memory usage information via xenbus.  It also implements
> > "selfballooning" which applies the memory information
> > locally to immediately adjust the balloon, giving up memory
> > when it is not needed and asking for it back when it is needed,
> > implementing a first-come-first-served system-wide ballooning
> > "policy".  When a domain asks for memory but none is
available,
> > it must use its own configured swap disk, resulting in
> > (potentially severe) performance degradation.  Naturally,
> > it is not recommended to turn on selfballooning in a domain
> > that has no swap disk or if performance is more important
> > than increasing the number of VMs runnable on a physical machine.
> >
> > A key assumption is that the Linux variable vm_committed_space
> > is a reasonable first approximation of memory needed by a domain.
> > This approximation will probably improve over time, but is
> > a good start for now.  The variable is bound on the lower end
> > by the recently submitted minimum_target() algorithm patch;
> > thus O-O-M conditions should not occur.
> >
> > The code is a bit complicated in a couple of places because of
> > race conditions involving xenstored startup relative to
> > turning on selfballooning locally.  Because the key variable
> > (vm_committed_space) is not exported by Linux, I implemented
> > a horrible hack which still allows the code to work in a
> > module, however I fully expect that this part of the patch
> > will not be accepted (which will limit the functionality to
> > pvm domains only... probably OK for now).
> >
> > Existing balloon functionality which is unchanged:
> > - Set target for VM from domain0
> > - Set target inside VM by writing to /proc/xen/balloon
> > Existing balloon info on xenbus which is unchanged:
> > - /local/domain/X/memory/target
> > To turn on selfballooning:
> > - Inside a VM: "echo 1 > /proc/xen/balloon"
> > - From domain0: "xenstore-write 
> /local/domain/X/memory/selfballoon 1"
> > To turn off selfballooning:
> > - Inside a VM: "echo 0 > /proc/xen/balloon"
> > - From domain0: "xenstore-write 
> /local/domain/X/memory/selfballoon 0"
> > New balloon info now on xenbus:
> > - /local/domain/X/memory/selfballoon [0 or 1]
> > - /local/domain/X/memory/actual [kB] *
> > - /local/domain/X/memory/minimum [kB] *
> > - /local/domain/X/memory/selftarget [kB] * (only valid if
> >   selfballoon==1)
> >  * writeable only by balloon driver in X when either
> >    selfballooning is first enabled, or target is changed
> >    by domain0
> >
> > Thanks,
> > Dan
> >
> > ==================================> > Thanks... for the memory
> > I really could use more / My throughput''s on the floor
> > The balloon is flat / My swap disk''s fat / I''ve
OOM''s in store
> > Overcommitted so much
> > (with apologies to the late great Bob Hope)
> 
> --
> Torben Viets                              w               
> Viets@work.de
> n@work Internet Informationssysteme GmbH  o      
> http://support.work.de
> Wandalenweg 5                             r   Tel.: +49 40 23 
> 88 09 - 0
> D-20097 Hamburg                           k   Fax:  +49 40 23 
> 88 09 -29
> HR B 61 668  - Amtsgericht Hamburg 	      	      Gf Jan Diegelmann
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - May 2008 - RE: [PATCH] balloon: selfballooning and post memory info via xenbus,

RE: [Xen-devel] [PATCH] balloon: selfballooning and post memory info via xenbus,

RE: [Xen-devel] [PATCH] balloon: selfballooning and post memory info via xenbus,