Xen devel - May 2008 - RE: [PATCH] linux/balloon:don''t allow ballooningdowna domain below a reasonable limit

>>> "Dan Magenheimer" <dan.magenheimer@oracle.com>
05/01/08 2:00 AM >>>
>OK, I think I am understanding it a bit better:
>the max_pfn part is just adding in some "slop"
>which is a fraction of total main memory which
>is growing smaller (roughly logarithmically)
>as memory grows larger.  I''m still not sure about
>the magic values in MB2PAGES though... I''m guessing
>these were gathered somehow experimentally?
I have to defer to the original author here - Kurt?
>With the "divide result of your algorithm by two",
>I was able to get thirteen 512MB domains (idle
>for now) running on a 2GB system.
You mean ballooned-down domains, right? Perhaps using your
self-ballooning change? I have to admit I''m a little nervous
about attempting to overcommit memory in this way in a
production environment, but as long as this depends on a
decision of the operator it''s certainly a good option to have.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> >OK, I think I am understanding it a bit better:
> >the max_pfn part is just adding in some "slop"
> >which is a fraction of total main memory which
> >is growing smaller (roughly logarithmically)
> >as memory grows larger.  I''m still not sure about
> >the magic values in MB2PAGES though... I''m guessing
> >these were gathered somehow experimentally?
> 
> I have to defer to the original author here - Kurt?
Eagerly awaiting... In addition to cutting it
in half, I subtracted another 10MB (in a memory=512
domain) and still didn''t see any OOMs, though my
testing was admittedly limited.
 
> >With the "divide result of your algorithm by two",
> >I was able to get thirteen 512MB domains (idle
> >for now) running on a 2GB system.
> 
> You mean ballooned-down domains, right? Perhaps using your
> self-ballooning change? I have to admit I''m a little nervous
> about attempting to overcommit memory in this way in a
> production environment, but as long as this depends on a
> decision of the operator it''s certainly a good option to have.
Yes, ballooned-down domains.  In fact with minimum_target()
modified as above (half of algorithm minus 10MB) and
a variable load (repeating { compile xen; sleep(30<rand<541) }),
I got fifteen 512MB domains running on a 2GB systems.

Agreed that there are many environments where this kind
of ballooning would cause performance problems (or worse).
However, there are certainly some environments (and some
competitive situations ;-) where one might choose to
tradeoff performance to run more VMs per physical machine.

Dan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

"Dan Magenheimer" <dan.magenheimer@oracle.com> writes:
>> >OK, I think I am understanding it a bit better:
>> >the max_pfn part is just adding in some "slop"
>> >which is a fraction of total main memory which
>> >is growing smaller (roughly logarithmically)
>> >as memory grows larger.  I''m still not sure about
>> >the magic values in MB2PAGES though... I''m guessing
>> >these were gathered somehow experimentally?
>> 
>> I have to defer to the original author here - Kurt?
>
> Eagerly awaiting... In addition to cutting it
> in half, I subtracted another 10MB (in a memory=512
> domain) and still didn''t see any OOMs, though my
> testing was admittedly limited.
>  
>> >With the "divide result of your algorithm by two",
>> >I was able to get thirteen 512MB domains (idle
>> >for now) running on a 2GB system.
>> 
>> You mean ballooned-down domains, right? Perhaps using your
>> self-ballooning change? I have to admit I''m a little nervous
>> about attempting to overcommit memory in this way in a
>> production environment, but as long as this depends on a
>> decision of the operator it''s certainly a good option to have.
>
> Yes, ballooned-down domains.  In fact with minimum_target()
> modified as above (half of algorithm minus 10MB) and
> a variable load (repeating { compile xen; sleep(30<rand<541) }),
> I got fifteen 512MB domains running on a 2GB systems.
>
> Agreed that there are many environments where this kind
> of ballooning would cause performance problems (or worse).
> However, there are certainly some environments (and some
> competitive situations ;-) where one might choose to
> tradeoff performance to run more VMs per physical machine.
>
> Dan
I for example have 6 domains to compile software for 32bit and 64bit
for Debian stable, testing and unstable each. They can benefit from
more memory when they build gcc for example. But 99.9% of the time
they just wait for something to do.

It would be real nice to autoballoon them down when idle. In fact I
would like to auto balloon them so that the domain that swaps least
gets sized down and the domain that swaps most gets the pages. To a
lesser degree cpu and IO usage should also play a part.

MfG
        Goswin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hmmm... it appears to me that minimum_target() doesn''t
work when balloon.c is built as a module (always returns 0).

Can you confirm/deny?

Also, still hoping to get more info on the algorithm
used in minimum_target().

Thanks,
Dan
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of Jan Beulich
> Sent: Friday, May 02, 2008 1:31 PM
> To: dan.magenheimer@oracle.com
> Cc: Ky Srinivasan; xen-devel@lists.xensource.com;
> keir.fraser@eu.citrix.com; garloff@suse.de
> Subject: RE: [Xen-devel] [PATCH] linux/balloon:don''t allow
> ballooningdowna domain below a reasonable limit
> 
> 
> >>> "Dan Magenheimer" <dan.magenheimer@oracle.com>
05/01/08
> 2:00 AM >>>
> >OK, I think I am understanding it a bit better:
> >the max_pfn part is just adding in some "slop"
> >which is a fraction of total main memory which
> >is growing smaller (roughly logarithmically)
> >as memory grows larger.  I''m still not sure about
> >the magic values in MB2PAGES though... I''m guessing
> >these were gathered somehow experimentally?
> 
> I have to defer to the original author here - Kurt?
> 
> >With the "divide result of your algorithm by two",
> >I was able to get thirteen 512MB domains (idle
> >for now) running on a 2GB system.
> 
> You mean ballooned-down domains, right? Perhaps using your
> self-ballooning change? I have to admit I''m a little nervous
> about attempting to overcommit memory in this way in a
> production environment, but as long as this depends on a
> decision of the operator it''s certainly a good option to have.
> 
> Jan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> "Dan Magenheimer" <dan.magenheimer@oracle.com>
09.05.08 22:38 >>>
>Hmmm... it appears to me that minimum_target() doesn''t
>work when balloon.c is built as a module (always returns 0).
>
>Can you confirm/deny?
Yes, that''s a change Keir did after noticing that I used an improper
variable (totalram_pages) to base the calculation upon in the modular case
(max_pfn is not exported anymore in newer kernels). It should be possible
though to base the calculation on num_physpages, as in the patch below.

Jan

--- head-2008-04-15.orig/drivers/xen/balloon/balloon.c
+++ head-2008-04-15/drivers/xen/balloon/balloon.c
@@ -198,8 +198,8 @@ static unsigned long current_target(void
 static unsigned long minimum_target(void)
 {
 #ifndef CONFIG_XEN
-	return 0;
-#else
+#define max_pfn num_physpages
+#endif
 	unsigned long min_pages, curr_pages = current_target();
 
 #define MB2PAGES(mb) ((mb) << (20 - PAGE_SHIFT))
@@ -227,7 +227,7 @@ static unsigned long minimum_target(void
 
 	/* Don''t enforce growth */
 	return min(min_pages, curr_pages);
-#endif
+#undef max_pfn
 }
 
 static int increase_reservation(unsigned long nr_pages)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I see.  Sorry I missed this in the earlier thread.  I can
confirm that the patch below works in a module.

Keir, the existing checked in balloon.c doesn''t work properly
when compiled as a module so Jan''s patch should be applied.
> -----Original Message-----
> From: Jan Beulich [mailto:jbeulich@novell.com]
> Sent: Tuesday, May 13, 2008 1:08 AM
> To: dan.magenheimer@oracle.com
> Cc: keir.fraser@eu.citrix.com; xen-devel@lists.xensource.com; Ky
> Srinivasan; garloff@suse.de
> Subject: RE: [Xen-devel] [PATCH] linux/balloon: don''t allow
ballooning
> down a domain below a reasonable limit
> 
> 
> >>> "Dan Magenheimer" <dan.magenheimer@oracle.com>
09.05.08 22:38 >>>
> >Hmmm... it appears to me that minimum_target() doesn''t
> >work when balloon.c is built as a module (always returns 0).
> >
> >Can you confirm/deny?
> 
> Yes, that''s a change Keir did after noticing that I used an
improper
> variable (totalram_pages) to base the calculation upon in the 
> modular case
> (max_pfn is not exported anymore in newer kernels). It should 
> be possible
> though to base the calculation on num_physpages, as in the 
> patch below.
> 
> Jan
> 
> --- head-2008-04-15.orig/drivers/xen/balloon/balloon.c
> +++ head-2008-04-15/drivers/xen/balloon/balloon.c
> @@ -198,8 +198,8 @@ static unsigned long current_target(void
>  static unsigned long minimum_target(void)
>  {
>  #ifndef CONFIG_XEN
> -	return 0;
> -#else
> +#define max_pfn num_physpages
> +#endif
>  	unsigned long min_pages, curr_pages = current_target();
>  
>  #define MB2PAGES(mb) ((mb) << (20 - PAGE_SHIFT))
> @@ -227,7 +227,7 @@ static unsigned long minimum_target(void
>  
>  	/* Don''t enforce growth */
>  	return min(min_pages, curr_pages);
> -#endif
> +#undef max_pfn
>  }
>  
>  static int increase_reservation(unsigned long nr_pages)
> 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel