Xen devel - May 2006 - [PATCH] Make "xm mem-set" be lower bound on domX-min-mem

This patch causes "xm mem-set" to be lower bound on domX-min-mem
option
in xend-config. Another configuration option called domU-min-mem has
been introduced, which works similarly to dom0-min-mem. This is prevents
users from freezing the system when doing "xm mem-set" on very low
values like 32M.

Signed-off-by: Aravindh Puthiyaparambil
<aravindh.puthiyaparambil@unisys.com>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I sent this patch in last week. It has not made it into the tree nor
have I heard any feedback about it.  Is there anything wrong with the
patch?

The reason for the patch is described in this discussion:
http://www.nabble.com/xm-mem-set-issues-t1603450.html#a4347490

Thanks,
Aravindh
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-
> bounces@lists.xensource.com] On Behalf Of Puthiyaparambil, Aravindh
> Sent: Friday, May 12, 2006 12:17 PM
> To: xen-devel@lists.xensource.com; Keir Fraser
> Subject: [Xen-devel] [PATCH] Make "xm mem-set" be lower bound on
domX-min-
> mem
> 
> This patch causes "xm mem-set" to be lower bound on domX-min-mem
option
> in xend-config. Another configuration option called domU-min-mem has
> been introduced, which works similarly to dom0-min-mem. This is
prevents
> users from freezing the system when doing "xm mem-set" on very
low
> values like 32M.
> 
> Signed-off-by: Aravindh Puthiyaparambil
> <aravindh.puthiyaparambil@unisys.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, May 12, 2006 at 12:16:57PM -0400, Puthiyaparambil, Aravindh wrote:
> This patch causes "xm mem-set" to be lower bound on domX-min-mem
option
> in xend-config. Another configuration option called domU-min-mem has
> been introduced, which works similarly to dom0-min-mem. This is prevents
> users from freezing the system when doing "xm mem-set" on very
low
> values like 32M.
> 
> Signed-off-by: Aravindh Puthiyaparambil
> <aravindh.puthiyaparambil@unisys.com>
> 
I''m not so sure about this one.  We''ve discussed this issue
recently, and
there are a number of people (notably some of the guys at IBM) who
don''t want
this limit to be imposed by Xend.  If you have one VM that you know to be safe
in 16MB, say, then you either need to set domU-min-mem to 0, defeating the
point of having it, or you need to have a "force" flag to override
domU-min-mem.  The latter option here was disliked, because it was felt that
tools for VM management would just end up using the force flag all the time
because they don''t know what the domU-min-mem setting is, and again,
this
would defeat the point.

The min-mem value should be a per-VM setting rather than a Xend setting, so
the check belongs with the tool that''s managing VMs for you.

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On SLES10 we are using simple minded patch that ensures that the admin
accidentally (or inadvertently) does not set the mem value below a
certain minimum value. The goal here is not to guarantee any level of
performance for the applications that may be running when the memory is
throttled down to the lowest minimum value; but rather that we have
enough memory to support the memory management overhead. Since Arvindh
was working on this, I did not submit the patch to xen-devl. If there is
interest, I can submit our patch for this problem.

Regards,

K. Y
>>> On Fri, May 19, 2006 at 11:29 am, in message
<20060519152923.GG27262@leeni.uk.xensource.com>, Ewan Mellor
<ewan@xensource.com> wrote: 
> On Fri, May 12, 2006 at 12:16:57PM - 0400, Puthiyaparambil, Aravindh
wrote:
> 
>> This patch causes "xm mem- set" to be lower bound on domX-
min- mem
option
>> in xend- config. Another configuration option called domU- min- mem
has
>> been introduced, which works similarly to dom0- min- mem. This is
prevents
>> users from freezing the system when doing "xm mem- set" on
very low
>> values like 32M.
>> 
>> Signed- off- by: Aravindh Puthiyaparambil
>> <aravindh.puthiyaparambil@unisys.com>
>> 
> 
> I''m not so sure about this one.  We''ve discussed this
issue recently,
and
> there are a number of people (notably some of the guys at IBM) who
don''t 
> want
> this limit to be imposed by Xend.  If you have one VM that you know
to be 
> safe
> in 16MB, say, then you either need to set domU- min- mem to 0,
defeating the
> point of having it, or you need to have a "force" flag to
override
> domU- min- mem.  The latter option here was disliked, because it was
felt that
> tools for VM management would just end up using the force flag all
the time
> because they don''t know what the domU- min- mem setting is, and
again, this
> would defeat the point.
> 
> The min- mem value should be a per- VM setting rather than a Xend
setting, so
> the check belongs with the tool that''s managing VMs for you.
> 
> Ewan.
> 
> _______________________________________________
> Xen- devel mailing list
> Xen- devel@lists.xensource.com
> http://lists.xensource.com/xen- devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

---------------------
Message: 4
Date: Fri, 19 May 2006 16:29:23 +0100
From: Ewan Mellor <ewan@xensource.com>
Subject: Re: [Xen-devel] [PATCH] Make "xm mem-set" be lower bound on
             domX-min-mem
To: "Puthiyaparambil, Aravindh"
<aravindh.puthiyaparambil@unisys.com>
Cc: xen-devel@lists.xensource.com
Message-ID: <20060519152923.GG27262@leeni.uk.xensource.com>
Content-Type: text/plain; charset=us-ascii

On Fri, May 12, 2006 at 12:16:57PM -0400, Puthiyaparambil, Aravindh wrote:
> This patch causes "xm mem-set" to be lower bound on domX-min-mem
option
> in xend-config. Another configuration option called domU-min-mem has
> been introduced, which works similarly to dom0-min-mem. This is prevents
> users from freezing the system when doing "xm mem-set" on very
low
> values like 32M.
>
> Signed-off-by: Aravindh Puthiyaparambil
> <aravindh.puthiyaparambil@unisys.com>
>
I''m not so sure about this one.  We''ve discussed this issue
recently, and
there are a number of people (notably some of the guys at IBM) who
don''t
want
this limit to be imposed by Xend.  If you have one VM that you know to be
safe
in 16MB, say, then you either need to set domU-min-mem to 0, defeating the
point of having it, or you need to have a "force" flag to override
domU-min-mem.  The latter option here was disliked, because it was felt
that
tools for VM management would just end up using the force flag all the time
because they don''t know what the domU-min-mem setting is, and again,
this
would defeat the point.

The min-mem value should be a per-VM setting rather than a Xend setting, so
the check belongs with the tool that''s managing VMs for you.

Ewan.
-------------------------------------


Unless there is an known *architectural* limit in Xen on the lower bound of
the memory for a guest DomU, then I agree - xend
shouldn''t impose an arbitrary one simply to act as ''hard
hint'' to prevent
stupid users from doing stupid things
(give ''em all the rope they want I say! :-) Care-and-feeding of naive
users
is best left to tools higher up the mgmt stack (IMO).

That said, I am very curious to understand what known or highly-recommended
minimums and maximums there are for memory (and all guest resources
allocated by Xen for that matter), both for Dom0 and DomU''s, so that
these
can be accurately reflected in the relevant min/max
MemoryResourceAllocationSettingData exposed via the CIM providers.

- Gareth

Dr. Gareth S. Bestor
IBM Linux Technology Center
M/S DES2-01
15300 SW Koll Parkway, Beaverton, OR 97006
503-578-3186, T/L 775-3186, Fax 503-578-3186

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>  Unless there is an known *architectural* limit in Xen on the lower 
> bound of the memory for a guest DomU, then I agree - xend
>  shouldn''t impose an arbitrary one simply to act as ''hard
hint'' to
> prevent stupid users from doing stupid things
>  (give ''em all the rope they want I say! :-) Care-and-feeding of
naive
> users is best left to tools higher up the mgmt stack (IMO).
I agree with this. I''m also not sure about putting a lower bound in the
balloon driver, but at least there we know that shrinking to <2% of 
original allocation is a very bad idea with very high probability.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>...shrinking to <2% of original allocation is a very bad idea
An absolute limit might be easier to handle - and expose to users - than a
relative one, especially up-front in CIM where it exposes min/max limits on
resource allocations. Or is it really <2% of whatever the original memory
allocation is when things go to kabluwey... ?

BTW - at the moment we are exposing a 16MB minimum DomU memory size thru
our CIM providers resource allocation defaults, although this is more a
hint than anything actually enforced; the mgmt client can still pass in
whatever value they like and we (CIM) will blindly pass it along to xm
create ...

- Gareth

Dr. Gareth S. Bestor
IBM Linux Technology Center
M/S DES2-01
15300 SW Koll Parkway, Beaverton, OR 97006
503-578-3186, T/L 775-3186, Fax 503-578-3186



                                                                           
             Keir Fraser                                                   
             <Keir.Fraser@cl.c                                             
             am.ac.uk>                                                  To 
                                       Gareth S Bestor/Beaverton/IBM@IBMUS 
             05/20/06 01:26 AM                                          cc 
                                       xen-devel@lists.xensource.com       
                                                                   Subject 
                                       Re: [Xen-devel] [PATCH] Make "xm    
                                       mem-set" be lower bound on          
                                       domX-min-mem                        
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           



>  Unless there is an known *architectural* limit in Xen on the lower
> bound of the memory for a guest DomU, then I agree - xend
>  shouldn''t impose an arbitrary one simply to act as ''hard
hint'' to
> prevent stupid users from doing stupid things
>  (give ''em all the rope they want I say! :-) Care-and-feeding of
naive
> users is best left to tools higher up the mgmt stack (IMO).
I agree with this. I''m also not sure about putting a lower bound in the
balloon driver, but at least there we know that shrinking to <2% of
original allocation is a very bad idea with very high probability.

  -- Keir






_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 20 May 2006, at 22:27, Gareth S Bestor wrote:
>  An absolute limit might be easier to handle - and expose to users - 
> than a relative one, especially up-front in CIM where it exposes 
> min/max limits on resource allocations. Or is it really <2% of 
> whatever the original memory allocation is when things go to 
> kabluwey... ?
Unfortunately so: some critical kernel data structures are sized 
proportional to the maximum amount of memory that may be allocated to 
the guest. These data structures do not shrink as you remove memory 
from the guest.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Bugger. :-)

But this is good to know! Obviously it stresses the need to keep track of
the initial memory allocation, to set limits on subsequent shrinking.
Fortunately we already do this in the (new) Xen CIM provider model by
maintaining both the so-called ''recorded'' memory allocation
size/settings
(aka initial size) as well as the ''current'' memory allocation
size/setting
(eg after ballooning). So at least we have the data needed to enforce a
lower limit.

thnx!

- Gareth

Dr. Gareth S. Bestor
IBM Linux Technology Center
M/S DES2-01
15300 SW Koll Parkway, Beaverton, OR 97006
503-578-3186, T/L 775-3186, Fax 503-578-3186



                                                                           
             Keir Fraser                                                   
             <Keir.Fraser@cl.c                                             
             am.ac.uk>                                                  To 
                                       Gareth S Bestor/Beaverton/IBM@IBMUS 
             05/21/06 01:45 AM                                          cc 
                                       xen-devel@lists.xensource.com       
                                                                   Subject 
                                       Re: [Xen-devel] [PATCH] Make "xm    
                                       mem-set" be lower bound on          
                                       domX-min-mem                        
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           





On 20 May 2006, at 22:27, Gareth S Bestor wrote:
>  An absolute limit might be easier to handle - and expose to users -
> than a relative one, especially up-front in CIM where it exposes
> min/max limits on resource allocations. Or is it really <2% of
> whatever the original memory allocation is when things go to
> kabluwey... ?
Unfortunately so: some critical kernel data structures are sized
proportional to the maximum amount of memory that may be allocated to
the guest. These data structures do not shrink as you remove memory
from the guest.

  -- Keir






_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sorry to bring this up again. I see that there is dom0-min-mem option in
xend-config. The only time it comes into play is when ballooning memory
away from Dom0 to be given to a DomU that is requesting memory. Xend
ensures that Dom0''s memory does not fall below dom0-min-mem.
Shouldn''t
this option also be taken care of by higher level mgmt tools? If not,
shouldn''t an "xm mem-set" on Dom0 also look at this value?

Thanks,
Aravindh
> -----Original Message-----
> From: Ewan Mellor [mailto:ewan@xensource.com]
> Sent: Friday, May 19, 2006 11:29 AM
> To: Puthiyaparambil, Aravindh
> Cc: xen-devel@lists.xensource.com; Keir Fraser
> Subject: Re: [Xen-devel] [PATCH] Make "xm mem-set" be lower bound
on
domX-min-mem
> 
> On Fri, May 12, 2006 at 12:16:57PM -0400, Puthiyaparambil, Aravindh
wrote:
> 
> > This patch causes "xm mem-set" to be lower bound on
domX-min-mem
option
> > in xend-config. Another configuration option called domU-min-mem has
> > been introduced, which works similarly to dom0-min-mem. This is
prevents
> > users from freezing the system when doing "xm mem-set" on
very low
> > values like 32M.
> >
> > Signed-off-by: Aravindh Puthiyaparambil
> > <aravindh.puthiyaparambil@unisys.com>
> >
> 
> I''m not so sure about this one.  We''ve discussed this
issue recently,
and
> there are a number of people (notably some of the guys at IBM) who
don''t want
> this limit to be imposed by Xend.  If you have one VM that you know to
be safe
> in 16MB, say, then you either need to set domU-min-mem to 0, defeating
the
> point of having it, or you need to have a "force" flag to
override
> domU-min-mem.  The latter option here was disliked, because it was
felt that
> tools for VM management would just end up using the force flag all the
time
> because they don''t know what the domU-min-mem setting is, and
again,
this
> would defeat the point.
> 
> The min-mem value should be a per-VM setting rather than a Xend
setting, so
> the check belongs with the tool that''s managing VMs for you.
> 
> Ewan.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel