Xen devel - Nov 2006 - [PATCH] Make ballooning work with maxmem > mem (i386 version)

Trying to start a guest with maxmem > mem and then balloon up to a value
greater than mem is currently failing. This have been already discovered
(patch sent some days ago) for x86_64. i386 suffers from the same
problem.

This patch fixes it.

-- 
Glauber de Oliveira Costa
Red Hat Inc.
"Free as in Freedom"


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Nov 10, 2006 at 03:23:15PM +0000, Keir Fraser
wrote:
> On 10/11/06 13:24, "Glauber de Oliveira Costa"
<gcosta@redhat.com> wrote:
> 
> > Trying to start a guest with maxmem > mem and then balloon up to a
value
> > greater than mem is currently failing. This have been already
discovered
> > (patch sent some days ago) for x86_64. i386 suffers from the same
> > problem.
> > 
> > This patch fixes it.
> 
> I took both patches and then changed my mind and immediately reverted them.
> There is a better way: we should support the XENMEM_memory_map hypercall.
> We should provide a hypercall (domctl) to set a memory_map_limit parameter
> and then Xen can use that to fake a memory map when XENMEM_memory_map is
> called. The tools can set that parameter from
config[''maxmem''].
And what happens when the hypercall ever returns ENOSYS, like a kernel
running in a bit old Hypervisor? 

IMHO,If we have to ever fallback into default assumptions, it seems wiser 
to extend the physicall map to maximum_reservation, not current_reservation. 

-- 
Glauber de Oliveira Costa
Red Hat Inc.
"Free as in Freedom"

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/11/06 15:33, "Glauber de Oliveira Costa"
<gcosta@redhat.com> wrote:
>> I took both patches and then changed my mind and immediately reverted
them.
>> There is a better way: we should support the XENMEM_memory_map
hypercall.
>> We should provide a hypercall (domctl) to set a memory_map_limit
parameter
>> and then Xen can use that to fake a memory map when XENMEM_memory_map
is
>> called. The tools can set that parameter from
config[''maxmem''].
> 
> And what happens when the hypercall ever returns ENOSYS, like a kernel
> running in a bit old Hypervisor?
> 
> IMHO,If we have to ever fallback into default assumptions, it seems wiser
> to extend the physicall map to maximum_reservation, not
current_reservation.
Maxmem will in future be fixed to track tot_pages. That was its original
purpose: to cap what memory the guest is allowed *now*, not to tell it the
max that it will ever be allowed. Also on e.g., dom0 the max is implemented
as ~0UL (no limit). "Fortunately" this looks like a -ve return code
from the
max_mem query hypercall, but if it didn''t (for example we masked off
the
high bit to make sure the return value was +ve; a fix I will probably
implement) then dom0 boot will be broken as it will try to allocate a
much-too-big p2m table.

We have a way to tell the guest its maximum-ever memory allowance, it''s
just
not implemented on the tools and hypervisor side yet. That should be fixed.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/11/06 15:33, "Glauber de Oliveira Costa"
<gcosta@redhat.com> wrote:
>> I took both patches and then changed my mind and immediately reverted
them.
>> There is a better way: we should support the XENMEM_memory_map
hypercall.
>> We should provide a hypercall (domctl) to set a memory_map_limit
parameter
>> and then Xen can use that to fake a memory map when XENMEM_memory_map
is
>> called. The tools can set that parameter from
config[''maxmem''].
> 
> And what happens when the hypercall ever returns ENOSYS, like a kernel
> running in a bit old Hypervisor?
Oh, and bear in mind you can always use ''memory='' as a boot
parameter to
force the p2m size and mem_map size.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Nov 10, 2006 at 03:44:07PM +0000, Keir Fraser
wrote:
> 
> 
> 
> On 10/11/06 15:33, "Glauber de Oliveira Costa"
<gcosta@redhat.com> wrote:
> 
> >> I took both patches and then changed my mind and immediately
reverted them.
> >> There is a better way: we should support the XENMEM_memory_map
hypercall.
> >> We should provide a hypercall (domctl) to set a memory_map_limit
parameter
> >> and then Xen can use that to fake a memory map when
XENMEM_memory_map is
> >> called. The tools can set that parameter from
config[''maxmem''].
> > 
> > And what happens when the hypercall ever returns ENOSYS, like a kernel
> > running in a bit old Hypervisor?
> 
> Oh, and bear in mind you can always use ''memory='' as a
boot parameter to
> force the p2m size and mem_map size.
That''s not particuarly helpful when booting a domain with pygrub
because
now everytime you change the memory settings in the config for the guest
you also need to update the grub config to actually make it take effect.
Or, if you''re not using pygrub, then every time you change the
''mem''
setting in the guest config, you also need to update the
''extra'' setting
for the command line.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Nov 10, 2006 at 03:43:10PM +0000, Keir Fraser
wrote:
> On 10/11/06 15:33, "Glauber de Oliveira Costa"
<gcosta@redhat.com> wrote:
> 
> >> I took both patches and then changed my mind and immediately
reverted them.
> >> There is a better way: we should support the XENMEM_memory_map
hypercall.
> >> We should provide a hypercall (domctl) to set a memory_map_limit
parameter
> >> and then Xen can use that to fake a memory map when
XENMEM_memory_map is
> >> called. The tools can set that parameter from
config[''maxmem''].
> > 
> > And what happens when the hypercall ever returns ENOSYS, like a kernel
> > running in a bit old Hypervisor?
> > 
> > IMHO,If we have to ever fallback into default assumptions, it seems
wiser
> > to extend the physicall map to maximum_reservation, not
current_reservation.
> 
> Maxmem will in future be fixed to track tot_pages. That was its original
> purpose: to cap what memory the guest is allowed *now*, not to tell it the
> max that it will ever be allowed. Also on e.g., dom0 the max is implemented
> as ~0UL (no limit). "Fortunately" this looks like a -ve return
code from the
> max_mem query hypercall, but if it didn''t (for example we masked
off the
> high bit to make sure the return value was +ve; a fix I will probably
> implement) then dom0 boot will be broken as it will try to allocate a
> much-too-big p2m table.
> 
> We have a way to tell the guest its maximum-ever memory allowance,
it''s just
> not implemented on the tools and hypervisor side yet. That should be fixed.
This is not going to be any help for guests running on existing 3.0.3 tree
though. Glauber''s patch makes the maxmem stuff work correctly for guest
kernels without requiring an upgrade of the HV which is much nicer for
compatability allowing both current & future guests to work on both current
and future HVs. With the increasingly widespread deployment of Xen I think
this kind of compatability is very important.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/11/06 15:54, "Daniel P. Berrange" <berrange@redhat.com>
wrote:
> This is not going to be any help for guests running on existing 3.0.3 tree
> though. Glauber''s patch makes the maxmem stuff work correctly for
guest
> kernels without requiring an upgrade of the HV which is much nicer for
> compatability allowing both current & future guests to work on both
current
> and future HVs. With the increasingly widespread deployment of Xen I think
> this kind of compatability is very important.
I''d rather have a scheme that worked with older guests than one which
worked
with older hypervisors. It also happens to be the more tasteful solution
here.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Nov 10, 2006 at 03:43:10PM +0000, Keir Fraser
wrote:
> On 10/11/06 15:33, "Glauber de Oliveira Costa"
<gcosta@redhat.com> wrote:
> 
> >> I took both patches and then changed my mind and immediately
reverted them.
> >> There is a better way: we should support the XENMEM_memory_map
hypercall.
> >> We should provide a hypercall (domctl) to set a memory_map_limit
parameter
> >> and then Xen can use that to fake a memory map when
XENMEM_memory_map is
> >> called. The tools can set that parameter from
config[''maxmem''].
> > 
> > And what happens when the hypercall ever returns ENOSYS, like a kernel
> > running in a bit old Hypervisor?
> > 
> > IMHO,If we have to ever fallback into default assumptions, it seems
wiser
> > to extend the physicall map to maximum_reservation, not
current_reservation.
> 
> Maxmem will in future be fixed to track tot_pages. That was its original
> purpose: to cap what memory the guest is allowed *now*, not to tell it the
> max that it will ever be allowed. 
In this scenario, what''s the purpose of current_reservation, as the
only
difference from it now, is that it returns tot_pages instead of
max_pages ?

-- 
Glauber de Oliveira Costa
Red Hat Inc.
"Free as in Freedom"

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/11/06 16:10, "Glauber de Oliveira Costa"
<gcosta@redhat.com> wrote:
>> Maxmem will in future be fixed to track tot_pages. That was its
original
>> purpose: to cap what memory the guest is allowed *now*, not to tell it
the
>> max that it will ever be allowed.
> 
> In this scenario, what''s the purpose of current_reservation, as
the only
> difference from it now, is that it returns tot_pages instead of
> max_pages ?
The idea is that when you ask a guest to balloon down you will change its
balloon target, then you will give it ''reasonable time'' to
drop its memory
allocation (reflected by tot_pages). If it fails to comply you take action
(e.g., warn user, suspend or kill the guest, etc). If it complies you drop
max_pages.

So you see that max_pages tracks behind tot_pages when you balloon down, and
tracks ahead when you balloon up. It has a distinct purpose separate from
that of tot_pages. That''s the theory anyway. It''s never been
properly
implemented in the python tools. But that may well change in future, so
I''m
not going to bake assumptions into guest kernels.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> We have a way to tell the guest its maximum-ever memory allowance,
it''s just
> not implemented on the tools and hypervisor side yet. That should be fixed.
Do we have any specification on how should this look like, and what kind
of behaviour it should provide?


-- 
Glauber de Oliveira Costa
Red Hat Inc.
"Free as in Freedom"

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 10/11/06 8:27 pm, "Glauber de Oliveira Costa"
<gcosta@redhat.com> wrote:
>> We have a way to tell the guest its maximum-ever memory allowance,
it''s just
>> not implemented on the tools and hypervisor side yet. That should be
fixed.
> 
> Do we have any specification on how should this look like, and what kind
> of behaviour it should provide?
XENMEM_set_memory_map taking a struct { domain, range_list }. This can
initialise a small array embedded in the domain structure.

Plumb through libxc and python xc.so. Probably make the call from
LinuxImageHandler (or whatever it''s called) in image.py.

Done. 

Most of the pain is in the tools plumbing (in particular arg parsing in the
python wrapper).

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel