James Dingwall
2013-Oct-29 10:26 UTC
Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
Hi,
I am having some memory ballooning problems which only seem to have
appeared since upgrading to Xen 4.3. In short if I set the domU
configuration file as:
memory = 512
maxmem = 1024
Then even under memory pressure the guest domain does not balloon past
the value of the ''memory'' parameter.
xl info shows plenty of available memory in dom0:
total_memory : 32767
free_memory : 12445
xl list -l on the domain shows:
"max_memkb": 1048576,
"target_memkb": 524288,
xl top shows:
NAME guestdom
STATE --b---
CPU(sec) 79
CPU(%) 1.2
MEM(k) 343116
MEM(%) 1.0
MAXMEM(k) 525312
MAXMEM(%) 1.6
VCPUS 2
NETS 0
NETTX(k) 0
NETRX(k) 0
VBDS 2
VBD_OO 0
VBD_RD 8639
VBD_WR 129666
VBD_RSECT 529846
VBD_WSECT 1622593
SSID 0
In the domU dmesg it seems as though the guest has some idea that it has
a ''maxmem'' size of 1024Mb:
[ 0.000000] Memory: 408932K/1048188K available (4562K kernel code,
541K rwdata, 1740K rodata, 924K init, 644K bss, 639256K reserved)
but /proc/iomap seems to have reduced it to ''memory'':
00000000-00000fff : reserved
00001000-0009ffff : System RAM
000a0000-000fffff : reserved
000f0000-000fffff : System ROM
00100000-1fffffff : System RAM
01000000-01474c38 : Kernel code
01474c39-016b167f : Kernel data
017a0000-01840fff : Kernel bss
20000000-d7feffff : Unusable memory
d7ff0000-d7ffdfff : ACPI Tables
d7ffe000-d7ffffff : ACPI Non-volatile Storage
100000000-11fffffff : System RAM
After running
xl mem-max guestdom 1024
xl list -l remains
"max_memkb": 1048576,
"target_memkb": 524288,
xl top has changed:
NAME guestdom
STATE --b---
CPU(sec) 98
CPU(%) 0.9
MEM(k) 343116
MEM(%) 1.0
MAXMEM(k) 1049600
MAXMEM(%) 3.1
VCPUS 2
NETS 0
NETTX(k) 0
NETRX(k) 0
VBDS 2
VBD_OO 0
VBD_RD 9068
VBD_WR 130257
VBD_RSECT 549640
VBD_WSECT 1628666
SSID 0
Although /proc/iomem doesn''t change the guest has no problem ballooning
up to the ''maxmem'' value. It seems that this is a bug with xl
not
setting the appropriate paramter somewhere when the domain is created.
The domU and dom0 kernel version is 3.11.6, Xen is 4.3. Please let me
know if examinging any other settings before/after the xl mem-max
command is run would be helpful.
Thanks,
James
Wei Liu
2013-Oct-29 10:59 UTC
Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote:> Hi, > > I am having some memory ballooning problems which only seem to have > appeared since upgrading to Xen 4.3. In short if I set the domU > configuration file as: > > memory = 512 > maxmem = 1024 > > Then even under memory pressure the guest domain does not balloon past > the value of the ''memory'' parameter. > > xl info shows plenty of available memory in dom0: > total_memory : 32767 > free_memory : 12445 > > > xl list -l on the domain shows: > "max_memkb": 1048576, > "target_memkb": 524288, >This is parsed from your config file so they should always look OK to you. Does the following patch help? Git blame tells me the change to set maxmem to target_memkb was introduced 4 years ago so I suspect there''s reason to do that. If we cannot fix it here we need to insert the corresponding call later. Wei. ----8<--- diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index 356f920..fb7965d 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -235,7 +235,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, libxl_domain_set_nodeaffinity(ctx, domid, &info->nodemap); libxl_set_vcpuaffinity_all(ctx, domid, info->max_vcpus, &info->cpumap); - xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT); + xc_domain_setmaxmem(ctx->xch, domid, info->max_memkb + LIBXL_MAXMEM_CONSTANT); xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL); state->store_domid = xs_domid ? atoi(xs_domid) : 0; free(xs_domid);
Andrew Cooper
2013-Oct-29 11:06 UTC
Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
On 29/10/13 10:59, Wei Liu wrote:> On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote: >> Hi, >> >> I am having some memory ballooning problems which only seem to have >> appeared since upgrading to Xen 4.3. In short if I set the domU >> configuration file as: >> >> memory = 512 >> maxmem = 1024 >> >> Then even under memory pressure the guest domain does not balloon past >> the value of the ''memory'' parameter. >> >> xl info shows plenty of available memory in dom0: >> total_memory : 32767 >> free_memory : 12445 >> >> >> xl list -l on the domain shows: >> "max_memkb": 1048576, >> "target_memkb": 524288, >> > This is parsed from your config file so they should always look OK to > you. > > Does the following patch help? > > Git blame tells me the change to set maxmem to target_memkb was > introduced 4 years ago so I suspect there''s reason to do that. If we > cannot fix it here we need to insert the corresponding call later.When setmaxmem sets a limit lower than current, the domain can strictly only balloon down until it is equal to or under the new limit. Performing a setmaxmem hypercall to info->target_memkb will prevent from domain from ballooning down then back up a bit, when it has been asked to balloon down a lot; i.e. it must strictly balloon down to the limit it has been given. Therefore, I am not sure this change is valid. ~Andrew> > Wei. > ----8<--- > diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c > index 356f920..fb7965d 100644 > --- a/tools/libxl/libxl_dom.c > +++ b/tools/libxl/libxl_dom.c > @@ -235,7 +235,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, > libxl_domain_set_nodeaffinity(ctx, domid, &info->nodemap); > libxl_set_vcpuaffinity_all(ctx, domid, info->max_vcpus, &info->cpumap); > > - xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT); > + xc_domain_setmaxmem(ctx->xch, domid, info->max_memkb + LIBXL_MAXMEM_CONSTANT); > xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL); > state->store_domid = xs_domid ? atoi(xs_domid) : 0; > free(xs_domid); > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
James Dingwall
2013-Oct-29 11:40 UTC
Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
On Tue, Oct 29, 2013 at 10:59:21AM +0000, Wei Liu wrote:> On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote: > > Hi, > > > > I am having some memory ballooning problems which only seem to have > > appeared since upgrading to Xen 4.3. In short if I set the domU > > configuration file as: > > > > memory = 512 > > maxmem = 1024 > > > > Then even under memory pressure the guest domain does not balloon past > > the value of the ''memory'' parameter. > > > > xl info shows plenty of available memory in dom0: > > total_memory : 32767 > > free_memory : 12445 > > > > > > xl list -l on the domain shows: > > "max_memkb": 1048576, > > "target_memkb": 524288, > > > > This is parsed from your config file so they should always look OK to > you. > > Does the following patch help? > > Git blame tells me the change to set maxmem to target_memkb was > introduced 4 years ago so I suspect there''s reason to do that. If we > cannot fix it here we need to insert the corresponding call later. > > Wei. > ----8<--- > diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c > index 356f920..fb7965d 100644 > --- a/tools/libxl/libxl_dom.c > +++ b/tools/libxl/libxl_dom.c > @@ -235,7 +235,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, > libxl_domain_set_nodeaffinity(ctx, domid, &info->nodemap); > libxl_set_vcpuaffinity_all(ctx, domid, info->max_vcpus, &info->cpumap); > > - xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT); > + xc_domain_setmaxmem(ctx->xch, domid, info->max_memkb + LIBXL_MAXMEM_CONSTANT); > xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL); > state->store_domid = xs_domid ? atoi(xs_domid) : 0; > free(xs_domid); >This patch solves the problem for me where maxmem in config file is not honoured during domain creation. With it applied xl top immediately shows the correct value in the MAXMEM column. I can see that xl_cmdimpl.c safely sets max_memkb for the case where the configuration file does not define maxmem. Thanks, James -- ------------------------------------------------------------------------ James Dingwall e: james@dingwall.me.uk w: http://www.dingwall.me.uk/ ------------------------------------------------------------------------
Wei Liu
2013-Oct-29 12:08 UTC
Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
On Tue, Oct 29, 2013 at 11:06:01AM +0000, Andrew Cooper wrote:> On 29/10/13 10:59, Wei Liu wrote: > > On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote: > >> Hi, > >> > >> I am having some memory ballooning problems which only seem to have > >> appeared since upgrading to Xen 4.3. In short if I set the domU > >> configuration file as: > >> > >> memory = 512 > >> maxmem = 1024 > >> > >> Then even under memory pressure the guest domain does not balloon past > >> the value of the ''memory'' parameter. > >> > >> xl info shows plenty of available memory in dom0: > >> total_memory : 32767 > >> free_memory : 12445 > >> > >> > >> xl list -l on the domain shows: > >> "max_memkb": 1048576, > >> "target_memkb": 524288, > >> > > This is parsed from your config file so they should always look OK to > > you. > > > > Does the following patch help? > > > > Git blame tells me the change to set maxmem to target_memkb was > > introduced 4 years ago so I suspect there''s reason to do that. If we > > cannot fix it here we need to insert the corresponding call later. > > When setmaxmem sets a limit lower than current, the domain can strictly > only balloon down until it is equal to or under the new limit. > > Performing a setmaxmem hypercall to info->target_memkb will prevent from > domain from ballooning down then back up a bit, when it has been asked > to balloon down a lot; i.e. it must strictly balloon down to the limit > it has been given. > > Therefore, I am not sure this change is valid. > > ~Andrew >Looking at the changeset 9905ac that introduced target_memkb, target_memkb was always set to max_memkb, so my guess is that the purpose was still to use max_memkb to be max memory limit. Probably later changes to that portion of code altered that behavior. The use of target_memkb in xc_domain_setmaxmem looks logically wrong to me anyway... Stefano, thoughts? Wei.
Konrad Rzeszutek Wilk
2013-Oct-29 14:49 UTC
Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
On Tue, Oct 29, 2013 at 12:08:04PM +0000, Wei Liu wrote:> On Tue, Oct 29, 2013 at 11:06:01AM +0000, Andrew Cooper wrote: > > On 29/10/13 10:59, Wei Liu wrote: > > > On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote: > > >> Hi, > > >> > > >> I am having some memory ballooning problems which only seem to have > > >> appeared since upgrading to Xen 4.3. In short if I set the domU > > >> configuration file as: > > >> > > >> memory = 512 > > >> maxmem = 1024 > > >> > > >> Then even under memory pressure the guest domain does not balloon past > > >> the value of the ''memory'' parameter. > > >> > > >> xl info shows plenty of available memory in dom0: > > >> total_memory : 32767 > > >> free_memory : 12445 > > >> > > >> > > >> xl list -l on the domain shows: > > >> "max_memkb": 1048576, > > >> "target_memkb": 524288, > > >> > > > This is parsed from your config file so they should always look OK to > > > you. > > > > > > Does the following patch help? > > > > > > Git blame tells me the change to set maxmem to target_memkb was > > > introduced 4 years ago so I suspect there''s reason to do that. If we > > > cannot fix it here we need to insert the corresponding call later. > > > > When setmaxmem sets a limit lower than current, the domain can strictly > > only balloon down until it is equal to or under the new limit. > > > > Performing a setmaxmem hypercall to info->target_memkb will prevent from > > domain from ballooning down then back up a bit, when it has been asked > > to balloon down a lot; i.e. it must strictly balloon down to the limit > > it has been given. > > > > Therefore, I am not sure this change is valid. > > > > ~Andrew > > > > Looking at the changeset 9905ac that introduced target_memkb, > target_memkb was always set to max_memkb, so my guess is that the > purpose was still to use max_memkb to be max memory limit. > > Probably later changes to that portion of code altered that behavior. > > The use of target_memkb in xc_domain_setmaxmem looks logically wrong to > me anyway... > > Stefano, thoughts?Lets also include Daniel in this - as he was trying to fix this in the past.
Daniel Kiper
2013-Oct-29 18:02 UTC
Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
On Tue, Oct 29, 2013 at 10:49:35AM -0400, Konrad Rzeszutek Wilk wrote:> On Tue, Oct 29, 2013 at 12:08:04PM +0000, Wei Liu wrote: > > On Tue, Oct 29, 2013 at 11:06:01AM +0000, Andrew Cooper wrote: > > > On 29/10/13 10:59, Wei Liu wrote: > > > > On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote: > > > >> Hi, > > > >> > > > >> I am having some memory ballooning problems which only seem to have > > > >> appeared since upgrading to Xen 4.3. In short if I set the domU > > > >> configuration file as: > > > >> > > > >> memory = 512 > > > >> maxmem = 1024 > > > >> > > > >> Then even under memory pressure the guest domain does not balloon past > > > >> the value of the ''memory'' parameter. > > > >> > > > >> xl info shows plenty of available memory in dom0: > > > >> total_memory : 32767 > > > >> free_memory : 12445 > > > >> > > > >> > > > >> xl list -l on the domain shows: > > > >> "max_memkb": 1048576, > > > >> "target_memkb": 524288, > > > >> > > > > This is parsed from your config file so they should always look OK to > > > > you. > > > > > > > > Does the following patch help? > > > > > > > > Git blame tells me the change to set maxmem to target_memkb was > > > > introduced 4 years ago so I suspect there''s reason to do that. If we > > > > cannot fix it here we need to insert the corresponding call later. > > > > > > When setmaxmem sets a limit lower than current, the domain can strictly > > > only balloon down until it is equal to or under the new limit. > > > > > > Performing a setmaxmem hypercall to info->target_memkb will prevent from > > > domain from ballooning down then back up a bit, when it has been asked > > > to balloon down a lot; i.e. it must strictly balloon down to the limit > > > it has been given. > > > > > > Therefore, I am not sure this change is valid. > > > > > > ~Andrew > > > > > > > Looking at the changeset 9905ac that introduced target_memkb, > > target_memkb was always set to max_memkb, so my guess is that the > > purpose was still to use max_memkb to be max memory limit. > > > > Probably later changes to that portion of code altered that behavior. > > > > The use of target_memkb in xc_domain_setmaxmem looks logically wrong to > > me anyway... > > > > Stefano, thoughts? > > Lets also include Daniel in this - as he was trying to fix this in the past.As I remember this behavior is by design. However, this is not in line with xm behavior which sometimes makes a problem. I have discussed this with IanJ and IanC (both CC-ed here) once. I have proposed some patch series to fix that issue but later I was not able to continue work on it. If you wish I could repost them once again with fixes requested by IanJ and IanC. You could also find latest version of patches here: http://lists.xenproject.org/archives/html/xen-devel/2013-04/msg03072.html Daniel
Wei Liu
2013-Oct-30 11:39 UTC
Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
On Tue, Oct 29, 2013 at 07:02:06PM +0100, Daniel Kiper wrote: [...]> > > > > > Stefano, thoughts? > > > > Lets also include Daniel in this - as he was trying to fix this in the past. > > As I remember this behavior is by design. However, this is not in line with > xm behavior which sometimes makes a problem. I have discussed this with IanJ > and IanC (both CC-ed here) once. I have proposed some patch series to fix that > issue but later I was not able to continue work on it. If you wish I could > repost them once again with fixes requested by IanJ and IanC. You could also > find latest version of patches here: > > http://lists.xenproject.org/archives/html/xen-devel/2013-04/msg03072.html >Oh, sure. If you have something new then please do. ;-) AFAICT from the latest series, the real meat that fixes this problem is in patch 2, which got no reply. Does that mean no objection was raised on that approach? Wei.> Daniel
Daniel Kiper
2013-Oct-31 13:35 UTC
Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
On Wed, Oct 30, 2013 at 11:39:51AM +0000, Wei Liu wrote:> On Tue, Oct 29, 2013 at 07:02:06PM +0100, Daniel Kiper wrote: > [...] > > > > > > > > Stefano, thoughts? > > > > > > Lets also include Daniel in this - as he was trying to fix this in the past. > > > > As I remember this behavior is by design. However, this is not in line with > > xm behavior which sometimes makes a problem. I have discussed this with IanJ > > and IanC (both CC-ed here) once. I have proposed some patch series to fix that > > issue but later I was not able to continue work on it. If you wish I could > > repost them once again with fixes requested by IanJ and IanC. You could also > > find latest version of patches here: > > > > http://lists.xenproject.org/archives/html/xen-devel/2013-04/msg03072.html > > > > Oh, sure. If you have something new then please do. ;-) > > AFAICT from the latest series, the real meat that fixes this problem is > in patch 2, which got no reply. Does that mean no objection was raisedIIRC patch number 1 is much more important in your case. If you can test it (and others if it is possible) then it will be nice. However, patches may require some tweaks to be applied on current unstable tree.> on that approach?In general there were only small issues left. I will fix them and repost when I will be sure that they solve your problems. Daniel