Hi, i''m having some trouble with assigning memory to my dom0. I''ve added "dom0_mem=8192M" to the xen command line yet "free -m" reports only 5686MB total. Running Xen 4.1.2 with kernel 3.2.0-1-amd64, debian testing. Attached are the outputs of dmesg, xm dmesg, xm info and xm list. Regards, Henrik Olsson _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org lists.xen.org/xen-devel
On Wed, Feb 22, 2012 at 23:19, Henrik Olsson <henrik@fixme.se> wrote:> Hi, i''m having some trouble with assigning memory to my dom0. > > I''ve added "dom0_mem=8192M" to the xen command line yet "free -m" > reports only 5686MB total. > > Running Xen 4.1.2 with kernel 3.2.0-1-amd64, debian testing. > > Attached are the outputs of dmesg, xm dmesg, xm info and xm list. > > Regards, > Henrik OlssonForgot to mention, issuing xm mem-set changes what xm list says but not free. root@muaddib:~# xm mem-set Domain-0 8192M root@muaddib:~# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 8192 2 r----- 598.1 root@muaddib:~# free -m total used free shared buffers cached Mem: 5686 2133 3552 0 54 884
On 22/02/12 22:19, Henrik Olsson wrote:> Hi, i''m having some trouble with assigning memory to my dom0. > > I''ve added "dom0_mem=8192M" to the xen command line yet "free -m" > reports only 5686MB total.[ 0.000000] Freeing 6f000-100000 pfn range: 593920 pages freed [ 0.000000] 1-1 mapping on 6f000->100000 [ 0.000000] Released 595312 pages of unused memory This accounts for most of your "missing" memory. To get it back you need to adjust the balloon driver''s target to the amount of memory you want. David
On Thu, Feb 23, 2012 at 19:55, David Vrabel <dvrabel@cantab.net> wrote:> On 22/02/12 22:19, Henrik Olsson wrote: >> Hi, i''m having some trouble with assigning memory to my dom0. >> >> I''ve added "dom0_mem=8192M" to the xen command line yet "free -m" >> reports only 5686MB total. > > [ 0.000000] Freeing 6f000-100000 pfn range: 593920 pages freed > [ 0.000000] 1-1 mapping on 6f000->100000 > [ 0.000000] Released 595312 pages of unused memory > > This accounts for most of your "missing" memory. To get it back you > need to adjust the balloon driver''s target to the amount of memory you want. > > DavidHi, i''m not sure i understand.. I''ve configured /etc/xen/xend-config.sxp with: (dom0-min-mem 8192) (enable-dom0-ballooning no) Shouldn''t this disable ballooning? Or do i need to pass some parameter somewhere? Regards, Henrik
Roderick Colenbrander
2012-Feb-27 19:04 UTC
Re: dom0 not seeing all of the assigned memory
On Mon, Feb 27, 2012 at 5:34 PM, Henrik Olsson <henrik@fixme.se> wrote:> On Thu, Feb 23, 2012 at 19:55, David Vrabel <dvrabel@cantab.net> wrote: >> On 22/02/12 22:19, Henrik Olsson wrote: >>> Hi, i''m having some trouble with assigning memory to my dom0. >>> >>> I''ve added "dom0_mem=8192M" to the xen command line yet "free -m" >>> reports only 5686MB total. >> >> [ 0.000000] Freeing 6f000-100000 pfn range: 593920 pages freed >> [ 0.000000] 1-1 mapping on 6f000->100000 >> [ 0.000000] Released 595312 pages of unused memory >> >> This accounts for most of your "missing" memory. To get it back you >> need to adjust the balloon driver''s target to the amount of memory you want. >> >> David > > Hi, i''m not sure i understand.. > > I''ve configured /etc/xen/xend-config.sxp with: > (dom0-min-mem 8192) > (enable-dom0-ballooning no) > > Shouldn''t this disable ballooning? Or do i need to pass some parameter > somewhere? > > Regards, > Henrik > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > lists.xen.org/xen-develMake sure you have added the following patch to your Xen if you are using a recent >=3.1 kernel. Unfortunately it isn''t in Xen 4.1.x, but I guess it should be added to there: xenbits.xen.org/hg/staging/xen-unstable.hg/rev/c56dd5eb0fa2 See if it helps. Roderick
Am 27.02.2012 20:04, schrieb Roderick Colenbrander:> On Mon, Feb 27, 2012 at 5:34 PM, Henrik Olsson <henrik@fixme.se> wrote: >> On Thu, Feb 23, 2012 at 19:55, David Vrabel <dvrabel@cantab.net> wrote: >>> On 22/02/12 22:19, Henrik Olsson wrote: >>>> Hi, i''m having some trouble with assigning memory to my dom0. >>>> >>>> I''ve added "dom0_mem=8192M" to the xen command line yet "free -m" >>>> reports only 5686MB total. >>> >>> [ 0.000000] Freeing 6f000-100000 pfn range: 593920 pages freed >>> [ 0.000000] 1-1 mapping on 6f000->100000 >>> [ 0.000000] Released 595312 pages of unused memory >>> >>> This accounts for most of your "missing" memory. To get it back you >>> need to adjust the balloon driver''s target to the amount of memory you want. >>> >>> David >> >> Hi, i''m not sure i understand.. >> >> I''ve configured /etc/xen/xend-config.sxp with: >> (dom0-min-mem 8192) >> (enable-dom0-ballooning no) >> >> Shouldn''t this disable ballooning? Or do i need to pass some parameter >> somewhere?I have the same problem. I believe this also happens even if the ballooning driver is disabled in dom0 kernel.>> Regards, >> Henrik > > Make sure you have added the following patch to your Xen if you are > using a recent >=3.1 kernel. Unfortunately it isn''t in Xen 4.1.x, but > I guess it should be added to there: > xenbits.xen.org/hg/staging/xen-unstable.hg/rev/c56dd5eb0fa2 > > See if it helps.I recompiled the hypervisor (this was a patch for the hypervisor, right?) and dom0 only has around 413MB inspite of dom0_mem=512M hypervisor parameter. free shows 413MB straight after booting into dom0 (no xend, xenstored or anything xen related stuff has started yet). Regards, Sven
Konrad Rzeszutek Wilk
2012-Mar-15 18:33 UTC
Re: dom0 not seeing all of the assigned memory
On Sun, Mar 11, 2012 at 11:20:02PM +0100, Sven Köhler wrote:> Am 27.02.2012 20:04, schrieb Roderick Colenbrander: > > On Mon, Feb 27, 2012 at 5:34 PM, Henrik Olsson <henrik@fixme.se> wrote: > >> On Thu, Feb 23, 2012 at 19:55, David Vrabel <dvrabel@cantab.net> wrote: > >>> On 22/02/12 22:19, Henrik Olsson wrote: > >>>> Hi, i''m having some trouble with assigning memory to my dom0. > >>>> > >>>> I''ve added "dom0_mem=8192M" to the xen command line yet "free -m" > >>>> reports only 5686MB total. > >>> > >>> [ 0.000000] Freeing 6f000-100000 pfn range: 593920 pages freed > >>> [ 0.000000] 1-1 mapping on 6f000->100000 > >>> [ 0.000000] Released 595312 pages of unused memory > >>> > >>> This accounts for most of your "missing" memory. To get it back you > >>> need to adjust the balloon driver''s target to the amount of memory you want. > >>> > >>> David > >> > >> Hi, i''m not sure i understand.. > >> > >> I''ve configured /etc/xen/xend-config.sxp with: > >> (dom0-min-mem 8192) > >> (enable-dom0-ballooning no) > >> > >> Shouldn''t this disable ballooning? Or do i need to pass some parameter > >> somewhere? > > I have the same problem. I believe this also happens even if the > ballooning driver is disabled in dom0 kernel. > > >> Regards, > >> Henrik > > > > Make sure you have added the following patch to your Xen if you are > > using a recent >=3.1 kernel. Unfortunately it isn''t in Xen 4.1.x, but > > I guess it should be added to there: > > xenbits.xen.org/hg/staging/xen-unstable.hg/rev/c56dd5eb0fa2 > > > > See if it helps. > > I recompiled the hypervisor (this was a patch for the hypervisor, > right?) and dom0 only has around 413MB inspite of dom0_mem=512M > hypervisor parameter. > > free shows 413MB straight after booting into dom0 (no xend, xenstored or > anything xen related stuff has started yet).<sigh> So it sounds like that the kernel should balloon up the released amount of memory. David, did we discuss this at some point and agreed that was the proper way?
On 15/03/12 18:33, Konrad Rzeszutek Wilk wrote:> On Sun, Mar 11, 2012 at 11:20:02PM +0100, Sven Köhler wrote: >> Am 27.02.2012 20:04, schrieb Roderick Colenbrander: >>> On Mon, Feb 27, 2012 at 5:34 PM, Henrik Olsson <henrik@fixme.se> wrote: >>>> On Thu, Feb 23, 2012 at 19:55, David Vrabel <dvrabel@cantab.net> wrote: >>>>> On 22/02/12 22:19, Henrik Olsson wrote: >>>>>> Hi, i''m having some trouble with assigning memory to my dom0. >>>>>> >>>>>> I''ve added "dom0_mem=8192M" to the xen command line yet "free -m" >>>>>> reports only 5686MB total. >>>>> >>>>> [ 0.000000] Freeing 6f000-100000 pfn range: 593920 pages freed >>>>> [ 0.000000] 1-1 mapping on 6f000->100000 >>>>> [ 0.000000] Released 595312 pages of unused memory >>>>> >>>>> This accounts for most of your "missing" memory. To get it back you >>>>> need to adjust the balloon driver''s target to the amount of memory you want. >>>>> >>>>> David >>>> >>>> Hi, i''m not sure i understand.. >>>> >>>> I''ve configured /etc/xen/xend-config.sxp with: >>>> (dom0-min-mem 8192) >>>> (enable-dom0-ballooning no) >>>> >>>> Shouldn''t this disable ballooning? Or do i need to pass some parameter >>>> somewhere? >> >> I have the same problem. I believe this also happens even if the >> ballooning driver is disabled in dom0 kernel.Well, yes. The balloon driver needs to be enabled and a new target set to get back the memory released during boot.>>> Make sure you have added the following patch to your Xen if you are >>> using a recent >=3.1 kernel. Unfortunately it isn''t in Xen 4.1.x, but >>> I guess it should be added to there: >>> xenbits.xen.org/hg/staging/xen-unstable.hg/rev/c56dd5eb0fa2 >>> >>> See if it helps. >> >> I recompiled the hypervisor (this was a patch for the hypervisor, >> right?) and dom0 only has around 413MB inspite of dom0_mem=512M >> hypervisor parameter.You need dom0_mem=max:512M>> free shows 413MB straight after booting into dom0 (no xend, xenstored or >> anything xen related stuff has started yet).This case isn''t because of the released memory (since 512M is well below the MMIO holes that cause lots of released memory) but because page tables are reserved to the amount of physical ram in the system.> <sigh> So it sounds like that the kernel should balloon up the released > amount of memory. David, did we discuss this at some point and agreed that > was the proper way?I don''t remember discussing it but it''s an interesting idea. Let me think about it. It will need to work without the balloon driver (since it might not exist). David
Am 15.03.2012 19:54, schrieb David Vrabel:> On 15/03/12 18:33, Konrad Rzeszutek Wilk wrote: >> On Sun, Mar 11, 2012 at 11:20:02PM +0100, Sven Köhler wrote: >>> >>> I have the same problem. I believe this also happens even if the >>> ballooning driver is disabled in dom0 kernel. > > Well, yes. The balloon driver needs to be enabled and a new target set > to get back the memory released during boot.The dom0 releases pages and passes them back to the hypervisor? But why does it do so in the first place?>>>> Make sure you have added the following patch to your Xen if you are >>>> using a recent >=3.1 kernel. Unfortunately it isn''t in Xen 4.1.x, but >>>> I guess it should be added to there: >>>> xenbits.xen.org/hg/staging/xen-unstable.hg/rev/c56dd5eb0fa2 >>>> >>>> See if it helps. >>> >>> I recompiled the hypervisor (this was a patch for the hypervisor, >>> right?) and dom0 only has around 413MB inspite of dom0_mem=512M >>> hypervisor parameter. > > You need dom0_mem=max:512MI can''t find documentation on what that does. But if I had to make an educated guess, I would assume that dom0_mem=max:512M sets an upper bound on the memory that dom0 may consume. It sounds like, the hypervisor may take memory away from dom0, so that dom0 drops below the specified 512MB. Here''s what I want to do (and what I though that dom0_mem=512M would actually do): I want dom0 to have _exactly_ 512MB of RAM. Neither less nor more than that. So what does dom0_mem=512M do? Does it set a lower bound? Recommendation for dom0_mem=512M are all over the internet (even wiki.xen.org) and it is often explained as statically assinging memory to dom0 (which IMHO implies that 512M is both the upper and lower bound).>>> free shows 413MB straight after booting into dom0 (no xend, xenstored or >>> anything xen related stuff has started yet). > > This case isn''t because of the released memory (since 512M is well below > the MMIO holes that cause lots of released memory) but because page > tables are reserved to the amount of physical ram in the system.OK, I understand. What I don''t understand is why the kernel thinks why it needs page tables for more than 512M.>> <sigh> So it sounds like that the kernel should balloon up the released >> amount of memory. David, did we discuss this at some point and agreed that >> was the proper way? > > I don''t remember discussing it but it''s an interesting idea. Let me > think about it. It will need to work without the balloon driver (since > it might not exist).I''m not sure what you are discussing. Can you explain a little more? I remember the good old times, kernel 2.6.18 and 2.6.38 with xen patches, where free shows a total of 524620 kilobytes and xl list shows 512 megabytes, with dom0_mem=512M. With 3.3, free shows a toal of 413920 and xl list shows 511 megabytes for dom0. Do you realize, that the numbers are very different? 524620 is very close to 512 * 1024 while 413920 is very very different from 511 * 1024. Regards, Sven
On Thu, 2012-03-15 at 20:07 +0000, Sven Köhler wrote:> Am 15.03.2012 19:54, schrieb David Vrabel:> > You need dom0_mem=max:512M > > I can''t find documentation on what that does.This option is documented in the source tree, and the in tree documentation is published at: xenbits.xen.org/docs/unstable/misc/xen-command-line.html Ian.
Am 15.03.2012 21:13, schrieb Ian Campbell:> On Thu, 2012-03-15 at 20:07 +0000, Sven Köhler wrote: >> Am 15.03.2012 19:54, schrieb David Vrabel: > >>> You need dom0_mem=max:512M >> >> I can''t find documentation on what that does. > > This option is documented in the source tree, and the in tree > documentation is published at: > xenbits.xen.org/docs/unstable/misc/xen-command-line.htmlThanks for the link. That is very useful! Is there any path that leads from xen.org/support/documentation.html to the page above in case I forget the URL? However: according to that documentation dom0_mem=512M implies dom0_mem=max:512M. Or am I wrong?
On Thu, 2012-03-15 at 20:29 +0000, Sven Köhler wrote:> Am 15.03.2012 21:13, schrieb Ian Campbell: > > On Thu, 2012-03-15 at 20:07 +0000, Sven Köhler wrote: > >> Am 15.03.2012 19:54, schrieb David Vrabel: > > > >>> You need dom0_mem=max:512M > >> > >> I can''t find documentation on what that does. > > > > This option is documented in the source tree, and the in tree > > documentation is published at: > > xenbits.xen.org/docs/unstable/misc/xen-command-line.html > > Thanks for the link. That is very useful! Is there any path that leads > from xen.org/support/documentation.html to the page above in > case I forget the URL?Click "Generated Xen 4.x Documentation" then "complete set of generated documents here".> However: according to that documentation dom0_mem=512M implies > dom0_mem=max:512M. Or am I wrong?Not if the machine only has 256M of RAM? Ian.
Am 15.03.2012 22:55, schrieb Ian Campbell:> On Thu, 2012-03-15 at 20:29 +0000, Sven Köhler wrote: >> However: according to that documentation dom0_mem=512M implies >> dom0_mem=max:512M. Or am I wrong? > > Not if the machine only has 256M of RAM?My interpretation is that dom0_mem=512M also implies dom0_mem=min:512M. Not sure how the hypervisor deals with that, if the machine only has 256M of RAM.
On 15/03/12 20:07, Sven Köhler wrote:> Am 15.03.2012 19:54, schrieb David Vrabel: >> On 15/03/12 18:33, Konrad Rzeszutek Wilk wrote: >>> On Sun, Mar 11, 2012 at 11:20:02PM +0100, Sven Köhler wrote: >>>> >>>> I have the same problem. I believe this also happens even if the >>>> ballooning driver is disabled in dom0 kernel. >> >> Well, yes. The balloon driver needs to be enabled and a new target set >> to get back the memory released during boot. > > The dom0 releases pages and passes them back to the hypervisor? > But why does it do so in the first place?Because the pages are not accessible. They overlap with BIOS or device memory. Later kernels are better at releasing more pages than earlier ones and so it may appear that the domain ends up with less pages but it really has the same number of usable pages.>>>>> Make sure you have added the following patch to your Xen if you are >>>>> using a recent >=3.1 kernel. Unfortunately it isn''t in Xen 4.1.x, but >>>>> I guess it should be added to there: >>>>> xenbits.xen.org/hg/staging/xen-unstable.hg/rev/c56dd5eb0fa2 >>>>> >>>>> See if it helps. >>>> >>>> I recompiled the hypervisor (this was a patch for the hypervisor, >>>> right?) and dom0 only has around 413MB inspite of dom0_mem=512M >>>> hypervisor parameter. >> >> You need dom0_mem=max:512M > > I can''t find documentation on what that does. But if I had to make an > educated guess, I would assume that dom0_mem=max:512M sets an upper > bound on the memory that dom0 may consume. It sounds like, the > hypervisor may take memory away from dom0, so that dom0 drops below the > specified 512MB.The documentation isn''t great. dom0_mem sets two memory related limits: a) the initial number of pages; and b) the maximum possible number of pages (maximum reservation). Without the ''max:'' option, the maximum reservation is unlimited. Since 3.0.5, Linux has used the minimum of the maximum reservation and the total amount of physical RAM to size its page tables etc. David
Am 19.03.2012 12:09, schrieb David Vrabel:> On 15/03/12 20:07, Sven Köhler wrote: >> Am 15.03.2012 19:54, schrieb David Vrabel: >>> On 15/03/12 18:33, Konrad Rzeszutek Wilk wrote: >>>> On Sun, Mar 11, 2012 at 11:20:02PM +0100, Sven Köhler wrote: >>>>> >>>>> I have the same problem. I believe this also happens even if the >>>>> ballooning driver is disabled in dom0 kernel. >>> >>> Well, yes. The balloon driver needs to be enabled and a new target set >>> to get back the memory released during boot. >> >> The dom0 releases pages and passes them back to the hypervisor? >> But why does it do so in the first place? > > Because the pages are not accessible. They overlap with BIOS or device > memory. > > Later kernels are better at releasing more pages than earlier ones and > so it may appear that the domain ends up with less pages but it really > has the same number of usable pages.This all makes sense. But I don''t have the feeling, this is a proper explanation of what I''m seeing. "xl list" shows that dom0 is still at 511M. If the pages were really released back to the hypervisor, then "xl list" should show an amount of memory somewhere near the the value that "free" shows. But this isn''t the case. Was it wrong when I said that the pages are passed back to the hypervisor?>>>>>> Make sure you have added the following patch to your Xen if you are >>>>>> using a recent >=3.1 kernel. Unfortunately it isn''t in Xen 4.1.x, but >>>>>> I guess it should be added to there: >>>>>> xenbits.xen.org/hg/staging/xen-unstable.hg/rev/c56dd5eb0fa2 >>>>>> >>>>>> See if it helps. >>>>> >>>>> I recompiled the hypervisor (this was a patch for the hypervisor, >>>>> right?) and dom0 only has around 413MB inspite of dom0_mem=512M >>>>> hypervisor parameter. >>> >>> You need dom0_mem=max:512M >> >> I can''t find documentation on what that does. But if I had to make an >> educated guess, I would assume that dom0_mem=max:512M sets an upper >> bound on the memory that dom0 may consume. It sounds like, the >> hypervisor may take memory away from dom0, so that dom0 drops below the >> specified 512MB. > > The documentation isn''t great. dom0_mem sets two memory related limits: > a) the initial number of pages; and b) the maximum possible number of > pages (maximum reservation).As far as I can see, there are three variables here: - the initial amount of pages that dom0 gets - the minimal amount of pages that dom0 should have - the maximal amount of pages that dom0 can have Please please fix the documentation! Rumors about what dom0_mem=X does are spreading. And most pages I found said that it would statically assign memory to dom0. What you''re saying sounds like dom0 is still allowed to grow if one does only use dom0_mem=X but not dom0_mem=max:X - which is more dynamic than static.> Without the ''max:'' option, the maximum reservation is unlimited. > > Since 3.0.5, Linux has used the minimum of the maximum reservation and > the total amount of physical RAM to size its page tables etc.I tried dom0_mem=max:512M as far as I remember. And it didn''t improve the situation (kernel 3.2.10). Also, when dom0 is booting without xen, the amount of memory shown by free was not nearly as far away from the amount of physical ram as when dom0 is booting with xen and dom0_mem=... Something still smells fishy to me! Anyway: I hope some of the developers have reproduced the problem and are about to fix it. Regards, Sven _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org lists.xen.org/xen-devel
Also with Wheezy, upstream kernel (amd64 debian package) and xen-unstable there is the same problem. -- View this message in context: xen.1045712.n5.nabble.com/dom0-not-seeing-all-of-the-assigned-memory-tp5506459p5576960.html Sent from the Xen - Dev mailing list archive at Nabble.com.
I''m having this problem in a vanillla debian squeeze install, too: ski@xen:~$ dpkg -l | grep -e linux-image -e hypervisor | grep xen ii linux-image-2.6.32-5-xen-amd64 2.6.32-41squeeze2 Linux 2.6.32 for 64-bit PCs, Xen dom0 support ii linux-image-xen-amd64 2.6.32+29 Linux for 64-bit PCs (meta-package), Xen dom0 support ii xen-hypervisor-4.0-amd64 4.0.1-4 The Xen Hypervisor on AMD64 ski@xen:~$ free -m total used free shared buffers cached Mem: 4373 2478 1895 0 16 79 -/+ buffers/cache: 2382 1991 Swap: 3814 0 3814 ski@xen:~$ sudo xm info host : xen.allafrica.com release : 2.6.32-5-xen-amd64 version : #1 SMP Thu Mar 22 21:14:26 UTC 2012 machine : x86_64 nr_cpus : 8 nr_nodes : 1 cores_per_socket : 4 threads_per_core : 1 cpu_mhz : 2493 hw_caps : bfebfbff:20100800:00000000:00000940:000ce3bd:00000000:00000001:00000000 virt_caps : hvm total_memory : 32762 free_memory : 29280 node_to_cpu : node0:0-7 node_to_memory : node0:29280 node_to_dma32_mem : node0:1974 max_node_id : 0 xen_major : 4 xen_minor : 0 xen_extra : .1 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable xen_commandline : placeholder com1=57600,8n1,0x2f8,3 console=com1,vga loglvl=all guest_loglvl=all dom0_mem=max:6144M cc_compiler : gcc version 4.4.5 (Debian 4.4.5-8) cc_compile_by : waldi cc_compile_domain : debian.org cc_compile_date : Thu Jun 9 18:38:03 UTC 2011 xend_config_format : 4 Cheers. -- View this message in context: xen.1045712.n5.nabble.com/dom0-not-seeing-all-of-the-assigned-memory-tp5506459p5645610.html Sent from the Xen - Dev mailing list archive at Nabble.com.
Also, if I xm mem-set Domain-0 6144, xm list is updated to the new value, but xm top continues to show the old (even after restarting it). -- View this message in context: xen.1045712.n5.nabble.com/dom0-not-seeing-all-of-the-assigned-memory-tp5506459p5645620.html Sent from the Xen - Dev mailing list archive at Nabble.com.
> From: Brian Szymanski [mailto:skibrianski@gmail.com] > Subject: Re: [Xen-devel] dom0 not seeing all of the assigned memory > > Also, if I xm mem-set Domain-0 6144, xm list is updated to the new value, but > xm top continues to show the old (even after restarting it).This is a very old annoying bug in xm. It is especially noticeable when a tmem-savvy guest (with selfballooning) is enabled. With xm now deprecated, it may never get fixed. I''m told it is fixed with "xl". If not, please report it. Thanks, Dan