Hi.
In my opinion, mem-max is not process correctly.
And information of xm top (xentop) and of XendDomainInfo is not
correspond correctly.
Actually, I seem that restriction of memory in xen handle pfn and pages
variables (ex. max_pfn, max_pages), but XendDomainInfo handles its local
variables with memory information.
xm list shows XenDomainInfo information and xmtop shows actual memory
informations.
xm mem-set and mem-max process renew of XenDomainInfo, and therefore
handle actual memory information.
In the case that memory modification is not possible, latter process
checks and restricts.
Therefore, information between xm list and xm top becomes different
information.
(Mem information in xm list is corresponded to requested target.)
When domain is initialized (started up), memory attribute secure needed
memory and memmax attribute is setup restriction and extra mem attribute
arranges balloon memory.
However, first restriction decide memory and memmax attributes, and
balloon secure by memory attributes, so mem-max reflects correctly.
It is nearly to memory attributes.
After domain waked up, new restriction changes by xm mem-max command.
So, memory information in xm list is not correctly, but show your
request.
Of course, it is necessary to fix this non-correspondence.
Satoshi Uchida
-----Original Message-----
From: Michael Heyse [mailto:mhk@designassembly.de]
Sent: Wednesday, January 11, 2006 9:10 PM
To: xen-users@lists.xensource.com
Subject: [Xen-users] Memory management questions
Hi,
please help me understand xen''s memory management and the balloon
driver
a little bit better.
setup: xen-3.0-testing-20051206 (installed with gentoo''s xen-3.0.0-r2
ebuild), Kernel 2.6.12.6-xen total RAM: 1G (no swap for this test) dom0
parameters:
- dom0_mem=768M (grub config)
- dom0-min-mem 196 (xend-config.sxp)
dom1 parameters: memory=128, extra="mem=768M"
dom2 parameters: memory=256, extra="mem=768M"
If I understood correctly how the ballon driver works, this should boot
dom0 with 768M available initially and automatically ballooning it down
to not less than 196M if memory is needed by other domains. dom1 and
dom2 should start with 128M and 256M respectively, with the option of
ballooning both to a max of 768M with the "xm mem-set" command run
from
dom0. After all domains are booted, `xm list` gives me
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 586 1 r----- 118.0
dom1 1 128 1 -b---- 32.6
dom2 2 256 1 -b---- 30.3
and `cat /proc/meminfo` shows a MemTotal of 775108 kB in dom1 and dom2
with reasonable MemFree values of approx. 80M and 220M respectively,
which looks all right to me. Did I get it until here?
Now suppose I want to compile glibc in dom1 which needs lots of memory.
So I do a
xm mem-set 1 768M
`xm list` now shows
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 586 1 r----- 120.3
dom1 1 768 1 -b---- 32.7
dom2 2 256 1 -b---- 30.3
which I don''t understand, because now the sum of memory used exceeds
the
1G system total. Shouldn''t xm complain here? Also, `xm top` still shows
a MAXMEM of 131072 for dom1, and `cat /proc/meminfo` in dom1 doesn''t
show a change at all. And there definitely isn''t more memory in
domavailable 1, because if I mount a tmpfs on ~/test and do a `dd
if=/dev/zero of=~/test/test` the oom-killer kicks in as soon as the file
size reaches approx. 100M. The same thing happens if I''m less greedy
and
do a `xm mem-set 1 256M`, which should work with the available memory.
So it looks like my ballooning attempt failed.
So what did I do wrong?
Thanks,
Michael
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Satoshi Uchida wrote:> When domain is initialized (started up), memory attribute secure needed > memory and memmax attribute is setup restriction and extra mem attribute > arranges balloon memory. > However, first restriction decide memory and memmax attributes, and > balloon secure by memory attributes, so mem-max reflects correctly. > It is nearly to memory attributes. > After domain waked up, new restriction changes by xm mem-max command. > > So, memory information in xm list is not correctly, but show your > request.OK, thanks. If I understood correctly, I have to do a `xm mem-max` before setting the amount of memory with `xm mem-set''. But now something really strange happens. I booted dom2 as described before (with memory=256 and extra="mem=768M"). Right after it''s up, `cat /proc/meminfo` in dom2 shows a MemTotal of 775108k as expected. Now I do a `xm mem-max 2 512M`, and `xm top` correctly shows this, `xm list` still shows the 256M. So now I do a `xm mem-set 2 512M`. Now `xm list` shows that the domain has 512M available. But `xm top` shows, that dom1 is using only 262116k with a MEMMAX of 524288k. And the strangest thing is: `cat /proc/meminfo` in dom2 now shows a MemTotal of only 262136k, so the virtual machine believes that its available memory has shrunk to 256M from 768M. I thought that the balloon driver only allocates/frees memory but doesn''t actually do memory hot(un)plugging? Could you please explain this to me? Thanks, Michael _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi.> I have to do a `xm mem-max` before setting the amount of memory with`xm mem-set''. Yes. mem-max is braking for increasing memory, but is not decrease for allocated memory. If amount of memory when mem-max is smaller than allocated memory, I seem to that system outputs error messages by other causes (ex.netfront).> Right after it''s up, `cat /proc/meminfo` in dom2 shows a MemTotal of775108k as expected Really? In my environments, MemTotal is shown correctly. (But, LowMemoryTotal is not correctly.)> I thought that the balloon driver only allocates/frees memory butdoesn''t actually do memory hot(un)plugging? I seem it too. In my understanding, First, balloon driver create list of available memory address from memory or mem attributes in extra. Then, by communication HyperVisor, available memory within list allocated into inactive list in OS or securing memory from allocated one is freed. Therefore, it is delicate that this mechanism called hot plug. In my guess, 1. If extra mem attributes in each domains is not all RAM size, by repeating to use balloon function, actual memory has free aria, but one domain can not use. (by hard limit, allocation of memory is overborne.) 2. If you add physical memory modules, you must reboot xen environments. In this The joy would be cut. Thanks, Satoshi UCHIDA -----Original Message----- From: Michael Heyse [mailto:mhk@designassembly.de] Sent: Saturday, January 14, 2006 3:23 AM To: Satoshi Uchida Cc: xen-users@lists.xensource.com Subject: Re: FW: [Xen-users] Memory management questions Satoshi Uchida wrote:> When domain is initialized (started up), memory attribute secure > needed memory and memmax attribute is setup restriction and extra mem > attribute arranges balloon memory. > However, first restriction decide memory and memmax attributes, and > balloon secure by memory attributes, so mem-max reflects correctly. > It is nearly to memory attributes. > After domain waked up, new restriction changes by xm mem-max command. > > So, memory information in xm list is not correctly, but show your > request.OK, thanks. If I understood correctly, I have to do a `xm mem-max` before setting the amount of memory with `xm mem-set''. But now something really strange happens. I booted dom2 as described before (with memory=256 and extra="mem=768M"). Right after it''s up, `cat /proc/meminfo` in dom2 shows a MemTotal of 775108k as expected. Now I do a `xm mem-max 2 512M`, and `xm top` correctly shows this, `xm list` still shows the 256M. So now I do a `xm mem-set 2 512M`. Now `xm list` shows that the domain has 512M available. But `xm top` shows, that dom1 is using only 262116k with a MEMMAX of 524288k. And the strangest thing is: `cat /proc/meminfo` in dom2 now shows a MemTotal of only 262136k, so the virtual machine believes that its available memory has shrunk to 256M from 768M. I thought that the balloon driver only allocates/frees memory but doesn''t actually do memory hot(un)plugging? Could you please explain this to me? Thanks, Michael _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users