CentOS virt - Oct 2011 - centos6 and xen4 dom0 memory allocation

Hi,

This is slightly OT as xen is not officially supported in EL6. But
maybe someone else is running it on centos6 as well and could help me
out here:

I have problems allocating a static amount of memory for my xen dom0.
It seems to reduce when new domUs are created, even though I *think*
ballooning is disabled. Can someone advise me what I'm doing wrong?

I have xen-4.1.1-3 installed from
http://xenbits.xen.org/people/mayoung/EL6.xen on my centos6 server. I
am running 2.6.32-131.12.1.el6.xendom0.x86_64 kernel available from
the same place. 'xl info' reports about available memory as follows:

total_memory           : 24567
free_memory            : 3698
xen_commandline        : dom0_mem=2048M

and in xend-config.sxp I have:

(dom0-min-mem 256)
(enable-dom0-ballooning no)
(total_available_memory 0)

Now one would think that I should have 2048M memory for dom0, right? I
started from there, but when starting domUs it has been reduced to
1140M even though I think ballooning is disabled and there is free
memory available? And the weird thing is that dom0 has done some
swapping:

# free -mt
             total       used       free     shared    buffers     cached
Mem:          1140       1052         88          0          2         12
-/+ buffers/cache:       1038        102
Swap:         8194         85       8109
Total:        9335       1138       8197

Shutting down domUs and making more memory available does not reverse
the effect, dom0 does not allocate more memory.

What am I missing here or is my xen setup acting strangely here?

Best regards,
Peter

On Thu, Oct 20, 2011 at 12:37:34AM +0300, Peter Peltonen
wrote:
> Hi,
> 
> This is slightly OT as xen is not officially supported in EL6. But
> maybe someone else is running it on centos6 as well and could help me
> out here:
> 
> I have problems allocating a static amount of memory for my xen dom0.
> It seems to reduce when new domUs are created, even though I *think*
> ballooning is disabled. Can someone advise me what I'm doing wrong?
> 
> I have xen-4.1.1-3 installed from
> http://xenbits.xen.org/people/mayoung/EL6.xen on my centos6 server. I
> am running 2.6.32-131.12.1.el6.xendom0.x86_64 kernel available from
> the same place. 'xl info' reports about available memory as
follows:
> 
> total_memory           : 24567
> free_memory            : 3698
> xen_commandline        : dom0_mem=2048M
> 
> and in xend-config.sxp I have:
> 
> (dom0-min-mem 256)
> (enable-dom0-ballooning no)
> (total_available_memory 0)
> 
> Now one would think that I should have 2048M memory for dom0, right? I
> started from there, but when starting domUs it has been reduced to
> 1140M even though I think ballooning is disabled and there is free
> memory available? And the weird thing is that dom0 has done some
> swapping:
> 
Did you check the amount of memory in dom0 right after boot? 
You *might* need to add also "mem=2028M" cmdline option for dom0 linux
kernel (vmlinuz).

Also Linux kernel is *supposed* to swap all inactive pages! 

So it's normal to have swap being used. If you have *active* swap usage 
all the time (pages in and out from swap), only then it means you don't have
enough memory allocated.

> # free -mt
>              total       used       free     shared    buffers     cached
> Mem:          1140       1052         88          0          2         12
> -/+ buffers/cache:       1038        102
> Swap:         8194         85       8109
> Total:        9335       1138       8197
> 
> Shutting down domUs and making more memory available does not reverse
> the effect, dom0 does not allocate more memory.
> 
dom0 is not supposed to allocate more memory!
dom0 is a virtual machine. Free memory goes into Xen hypervisor, check with
"xl info" or "xm info".
> What am I missing here or is my xen setup acting strangely here?
> 
You're missing the fact that dom0 is a VM.

-- Pasi