stalks
2010-Apr-04 02:44 UTC
[Xen-users] dom0 memory usage >1GB at startup (no domU''s configured)
Dear list,
I have a new install of Debian Squeeze.
I have followed the hints from the ParaVirtOps wiki page to grab
Jeremy''s kernel sources from kernel.org and compiled a 2.6.31.x (and
2.6.33.x) kernel.
I have yet to install any significant software other than package
"xen-hypervisor-3.4-amd64".
The machine is a Core i7 920 with 8GB RAM.
If I boot the plain kernel (without xen-3.4.gz) I get ~65MB ram usage
shown in free -m.
If I boot with full xen, bear in mind I have no domU''s yet, I get the
following:
> free -m
total used free shared buffers cached
Mem: 7948 1412 6535 0 9 107
-/+ buffers/cache: 1295 6652
Swap: 0 0 0
> xm info
host : oracle
release : 2.6.33.2-amd64-xen
version : #1 SMP Sun Apr 4 00:21:54 BST 2010
machine : x86_64
nr_cpus : 8
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 2
cpu_mhz : 2660
hw_caps :
bfebfbff:28100800:00000000:00000340:0098e3bd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 8183
free_memory : 2174
node_to_cpu : node0:0-7
node_to_memory : node0:2174
xen_major : 3
xen_minor : 4
xen_extra :
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
cc_compiler : gcc version 4.4.3 (Debian 4.4.3-2)
cc_compile_by : waldi
cc_compile_domain : debian.org
cc_compile_date : Mon Mar 1 21:16:18 UTC 2010
xend_config_format : 4
> xm top
xentop - 03:33:21 Xen 3.4
1 domains: 1 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0
shutdown
Mem: 8379516k total, 6152576k used, 2226940k free CPUs: 8 @ 2660MHz
(free -m): Over 1GB used inside dom0 already?
(xm info): Only 2GB available for guests?
(xm top): Over 6GB used?
The only processes running outside of bare minimum are xinetd, sendmail,
mdadm and ssh.
I can stop xend and xendomains from starting, and I still have 1GB used
at startup.
The same setup on a lesser spec''d machine is working fine with ~140Mb
used at boot. I''ve spent a few hours on this and am at a loss.
Some info can be found at the following sources:
http://www.nooblet.org/other/xen-users/lspci.txt
http://www.nooblet.org/other/xen-users/lspci-vv.txt
http://www.nooblet.org/other/xen-users/cpuinfo.txt
http://www.nooblet.org/other/xen-users/grub.txt
http://www.nooblet.org/other/xen-users/dmesg.txt
http://www.nooblet.org/other/xen-users/meminfo.txt
http://www.nooblet.org/other/xen-users/ps-awwux.txt
Someone show me the light!
Regards,
Steve.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
stalks
2010-Apr-04 11:52 UTC
Re: [Xen-users] dom0 memory usage >1GB at startup (no domU''s configured)
On 04/04/2010 03:44, stalks wrote:> Dear list, > > I have a new install of Debian Squeeze. > > I have followed the hints from the ParaVirtOps wiki page to grab > Jeremy''s kernel sources from kernel.org and compiled a 2.6.31.x (and > 2.6.33.x) kernel. > > I have yet to install any significant software other than package > "xen-hypervisor-3.4-amd64". > > The machine is a Core i7 920 with 8GB RAM. > > If I boot the plain kernel (without xen-3.4.gz) I get ~65MB ram usage > shown in free -m. > > If I boot with full xen, bear in mind I have no domU''s yet, I get the > following: > > > free -m > total used free shared buffers cached > Mem: 7948 1412 6535 0 9 107 > -/+ buffers/cache: 1295 6652 > Swap: 0 0 0 > > > xm info > host : oracle > release : 2.6.33.2-amd64-xen > version : #1 SMP Sun Apr 4 00:21:54 BST 2010 > machine : x86_64 > nr_cpus : 8 > nr_nodes : 1 > cores_per_socket : 4 > threads_per_core : 2 > cpu_mhz : 2660 > hw_caps : > bfebfbff:28100800:00000000:00000340:0098e3bd:00000000:00000001:00000000 > virt_caps : hvm > total_memory : 8183 > free_memory : 2174 > node_to_cpu : node0:0-7 > node_to_memory : node0:2174 > xen_major : 3 > xen_minor : 4 > xen_extra : > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 > hvm-3.0-x86_32p hvm-3.0-x86_64 > xen_scheduler : credit > xen_pagesize : 4096 > platform_params : virt_start=0xffff800000000000 > xen_changeset : unavailable > cc_compiler : gcc version 4.4.3 (Debian 4.4.3-2) > cc_compile_by : waldi > cc_compile_domain : debian.org > cc_compile_date : Mon Mar 1 21:16:18 UTC 2010 > xend_config_format : 4 > > > xm top > xentop - 03:33:21 Xen 3.4 > 1 domains: 1 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0 > shutdown > Mem: 8379516k total, 6152576k used, 2226940k free CPUs: 8 @ > 2660MHz > > (free -m): Over 1GB used inside dom0 already? > (xm info): Only 2GB available for guests? > (xm top): Over 6GB used? > > The only processes running outside of bare minimum are xinetd, > sendmail, mdadm and ssh. > > I can stop xend and xendomains from starting, and I still have 1GB > used at startup. > > The same setup on a lesser spec''d machine is working fine with ~140Mb > used at boot. I''ve spent a few hours on this and am at a loss. > > Some info can be found at the following sources: > > http://www.nooblet.org/other/xen-users/lspci.txt > http://www.nooblet.org/other/xen-users/lspci-vv.txt > http://www.nooblet.org/other/xen-users/cpuinfo.txt > http://www.nooblet.org/other/xen-users/grub.txt > http://www.nooblet.org/other/xen-users/dmesg.txt > http://www.nooblet.org/other/xen-users/meminfo.txt > http://www.nooblet.org/other/xen-users/ps-awwux.txt > > Someone show me the light! > > Regards, > Steve. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-usersI have just tried 4.0.0 rc9 (xen-unstable.hg) and 2.6.32.10 with the same results. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
peter_booth@me.com
2010-Apr-04 13:02 UTC
Re: [Xen-users] dom0 memory usage >1GB at startup (no domU''s configured)
I think there is probably more time wasted through investigation of apparent memory issues in OSes than any other performance issue. Frequently these are non-issues. Here''s what you should do: 1. Dedicate 512MB ram and a single VCPU to dom0 2. Restart and continue happily Sent from my iPhone On Apr 4, 2010, at 7:52 AM, stalks <xen-users@stalks.nooblet.org> wrote:> On 04/04/2010 03:44, stalks wrote: >> Dear list, >> >> I have a new install of Debian Squeeze. >> >> I have followed the hints from the ParaVirtOps wiki page to grab >> Jeremy''s kernel sources from kernel.org and compiled a 2.6.31.x >> (and 2.6.33.x) kernel. >> >> I have yet to install any significant software other than package >> "xen-hypervisor-3.4-amd64". >> >> The machine is a Core i7 920 with 8GB RAM. >> >> If I boot the plain kernel (without xen-3.4.gz) I get ~65MB ram >> usage shown in free -m. >> >> If I boot with full xen, bear in mind I have no domU''s yet, I get >> the following: >> >> > free -m >> total used free shared buffers >> cached >> Mem: 7948 1412 6535 0 >> 9 107 >> -/+ buffers/cache: 1295 6652 >> Swap: 0 0 0 >> >> > xm info >> host : oracle >> release : 2.6.33.2-amd64-xen >> version : #1 SMP Sun Apr 4 00:21:54 BST 2010 >> machine : x86_64 >> nr_cpus : 8 >> nr_nodes : 1 >> cores_per_socket : 4 >> threads_per_core : 2 >> cpu_mhz : 2660 >> hw_caps : bfebfbff: >> 28100800:00000000:00000340:0098e3bd:00000000:00000001:00000000 >> virt_caps : hvm >> total_memory : 8183 >> free_memory : 2174 >> node_to_cpu : node0:0-7 >> node_to_memory : node0:2174 >> xen_major : 3 >> xen_minor : 4 >> xen_extra : >> xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0- >> x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 >> xen_scheduler : credit >> xen_pagesize : 4096 >> platform_params : virt_start=0xffff800000000000 >> xen_changeset : unavailable >> cc_compiler : gcc version 4.4.3 (Debian 4.4.3-2) >> cc_compile_by : waldi >> cc_compile_domain : debian.org >> cc_compile_date : Mon Mar 1 21:16:18 UTC 2010 >> xend_config_format : 4 >> >> > xm top >> xentop - 03:33:21 Xen 3.4 >> 1 domains: 1 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0 >> shutdown >> Mem: 8379516k total, 6152576k used, 2226940k free CPUs: 8 @ >> 2660MHz >> >> (free -m): Over 1GB used inside dom0 already? >> (xm info): Only 2GB available for guests? >> (xm top): Over 6GB used? >> >> The only processes running outside of bare minimum are xinetd, >> sendmail, mdadm and ssh. >> >> I can stop xend and xendomains from starting, and I still have 1GB >> used at startup. >> >> The same setup on a lesser spec''d machine is working fine with >> ~140Mb used at boot. I''ve spent a few hours on this and am at a loss. >> >> Some info can be found at the following sources: >> >> http://www.nooblet.org/other/xen-users/lspci.txt >> http://www.nooblet.org/other/xen-users/lspci-vv.txt >> http://www.nooblet.org/other/xen-users/cpuinfo.txt >> http://www.nooblet.org/other/xen-users/grub.txt >> http://www.nooblet.org/other/xen-users/dmesg.txt >> http://www.nooblet.org/other/xen-users/meminfo.txt >> http://www.nooblet.org/other/xen-users/ps-awwux.txt >> >> Someone show me the light! >> >> Regards, >> Steve. >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xensource.com >> http://lists.xensource.com/xen-users > > > I have just tried 4.0.0 rc9 (xen-unstable.hg) and 2.6.32.10 with the > same results. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Nick Couchman
2010-Apr-04 14:32 UTC
Re: [Xen-users] dom0 memory usage >1GB at startup (no domU''s configured)
Use "top" and sort by memory usage to see what''s taking up the memory. Also, you might consider specifying "dom0_mem" on the xen kernel boot line, as this will limit the memory that dom0 sees, which will affect how memory is allocated on the system. Don''t be fooled by the "xm info" output, or the "xm top" output, for that matter. No, this does not mean that only 2GB of RAM are available for guests. I''m assuming you''ve already started up one domU on that machine? Xen automatically balloons memory on dom0 as guests (domUs) are started - so, if you start a guest that takes up 4GB, you''ll see the dom0 memory decrease to 4GB and the rest will be allocated to the domU. Xen does *not* automatically reassign memory back to dom0 after a domU is shut down - dom0s memory remains lower and the memory is marked as "free" to Xen. When you first start up your machine, before you launch any domUs, you should see close to 0 MB listed as "free" in the "xm info" screen. This is another reason why it''s good to set dom0_mem at boot time - it''ll give you a much better idea of how much memory you actually have available using the xm commands. -Nick>>> stalks 04/03/10 8:44 PM >>>Dear list, I have a new install of Debian Squeeze. I have followed the hints from the ParaVirtOps wiki page to grab Jeremy''s kernel sources from kernel.org and compiled a 2.6.31.x (and 2.6.33.x) kernel. I have yet to install any significant software other than package "xen-hypervisor-3.4-amd64". The machine is a Core i7 920 with 8GB RAM. If I boot the plain kernel (without xen-3.4.gz) I get ~65MB ram usage shown in free -m. If I boot with full xen, bear in mind I have no domU''s yet, I get the following: > free -m total used free shared buffers cached Mem: 7948 1412 6535 0 9 107 -/+ buffers/cache: 1295 6652 Swap: 0 0 0 > xm info host : oracle release : 2.6.33.2-amd64-xen version : #1 SMP Sun Apr 4 00:21:54 BST 2010 machine : x86_64 nr_cpus : 8 nr_nodes : 1 cores_per_socket : 4 threads_per_core : 2 cpu_mhz : 2660 hw_caps : bfebfbff:28100800:00000000:00000340:0098e3bd:00000000:00000001:00000000 virt_caps : hvm total_memory : 8183 free_memory : 2174 node_to_cpu : node0:0-7 node_to_memory : node0:2174 xen_major : 3 xen_minor : 4 xen_extra : xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable cc_compiler : gcc version 4.4.3 (Debian 4.4.3-2) cc_compile_by : waldi cc_compile_domain : debian.org cc_compile_date : Mon Mar 1 21:16:18 UTC 2010 xend_config_format : 4 > xm top xentop - 03:33:21 Xen 3.4 1 domains: 1 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown Mem: 8379516k total, 6152576k used, 2226940k free CPUs: 8 @ 2660MHz (free -m): Over 1GB used inside dom0 already? (xm info): Only 2GB available for guests? (xm top): Over 6GB used? The only processes running outside of bare minimum are xinetd, sendmail, mdadm and ssh. I can stop xend and xendomains from starting, and I still have 1GB used at startup. The same setup on a lesser spec''d machine is working fine with ~140Mb used at boot. I''ve spent a few hours on this and am at a loss. Some info can be found at the following sources: http://www.nooblet.org/other/xen-users/lspci.txt http://www.nooblet.org/other/xen-users/lspci-vv.txt http://www.nooblet.org/other/xen-users/cpuinfo.txt http://www.nooblet.org/other/xen-users/grub.txt http://www.nooblet.org/other/xen-users/dmesg.txt http://www.nooblet.org/other/xen-users/meminfo.txt http://www.nooblet.org/other/xen-users/ps-awwux.txt Someone show me the light! Regards, Steve. -------- This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Markus Hochholdinger
2010-Apr-15 14:44 UTC
Re: [Xen-users] dom0 memory usage >1GB at startup (no domU''s configured)
Hello, Am 04.04.2010 um 04:44 Uhr schrieb stalks <xen-users@stalks.nooblet.org>:> I have a new install of Debian Squeeze. > I have followed the hints from the ParaVirtOps wiki page to grab > Jeremy''s kernel sources from kernel.org and compiled a 2.6.31.x (and > 2.6.33.x) kernel.you could also install the debian kernels: http://packages.debian.org/sid/linux-image-2.6.32-4-xen-686 or http://packages.debian.org/sid/linux-image-2.6.32-4-xen-amd64> I have yet to install any significant software other than package > "xen-hypervisor-3.4-amd64". > The machine is a Core i7 920 with 8GB RAM. > If I boot the plain kernel (without xen-3.4.gz) I get ~65MB ram usage > shown in free -m. > If I boot with full xen, bear in mind I have no domU''s yet, I get the > following: > > free -m > total used free shared buffers cached > Mem: 7948 1412 6535 0 9 107 > -/+ buffers/cache: 1295 6652 > Swap: 0 0 0 > > xm info > host : oracle > release : 2.6.33.2-amd64-xen > version : #1 SMP Sun Apr 4 00:21:54 BST 2010 > machine : x86_64 > nr_cpus : 8 > nr_nodes : 1 > cores_per_socket : 4 > threads_per_core : 2 > cpu_mhz : 2660 > hw_caps : > bfebfbff:28100800:00000000:00000340:0098e3bd:00000000:00000001:00000000 > virt_caps : hvm > total_memory : 8183 > free_memory : 2174 > node_to_cpu : node0:0-7 > node_to_memory : node0:2174 > xen_major : 3 > xen_minor : 4 > xen_extra : > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 > hvm-3.0-x86_32p hvm-3.0-x86_64 > xen_scheduler : credit > xen_pagesize : 4096 > platform_params : virt_start=0xffff800000000000 > xen_changeset : unavailable > cc_compiler : gcc version 4.4.3 (Debian 4.4.3-2) > cc_compile_by : waldi > cc_compile_domain : debian.org > cc_compile_date : Mon Mar 1 21:16:18 UTC 2010 > xend_config_format : 4 > > xm top > xentop - 03:33:21 Xen 3.4 > 1 domains: 1 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0 > shutdown > Mem: 8379516k total, 6152576k used, 2226940k free CPUs: 8 @ 2660MHz > (free -m): Over 1GB used inside dom0 already?It''s possible that some drivers or programs are preserving memory as percentage of the whole memory it sees.> (xm info): Only 2GB available for guests? > (xm top): Over 6GB used?Sure? Tried to start a domU with 3GB RAM? It seems this is the auto balooning of the dom0. It works as follows: Take the whole memory for dom0 and give away memory if domUs need them, e.g. when starting a new domU. I prevent this with (dom0-min-mem 0) in my /etc/xen/xend-config.sxp and giving dom0_mem=256M as parameter to the hypervisor in my grub config, this looks like this for me: title Xen 3.4-amd64 / Debian GNU/Linux, kernel 2.6.32-4-xen-686 root (hd0,0) kernel /boot/xen-3.4-amd64.gz dom0_mem=256M module /boot/vmlinuz-2.6.32-4-xen-686 root=/dev/md0 elevator=deadline ro pci=nomsi console=tty0 module /boot/initrd.img-2.6.32-4-xen-686> The only processes running outside of bare minimum are xinetd, sendmail, > mdadm and ssh. > I can stop xend and xendomains from starting, and I still have 1GB used > at startup. > The same setup on a lesser spec''d machine is working fine with ~140Mb > used at boot. I''ve spent a few hours on this and am at a loss.Try to limit the memory for your dom0 as seen above and look again.> Some info can be found at the following sources: > http://www.nooblet.org/other/xen-users/lspci.txt > http://www.nooblet.org/other/xen-users/lspci-vv.txt > http://www.nooblet.org/other/xen-users/cpuinfo.txt > http://www.nooblet.org/other/xen-users/grub.txt > http://www.nooblet.org/other/xen-users/dmesg.txt > http://www.nooblet.org/other/xen-users/meminfo.txt > http://www.nooblet.org/other/xen-users/ps-awwux.txt > Someone show me the light!-- greetings eMHa _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users