Dennis Storm - Drecomm BV
2011-Sep-01 08:20 UTC
[Xen-users] Physical memory not fully available to DomU''s
Hi,
I''m having a weird problem with one of my Xen machines. I have a 96Gb
ram server, running on Debian 6 with Xen kernel. However, I can only
assign memory out of a pool of 24GB, the rest is unavailable for
assignment. When trying to run a new DomU i''m getting the error that
there isn''t enough free ram. However, when I do the math myself I can
see that there''s 22GB of ram in use, which leave 74GB available (minus
Dom0 minimum limit, which is set at 8GB).
I''ve tried disabling ballooning, i als saw posts of ram missing due to
the balloondriver, but nothing in the likes of this volume of missing ram.
Any help would be very much appreciated. Seems a shame to let 74GB go to
waste. If you need any more info, please let me know!
XM info output:
--------------
host : <blotted out>
release : 2.6.32-5-xen-amd64
version : #1 SMP Tue Jun 14 12:46:30 UTC 2011
machine : x86_64
nr_cpus : 16
nr_nodes : 2
cores_per_socket : 4
threads_per_core : 2
cpu_mhz : 2400
hw_caps :
bfebfbff:2c100800:00000000:00001f40:009ee3fd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 98295
free_memory : 19
node_to_cpu : node0:0-3,8-11
node1:4-7,12-15
node_to_memory : node0:19
node1:0
node_to_dma32_mem : node0:0
node1:0
max_node_id : 1
xen_major : 4
xen_minor : 0
xen_extra : .1
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline : placeholder
cc_compiler : gcc version 4.4.5 (Debian 4.4.5-10)
cc_compile_by : waldi
cc_compile_domain : debian.org
cc_compile_date : Wed Jan 12 14:04:06 UTC 2011
xend_config_format : 4
--------------
XM List output:
-----------
Name ID Mem VCPUs State
Time(s)
Domain-0 0 11257 16 r-----
24477.0
<hostname blotted out> 25 256 8 -b----
4233.8
<hostname blotted out> 11 1024 8 -b----
1725.5
<hostname blotted out> 2 4192 8 r-----
66563.5
<hostname blotted out> 3 512 16 -b----
15330.1
<hostname blotted out> 17 512 16 -b----
5249.6
<hostname blotted out> 18 512 16 -b----
12292.2
<hostname blotted out> 19 512 1 -b----
6620.5
<hostname blotted out> 8 512 16 -b----
1554.6
<hostname blotted out> 9 1024 4 -b----
1837.1
<hostname blotted out> 12 1024 4 -b----
5932.5
<hostname blotted out> 13 1024 16 -b----
35491.3
<hostname blotted out> 22 1024 8 -b----
5505.7
<hostname blotted out> 20 512 1 -b----
15667.6
<hostname blotted out> 23 2816 8 -b----
6621.9
<hostname blotted out> 26 512 1 -b----
11633.6
<hostname blotted out> 28 512 16 -b----
906.9
<hostname blotted out> 15 512 1 -b----
17502.6
<hostname blotted out> 27 256 1 -b----
3924.0
<hostname blotted out> 7 512 1 -b----
12216.5
<hostname blotted out> 6 512 1 -b----
9732.1
<hostname blotted out> 29 2048 16 -b----
1738.4
<hostname blotted out> 4 256 8 -b----
4346.0
<hostname blotted out> 30 2048 8 -b----
4066.8
Total ram used for all DomU''s: 22624MB
----------------------
Config for DomU (generic for all running DomU''s, except ram and core
assignments)
----------------------
#
# Kernel + memory size
#
kernel = ''/boot/vmlinuz-2.6.32-5-xen-amd64''
ramdisk = ''/boot/initrd.img-2.6.32-5-xen-amd64''
memory = ''2048''
vcpus = ''8''
extra = ''xencons=tty''
#
# Disk device(s).
#
root = ''/dev/xvda1 ro''
disk = [ ''file:/storage/target/disk.img,xvda,w'',
''file:/storage/target/swap.img,xvdb,w'' ]
#
# Hostname
#
name = ''<hostname blotted out>''
#
# Networking
#
vif = [ ''ip=<obscured>'' ]
#
# Behaviour
#
on_poweroff = ''destroy''
on_reboot = ''restart''
on_crash = ''restart''
--------
xend-config.sxp
--------
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 196)
(enable-dom0-ballooning yes)
(total_available_memory 0)
(dom0-cpus 0)
----------
Kind regards,
Dennis Storm - Drecomm BV
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Dennis Storm
2011-Sep-02 19:44 UTC
[Xen-users] Physical memory not fully available to DomU''s
Hi,
I''m having a weird problem with one of my Xen machines. I have a 96Gb
ram server, running on Debian 6 with Xen kernel. However, I can only
assign memory out of a pool of 24GB, the rest is unavailable for
assignment. When trying to run a new DomU i''m getting the error that
there isn''t enough free ram. However, when I do the math myself I can
see that there''s 22GB of ram in use, which leave 74GB available (minus
Dom0 minimum limit, which is set at 8GB).
I''ve tried disabling ballooning, i als saw posts of ram missing due to
the balloondriver, but nothing in the likes of this volume of missing ram.
Any help would be very much appreciated. Seems a shame to let 74GB go to
waste. If you need any more info, please let me know!
XM info output:
--------------
host : <blotted out>
release : 2.6.32-5-xen-amd64
version : #1 SMP Tue Jun 14 12:46:30 UTC 2011
machine : x86_64
nr_cpus : 16
nr_nodes : 2
cores_per_socket : 4
threads_per_core : 2
cpu_mhz : 2400
hw_caps :
bfebfbff:2c100800:00000000:00001f40:009ee3fd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 98295
free_memory : 19
node_to_cpu : node0:0-3,8-11
node1:4-7,12-15
node_to_memory : node0:19
node1:0
node_to_dma32_mem : node0:0
node1:0
max_node_id : 1
xen_major : 4
xen_minor : 0
xen_extra : .1
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline : placeholder
cc_compiler : gcc version 4.4.5 (Debian 4.4.5-10)
cc_compile_by : waldi
cc_compile_domain : debian.org
cc_compile_date : Wed Jan 12 14:04:06 UTC 2011
xend_config_format : 4
--------------
XM List output:
-----------
Name ID Mem VCPUs State
Time(s)
Domain-0 0 11257 16 r-----
24477.0
<hostname blotted out> 25 256 8 -b----
4233.8
<hostname blotted out> 11 1024 8 -b----
1725.5
<hostname blotted out> 2 4192 8 r-----
66563.5
<hostname blotted out> 3 512 16 -b----
15330.1
<hostname blotted out> 17 512 16 -b----
5249.6
<hostname blotted out> 18 512 16 -b----
12292.2
<hostname blotted out> 19 512 1 -b----
6620.5
<hostname blotted out> 8 512 16 -b----
1554.6
<hostname blotted out> 9 1024 4 -b----
1837.1
<hostname blotted out> 12 1024 4 -b----
5932.5
<hostname blotted out> 13 1024 16 -b----
35491.3
<hostname blotted out> 22 1024 8 -b----
5505.7
<hostname blotted out> 20 512 1 -b----
15667.6
<hostname blotted out> 23 2816 8 -b----
6621.9
<hostname blotted out> 26 512 1 -b----
11633.6
<hostname blotted out> 28 512 16 -b----
906.9
<hostname blotted out> 15 512 1 -b----
17502.6
<hostname blotted out> 27 256 1 -b----
3924.0
<hostname blotted out> 7 512 1 -b----
12216.5
<hostname blotted out> 6 512 1 -b----
9732.1
<hostname blotted out> 29 2048 16 -b----
1738.4
<hostname blotted out> 4 256 8 -b----
4346.0
<hostname blotted out> 30 2048 8 -b----
4066.8
Total ram used for all DomU''s: 22624MB
----------------------
Config for DomU (generic for all running DomU''s, except ram and core
assignments)
----------------------
#
# Kernel + memory size
#
kernel = ''/boot/vmlinuz-2.6.32-5-xen-amd64''
ramdisk = ''/boot/initrd.img-2.6.32-5-xen-amd64''
memory = ''2048''
vcpus = ''8''
extra = ''xencons=tty''
#
# Disk device(s).
#
root = ''/dev/xvda1 ro''
disk = [ ''file:/storage/target/disk.img,xvda,w'',
''file:/storage/target/swap.img,xvdb,w'' ]
#
# Hostname
#
name = ''<hostname blotted out>''
#
# Networking
#
vif = [ ''ip=<obscured>'' ]
#
# Behaviour
#
on_poweroff = ''destroy''
on_reboot = ''restart''
on_crash = ''restart''
--------
xend-config.sxp
--------
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 196)
(enable-dom0-ballooning yes)
(total_available_memory 0)
(dom0-cpus 0)
----------
Kind regards,
Dennis Storm - Drecomm BV
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Ian Tobin
2011-Sep-02 20:44 UTC
RE: [Xen-users] Physical memory not fully available to DomU''s
Hi,
Same issue here, running Debian 6 with 2.6.32.43 and xen 4.1.1.
Not found a solution yet though ive not had time to test.
Thanks
Ian
-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Dennis Storm
Sent: 02 September 2011 20:45
To: xen-users@lists.xensource.com
Subject: [Xen-users] Physical memory not fully available to DomU's
Hi,
I'm having a weird problem with one of my Xen machines. I have a 96Gb ram
server, running on Debian 6 with Xen kernel. However, I can only assign memory
out of a pool of 24GB, the rest is unavailable for assignment. When trying to
run a new DomU i'm getting the error that there isn't enough free ram.
However, when I do the math myself I can see that there's 22GB of ram in
use, which leave 74GB available (minus
Dom0 minimum limit, which is set at 8GB).
I've tried disabling ballooning, i als saw posts of ram missing due to the
balloondriver, but nothing in the likes of this volume of missing ram.
Any help would be very much appreciated. Seems a shame to let 74GB go to waste.
If you need any more info, please let me know!
XM info output:
--------------
host : <blotted out>
release : 2.6.32-5-xen-amd64
version : #1 SMP Tue Jun 14 12:46:30 UTC 2011
machine : x86_64
nr_cpus : 16
nr_nodes : 2
cores_per_socket : 4
threads_per_core : 2
cpu_mhz : 2400
hw_caps :
bfebfbff:2c100800:00000000:00001f40:009ee3fd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 98295
free_memory : 19
node_to_cpu : node0:0-3,8-11
node1:4-7,12-15
node_to_memory : node0:19
node1:0
node_to_dma32_mem : node0:0
node1:0
max_node_id : 1
xen_major : 4
xen_minor : 0
xen_extra : .1
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline : placeholder
cc_compiler : gcc version 4.4.5 (Debian 4.4.5-10)
cc_compile_by : waldi
cc_compile_domain : debian.org
cc_compile_date : Wed Jan 12 14:04:06 UTC 2011
xend_config_format : 4
--------------
XM List output:
-----------
Name ID Mem VCPUs State
Time(s)
Domain-0 0 11257 16 r-----
24477.0
<hostname blotted out> 25 256 8 -b----
4233.8
<hostname blotted out> 11 1024 8 -b----
1725.5
<hostname blotted out> 2 4192 8 r-----
66563.5
<hostname blotted out> 3 512 16 -b----
15330.1
<hostname blotted out> 17 512 16 -b----
5249.6
<hostname blotted out> 18 512 16 -b----
12292.2
<hostname blotted out> 19 512 1 -b----
6620.5
<hostname blotted out> 8 512 16 -b----
1554.6
<hostname blotted out> 9 1024 4 -b----
1837.1
<hostname blotted out> 12 1024 4 -b----
5932.5
<hostname blotted out> 13 1024 16 -b----
35491.3
<hostname blotted out> 22 1024 8 -b----
5505.7
<hostname blotted out> 20 512 1 -b----
15667.6
<hostname blotted out> 23 2816 8 -b----
6621.9
<hostname blotted out> 26 512 1 -b----
11633.6
<hostname blotted out> 28 512 16 -b----
906.9
<hostname blotted out> 15 512 1 -b----
17502.6
<hostname blotted out> 27 256 1 -b----
3924.0
<hostname blotted out> 7 512 1 -b----
12216.5
<hostname blotted out> 6 512 1 -b----
9732.1
<hostname blotted out> 29 2048 16 -b----
1738.4
<hostname blotted out> 4 256 8 -b----
4346.0
<hostname blotted out> 30 2048 8 -b----
4066.8
Total ram used for all DomU's: 22624MB
----------------------
Config for DomU (generic for all running DomU's, except ram and core
assignments)
----------------------
#
# Kernel + memory size
#
kernel = '/boot/vmlinuz-2.6.32-5-xen-amd64'
ramdisk = '/boot/initrd.img-2.6.32-5-xen-amd64'
memory = '2048'
vcpus = '8'
extra = 'xencons=tty'
#
# Disk device(s).
#
root = '/dev/xvda1 ro'
disk = [ 'file:/storage/target/disk.img,xvda,w',
'file:/storage/target/swap.img,xvdb,w' ]
#
# Hostname
#
name = '<hostname blotted out>'
#
# Networking
#
vif = [ 'ip=<obscured>' ]
#
# Behaviour
#
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
--------
xend-config.sxp
--------
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 196)
(enable-dom0-ballooning yes)
(total_available_memory 0)
(dom0-cpus 0)
----------
Kind regards,
Dennis Storm - Drecomm BV
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Joseph Glanville
2011-Sep-05 16:27 UTC
Re: [Xen-users] Physical memory not fully available to DomU''s
Hmm I don''t have this issue with my hosts (I don''t think so anyways...), seem my xm info below: Does this only occur when you have the hosts loaded? I might go check some of my more fully loaded machines. host : scrubbed release : 2.6.32-5-xen-amd64 version : #1 SMP Tue Mar 8 00:01:30 UTC 2011 machine : x86_64 nr_cpus : 24 nr_nodes : 1 cores_per_socket : 6 threads_per_core : 2 cpu_mhz : 2666 hw_caps : bfebfbff:2c100800:00000000:00001f40:029ee3ff:00000000:00000001:00000000 virt_caps : hvm total_memory : 98294 free_memory : 78701 node_to_cpu : node0:0-23 node_to_memory : node0:78701 node_to_dma32_mem : node0:2993 max_node_id : 0 xen_major : 4 xen_minor : 0 xen_extra : .1 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xffff800000000000 xen_changeset : unavailable xen_commandline : placeholder dom0_mem=2048M dom0_max_mem=2048M cc_compiler : gcc version 4.4.5 (Debian 4.4.5-10) cc_compile_by : waldi cc_compile_domain : debian.org cc_compile_date : Wed Jan 12 14:04:06 UTC 2011 xend_config_format : 4 On 3 September 2011 06:44, Ian Tobin <itobin@tidyhosts.com> wrote:> Hi, > > Same issue here, running Debian 6 with 2.6.32.43 and xen 4.1.1. > > Not found a solution yet though ive not had time to test. > > Thanks > > Ian > > > > -----Original Message----- > From: xen-users-bounces@lists.xensource.com [mailto: > xen-users-bounces@lists.xensource.com] On Behalf Of Dennis Storm > Sent: 02 September 2011 20:45 > To: xen-users@lists.xensource.com > Subject: [Xen-users] Physical memory not fully available to DomU''s > > Hi, > > I''m having a weird problem with one of my Xen machines. I have a 96Gb ram > server, running on Debian 6 with Xen kernel. However, I can only assign > memory out of a pool of 24GB, the rest is unavailable for assignment. When > trying to run a new DomU i''m getting the error that there isn''t enough free > ram. However, when I do the math myself I can see that there''s 22GB of ram > in use, which leave 74GB available (minus > Dom0 minimum limit, which is set at 8GB). > > I''ve tried disabling ballooning, i als saw posts of ram missing due to the > balloondriver, but nothing in the likes of this volume of missing ram. > > Any help would be very much appreciated. Seems a shame to let 74GB go to > waste. If you need any more info, please let me know! > > XM info output: > -------------- > host : <blotted out> > release : 2.6.32-5-xen-amd64 > version : #1 SMP Tue Jun 14 12:46:30 UTC 2011 > machine : x86_64 > nr_cpus : 16 > nr_nodes : 2 > cores_per_socket : 4 > threads_per_core : 2 > cpu_mhz : 2400 > hw_caps : > bfebfbff:2c100800:00000000:00001f40:009ee3fd:00000000:00000001:00000000 > virt_caps : hvm > total_memory : 98295 > free_memory : 19 > node_to_cpu : node0:0-3,8-11 > node1:4-7,12-15 > node_to_memory : node0:19 > node1:0 > node_to_dma32_mem : node0:0 > node1:0 > max_node_id : 1 > xen_major : 4 > xen_minor : 0 > xen_extra : .1 > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 > hvm-3.0-x86_32p hvm-3.0-x86_64 > xen_scheduler : credit > xen_pagesize : 4096 > platform_params : virt_start=0xffff800000000000 > xen_changeset : unavailable > xen_commandline : placeholder > cc_compiler : gcc version 4.4.5 (Debian 4.4.5-10) > cc_compile_by : waldi > cc_compile_domain : debian.org > cc_compile_date : Wed Jan 12 14:04:06 UTC 2011 > xend_config_format : 4 > -------------- > > XM List output: > ----------- > Name ID Mem VCPUs State > Time(s) > Domain-0 0 11257 16 r----- > 24477.0 > <hostname blotted out> 25 256 8 -b---- > 4233.8 > <hostname blotted out> 11 1024 8 -b---- > 1725.5 > <hostname blotted out> 2 4192 8 r----- > 66563.5 > <hostname blotted out> 3 512 16 -b---- > 15330.1 > <hostname blotted out> 17 512 16 -b---- > 5249.6 > <hostname blotted out> 18 512 16 -b---- > 12292.2 > <hostname blotted out> 19 512 1 -b---- > 6620.5 > <hostname blotted out> 8 512 16 -b---- > 1554.6 > <hostname blotted out> 9 1024 4 -b---- > 1837.1 > <hostname blotted out> 12 1024 4 -b---- > 5932.5 > <hostname blotted out> 13 1024 16 -b---- > 35491.3 > <hostname blotted out> 22 1024 8 -b---- > 5505.7 > <hostname blotted out> 20 512 1 -b---- > 15667.6 > <hostname blotted out> 23 2816 8 -b---- > 6621.9 > <hostname blotted out> 26 512 1 -b---- > 11633.6 > <hostname blotted out> 28 512 16 -b---- > 906.9 > <hostname blotted out> 15 512 1 -b---- > 17502.6 > <hostname blotted out> 27 256 1 -b---- > 3924.0 > <hostname blotted out> 7 512 1 -b---- > 12216.5 > <hostname blotted out> 6 512 1 -b---- > 9732.1 > <hostname blotted out> 29 2048 16 -b---- > 1738.4 > <hostname blotted out> 4 256 8 -b---- > 4346.0 > <hostname blotted out> 30 2048 8 -b---- > 4066.8 > > Total ram used for all DomU''s: 22624MB > ---------------------- > > Config for DomU (generic for all running DomU''s, except ram and core > assignments) > ---------------------- > > # > # Kernel + memory size > # > kernel = ''/boot/vmlinuz-2.6.32-5-xen-amd64'' > ramdisk = ''/boot/initrd.img-2.6.32-5-xen-amd64'' > > memory = ''2048'' > vcpus = ''8'' > extra = ''xencons=tty'' > > # > # Disk device(s). > # > root = ''/dev/xvda1 ro'' > disk = [ ''file:/storage/target/disk.img,xvda,w'', > ''file:/storage/target/swap.img,xvdb,w'' ] > > # > # Hostname > # > name = ''<hostname blotted out>'' > > # > # Networking > # > vif = [ ''ip=<obscured>'' ] > > # > # Behaviour > # > on_poweroff = ''destroy'' > on_reboot = ''restart'' > on_crash = ''restart'' > > -------- > > xend-config.sxp > -------- > (network-script network-bridge) > (vif-script vif-bridge) > (dom0-min-mem 196) > (enable-dom0-ballooning yes) > (total_available_memory 0) > (dom0-cpus 0) > ---------- > > Kind regards, > > Dennis Storm - Drecomm BV > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >-- * Founder | Director | VP Research Orion Virtualisation Solutions* | www.orionvm.com.au | Phone: 1300 56 99 52 | Mobile: 0428 754 846 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Dennis Storm - Drecomm BV
2011-Sep-06 06:51 UTC
Re: [Xen-users] Physical memory not fully available to DomU''s
Hmm, just one reply of somebody who has the same issue. No one else encountered this problem? Or has any kind of clue? Met vriendelijke groet, Dennis Storm Senior Systeembeheerder ------------------------------------------------------------------------ E d.storm@drecomm.nl T 050-577 58 22 (doorkiesnummer: 9701) M 06 4141 0181 drecomm ------------------------------------------------------------------------ Vestiging Groningen Hoendiep 208 T +31 (0)50 577 58 22 Bekijk onze actuele projecten 9745 ED Groningen F +31 (0)50 577 58 23 www.drecomm.nl <http://www.drecomm.nl> ------------------------------------------------------------------------ Op 2-9-2011 21:44, Dennis Storm schreef:> Hi, > > I''m having a weird problem with one of my Xen machines. I have a 96Gb > ram server, running on Debian 6 with Xen kernel. However, I can only > assign memory out of a pool of 24GB, the rest is unavailable for > assignment. When trying to run a new DomU i''m getting the error that > there isn''t enough free ram. However, when I do the math myself I can > see that there''s 22GB of ram in use, which leave 74GB available (minus > Dom0 minimum limit, which is set at 8GB). > > I''ve tried disabling ballooning, i als saw posts of ram missing due to > the balloondriver, but nothing in the likes of this volume of missing ram. > > Any help would be very much appreciated. Seems a shame to let 74GB go to > waste. If you need any more info, please let me know! > > XM info output: > -------------- > host :<blotted out> > release : 2.6.32-5-xen-amd64 > version : #1 SMP Tue Jun 14 12:46:30 UTC 2011 > machine : x86_64 > nr_cpus : 16 > nr_nodes : 2 > cores_per_socket : 4 > threads_per_core : 2 > cpu_mhz : 2400 > hw_caps : > bfebfbff:2c100800:00000000:00001f40:009ee3fd:00000000:00000001:00000000 > virt_caps : hvm > total_memory : 98295 > free_memory : 19 > node_to_cpu : node0:0-3,8-11 > node1:4-7,12-15 > node_to_memory : node0:19 > node1:0 > node_to_dma32_mem : node0:0 > node1:0 > max_node_id : 1 > xen_major : 4 > xen_minor : 0 > xen_extra : .1 > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 > hvm-3.0-x86_32p hvm-3.0-x86_64 > xen_scheduler : credit > xen_pagesize : 4096 > platform_params : virt_start=0xffff800000000000 > xen_changeset : unavailable > xen_commandline : placeholder > cc_compiler : gcc version 4.4.5 (Debian 4.4.5-10) > cc_compile_by : waldi > cc_compile_domain : debian.org > cc_compile_date : Wed Jan 12 14:04:06 UTC 2011 > xend_config_format : 4 > -------------- > > XM List output: > ----------- > Name ID Mem VCPUs State > Time(s) > Domain-0 0 11257 16 r----- > 24477.0 > <hostname blotted out> 25 256 8 -b---- > 4233.8 > <hostname blotted out> 11 1024 8 -b---- > 1725.5 > <hostname blotted out> 2 4192 8 r----- > 66563.5 > <hostname blotted out> 3 512 16 -b---- > 15330.1 > <hostname blotted out> 17 512 16 -b---- > 5249.6 > <hostname blotted out> 18 512 16 -b---- > 12292.2 > <hostname blotted out> 19 512 1 -b---- > 6620.5 > <hostname blotted out> 8 512 16 -b---- > 1554.6 > <hostname blotted out> 9 1024 4 -b---- > 1837.1 > <hostname blotted out> 12 1024 4 -b---- > 5932.5 > <hostname blotted out> 13 1024 16 -b---- > 35491.3 > <hostname blotted out> 22 1024 8 -b---- > 5505.7 > <hostname blotted out> 20 512 1 -b---- > 15667.6 > <hostname blotted out> 23 2816 8 -b---- > 6621.9 > <hostname blotted out> 26 512 1 -b---- > 11633.6 > <hostname blotted out> 28 512 16 -b---- > 906.9 > <hostname blotted out> 15 512 1 -b---- > 17502.6 > <hostname blotted out> 27 256 1 -b---- > 3924.0 > <hostname blotted out> 7 512 1 -b---- > 12216.5 > <hostname blotted out> 6 512 1 -b---- > 9732.1 > <hostname blotted out> 29 2048 16 -b---- > 1738.4 > <hostname blotted out> 4 256 8 -b---- > 4346.0 > <hostname blotted out> 30 2048 8 -b---- > 4066.8 > > Total ram used for all DomU''s: 22624MB > ---------------------- > > Config for DomU (generic for all running DomU''s, except ram and core > assignments) > ---------------------- > > # > # Kernel + memory size > # > kernel = ''/boot/vmlinuz-2.6.32-5-xen-amd64'' > ramdisk = ''/boot/initrd.img-2.6.32-5-xen-amd64'' > > memory = ''2048'' > vcpus = ''8'' > extra = ''xencons=tty'' > > # > # Disk device(s). > # > root = ''/dev/xvda1 ro'' > disk = [ ''file:/storage/target/disk.img,xvda,w'', > ''file:/storage/target/swap.img,xvdb,w'' ] > > # > # Hostname > # > name = ''<hostname blotted out>'' > > # > # Networking > # > vif = [ ''ip=<obscured>'' ] > > # > # Behaviour > # > on_poweroff = ''destroy'' > on_reboot = ''restart'' > on_crash = ''restart'' > > -------- > > xend-config.sxp > -------- > (network-script network-bridge) > (vif-script vif-bridge) > (dom0-min-mem 196) > (enable-dom0-ballooning yes) > (total_available_memory 0) > (dom0-cpus 0) > ---------- > > Kind regards, > > Dennis Storm - Drecomm BV > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pim van Riezen
2011-Sep-06 08:53 UTC
Re: [Xen-users] Physical memory not fully available to DomU''s
On Sep 6, 2011, at 8:51 , Dennis Storm - Drecomm BV wrote:> Hmm, just one reply of somebody who has the same issue. No one else encountered this problem? Or has any kind of clue?Dennis, We''re not seeing this issue and we''re scaling up to 128GB RAM. We do, as a standard, limit dom0_mem to 2/4GB on the grub line for the hypervisor. I wouldn''t be surprised if this were a factor. Cheers, Pim _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
James Dingwall
2011-Sep-06 10:12 UTC
RE: [Xen-users] Physical memory not fully available to DomU''s
>> Hmm, just one reply of somebody who has the same issue. No one else encountered this problem? Or has any kind of clue? > >Dennis, > >We''re not seeing this issue and we''re scaling up to 128GB RAM. We do, as a standard, limit dom0_mem to 2/4GB on the grub line for the >hypervisor. I wouldn''t be surprised if this were a factor.What do you have CONFIG_XEN_MAX_DOMAIN_MEMORY set to in your kernel config? I thought that this parameter was only about how much memory any individual domU could address but... This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pim van Riezen
2011-Sep-06 10:14 UTC
Re: [Xen-users] Physical memory not fully available to DomU''s
On Sep 6, 2011, at 12:12 , James Dingwall wrote:> >>> Hmm, just one reply of somebody who has the same issue. No one else encountered this problem? Or has any kind of clue? >> >> Dennis, >> >> We''re not seeing this issue and we''re scaling up to 128GB RAM. We do, as a standard, limit dom0_mem to 2/4GB on the grub line for the >hypervisor. I wouldn''t be surprised if this were a factor. > > What do you have CONFIG_XEN_MAX_DOMAIN_MEMORY set to in your kernel config? I thought that this parameter was only about how much memory any individual domU could address but... > This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, > you may review at http://www.amdocs.com/email_disclaimer.aspWe have been using stock RHEL xen-kernels and experimenting with 3.0.x, in both cases these were kept at vanilla settings. Pi _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Le mardi 06 septembre 2011 à 08:51 +0200, Dennis Storm - Drecomm BV a écrit : Hello, If your machine is fitted with 96GB why are you using (enable-dom0-ballooning yes) ? If your DOM0 starts with 8GB put the same amount for "dom0_min_mem" and ballooning = no. I have had problems with "ballooning" ! So I am very cautious with its usage. Regards JP P> Hmm, just one reply of somebody who has the same issue. No one else > encountered this problem? Or has any kind of clue? > > Met vriendelijke groet, > > Dennis Storm > Senior Systeembeheerder > > > > ______________________________________________________________________ > E d.storm@drecomm.nl > > T 050-577 58 22 (doorkiesnummer: > 9701) > > M 06 4141 0181 > > > drecomm > > ______________________________________________________________________ > Vestiging Groningen > Hoendiep 208 > T +31 (0)50 577 58 22 > Bekijk onze actuele > projecten > 9745 ED Groningen > F +31 (0)50 577 58 23 > www.drecomm.nl > > ______________________________________________________________________ > > Op 2-9-2011 21:44, Dennis Storm schreef: > > Hi, > > > > I''m having a weird problem with one of my Xen machines. I have a 96Gb > > ram server, running on Debian 6 with Xen kernel. However, I can only > > assign memory out of a pool of 24GB, the rest is unavailable for > > assignment. When trying to run a new DomU i''m getting the error that > > there isn''t enough free ram. However, when I do the math myself I can > > see that there''s 22GB of ram in use, which leave 74GB available (minus > > Dom0 minimum limit, which is set at 8GB). > > > > I''ve tried disabling ballooning, i als saw posts of ram missing due to > > the balloondriver, but nothing in the likes of this volume of missing ram. > > > > Any help would be very much appreciated. Seems a shame to let 74GB go to > > waste. If you need any more info, please let me know! > > > > XM info output: > > -------------- > > host : <blotted out> > > release : 2.6.32-5-xen-amd64 > > version : #1 SMP Tue Jun 14 12:46:30 UTC 2011 > > machine : x86_64 > > nr_cpus : 16 > > nr_nodes : 2 > > cores_per_socket : 4 > > threads_per_core : 2 > > cpu_mhz : 2400 > > hw_caps : > > bfebfbff:2c100800:00000000:00001f40:009ee3fd:00000000:00000001:00000000 > > virt_caps : hvm > > total_memory : 98295 > > free_memory : 19 > > node_to_cpu : node0:0-3,8-11 > > node1:4-7,12-15 > > node_to_memory : node0:19 > > node1:0 > > node_to_dma32_mem : node0:0 > > node1:0 > > max_node_id : 1 > > xen_major : 4 > > xen_minor : 0 > > xen_extra : .1 > > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 > > hvm-3.0-x86_32p hvm-3.0-x86_64 > > xen_scheduler : credit > > xen_pagesize : 4096 > > platform_params : virt_start=0xffff800000000000 > > xen_changeset : unavailable > > xen_commandline : placeholder > > cc_compiler : gcc version 4.4.5 (Debian 4.4.5-10) > > cc_compile_by : waldi > > cc_compile_domain : debian.org > > cc_compile_date : Wed Jan 12 14:04:06 UTC 2011 > > xend_config_format : 4 > > -------------- > > > > XM List output: > > ----------- > > Name ID Mem VCPUs State > > Time(s) > > Domain-0 0 11257 16 r----- > > 24477.0 > > <hostname blotted out> 25 256 8 -b---- > > 4233.8 > > <hostname blotted out> 11 1024 8 -b---- > > 1725.5 > > <hostname blotted out> 2 4192 8 r----- > > 66563.5 > > <hostname blotted out> 3 512 16 -b---- > > 15330.1 > > <hostname blotted out> 17 512 16 -b---- > > 5249.6 > > <hostname blotted out> 18 512 16 -b---- > > 12292.2 > > <hostname blotted out> 19 512 1 -b---- > > 6620.5 > > <hostname blotted out> 8 512 16 -b---- > > 1554.6 > > <hostname blotted out> 9 1024 4 -b---- > > 1837.1 > > <hostname blotted out> 12 1024 4 -b---- > > 5932.5 > > <hostname blotted out> 13 1024 16 -b---- > > 35491.3 > > <hostname blotted out> 22 1024 8 -b---- > > 5505.7 > > <hostname blotted out> 20 512 1 -b---- > > 15667.6 > > <hostname blotted out> 23 2816 8 -b---- > > 6621.9 > > <hostname blotted out> 26 512 1 -b---- > > 11633.6 > > <hostname blotted out> 28 512 16 -b---- > > 906.9 > > <hostname blotted out> 15 512 1 -b---- > > 17502.6 > > <hostname blotted out> 27 256 1 -b---- > > 3924.0 > > <hostname blotted out> 7 512 1 -b---- > > 12216.5 > > <hostname blotted out> 6 512 1 -b---- > > 9732.1 > > <hostname blotted out> 29 2048 16 -b---- > > 1738.4 > > <hostname blotted out> 4 256 8 -b---- > > 4346.0 > > <hostname blotted out> 30 2048 8 -b---- > > 4066.8 > > > > Total ram used for all DomU''s: 22624MB > > ---------------------- > > > > Config for DomU (generic for all running DomU''s, except ram and core > > assignments) > > ---------------------- > > > > # > > # Kernel + memory size > > # > > kernel = ''/boot/vmlinuz-2.6.32-5-xen-amd64'' > > ramdisk = ''/boot/initrd.img-2.6.32-5-xen-amd64'' > > (enable-dom0-ballooning yes) > > memory = ''2048'' > > vcpus = ''8'' > > extra = ''xencons=tty'' > > > > # > > # Disk device(s).http://www.virtuatopia.com/index.php/Installing_and_Running_Windows_7_as_a_Xen_HVM_domainU_Guest > > # > > root = ''/dev/xvda1 ro'' > > disk = [ ''file:/storage/target/disk.img,xvda,w'', > > ''file:/storage/target/swap.img,xvdb,w'' ] > > > > # > > # Hostname > > # > > name = ''<hostname blotted out>'' > > > > # > > # Networking > > # > > vif = [ ''ip=<obscured>'' ] > > > > # > > # Behaviour > > # > > on_poweroff = ''destroy'' > > on_reboot = ''restart'' > > on_crash = ''restart'' > > > > -------- > > > > xend-config.sxp > > -------- > > (network-script network-bridge) > > (vif-script vif-bridge) > > (dom0-min-mem 196) > > (enable-dom0-ballooning yes) > > (total_available_memory 0) > > (dom0-cpus 0) > > ---------- > > > > Kind regards, > > > > Dennis Storm - Drecomm BV > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Dennis Storm - Drecomm BV
2011-Sep-07 07:29 UTC
Re: [Xen-users] Physical memory not fully available to DomU''s
Hmm, I''ll give it a go. Since it is a production machine I''ll try and make it happen this evening. I''ve set the kernel parameter in grub2 config, and disabled the balloon driver. I''ll let you guys know, thanks for the tips so far! Met vriendelijke groet, Dennis Storm Senior Systeembeheerder ------------------------------------------------------------------------ E d.storm@drecomm.nl T 050-577 58 22 (doorkiesnummer: 9701) M 06 4141 0181 drecomm ------------------------------------------------------------------------ Vestiging Groningen Hoendiep 208 T +31 (0)50 577 58 22 Bekijk onze actuele projecten 9745 ED Groningen F +31 (0)50 577 58 23 www.drecomm.nl <http://www.drecomm.nl> ------------------------------------------------------------------------ Op 6-9-2011 10:53, Pim van Riezen schreef:> On Sep 6, 2011, at 8:51 , Dennis Storm - Drecomm BV wrote: > >> Hmm, just one reply of somebody who has the same issue. No one else encountered this problem? Or has any kind of clue? > Dennis, > > We''re not seeing this issue and we''re scaling up to 128GB RAM. We do, as a standard, limit dom0_mem to 2/4GB on the grub line for the hypervisor. I wouldn''t be surprised if this were a factor. > > Cheers, > Pim > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users