Hello,
I''m running a FC6 dom0, with kernel version,
kernel-xen-2.6.19-1.2895.fc6 on a Dell PowerEdge 1600SC. I just
upgraded it from 1 GB (2 x 512 MB modules) to 3 GB (2 x 1 GB + 2 x 512
MB). The BIOS sees all of the ram, but kernel does not. All I see when
I run free is this:
total used free shared buffers
cached
Mem: 2454528 207876 2246652 0 11032
47380
-/+ buffers/cache: 149464 2305064
Swap: 524280 0 524280
It looks like 512 MB is missing.
Here''s a bit more detail from dmesg:
Memory: 2874124k/2987900k available (2166k kernel code, 104532k
reserved, 1100k data, 184k init, 2242436k highmem)
So, what happened to the rest of the RAM? I should add that the server
is updated with the latest firmware and BIOS.
BTW, I appear to be having problems with one of my domUs now too (after
the ram upgrade). A PCI device I was handing to the domU is now causing
the domU kernel to panic, crashing the domU. I''ll try to get a copy of
the error message.
Any help appreciated.
Regards,
Ranbir
--
Kanwar Ranbir Sandhu
Linux 2.6.20-1.2944.fc6 i686 GNU/Linux
00:17:27 up 10:25, 2 users, load average: 0.68, 0.43, 0.26
try a new xen kernel run a yum update -------------------- Itamar Reis Peixoto e-mail: itamar@ispbrasil.com.br msn: itamarjp@starmedia.com skype: itamarjp icq: 81053601 +55 34 3238 3845 +55 11 4063 5033 ----- Original Message ----- From: "Kanwar Ranbir Sandhu" <m3freak@thesandhufamily.ca> To: <fedora-xen@redhat.com> Sent: Tuesday, July 03, 2007 1:27 AM Subject: [Fedora-xen] Not seeing all RAM> Hello, > > I''m running a FC6 dom0, with kernel version, > kernel-xen-2.6.19-1.2895.fc6 on a Dell PowerEdge 1600SC. I just > upgraded it from 1 GB (2 x 512 MB modules) to 3 GB (2 x 1 GB + 2 x 512 > MB). The BIOS sees all of the ram, but kernel does not. All I see when > I run free is this: > > total used free shared buffers > cached > Mem: 2454528 207876 2246652 0 11032 > 47380 > -/+ buffers/cache: 149464 2305064 > Swap: 524280 0 524280 > > It looks like 512 MB is missing. > > Here''s a bit more detail from dmesg: > > Memory: 2874124k/2987900k available (2166k kernel code, 104532k > reserved, 1100k data, 184k init, 2242436k highmem) > > So, what happened to the rest of the RAM? I should add that the server > is updated with the latest firmware and BIOS. > > BTW, I appear to be having problems with one of my domUs now too (after > the ram upgrade). A PCI device I was handing to the domU is now causing > the domU kernel to panic, crashing the domU. I''ll try to get a copy of > the error message. > > Any help appreciated. > > Regards, > > Ranbir > -- > Kanwar Ranbir Sandhu > Linux 2.6.20-1.2944.fc6 i686 GNU/Linux > 00:17:27 up 10:25, 2 users, load average: 0.68, 0.43, 0.26 > > > -- > Fedora-xen mailing list > Fedora-xen@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-xen >
Kanwar Ranbir Sandhu wrote:> Hello, > > I''m running a FC6 dom0, with kernel version, > kernel-xen-2.6.19-1.2895.fc6 on a Dell PowerEdge 1600SC. I just > upgraded it from 1 GB (2 x 512 MB modules) to 3 GB (2 x 1 GB + 2 x 512 > MB).If this still happens after you''ve updated to the latest FC6 (IIRC, kernel-xen 2.6.20-1.2952) then please file a bug here: https://bugzilla.redhat.com/bugzilla/index.cgi There are various problems with Xen & memory, although they don''t normally happen below the 4 GB boundary (and mostly happen for people with absolutely enormous amounts of memory, like > 60 GB). We''ll try and fix them. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Tue, 2007-07-03 at 10:17 +0100, Richard W.M. Jones wrote:> If this still happens after you''ve updated to the latest FC6 (IIRC, > kernel-xen 2.6.20-1.2952) then please file a bug here: > https://bugzilla.redhat.com/bugzilla/index.cgiSome improvement, but still missing RAM: [root@xenmaster ~]# free total used free shared buffers cached Mem: 2716672 211452 2505220 0 11056 48608 -/+ buffers/cache: 151788 2564884 Swap: 524280 0 524280 [root@xenmaster ~]# grep -i memory /var/log/dmesg Memory: 2874636k/2987900k available (2135k kernel code, 104060k reserved, 1130k data, 176k init, 2242436k highmem) The BIOS reports 3072 MB. I''ll file that bug report. Regards, Ranbir -- Kanwar Ranbir Sandhu Linux 2.6.20-1.2944.fc6 i686 GNU/Linux 10:06:21 up 20:14, 2 users, load average: 0.18, 0.20, 0.23
On Tue, Jul 03, 2007 at 12:27:54AM -0400, Kanwar Ranbir Sandhu wrote:> Hello, > > I''m running a FC6 dom0, with kernel version, > kernel-xen-2.6.19-1.2895.fc6 on a Dell PowerEdge 1600SC. I just > upgraded it from 1 GB (2 x 512 MB modules) to 3 GB (2 x 1 GB + 2 x 512 > MB). The BIOS sees all of the ram, but kernel does not. All I see when > I run free is this: > > total used free shared buffers > cached > Mem: 2454528 207876 2246652 0 11032 > 47380 > -/+ buffers/cache: 149464 2305064 > Swap: 524280 0 524280 > > It looks like 512 MB is missing.Let me guest - you are running a 32-bit kernel on the machine ? If so the problem is likely that the BOIS is remapping a portion of memory above the 4 GB mark, to make room for PCI device DMA zones. A 32-bit kernel though can only access physical addrs upto 4 GB, so it can''t see the remapped memory.> BTW, I appear to be having problems with one of my domUs now too (after > the ram upgrade). A PCI device I was handing to the domU is now causing > the domU kernel to panic, crashing the domU. I''ll try to get a copy of > the error message.PCI device mapping to DomUs is insanely fragile, whether it works or not is mostly luck. In this case since you added more memory, the locaiton of the PCI device''s DMA zone probably changed and by (bad) luck is no longer accessible to the DomU or something like that. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Tue, 2007-07-03 at 15:23 +0100, Daniel P. Berrange wrote:> Let me guest - you are running a 32-bit kernel on the machine ? If so > the problem is likely that the BOIS is remapping a portion of memory > above the 4 GB mark, to make room for PCI device DMA zones. A 32-bit > kernel though can only access physical addrs upto 4 GB, so it can''t > see the remapped memory.I''m aware of that, but in my case it doesn''t make sense: the server only has 3 GB of RAM. The kernel just doesn''t see all of it. I was using kernel-xen-2.6.19-1.2895.fc6 and about 512 MB were missing. After upgrading to kernel-xen-2.6.20-1.2962.fc6, the kernel can see more memory, but I''m still missing about 256 MB. I''m going to open a new bug report for this.> PCI device mapping to DomUs is insanely fragile, whether it works or not > is mostly luck. In this case since you added more memory, the locaiton of > the PCI device''s DMA zone probably changed and by (bad) luck is no longer > accessible to the DomU or something like that.That''s what I figured, though I don''t know all of the details. However, with the new kernel being able to see more of the 3 GB, the domU is working again. The pci device I''m hiding from dom0 is being properly handed to the domU. Regards, Ranbir -- Kanwar Ranbir Sandhu Linux 2.6.20-1.2944.fc6 i686 GNU/Linux 10:42:35 up 20:50, 2 users, load average: 2.15, 2.16, 1.45
with non xen kernel are you able to see all amount of memory ? have you take a look on your bios to check if your video card is not using your memory ? -------------------- Itamar Reis Peixoto e-mail: itamar@ispbrasil.com.br msn: itamarjp@starmedia.com skype: itamarjp icq: 81053601 +55 34 3238 3845 +55 11 4063 5033 ----- Original Message ----- From: "Kanwar Ranbir Sandhu" <m3freak@thesandhufamily.ca> To: "Daniel P. Berrange" <berrange@redhat.com> Cc: <fedora-xen@redhat.com> Sent: Tuesday, July 03, 2007 11:42 AM Subject: Re: [Fedora-xen] Not seeing all RAM> On Tue, 2007-07-03 at 15:23 +0100, Daniel P. Berrange wrote: >> Let me guest - you are running a 32-bit kernel on the machine ? If so >> the problem is likely that the BOIS is remapping a portion of memory >> above the 4 GB mark, to make room for PCI device DMA zones. A 32-bit >> kernel though can only access physical addrs upto 4 GB, so it can''t >> see the remapped memory. > > I''m aware of that, but in my case it doesn''t make sense: the server only > has 3 GB of RAM. The kernel just doesn''t see all of it. > > I was using kernel-xen-2.6.19-1.2895.fc6 and about 512 MB were missing. > After upgrading to kernel-xen-2.6.20-1.2962.fc6, the kernel can see more > memory, but I''m still missing about 256 MB. > > I''m going to open a new bug report for this. > >> PCI device mapping to DomUs is insanely fragile, whether it works or not >> is mostly luck. In this case since you added more memory, the locaiton of >> the PCI device''s DMA zone probably changed and by (bad) luck is no longer >> accessible to the DomU or something like that. > > That''s what I figured, though I don''t know all of the details. However, > with the new kernel being able to see more of the 3 GB, the domU is > working again. The pci device I''m hiding from dom0 is being properly > handed to the domU. > > Regards, > > Ranbir > -- > Kanwar Ranbir Sandhu > Linux 2.6.20-1.2944.fc6 i686 GNU/Linux > 10:42:35 up 20:50, 2 users, load average: 2.15, 2.16, 1.45 > > > -- > Fedora-xen mailing list > Fedora-xen@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-xen >
On Tue, 2007-07-03 at 14:03 -0300, Itamar Reis Peixoto wrote:> with non xen kernel are you able to see all amount of memory ?I haven''t tried that, but I will tonight when I get home. I''m expecting the non-xen kernel to see all the RAM.> have you take a look on your bios to check if your video card is not using > your memory ?It could be, but even if it was, it wouldn''t be much. I''m running the dom0 on a real server, and the embedded graphics controller is very simple. At most, it''s probably using 8 MB if it doesn''t have it''s own dedicated memory. I''m fairly certain the real problem is a bug in Fedora Core 6''s xen kernel. When I upgraded to the lastest kernel version, the kernel picked up more of the RAM. There''s still about 256 missing, but it''s better than the 512 MB that was missing with the older xen kernel. Regards, Ranbir -- Kanwar Ranbir Sandhu Linux 2.6.20-1.2962.fc6 i686 GNU/Linux 16:54:39 up 1 day, 5:38, 3 users, load average: 0.20, 0.11, 0.09
On Wednesday 04 July 2007 22:55:01 Kanwar Ranbir Sandhu wrote:> On Tue, 2007-07-03 at 14:03 -0300, Itamar Reis Peixoto wrote: > > with non xen kernel are you able to see all amount of memory ? > > I haven''t tried that, but I will tonight when I get home. I''m expecting > the non-xen kernel to see all the RAM. >I expect that too! I have exactly the same problem. The difference is that I''m on Debian 4.0. So this may not be a Fedora-specific problem at all. If I boot the standard Debian 2.6.18-4-amd64 then I have 2061340k total using "top". If I boot Xen kernel 2.6.18-4-xen-amd64 the I only get 1793024k total. Something''s eating memory. I''m going to file a report at Debian too. Just thought I drop the note here as well. HTH, Mikael
On Thu, 2007-07-05 at 15:59 +0200, Mikael Bak wrote:> I expect that too! > I have exactly the same problem. The difference is that I''m on Debian 4.0. So > this may not be a Fedora-specific problem at all. > > If I boot the standard Debian 2.6.18-4-amd64 then I have 2061340k total > using "top". > If I boot Xen kernel 2.6.18-4-xen-amd64 the I only get 1793024k total.I booted the server with the non-xen kernel, and this is what free reports: [root@xenmaster ~]# free total used free shared buffers cached Mem: 3109504 82904 3026600 0 7084 33652 -/+ buffers/cache: 42168 3067336 Swap: 524280 0 524280 As you can see, all of the RAM is available. So, it''s most probably a bug in the xen kernel. Regards, Ranbir -- Kanwar Ranbir Sandhu Linux 2.6.20-1.2962.fc6 i686 GNU/Linux 12:52:10 up 6 days, 1:34, 3 users, load average: 0.50, 0.41, 0.31