thr3ads.net - Xen users - [Xen-users] Xen 3 not seeing the right amount of RAM on x86

If this information is useful, please help other people find it:
Share via:

Chuck McIntyre

2007-Nov-28 04:47 UTC

[Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Hi,

I am having a problem where Xen is not seeing the appropriate amount
of memory allocated. At first I thought this was similar to previously
described issues with the BIOS remapped hole for PCI devices (such as
in a post here
http://lists.xensource.com/archives/html/xen-users/2005-07/msg00177.html),
however, we only see about 3.5GB (see below) of RAM regardless of
whether we have a system with 4GB of RAM or 8GB of RAM (as seen by the
BIOS). Regardless, when we boot into the non-xen kernel available on
the same box, the OS then sees all the available RAM.

I have tried to provide as much relevant information as I could
without being spammy. Please let me know if I need to provide
additional details. I would appreciate any help here, as this one has
us stumped, or do we have a bug with our hardware version?

__4GB system___

[me@molotov ~]$ uname -a
Linux molotov.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[me@molotov ~]$ rpm -qa | grep xen
xen-3.0.3-25.0.4.el5
xen-libs-3.0.3-25.0.4.el5
xen-libs-3.0.3-25.0.4.el5
kernel-xen-2.6.18-8.1.15.el5.centos.plus
[me@molotov ~]$ grep MemTotal /proc/meminfo
MemTotal:      3428400 kB
top gives me:
Mem:   3428400k total,   422916k used,  3005484k free,    21648k buffers
[root@molotov ~]# xm info 2>&1 | grep memory
total_memory           : 3535
free_memory            : 129
xentop gives me:
Mem: 3620404k total, 3488052k used, 132352k free    CPUs: 4 @ 2200MHz
dmesg
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000d1d4c000 (usable)
On node 0 totalpages: 859468
  DMA zone: 859468 pages, LIFO batch:31
DMI present.

-=After reboot=-

BIOS gives me:
Initializing USB Controllers .. Done.
4096MB OK
[me@molotov ~]$ grep MemTotal /proc/meminfo
MemTotal:      4050520 kB
top gives
Mem:   4050520k total,   264668k used,  3785852k free,    20612k buffers
dmesg
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
 BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000dcff0000 (usable)
 BIOS-e820: 00000000dcff0000 - 00000000dcffe000 (ACPI data)
 BIOS-e820: 00000000dcffe000 - 00000000dd000000 (ACPI NVS)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
 BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000123000000 (usable)
DMI present.
Bootmem setup node 0 0000000000000000-0000000080000000
Bootmem setup node 1 0000000080000000-0000000123000000
On node 0 totalpages: 515808
  DMA zone: 2728 pages, LIFO batch:0
  DMA32 zone: 513080 pages, LIFO batch:31
On node 1 totalpages: 515144
  DMA32 zone: 373744 pages, LIFO batch:31
  Normal zone: 141400 pages, LIFO batch:31



__ 8GB System __

[me@molotov ~]$ uname -a
Linux benelli.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[me@molotov ~]$ rpm -qa |  grep xen
kernel-xen-2.6.18-8.1.15.el5.centos.plus
xen-libs-3.0.3-25.0.4.el5
xen-libs-3.0.3-25.0.4.el5
xen-3.0.3-25.0.4.el5
[me@benelli ~]$ grep MemTotal /proc/meminfo
MemTotal:      3428336 kB
top gives me:
Mem:   3428336k total,   437888k used,  2990448k free,    16580k buffers
[root@benelli ~]# xm info 2>&1| grep memory
total_memory           : 3535
free_memory            : 0
xentop gives me:
Mem: 3620336k total, 3620064k used, 272k free    CPUs: 4 @ 2200MHz
dmesg
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000d1d3c000 (usable)
DMI present.

-=After Reboot=-

BIOS gives me:
Initializing USB Controllers .. Done.
8192MB OK

After reboot into non-xen kernel:

[root@benelli ~]# grep MemTotal /proc/meminfo
MemTotal:      8181124 kB
Top gives
Mem:   8181124k total,   284732k used,  7896392k free,    16944k buffers
dmesg
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009c400 (usable)
 BIOS-e820: 000000000009c400 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000dcfe0000 (usable)
 BIOS-e820: 00000000dcfe0000 - 00000000dcfee000 (ACPI data)
 BIOS-e820: 00000000dcfee000 - 00000000dcff0000 (ACPI NVS)
 BIOS-e820: 00000000dcff0000 - 00000000dcffe000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
 BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000223000000 (usable)
DMI present.
Bootmem setup node 0 0000000000000000-0000000123000000
Bootmem setup node 1 0000000123000000-0000000223000000
On node 0 totalpages: 1030931
  DMA zone: 2723 pages, LIFO batch:0
  DMA32 zone: 886808 pages, LIFO batch:31
  Normal zone: 141400 pages, LIFO batch:31
On node 1 totalpages: 1034240
  Normal zone: 1034240 pages, LIFO batch:31


__Both Systems__

Here is grub.conf (equivalent on both systems):
default=0
timeout=5
serial --unit=0 --speed=9600
terminal --timeout=5 serial console
title CentOS (2.6.18-8.1.15.el5.centos.plusxen)
        root (hd0,0)
        kernel /boot/xen.gz-2.6.18-8.1.15.el5.centos.plus console=vga
        module /boot/vmlinuz-2.6.18-8.1.15.el5.centos.plusxen ro
root=/dev/md0 console=ttyS0 console=ttyS0,9600n8 console=tty0
xencons=tty
        module /boot/initrd-2.6.18-8.1.15.el5.centos.plusxen.img
title CentOS (2.6.18-8.1.15.el5.centos.plus)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.18-8.1.15.el5.centos.plus ro
root=/dev/md0 console=ttyS0 console=ttyS0,9600n8 console=tty0
xencons=tty
        initrd /boot/initrd-2.6.18-8.1.15.el5.centos.plus.img

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Daniel Mealha Cabrita

2007-Nov-28 05:24 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

It looks like you''re using a 32-bit Xen kernel without compiled PAE
support.
Also looks like your non-Xen kernel is a 32-bit one with PAE support.

Em Qua 28 Nov 2007 02:47, Chuck McIntyre escreveu:> Hi,
>
> I am having a problem where Xen is not seeing the appropriate amount
> of memory allocated. At first I thought this was similar to previously
> described issues with the BIOS remapped hole for PCI devices (such as
> in a post here
> http://lists.xensource.com/archives/html/xen-users/2005-07/msg00177.html),
> however, we only see about 3.5GB (see below) of RAM regardless of
> whether we have a system with 4GB of RAM or 8GB of RAM (as seen by the
> BIOS). Regardless, when we boot into the non-xen kernel available on
> the same box, the OS then sees all the available RAM.
>
> I have tried to provide as much relevant information as I could
> without being spammy. Please let me know if I need to provide
> additional details. I would appreciate any help here, as this one has
> us stumped, or do we have a bug with our hardware version?
>
> __4GB system___
>
> [me@molotov ~]$ uname -a
> Linux molotov.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
> 25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
> [me@molotov ~]$ rpm -qa | grep xen
> xen-3.0.3-25.0.4.el5
> xen-libs-3.0.3-25.0.4.el5
> xen-libs-3.0.3-25.0.4.el5
> kernel-xen-2.6.18-8.1.15.el5.centos.plus
> [me@molotov ~]$ grep MemTotal /proc/meminfo
> MemTotal:      3428400 kB
> top gives me:
> Mem:   3428400k total,   422916k used,  3005484k free,    21648k buffers
> [root@molotov ~]# xm info 2>&1 | grep memory
> total_memory           : 3535
> free_memory            : 129
> xentop gives me:
> Mem: 3620404k total, 3488052k used, 132352k free    CPUs: 4 @ 2200MHz
> dmesg
> BIOS-provided physical RAM map:
>  Xen: 0000000000000000 - 00000000d1d4c000 (usable)
> On node 0 totalpages: 859468
>   DMA zone: 859468 pages, LIFO batch:31
> DMI present.
>
> -=After reboot=-
>
> BIOS gives me:
> Initializing USB Controllers .. Done.
> 4096MB OK
> [me@molotov ~]$ grep MemTotal /proc/meminfo
> MemTotal:      4050520 kB
> top gives
> Mem:   4050520k total,   264668k used,  3785852k free,    20612k buffers
> dmesg
> BIOS-provided physical RAM map:
>  BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
>  BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
>  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
>  BIOS-e820: 0000000000100000 - 00000000dcff0000 (usable)
>  BIOS-e820: 00000000dcff0000 - 00000000dcffe000 (ACPI data)
>  BIOS-e820: 00000000dcffe000 - 00000000dd000000 (ACPI NVS)
>  BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
>  BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
>  BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
>  BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
>  BIOS-e820: 0000000100000000 - 0000000123000000 (usable)
> DMI present.
> Bootmem setup node 0 0000000000000000-0000000080000000
> Bootmem setup node 1 0000000080000000-0000000123000000
> On node 0 totalpages: 515808
>   DMA zone: 2728 pages, LIFO batch:0
>   DMA32 zone: 513080 pages, LIFO batch:31
> On node 1 totalpages: 515144
>   DMA32 zone: 373744 pages, LIFO batch:31
>   Normal zone: 141400 pages, LIFO batch:31
>
>
>
> __ 8GB System __
>
> [me@molotov ~]$ uname -a
> Linux benelli.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
> 25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
> [me@molotov ~]$ rpm -qa |  grep xen
> kernel-xen-2.6.18-8.1.15.el5.centos.plus
> xen-libs-3.0.3-25.0.4.el5
> xen-libs-3.0.3-25.0.4.el5
> xen-3.0.3-25.0.4.el5
> [me@benelli ~]$ grep MemTotal /proc/meminfo
> MemTotal:      3428336 kB
> top gives me:
> Mem:   3428336k total,   437888k used,  2990448k free,    16580k buffers
> [root@benelli ~]# xm info 2>&1| grep memory
> total_memory           : 3535
> free_memory            : 0
> xentop gives me:
> Mem: 3620336k total, 3620064k used, 272k free    CPUs: 4 @ 2200MHz
> dmesg
> BIOS-provided physical RAM map:
>  Xen: 0000000000000000 - 00000000d1d3c000 (usable)
> DMI present.
>
> -=After Reboot=-
>
> BIOS gives me:
> Initializing USB Controllers .. Done.
> 8192MB OK
>
> After reboot into non-xen kernel:
>
> [root@benelli ~]# grep MemTotal /proc/meminfo
> MemTotal:      8181124 kB
> Top gives
> Mem:   8181124k total,   284732k used,  7896392k free,    16944k buffers
> dmesg
> BIOS-provided physical RAM map:
>  BIOS-e820: 0000000000000000 - 000000000009c400 (usable)
>  BIOS-e820: 000000000009c400 - 00000000000a0000 (reserved)
>  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
>  BIOS-e820: 0000000000100000 - 00000000dcfe0000 (usable)
>  BIOS-e820: 00000000dcfe0000 - 00000000dcfee000 (ACPI data)
>  BIOS-e820: 00000000dcfee000 - 00000000dcff0000 (ACPI NVS)
>  BIOS-e820: 00000000dcff0000 - 00000000dcffe000 (reserved)
>  BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
>  BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
>  BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
>  BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
>  BIOS-e820: 0000000100000000 - 0000000223000000 (usable)
> DMI present.
> Bootmem setup node 0 0000000000000000-0000000123000000
> Bootmem setup node 1 0000000123000000-0000000223000000
> On node 0 totalpages: 1030931
>   DMA zone: 2723 pages, LIFO batch:0
>   DMA32 zone: 886808 pages, LIFO batch:31
>   Normal zone: 141400 pages, LIFO batch:31
> On node 1 totalpages: 1034240
>   Normal zone: 1034240 pages, LIFO batch:31
>
>
> __Both Systems__
>
> Here is grub.conf (equivalent on both systems):
> default=0
> timeout=5
> serial --unit=0 --speed=9600
> terminal --timeout=5 serial console
> title CentOS (2.6.18-8.1.15.el5.centos.plusxen)
>         root (hd0,0)
>         kernel /boot/xen.gz-2.6.18-8.1.15.el5.centos.plus console=vga
>         module /boot/vmlinuz-2.6.18-8.1.15.el5.centos.plusxen ro
> root=/dev/md0 console=ttyS0 console=ttyS0,9600n8 console=tty0
> xencons=tty
>         module /boot/initrd-2.6.18-8.1.15.el5.centos.plusxen.img
> title CentOS (2.6.18-8.1.15.el5.centos.plus)
>         root (hd0,0)
>         kernel /boot/vmlinuz-2.6.18-8.1.15.el5.centos.plus ro
> root=/dev/md0 console=ttyS0 console=ttyS0,9600n8 console=tty0
> xencons=tty
>         initrd /boot/initrd-2.6.18-8.1.15.el5.centos.plus.img
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
-- 
 Daniel Mealha Cabrita
 Divisao de Suporte Tecnico
 AINFO / Reitoria / UTFPR
 http://www.utfpr.edu.br

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Sadique Puthen

2007-Nov-28 10:43 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Daniel Mealha Cabrita wrote:> It looks like you''re using a 32-bit Xen kernel without compiled
PAE support.
>   
I don''t think so from the output of "uname -a".  But it looks
strange.
 
--Sadique> Also looks like your non-Xen kernel is a 32-bit one with PAE support.
>
> Em Qua 28 Nov 2007 02:47, Chuck McIntyre escreveu:
>   
>> Hi,
>>
>> I am having a problem where Xen is not seeing the appropriate amount
>> of memory allocated. At first I thought this was similar to previously
>> described issues with the BIOS remapped hole for PCI devices (such as
>> in a post here
>>
http://lists.xensource.com/archives/html/xen-users/2005-07/msg00177.html),
>> however, we only see about 3.5GB (see below) of RAM regardless of
>> whether we have a system with 4GB of RAM or 8GB of RAM (as seen by the
>> BIOS). Regardless, when we boot into the non-xen kernel available on
>> the same box, the OS then sees all the available RAM.
>>
>> I have tried to provide as much relevant information as I could
>> without being spammy. Please let me know if I need to provide
>> additional details. I would appreciate any help here, as this one has
>> us stumped, or do we have a bug with our hardware version?
>>
>> __4GB system___
>>
>> [me@molotov ~]$ uname -a
>> Linux molotov.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
>> 25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
>> [me@molotov ~]$ rpm -qa | grep xen
>> xen-3.0.3-25.0.4.el5
>> xen-libs-3.0.3-25.0.4.el5
>> xen-libs-3.0.3-25.0.4.el5
>> kernel-xen-2.6.18-8.1.15.el5.centos.plus
>> [me@molotov ~]$ grep MemTotal /proc/meminfo
>> MemTotal:      3428400 kB
>> top gives me:
>> Mem:   3428400k total,   422916k used,  3005484k free,    21648k
buffers
>> [root@molotov ~]# xm info 2>&1 | grep memory
>> total_memory           : 3535
>> free_memory            : 129
>> xentop gives me:
>> Mem: 3620404k total, 3488052k used, 132352k free    CPUs: 4 @ 2200MHz
>> dmesg
>> BIOS-provided physical RAM map:
>>  Xen: 0000000000000000 - 00000000d1d4c000 (usable)
>> On node 0 totalpages: 859468
>>   DMA zone: 859468 pages, LIFO batch:31
>> DMI present.
>>
>> -=After reboot=-
>>
>> BIOS gives me:
>> Initializing USB Controllers .. Done.
>> 4096MB OK
>> [me@molotov ~]$ grep MemTotal /proc/meminfo
>> MemTotal:      4050520 kB
>> top gives
>> Mem:   4050520k total,   264668k used,  3785852k free,    20612k
buffers
>> dmesg
>> BIOS-provided physical RAM map:
>>  BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
>>  BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
>>  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
>>  BIOS-e820: 0000000000100000 - 00000000dcff0000 (usable)
>>  BIOS-e820: 00000000dcff0000 - 00000000dcffe000 (ACPI data)
>>  BIOS-e820: 00000000dcffe000 - 00000000dd000000 (ACPI NVS)
>>  BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
>>  BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
>>  BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
>>  BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
>>  BIOS-e820: 0000000100000000 - 0000000123000000 (usable)
>> DMI present.
>> Bootmem setup node 0 0000000000000000-0000000080000000
>> Bootmem setup node 1 0000000080000000-0000000123000000
>> On node 0 totalpages: 515808
>>   DMA zone: 2728 pages, LIFO batch:0
>>   DMA32 zone: 513080 pages, LIFO batch:31
>> On node 1 totalpages: 515144
>>   DMA32 zone: 373744 pages, LIFO batch:31
>>   Normal zone: 141400 pages, LIFO batch:31
>>
>>
>>
>> __ 8GB System __
>>
>> [me@molotov ~]$ uname -a
>> Linux benelli.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
>> 25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
>> [me@molotov ~]$ rpm -qa |  grep xen
>> kernel-xen-2.6.18-8.1.15.el5.centos.plus
>> xen-libs-3.0.3-25.0.4.el5
>> xen-libs-3.0.3-25.0.4.el5
>> xen-3.0.3-25.0.4.el5
>> [me@benelli ~]$ grep MemTotal /proc/meminfo
>> MemTotal:      3428336 kB
>> top gives me:
>> Mem:   3428336k total,   437888k used,  2990448k free,    16580k
buffers
>> [root@benelli ~]# xm info 2>&1| grep memory
>> total_memory           : 3535
>> free_memory            : 0
>> xentop gives me:
>> Mem: 3620336k total, 3620064k used, 272k free    CPUs: 4 @ 2200MHz
>> dmesg
>> BIOS-provided physical RAM map:
>>  Xen: 0000000000000000 - 00000000d1d3c000 (usable)
>> DMI present.
>>
>> -=After Reboot=-
>>
>> BIOS gives me:
>> Initializing USB Controllers .. Done.
>> 8192MB OK
>>
>> After reboot into non-xen kernel:
>>
>> [root@benelli ~]# grep MemTotal /proc/meminfo
>> MemTotal:      8181124 kB
>> Top gives
>> Mem:   8181124k total,   284732k used,  7896392k free,    16944k
buffers
>> dmesg
>> BIOS-provided physical RAM map:
>>  BIOS-e820: 0000000000000000 - 000000000009c400 (usable)
>>  BIOS-e820: 000000000009c400 - 00000000000a0000 (reserved)
>>  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
>>  BIOS-e820: 0000000000100000 - 00000000dcfe0000 (usable)
>>  BIOS-e820: 00000000dcfe0000 - 00000000dcfee000 (ACPI data)
>>  BIOS-e820: 00000000dcfee000 - 00000000dcff0000 (ACPI NVS)
>>  BIOS-e820: 00000000dcff0000 - 00000000dcffe000 (reserved)
>>  BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
>>  BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
>>  BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
>>  BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
>>  BIOS-e820: 0000000100000000 - 0000000223000000 (usable)
>> DMI present.
>> Bootmem setup node 0 0000000000000000-0000000123000000
>> Bootmem setup node 1 0000000123000000-0000000223000000
>> On node 0 totalpages: 1030931
>>   DMA zone: 2723 pages, LIFO batch:0
>>   DMA32 zone: 886808 pages, LIFO batch:31
>>   Normal zone: 141400 pages, LIFO batch:31
>> On node 1 totalpages: 1034240
>>   Normal zone: 1034240 pages, LIFO batch:31
>>
>>
>> __Both Systems__
>>
>> Here is grub.conf (equivalent on both systems):
>> default=0
>> timeout=5
>> serial --unit=0 --speed=9600
>> terminal --timeout=5 serial console
>> title CentOS (2.6.18-8.1.15.el5.centos.plusxen)
>>         root (hd0,0)
>>         kernel /boot/xen.gz-2.6.18-8.1.15.el5.centos.plus console=vga
>>         module /boot/vmlinuz-2.6.18-8.1.15.el5.centos.plusxen ro
>> root=/dev/md0 console=ttyS0 console=ttyS0,9600n8 console=tty0
>> xencons=tty
>>         module /boot/initrd-2.6.18-8.1.15.el5.centos.plusxen.img
>> title CentOS (2.6.18-8.1.15.el5.centos.plus)
>>         root (hd0,0)
>>         kernel /boot/vmlinuz-2.6.18-8.1.15.el5.centos.plus ro
>> root=/dev/md0 console=ttyS0 console=ttyS0,9600n8 console=tty0
>> xencons=tty
>>         initrd /boot/initrd-2.6.18-8.1.15.el5.centos.plus.img
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>     
>
>   

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Chuck McIntyre

2007-Nov-28 12:13 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

On Nov 27, 2007 9:24 PM, Daniel Mealha Cabrita <dancab@utfpr.edu.br>
wrote:>
> It looks like you''re using a 32-bit Xen kernel without compiled
PAE support.
> Also looks like your non-Xen kernel is a 32-bit one with PAE support.
definitely not a 32bit kernel, I realize my copy/pastes were odd
before, so here are the revelvant bits again (munged only to hide
hostnames/usernames):

8GB system
[me@benelli ~]$ uname -a
Linux benelli.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[me@benelli ~]$ yum list *xen*
Loading "installonlyn" plugin
Setting up repositories
Reading repository metadata in from local files
Installed Packages
kernel-xen.x86_64                        2.6.18-8.1.15.el5.cent installed
xen.x86_64                               3.0.3-25.0.4.el5       installed
xen-libs.i386                            3.0.3-25.0.4.el5       installed
xen-libs.x86_64                          3.0.3-25.0.4.el5       installed

4GB system
[me@molotov ~]$ uname -a
Linux molotov.mydomain 2.6.18-8.1.15.el5.centos.plus #1 SMP Thu Oct 25
06:48:46 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[me@molotov ~]$ yum list *xen*
Loading "installonlyn" plugin
Setting up repositories
Reading repository metadata in from local files
Installed Packages
kernel-xen.x86_64                        2.6.18-8.1.15.el5.cent installed
xen.x86_64                               3.0.3-25.0.4.el5       installed
xen-libs.i386                            3.0.3-25.0.4.el5       installed
xen-libs.x86_64                          3.0.3-25.0.4.el5       installed

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Chuck McIntyre

2007-Nov-28 12:19 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

On Nov 28, 2007 4:13 AM, Chuck McIntyre <chuck.mcintyre@gmail.com>
wrote:> On Nov 27, 2007 9:24 PM, Daniel Mealha Cabrita <dancab@utfpr.edu.br>
wrote:
> >
> > It looks like you''re using a 32-bit Xen kernel without
compiled PAE support.
> > Also looks like your non-Xen kernel is a 32-bit one with PAE support.
>
> definitely not a 32bit kernel, I realize my copy/pastes were odd
> before, so here are the revelvant bits again (munged only to hide
> hostnames/usernames):
[SNIP]
> 4GB system
> [me@molotov ~]$ uname -a
> Linux molotov.mydomain 2.6.18-8.1.15.el5.centos.plus #1 SMP Thu Oct 25
> 06:48:46 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
>
And the xen kernel version on the 4GB system:

[me@molotov ~]$ uname -a
Linux molotov.corp.responsys.com 2.6.18-8.1.15.el5.centos.plusxen #1
SMP Thu Oct 25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Rafał Kupka

2007-Nov-28 13:24 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

On Wed, Nov 28, 2007 at 04:13:54AM -0800, Chuck McIntyre
wrote:> On Nov 27, 2007 9:24 PM, Daniel Mealha Cabrita <dancab@utfpr.edu.br>
wrote:Hello,
> > It looks like you''re using a 32-bit Xen kernel without
compiled PAE support.
> > Also looks like your non-Xen kernel is a 32-bit one with PAE support.
> 
> definitely not a 32bit kernel, I realize my copy/pastes were odd
> before, so here are the revelvant bits again (munged only to hide
> hostnames/usernames):
Could you send e820 memory map as seen by Xen?
$ xm dmesg
lines after "Xen-e820 RAM map:"

Did grub command displaymem shows all memory?

Kupson
-- 
Great software without the knowledge to run it is pretty useless.
(Linux Gazette #1)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Chuck McIntyre

2007-Nov-28 18:41 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

On Nov 28, 2007 5:24 AM, Rafał Kupka <rkupka+Listy.Xen@pronet.com.pl>
wrote:> Could you send e820 memory map as seen by Xen?
> $ xm dmesg
> lines after "Xen-e820 RAM map:"
>
Here is the output form the 4GB system

(XEN) Physical RAM map:
(XEN)  0000000000000000 - 000000000009dc00 (usable)
(XEN)  0000000000100000 - 00000000dcff0000 (usable)
(XEN) System RAM: 3535MB (3620404kB)
(XEN) Xen heap: 14MB (14340kB)
(XEN) found SMP MP-table at 000ff780
(XEN) DMI present.
> Did grub command displaymem shows all memory?
No, grub displaymem shows:

grub> displaymem
 EISA Memory BIOS Interface is present
 Address Map BIOS Interface is present
 Lower memory: 631K, Upper memory (to first chipset hole): 3619776K

grub>

This is again from the 4GB system. Do you need me to provide the same
details for the 8GB system? I can but it's more difficult as it is
nominally in use as a test system for someone right now (albeit not
running as many images as we would like).

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Boudreau Luc

2007-Nov-28 19:17 UTC

head link

[Xen-users] IRQ conflict between dom0 and domU

Hi,

I''ve had trouble lately with network hangs on a CentOS 5 dom0/domU.
After more research, I discovered a conflict between a PCI card
dedicated to a domU using IRQ 16 and my dom0 which also used the IRQ 16
for the usb hub. I''d like correct this but I don''t know where
to start.

What are my options to make the domU use the card on another IRQ ??

Thanks..

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

[STD]Ein

2007-Nov-29 00:53 UTC

head link

RE: [Xen-users] IRQ conflict between dom0 and domU

Add a ''metoo'' to that.

It''s amazing that modern hardware sporting ioapic''s still
manage to bind
multiple devices to the same irq, my machine at one point had two pci
nic''s,
onboard audio, onboard usb, and smbus all on the same irq.  As you can
imagine, it was very unstable.  Still looking for a proper solution myself,
disabling audio and usb and moving one of my nic''s reduced the sharing
to a
level that brought back system stability, but I''d really rather have
all
that back.

Also, I''ve found sharing irq''s between dom0 and domU (and
likely domU-domU)
leads to massive instability (at least in my experience.. with one machine),
so switching the pci slot to move the domU nic onto a different irq (and
probably forcing sharing for another device) might help you.

-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Boudreau Luc
Sent: November 28, 2007 1:18 PM
To: xen-users@lists.xensource.com
Subject: [Xen-users] IRQ conflict between dom0 and domU

Hi,

I''ve had trouble lately with network hangs on a CentOS 5 dom0/domU.
After more research, I discovered a conflict between a PCI card
dedicated to a domU using IRQ 16 and my dom0 which also used the IRQ 16
for the usb hub. I''d like correct this but I don''t know where
to start.

What are my options to make the domU use the card on another IRQ ??

Thanks..

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Rafał Kupka

2007-Nov-29 10:09 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

On Wed, Nov 28, 2007 at 10:41:17AM -0800, Chuck McIntyre
wrote:> On Nov 28, 2007 5:24 AM, Rafał Kupka <rkupka+Listy.Xen@pronet.com.pl>
wrote:Hello,
> > Could you send e820 memory map as seen by Xen?
> > $ xm dmesg
> > lines after "Xen-e820 RAM map:"
> >
> 
> Here is the output form the 4GB system
> 
> (XEN) Physical RAM map:
> (XEN)  0000000000000000 - 000000000009dc00 (usable)
> (XEN)  0000000000100000 - 00000000dcff0000 (usable)
> (XEN) System RAM: 3535MB (3620404kB)
> (XEN) Xen heap: 14MB (14340kB)
> (XEN) found SMP MP-table at 000ff780
> (XEN) DMI present.
Xen gets memory map from grub, unlike linux[1].
> > Did grub command displaymem shows all memory?
> 
> No, grub displaymem shows:
> 
> grub> displaymem
>  EISA Memory BIOS Interface is present
>  Address Map BIOS Interface is present
>  Lower memory: 631K, Upper memory (to first chipset hole): 3619776K
Yes, it''s this strange bug in grub I spot some time ago.
Please look at Debian bug #419994:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=419994

Can you try patch (with volatile unsigned long cont) and tell if it make
any difference?[2]
> This is again from the 4GB system. Do you need me to provide the same
> details for the 8GB system? I can but it''s more difficult as it is
> nominally in use as a test system for someone right now (albeit not
> running as many images as we would like).
It''s probably very same problem here.

Kupson
[1] But linux and grub both use int 15h...
[2] Make sure that files in /boot/grub directory are new, some systems
(Debian) don''t update it during grub package update.
-- 
Great software without the knowledge to run it is pretty useless.
(Linux Gazette #1)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Chuck McIntyre

2007-Nov-30 19:40 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

On Nov 29, 2007 2:09 AM, Rafał Kupka <rkupka+Listy.Xen@pronet.com.pl>
wrote:> Hello,
Hi!

Thanks for your help, see my responses below.
> > (XEN) Physical RAM map:
> > (XEN)  0000000000000000 - 000000000009dc00 (usable)
> > (XEN)  0000000000100000 - 00000000dcff0000 (usable)
> > (XEN) System RAM: 3535MB (3620404kB)
> > (XEN) Xen heap: 14MB (14340kB)
> > (XEN) found SMP MP-table at 000ff780
> > (XEN) DMI present.
>
> Xen gets memory map from grub, unlike linux[1].
Do you have a reference for this? Is this intentional? My
understanding is that grub doesn't have any way to map around PCI BIOS
created RAM holes because it is such a low level (boot) piece of
software. Is this, then, a bug? Is there a way around it?
> Yes, it's this strange bug in grub I spot some time ago.
> Please look at Debian bug #419994:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=419994
I'm not sure it's a grub bug though, since it works correctly in a
non-xen kernel.

Should I try a different boot loader?
> Can you try patch (with volatile unsigned long cont) and tell if it make
> any difference?[2]
Will try this, and see, just to be sure, in the meantime, if you could
provide reference for your assertion that Xen uses the boot loader to
get its RAM map, that would be very helpful.

Thanks again,
Chuck

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Rafał Kupka

2007-Nov-30 21:07 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

On Fri, Nov 30, 2007 at 11:40:52AM -0800, Chuck McIntyre
wrote:> On Nov 29, 2007 2:09 AM, Rafał Kupka <rkupka+Listy.Xen@pronet.com.pl>
wrote:Hello,
> Thanks for your help, see my responses below.
> 
> > > (XEN) Physical RAM map:
> > > (XEN)  0000000000000000 - 000000000009dc00 (usable)
> > > (XEN)  0000000000100000 - 00000000dcff0000 (usable)
> > > (XEN) System RAM: 3535MB (3620404kB)
> > > (XEN) Xen heap: 14MB (14340kB)
> > > (XEN) found SMP MP-table at 000ff780
> > > (XEN) DMI present.
> >
> > Xen gets memory map from grub, unlike linux[1].
> 
> Do you have a reference for this? Is this intentional?
Yes, it''s intentional. Xen hypervisor (unlike linux) uses GRUB
multiboot
protocol instead of custom one.

http://www.gnu.org/software/grub/manual/multiboot/multiboot.html.gz#Boot-information-format
Look at description of mmap_* fields.
>    My
> understanding is that grub doesn''t have any way to map around PCI
BIOS
> created RAM holes because it is such a low level (boot) piece of
> software. Is this, then, a bug? Is there a way around it?
The whole e820 memory map is a BIOS thing.
http://www.uruk.org/orig-grub/mem64mb.html 
> > Yes, it''s this strange bug in grub I spot some time ago.
> > Please look at Debian bug #419994:
> > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=419994
> 
> I''m not sure it''s a grub bug though, since it works
correctly in a
> non-xen kernel.
Non Xen kernel gets memory map differently, as I said before.
> Should I try a different boot loader?
Not much choice to boot Xen. AFAIK it''s grub or grub2.
Don''t know if grub2 is affected.
> > Can you try patch (with volatile unsigned long cont) and tell if it
make
> > any difference?[2]
> 
> Will try this, and see, just to be sure, in the meantime, if you could
> provide reference for your assertion that Xen uses the boot loader to
> get its RAM map, that would be very helpful.
Look at Xen hypervisor source.

Kupson
-- 
Great software without the knowledge to run it is pretty useless.
(Linux Gazette #1)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Robert A. Klahn

2007-Nov-30 22:18 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Chuck:

If you are running into this problem under Fedora, can you please
mention it in this bug report:

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=250299

If enough people pile on, maybe this will get fixed in a future release.
:)

Thanks....Bob.


On Fri, 30 Nov 2007 16:54:33 -0500 (EST),
xen-users-request@lists.xensource.com said:> Message: 1
> Date: Fri, 30 Nov 2007 11:40:52 -0800
> From: "Chuck McIntyre" <chuck.mcintyre@gmail.com>
> Subject: Re: [Xen-users] Xen 3 not seeing the right amount of RAM on
> 	x86_64
> To: xen-users@lists.xensource.com
> Message-ID:
> 	<b62bf9fa0711301140i1d71cc6bs538af801798885d3@mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-2
> 
> On Nov 29, 2007 2:09 AM, Rafa³ Kupka <rkupka+Listy.Xen@pronet.com.pl>
> wrote:
> > Hello,
> 
> Hi!
> 
> Thanks for your help, see my responses below.
> 
> > > (XEN) Physical RAM map:
> > > (XEN)  0000000000000000 - 000000000009dc00 (usable)
> > > (XEN)  0000000000100000 - 00000000dcff0000 (usable)
> > > (XEN) System RAM: 3535MB (3620404kB)
> > > (XEN) Xen heap: 14MB (14340kB)
> > > (XEN) found SMP MP-table at 000ff780
> > > (XEN) DMI present.
> >
> > Xen gets memory map from grub, unlike linux[1].
> 
> Do you have a reference for this? Is this intentional? My
> understanding is that grub doesn''t have any way to map around PCI
BIOS
> created RAM holes because it is such a low level (boot) piece of
> software. Is this, then, a bug? Is there a way around it?
> 
> > Yes, it''s this strange bug in grub I spot some time ago.
> > Please look at Debian bug #419994:
> > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=419994
> 
> I''m not sure it''s a grub bug though, since it works
correctly in a
> non-xen kernel.
> 
> Should I try a different boot loader?
> 
> > Can you try patch (with volatile unsigned long cont) and tell if it
make
> > any difference?[2]
> 
> Will try this, and see, just to be sure, in the meantime, if you could
> provide reference for your assertion that Xen uses the boot loader to
> get its RAM map, that would be very helpful.
> 
> Thanks again,
> Chuck
> -----------------------------------------------------------
Robert A. Klahn         robert@kint.org         AIM: rklahn

"Hope has two beautiful daughters: Anger and Courage. Anger
 at the way things are, and Courage to struggle to create 
 things as they should be." -- St. Augustine


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Chuck McIntyre

2007-Nov-30 23:14 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

On Nov 30, 2007 1:07 PM, Rafał Kupka <rkupka+Listy.Xen@pronet.com.pl>
wrote:> Yes, it's intentional. Xen hypervisor (unlike linux) uses GRUB
multiboot
> protocol instead of custom one.
>
>
http://www.gnu.org/software/grub/manual/multiboot/multiboot.html.gz#Boot-information-format
> Look at description of mmap_* fields.
Thanks this is excellent information, and definitely puts me on the track.

Does this seem at all, unusual, to you? To me it sounds like it was by
design, but it is clearly buggy (from the perspective of those of us
trying to use hardware virtualization on machines which have this
problem). I wonder if this is going to change at some point.

In any case, I will take your advice and try to patch grub to see if
it fixes my problem. Will post an update here with my progress.

Thanks again,
Chuck

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Chuck McIntyre

2007-Dec-03 18:15 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Hi folks,

Just a quick update to this issue, which seems to be resolved (will
send out more information once we''ve tried the solution on other
systems as well). Originally we were running the following Xen
version:

xen-3.0.3-25.0.4.el5
xen-libs-3.0.3-25.0.4.el5
xen-libs-3.0.3-25.0.4.el5
kernel-xen-2.6.18-8.1.15.el5.centos.plus

But were unable to see the correct RAM within Xen (3.5GB on a 4GB
system, for example) on 64bit CentOS linux with hardware
virtualization via the AMD Opteron 2200 chipset.

It was suggested that we try patching grub or upgrading to grub2,
since Xen appears to get its memory information from grub, it was also
suggested we try running the latest Xen.

As a first step we grabbed xen 3.1.2-pre and applied it to one of the
4GB systems. Suddenly everything worked!

[root@bouncingbetty ~]# uname -a
Linux bouncingbetty.mydomain 2.6.18-xen #1 SMP Fri Nov 30 17:48:44 PST
2007 x86_64 x86_64 x86_64 GNU/Linux
[root@bouncingbetty ~]# xm info | grep -i total_memory
total_memory           : 4095>From xm top:Mem: 4193844k total, 4190068k used, 3776k free    CPUs: 4 @ 2200MHz

We are going to try to apply this to one of our 8GB systems today, to
see if this fixes systems across the board.

As an added bonus, this seems to have also fixed the time/ntp issues
we were having in hardware virtualized linux.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Chuck McIntyre

2007-Dec-04 21:09 UTC

head link

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

I just confirmed, all my issues with both RAM and time (which I hadn''t
even gotten around to posting yet) seem to have been resolved by the
latest Xen (3.1.2).

On Dec 3, 2007 10:15 AM, Chuck McIntyre <chuck.mcintyre@gmail.com>
wrote:> Hi folks,
>
> Just a quick update to this issue, which seems to be resolved (will
> send out more information once we''ve tried the solution on other
> systems as well). Originally we were running the following Xen
> version:
>
> xen-3.0.3-25.0.4.el5
> xen-libs-3.0.3-25.0.4.el5
> xen-libs-3.0.3-25.0.4.el5
> kernel-xen-2.6.18-8.1.15.el5.centos.plus
>
> But were unable to see the correct RAM within Xen (3.5GB on a 4GB
> system, for example) on 64bit CentOS linux with hardware
> virtualization via the AMD Opteron 2200 chipset.
>
> It was suggested that we try patching grub or upgrading to grub2,
> since Xen appears to get its memory information from grub, it was also
> suggested we try running the latest Xen.
>
> As a first step we grabbed xen 3.1.2-pre and applied it to one of the
> 4GB systems. Suddenly everything worked!
>
> [root@bouncingbetty ~]# uname -a
> Linux bouncingbetty.mydomain 2.6.18-xen #1 SMP Fri Nov 30 17:48:44 PST
> 2007 x86_64 x86_64 x86_64 GNU/Linux
> [root@bouncingbetty ~]# xm info | grep -i total_memory
> total_memory           : 4095
> From xm top:
> Mem: 4193844k total, 4190068k used, 3776k free    CPUs: 4 @ 2200MHz
>
> We are going to try to apply this to one of our 8GB systems today, to
> see if this fixes systems across the board.
>
> As an added bonus, this seems to have also fixed the time/ntp issues
> we were having in hardware virtualized linux.
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Xen users - Nov 2007 - Xen 3 not seeing the right amount of RAM on x86_64

[Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

[Xen-users] IRQ conflict between dom0 and domU

RE: [Xen-users] IRQ conflict between dom0 and domU

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

Re: [Xen-users] Xen 3 not seeing the right amount of RAM on x86_64