Christopher S. Aker
2006-Apr-25 21:28 UTC
[Xen-devel] Yet another "Why can''t I see all my RAM" question
I know this has been asked a million times, and the typical answer is: that specific area of unallocable memory is used for ACPI data and framebuffers. However, I''ve got two identical machines, one running Xen-PAE, one running vanilla 2.6.16: Vanilla: MemTotal: 16623588 kB Xen(PAE) total_memory: 14336 (MB) I was a little disappointed to discover that my investment in RAM is going to be underutilized in Xen. I would think if vanilla Linux can utilize all the available RAM that Xen would be able to, also. (XEN) WARNING: Only the first 16 GB of the physical memory map can be accessed (XEN) by Xen in 32-bit mode. Truncating the memory map... (XEN) Physical RAM map: (XEN) 0000000000000000 - 000000000009fc00 (usable) (XEN) 000000000009fc00 - 00000000000a0000 (reserved) (XEN) 00000000000e8000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 000000007fff0000 (usable) (XEN) 000000007fff0000 - 000000007ffff000 (ACPI data) (XEN) 000000007ffff000 - 0000000080000000 (ACPI NVS) (XEN) 00000000ff780000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000000400000000 (usable) (XEN) System RAM: 14335MB (14679612kB) (XEN) Xen heap: 9MB (10044kB) (XEN) Using scheduler: Simple EDF Scheduler (sedf) (XEN) PAE enabled, limit: 16 GB (XEN) found SMP MP-table at 000ff780 (XEN) DMI 2.3 present. (XEN) Using APIC driver default (XEN) ACPI: RSDP (v002 ACPIAM ) @ 0x000f9760 (XEN) ACPI: XSDT (v001 A M I OEMXSDT 0x08000508 MSFT 0x00000097) @ 0x7fff0100 (XEN) ACPI: FADT (v003 A M I OEMFACP 0x08000508 MSFT 0x00000097) @ 0x7fff0281 (XEN) ACPI: MADT (v001 A M I OEMAPIC 0x08000508 MSFT 0x00000097) @ 0x7fff0380 (XEN) ACPI: SPCR (v001 A M I OEMSPCR 0x08000508 MSFT 0x00000097) @ 0x7fff0410 (XEN) ACPI: OEMB (v001 A M I OEMBIOS 0x08000508 MSFT 0x00000097) @ 0x7ffff040 (XEN) ACPI: SRAT (v001 A M I OEMSRAT 0x08000508 MSFT 0x00000097) @ 0x7fff3550 (XEN) ACPI: DSDT (v001 H8DAR H8DAR010 0x00000000 INTL 0x02002026) @ 0x00000000 ... (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000007800000->0000000008000000 (63488 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: c0100000->c06152e4 (XEN) Init. ramdisk: c0616000->c0616000 (XEN) Phys-Mach map: c0616000->c0656000 (XEN) Start info: c0656000->c0657000 (XEN) Page tables: c0657000->c0660000 (XEN) Boot stack: c0660000->c0661000 (XEN) TOTAL: c0000000->c0800000 (XEN) ENTRY ADDRESS: c0100000 So what''s the scoop here? Has anyone actually been able to use the full 16GB in PAE mode? Is there something I can tweak to get that final 1GB back? Thanks, -Chris _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt
2006-Apr-25 21:43 UTC
RE: [Xen-devel] Yet another "Why can''t I see all my RAM" question
> I know this has been asked a million times, and the typical > answer is: > that specific area of unallocable memory is used for ACPI > data and framebuffers. However, I''ve got two identical > machines, one running Xen-PAE, one running vanilla 2.6.16: > > Vanilla: > MemTotal: 16623588 kB > > Xen(PAE) > total_memory: 14336 (MB) > > I was a little disappointed to discover that my investment in > RAM is going to be underutilized in Xen. I would think if > vanilla Linux can utilize all the available RAM that Xen > would be able to, also.PAE Xen currently won''t see memory mapped above 16GB. Native Linux doesn''t have this restriction, it sees memory up to 64GB. It would actually be fairly easy to make the Xen restriction "16GB plus a bit" so that you''d get the full 16GB of RAM even with a large MMIO whole in the address space. We might implement that, but using a 64b hypervisor is definitely the preferred way forward. Best, Ian> (XEN) WARNING: Only the first 16 GB of the physical memory > map can be accessed > (XEN) by Xen in 32-bit mode. Truncating the memory map... > (XEN) Physical RAM map: > (XEN) 0000000000000000 - 000000000009fc00 (usable) > (XEN) 000000000009fc00 - 00000000000a0000 (reserved) > (XEN) 00000000000e8000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 000000007fff0000 (usable) > (XEN) 000000007fff0000 - 000000007ffff000 (ACPI data) > (XEN) 000000007ffff000 - 0000000080000000 (ACPI NVS) > (XEN) 00000000ff780000 - 0000000100000000 (reserved) > (XEN) 0000000100000000 - 0000000400000000 (usable) > (XEN) System RAM: 14335MB (14679612kB) > (XEN) Xen heap: 9MB (10044kB) > (XEN) Using scheduler: Simple EDF Scheduler (sedf) > (XEN) PAE enabled, limit: 16 GB > (XEN) found SMP MP-table at 000ff780 > (XEN) DMI 2.3 present. > (XEN) Using APIC driver default > (XEN) ACPI: RSDP (v002 ACPIAM > ) @ 0x000f9760 > (XEN) ACPI: XSDT (v001 A M I OEMXSDT 0x08000508 MSFT > 0x00000097) @ 0x7fff0100 > (XEN) ACPI: FADT (v003 A M I OEMFACP 0x08000508 MSFT 0x00000097) @ > 0x7fff0281 > (XEN) ACPI: MADT (v001 A M I OEMAPIC 0x08000508 MSFT > 0x00000097) @ 0x7fff0380 > (XEN) ACPI: SPCR (v001 A M I OEMSPCR 0x08000508 MSFT > 0x00000097) @ 0x7fff0410 > (XEN) ACPI: OEMB (v001 A M I OEMBIOS 0x08000508 MSFT > 0x00000097) @ 0x7ffff040 > (XEN) ACPI: SRAT (v001 A M I OEMSRAT 0x08000508 MSFT > 0x00000097) @ 0x7fff3550 > (XEN) ACPI: DSDT (v001 H8DAR H8DAR010 0x00000000 INTL > 0x02002026) @ 0x00000000 ... > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000007800000->0000000008000000 > (63488 pages to > be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: c0100000->c06152e4 > (XEN) Init. ramdisk: c0616000->c0616000 > (XEN) Phys-Mach map: c0616000->c0656000 > (XEN) Start info: c0656000->c0657000 > (XEN) Page tables: c0657000->c0660000 > (XEN) Boot stack: c0660000->c0661000 > (XEN) TOTAL: c0000000->c0800000 > (XEN) ENTRY ADDRESS: c0100000 > > So what''s the scoop here? Has anyone actually been able to > use the full 16GB in PAE mode? Is there something I can > tweak to get that final 1GB back? > > Thanks, > -Chris > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Christopher S. Aker
2006-Apr-25 23:54 UTC
Re: [Xen-devel] Yet another "Why can''t I see all my RAM" question
Ian Pratt wrote:>> I know this has been asked a million times, and the typical >> answer is: >> that specific area of unallocable memory is used for ACPI >> data and framebuffers. However, I''ve got two identical >> machines, one running Xen-PAE, one running vanilla 2.6.16: >> >> Vanilla: >> MemTotal: 16623588 kB >> >> Xen(PAE) >> total_memory: 14336 (MB) >> >> I was a little disappointed to discover that my investment in >> RAM is going to be underutilized in Xen. I would think if >> vanilla Linux can utilize all the available RAM that Xen >> would be able to, also. > > PAE Xen currently won''t see memory mapped above 16GB. Native Linux > doesn''t have this restriction, it sees memory up to 64GB. > > It would actually be fairly easy to make the Xen restriction "16GB plus > a bit" so that you''d get the full 16GB of RAM even with a large MMIO > whole in the address space. We might implement that, but using a 64b > hypervisor is definitely the preferred way forward.OK. The problem there is supporting all of our existing 32b clients without forcing them to upgrade libs, etc (nightmare). What''s more likely to be implemented first? Support for 32b(w/pae?) domains on 64b Xen, or this extra bit support? Also, any feeling for when either of those would be implemented? Thanks, -Chris _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt
2006-Apr-26 00:41 UTC
RE: [Xen-devel] Yet another "Why can''t I see all my RAM" question
> > It would actually be fairly easy to make the Xen restriction "16GB > > plus a bit" so that you''d get the full 16GB of RAM even > with a large > > MMIO whole in the address space. We might implement that, > but using a > > 64b hypervisor is definitely the preferred way forward. > > OK. The problem there is supporting all of our existing 32b > clients without forcing them to upgrade libs, etc (nightmare).You can use a 64b kernel on top of a 33b distro just fine. You also might want to see if there''s a BIOS upgrade for your system. I''ve never seen a memory hole anywhere near that size before (not that its usually a particular problem). Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel