Xen developers: My employers are interested in deploying large (specifically Sun Fire V40z) systems for use as virtual private server hosting platforms using Xen. However, in playing with Xen on another Opteron based system (with only 5 GB of RAM), I noticed that the PAE-enabled Xen build claims a 16 GB RAM ceiling. Some poking around in the source of the hyperkernel (specifically xen/include/asm/config.h) revealed that MACHPHYS_MBYTES seems to be the key limiting value, capping the system RAM at 16 gigabytes. Can this safely be increased? From my reading of the code, it appears that it is set as it is because each gigabyte of installed RAM requires a megabyte of RAM to be set aside for mapping tables and such. If I just increase the value of MACHPHYS_MBYTES to 32 (for example), should Xen boot and work happily, just consuming more RAM for its mapping tables? Please Cc: me with your reply, as I''m not subscribed to this list currently. -- Derrik Pates demon@devrandom.net _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 6 Jan 2006, at 22:21, Derrik Pates wrote:> Can this safely be increased? From my reading of the code, it appears > that it is set as it is because each gigabyte of installed RAM requires > a megabyte of RAM to be set aside for mapping tables and such. If I > just > increase the value of MACHPHYS_MBYTES to 32 (for example), should Xen > boot and work happily, just consuming more RAM for its mapping tables? > > Please Cc: me with your reply, as I''m not subscribed to this list > currently.The memory layout of Xen is hardcoded in asm-x86/config.h, and also in public/arch-x86_32.h. If the addresses of certain critical structures are modified appropriately then things should work fine. Of course Linux will end up with less kernel virtual address space and hence less lowmem. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > Keir Fraser > Sent: 09 January 2006 10:30 > To: Derrik Pates > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Xen 3.0 x86 + PAE with > 16 GB RAM? > > > On 6 Jan 2006, at 22:21, Derrik Pates wrote: > > > Can this safely be increased? From my reading of the code, > it appears > > that it is set as it is because each gigabyte of installed RAM > > requires a megabyte of RAM to be set aside for mapping tables and > > such. If I just increase the value of MACHPHYS_MBYTES to 32 (for > > example), should Xen boot and work happily, just consuming more RAM > > for its mapping tables? > > > > Please Cc: me with your reply, as I''m not subscribed to this list > > currently. > > The memory layout of Xen is hardcoded in asm-x86/config.h, > and also in public/arch-x86_32.h. If the addresses of certain > critical structures are modified appropriately then things > should work fine. Of course Linux will end up with less > kernel virtual address space and hence less lowmem.If I understand things right, the V40z is an AMD Opteron based system, which means that you could run x86-64. Is there any particular reason you don''t want to do this? This would give you 63TB (or is that 63-7 56TB?) or addressable memory, with full 32-bit compatibility for user-mode code. You''d obviously have to run the Guest Kernels in 64-bit mode as well... I''m not aware of any reason why this shouldn''t be a "better" solution? -- Mats> > -- Keir > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Petersson, Mats wrote:> If I understand things right, the V40z is an AMD Opteron based system, > which means that you could run x86-64.This is correct, yes.> Is there any particular reason > you don''t want to do this? This would give you 63TB (or is that 63-7 > 56TB?) or addressable memory, with full 32-bit compatibility for > user-mode code. You''d obviously have to run the Guest Kernels in 64-bit > mode as well...I''m considering this as well, but it''s not quite so simple when running a 32-bit guest OS - 64-bit guests are obviously a non-issue, but 32-bit OSes need several userspace tools replaced with equivalents that understand certain internal structures of the 64-bit kernel. I haven''t decided yet if the necessary hacking will be worthwhile to modify the userspace tools for several OSes, or tweak the hypervisor (and apparently the guest memory layouts as well, based on Keir''s statements) and use that instead.> I''m not aware of any reason why this shouldn''t be a "better" solution?I''ll agree it should be a higher-performance solution, at least, but I have to make sure it all works right, as well... :) The fact is the hardware in question hasn''t even arrived yet, so it''s something I still have to look into. -- Derrik Pates demon@devrandom.net _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> -----Original Message----- > From: Derrik Pates [mailto:demon@devrandom.net] > Sent: 09 January 2006 13:24 > To: Petersson, Mats > Cc: Keir Fraser; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] Xen 3.0 x86 + PAE with > 16 GB RAM? > > Petersson, Mats wrote: > > If I understand things right, the V40z is an AMD Opteron > based system, > > which means that you could run x86-64. > > This is correct, yes. > > > Is there any > particular reason > > you don''t want to do this? This would give you 63TB (or is > that 63-7 > > 56TB?) or addressable memory, with full 32-bit compatibility for > > user-mode code. You''d obviously have to run the Guest Kernels in > > 64-bit mode as well... > > I''m considering this as well, but it''s not quite so simple > when running a 32-bit guest OS - 64-bit guests are obviously > a non-issue, but 32-bit OSes need several userspace tools > replaced with equivalents that understand certain internal > structures of the 64-bit kernel. I haven''t decided yet if the > necessary hacking will be worthwhile to modify the userspace > tools for several OSes, or tweak the hypervisor (and > apparently the guest memory layouts as well, based on Keir''s > statements) and use that instead. >Hmm, so these are some custom tools? Or something that normally is distributed with the OS distribution you''re planning to use? All software distributed with the OS will, of course, be compiled to fit the kernel - whether it''s in 32-bit or 64-bit. Also, any tools not included in the distribution would be perfectly fine to run in 32-bit mode [with the relevant 32-bit restrictions, but that''s the same if you''re running PAE], as all OS interfaces that are available to the 32-bit app within a 32-bit OS are also made available, through a thin translation layer, in the 64-bit OS. As far as I''m aware, running 32-bit apps/tools will work exactly the same in 64-bit OS as it does in 32-bit OS. I''m sure there are some, rare, special cases where this is not true, but I''m not aware of any at this time. -- Mats> > I''m not aware of any reason why this shouldn''t be a > "better" solution? > > I''ll agree it should be a higher-performance solution, at > least, but I have to make sure it all works right, as well... > :) The fact is the hardware in question hasn''t even arrived > yet, so it''s something I still have to look into. > > -- > Derrik Pates > demon@devrandom.net > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel