Hi--I am currently running i386 version of xen 3.0.2 [root@fermigrid0 ~]# uname -a Linux fermigrid0.fnal.gov 2.6.16-xen3_86.1_rhel4.1 #1 SMP Thu Apr 13 06:52:14 PDT 2006 i686 i686 i386 GNU/Linux The machine previously had 4GB of RAM, and was booted with option dom0_mem=1024000 to ensure at least 1GB of RAM in dom0. I am also running five Xen instances using 500MB of RAM apiece. Today I upgraded the RAM to 12GB. I attempted to leave dom0 at 1GB and increase each of the Xen instances to 1GB for starters. What actually happened is by the time I tried to start the third 1GB instance I ran out of memory, it said there were only 168MB available. Is there something in this kernel that limits us to only 4GB of RAM? If so, what is the way around it? I can upgrade if necessary. (For that matter even before this, the sum of dom0 plus the 5 xen instances didn''t add up to the total amount of memory in the machine.) [root@fermigrid0 ~]# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 538 4 r----- 2694.1 fermigrid0-xen1 5 500 2 -b---- 1107.0 fermigrid0-xen2 6 500 2 -b---- 41.2 fermigrid0-xen3 7 500 2 -b---- 86.2 fermigrid0-xen4 8 500 2 -b---- 41.3 fermigrid0-xen5 9 500 2 r----- 42.1 Steve -- ------------------------------------------------------------------ Steven C. Timm, Ph.D (630) 840-8525 timm@fnal.gov http://home.fnal.gov/~timm/ Fermilab Computing Division, Scientific Computing Facilities, Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
You need to be running a PAE release of Xen to use more than 4Gig of memory. I''m guessing you''re not, right? I''m afraid that means upgrading both your Xen and you dom0 / domU kernels, which I realise is a pain. The RPMs of Xen 3.1 may provide what you need (plus an upgrade to the latest stable release of Xen). They''re compiled for PAE and are specifically for RHEL 4.5 although they''re not officially supported by anyone... The RedHat RHEL 4.5 upgrade includes a domU kernel which will also be PAE, so upgrading Xen to PAE will enable you to use the official RH kernels in the domUs. Cheers, Mark On Wednesday 27 June 2007, Steven Timm wrote:> Hi--I am currently running i386 version of xen 3.0.2 > [root@fermigrid0 ~]# uname -a > Linux fermigrid0.fnal.gov 2.6.16-xen3_86.1_rhel4.1 #1 SMP Thu Apr 13 > 06:52:14 PDT 2006 i686 i686 i386 GNU/Linux > > > The machine previously had 4GB of RAM, and was booted > with option dom0_mem=1024000 to ensure at least 1GB of RAM in > dom0. I am also running five Xen instances using 500MB of RAM apiece. > > Today I upgraded the RAM to 12GB. I attempted to leave dom0 at 1GB > and increase each of the Xen instances to 1GB for starters. > What actually happened is by the time I tried to start the > third 1GB instance I ran out of memory, it said there were only > 168MB available. > > Is there something in this kernel that limits us to only 4GB of RAM? > If so, what is the way around it? I can upgrade if necessary. > (For that matter even before this, the sum of dom0 plus the 5 xen > instances didn''t add up to the total amount of memory in the machine.) > > [root@fermigrid0 ~]# xm list > Name ID Mem(MiB) VCPUs State Time(s) > Domain-0 0 538 4 r----- 2694.1 > fermigrid0-xen1 5 500 2 -b---- 1107.0 > fermigrid0-xen2 6 500 2 -b---- 41.2 > fermigrid0-xen3 7 500 2 -b---- 86.2 > fermigrid0-xen4 8 500 2 -b---- 41.3 > fermigrid0-xen5 9 500 2 r----- 42.1 > > > > Steve-- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 6/27/07, Steven Timm <timm@fnal.gov> wrote:> > Hi--I am currently running i386 version of xen 3.0.2 > [root@fermigrid0 ~]# uname -a > Linux fermigrid0.fnal.gov 2.6.16-xen3_86.1_rhel4.1 #1 SMP Thu Apr 13 > 06:52:14 PDT 2006 i686 i686 i386 GNU/Linux> Is there something in this kernel that limits us to only 4GB of RAM? > If so, what is the way around it? I can upgrade if necessary.Yes, and that something is called PAE. IIRC, standard 32 bit kernels only support up to 4GB, and you need PAE enabled to reach up to 16GB on a 32bit system. Your uname looks like you compiled that kernel yourself. I would suggest recompiling it and make sure PAE is enabled (if that version even supports PAE) or use the PAE enabled kernel from the distro you are using. Sorry, but that''s all I can tell you. I use SLES and RHEL for my Xen testing, and they both have PAE enabled kernels by default, so I really haven''t had to deal with this for some time now. Cheers Jeff -- ------------------> Jeffrey Lane - W4KDH <------------------- www.jefflane.org Another cog in the great Corporate Wheel The internet has no government, no constitution, no laws, no rights, no police, no courts. Don''t talk about fairness or innocence, and don''t talk about what should be done. Instead, talk about what is being done and what will be done by the amorphous unreachable undefinable blob called "the internet user base." -Paul Vixie _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Jun 27, 2007 at 12:11:48PM -0500, Steven Timm wrote:> > Hi--I am currently running i386 version of xen 3.0.2 > [root@fermigrid0 ~]# uname -a > Linux fermigrid0.fnal.gov 2.6.16-xen3_86.1_rhel4.1 #1 SMP Thu Apr 13 > 06:52:14 PDT 2006 i686 i686 i386 GNU/Linux > > > The machine previously had 4GB of RAM, and was booted > with option dom0_mem=1024000 to ensure at least 1GB of RAM in > dom0. I am also running five Xen instances using 500MB of RAM apiece. > > Today I upgraded the RAM to 12GB. I attempted to leave dom0 at 1GB > and increase each of the Xen instances to 1GB for starters. > What actually happened is by the time I tried to start the > third 1GB instance I ran out of memory, it said there were only > 168MB available. > > Is there something in this kernel that limits us to only 4GB of RAM?Xen 3.0.2 did not support PAE, hence is limited to 4 GB. Upgrade to newer Xen - at least Xen 3.0.3 Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, 27 Jun 2007, Mark Williamson wrote:> You need to be running a PAE release of Xen to use more than 4Gig of memory.How do you tell if PAE is enabled or not? Was there any PAE in Xen 3.0.2 at all?> I''m guessing you''re not, right? I''m afraid that means upgrading both your > Xen and you dom0 / domU kernels, which I realise is a pain. > > The RPMs of Xen 3.1 may provide what you need (plus an upgrade to the latest > stable release of Xen). They''re compiled for PAE and are specifically for > RHEL 4.5 although they''re not officially supported by anyone...We''ll be going there eventually, maybe we should just bite the bullet and do it now? Steve Timm> > The RedHat RHEL 4.5 upgrade includes a domU kernel which will also be PAE, so > upgrading Xen to PAE will enable you to use the official RH kernels in the > domUs. > > Cheers, > Mark > > On Wednesday 27 June 2007, Steven Timm wrote: >> Hi--I am currently running i386 version of xen 3.0.2 >> [root@fermigrid0 ~]# uname -a >> Linux fermigrid0.fnal.gov 2.6.16-xen3_86.1_rhel4.1 #1 SMP Thu Apr 13 >> 06:52:14 PDT 2006 i686 i686 i386 GNU/Linux >> >> >> The machine previously had 4GB of RAM, and was booted >> with option dom0_mem=1024000 to ensure at least 1GB of RAM in >> dom0. I am also running five Xen instances using 500MB of RAM apiece. >> >> Today I upgraded the RAM to 12GB. I attempted to leave dom0 at 1GB >> and increase each of the Xen instances to 1GB for starters. >> What actually happened is by the time I tried to start the >> third 1GB instance I ran out of memory, it said there were only >> 168MB available. >> >> Is there something in this kernel that limits us to only 4GB of RAM? >> If so, what is the way around it? I can upgrade if necessary. >> (For that matter even before this, the sum of dom0 plus the 5 xen >> instances didn''t add up to the total amount of memory in the machine.) >> >> [root@fermigrid0 ~]# xm list >> Name ID Mem(MiB) VCPUs State Time(s) >> Domain-0 0 538 4 r----- 2694.1 >> fermigrid0-xen1 5 500 2 -b---- 1107.0 >> fermigrid0-xen2 6 500 2 -b---- 41.2 >> fermigrid0-xen3 7 500 2 -b---- 86.2 >> fermigrid0-xen4 8 500 2 -b---- 41.3 >> fermigrid0-xen5 9 500 2 r----- 42.1 >> >> >> >> Steve > > > >-- ------------------------------------------------------------------ Steven C. Timm, Ph.D (630) 840-8525 timm@fnal.gov http://home.fnal.gov/~timm/ Fermilab Computing Division, Scientific Computing Facilities, Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> On Wed, 27 Jun 2007, Mark Williamson wrote: > > You need to be running a PAE release of Xen to use more than 4Gig of > > memory. > > How do you tell if PAE is enabled or not? Was there any PAE in Xen 3.0.2 > at all?Daniel says not, so I''ll defer to his superior knowledge. In general, you could run "xm info" and see if you support x86_32p guests in the xen_caps field. If you don''t, then you don''t have PAE support.> > I''m guessing you''re not, right? I''m afraid that means upgrading both > > your Xen and you dom0 / domU kernels, which I realise is a pain. > > > > The RPMs of Xen 3.1 may provide what you need (plus an upgrade to the > > latest stable release of Xen). They''re compiled for PAE and are > > specifically for RHEL 4.5 although they''re not officially supported by > > anyone... > > We''ll be going there eventually, maybe we should just bite the bullet > and do it now?If you want to support the extra memory, then definitely you''ll need to upgrade to a PAE release (or 64-bit if that''s available), either from source or from the RPMs (or conceivably by upgrading to RHEL5, which isn''t based on 3.1 but does support PAE). I don''t know how up-to-date the XenSource RPMs are kept. Supporting PAE should allow the default fedora Xen kernels to run on your system easily too. Cheers, Mark> Steve Timm > > > The RedHat RHEL 4.5 upgrade includes a domU kernel which will also be > > PAE, so upgrading Xen to PAE will enable you to use the official RH > > kernels in the domUs. > > > > Cheers, > > Mark > > > > On Wednesday 27 June 2007, Steven Timm wrote: > >> Hi--I am currently running i386 version of xen 3.0.2 > >> [root@fermigrid0 ~]# uname -a > >> Linux fermigrid0.fnal.gov 2.6.16-xen3_86.1_rhel4.1 #1 SMP Thu Apr 13 > >> 06:52:14 PDT 2006 i686 i686 i386 GNU/Linux > >> > >> > >> The machine previously had 4GB of RAM, and was booted > >> with option dom0_mem=1024000 to ensure at least 1GB of RAM in > >> dom0. I am also running five Xen instances using 500MB of RAM apiece. > >> > >> Today I upgraded the RAM to 12GB. I attempted to leave dom0 at 1GB > >> and increase each of the Xen instances to 1GB for starters. > >> What actually happened is by the time I tried to start the > >> third 1GB instance I ran out of memory, it said there were only > >> 168MB available. > >> > >> Is there something in this kernel that limits us to only 4GB of RAM? > >> If so, what is the way around it? I can upgrade if necessary. > >> (For that matter even before this, the sum of dom0 plus the 5 xen > >> instances didn''t add up to the total amount of memory in the machine.) > >> > >> [root@fermigrid0 ~]# xm list > >> Name ID Mem(MiB) VCPUs State Time(s) > >> Domain-0 0 538 4 r----- 2694.1 > >> fermigrid0-xen1 5 500 2 -b---- 1107.0 > >> fermigrid0-xen2 6 500 2 -b---- 41.2 > >> fermigrid0-xen3 7 500 2 -b---- 86.2 > >> fermigrid0-xen4 8 500 2 -b---- 41.3 > >> fermigrid0-xen5 9 500 2 r----- 42.1 > >> > >> > >> > >> Steve-- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Kriz, Mike
2007-Aug-02 19:29 UTC
[Xen-users] RHEL5 Xen can''t allocate more then about 1.7GB RAM to HVM guest
I''m trying to create guests on a 32 bit system that has 12GB of RAM running RHEL5. I want to give each guest 2GB, however I am running into the following error upon creating the guests: malloc returned error 12 Based on some searching, I''ve tried setting the dom0_mem at boot manually to 1GB, as it seems this may be caused by not being able to find enough continuous address space. Not sure if the current RHEL5 xen kernel has implemented the patch for that or not. Here is my xm info: release : 2.6.18-8.1.8.el5xen version : #1 SMP Mon Jun 25 17:33:42 EDT 2007 machine : i686 nr_cpus : 4 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 2 threads_per_core : 1 cpu_mhz : 1995 hw_caps : bfebfbff:20100000:00000000:00000140:0004e33d:00000000:00000001 total_memory : 12287 free_memory : 4060 xen_major : 3 xen_minor : 0 xen_extra : .3-rc5-8.1.8.el xen_caps : xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p xen_pagesize : 4096 platform_params : virt_start=0xf5800000 xen_changeset : unavailable cc_compiler : gcc version 4.1.1 20070105 (Red Hat 4.1.1-52) cc_compile_by : brewbuilder cc_compile_domain : build.redhat.com cc_compile_date : Mon Jun 25 17:03:19 EDT 2007 xend_config_format : 2 I can give each guest up to about 1750MB, but once I approach the 1800MB mark I run into that error. Anyone have any ideas? Thanks! This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Daniel Ap. Martins Rosa
2007-Aug-21 18:42 UTC
Re: [Xen-users] RHEL5 Xen can''t allocate more then about 1.7GB RAM toHVM guest
I ´m with the same problem. Exists solution for this in version 3.0.3 of Xen ? Regards, Daniel Ap. Martins Rosa Ribeirão Preto/SP - Brazil ----- Original Message ----- From: "Kriz, Mike" <Mike_Kriz@adp.com> To: <xen-users@lists.xensource.com> Sent: Thursday, August 02, 2007 4:29 PM Subject: [Xen-users] RHEL5 Xen can''t allocate more then about 1.7GB RAM toHVM guest I''m trying to create guests on a 32 bit system that has 12GB of RAM running RHEL5. I want to give each guest 2GB, however I am running into the following error upon creating the guests: malloc returned error 12 Based on some searching, I''ve tried setting the dom0_mem at boot manually to 1GB, as it seems this may be caused by not being able to find enough continuous address space. Not sure if the current RHEL5 xen kernel has implemented the patch for that or not. Here is my xm info: release : 2.6.18-8.1.8.el5xen version : #1 SMP Mon Jun 25 17:33:42 EDT 2007 machine : i686 nr_cpus : 4 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 2 threads_per_core : 1 cpu_mhz : 1995 hw_caps : bfebfbff:20100000:00000000:00000140:0004e33d:00000000:00000001 total_memory : 12287 free_memory : 4060 xen_major : 3 xen_minor : 0 xen_extra : .3-rc5-8.1.8.el xen_caps : xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p xen_pagesize : 4096 platform_params : virt_start=0xf5800000 xen_changeset : unavailable cc_compiler : gcc version 4.1.1 20070105 (Red Hat 4.1.1-52) cc_compile_by : brewbuilder cc_compile_domain : build.redhat.com cc_compile_date : Mon Jun 25 17:03:19 EDT 2007 xend_config_format : 2 I can give each guest up to about 1750MB, but once I approach the 1800MB mark I run into that error. Anyone have any ideas? Thanks! This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Daniel P. Berrange
2007-Aug-21 18:52 UTC
Re: [Xen-users] RHEL5 Xen can''t allocate more then about 1.7GB RAM toHVM guest
On Tue, Aug 21, 2007 at 03:42:25PM -0300, Daniel Ap. Martins Rosa wrote:> I ´m with the same problem. > Exists solution for this in version 3.0.3 of Xen ?Upgrade. There were bugs in Xen 3.0.3 which prevented you assigning more than 2 GB ram to HVM guests. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Daniel Ap. Martins Rosa
2007-Aug-21 19:12 UTC
Re: [Xen-users] RHEL5 Xen can''t allocate more then about 1.7GB RAMtoHVM guest
Hi Daniel P. Berrange, Thank´s for reply. You know to say when the Red Hat you will be ready for use update1 of RHEL5 AP, and which version of the Xen will come in this update? Regards, Daniel Ap. Martins Rosa ----- Original Message ----- From: "Daniel P. Berrange" <berrange@redhat.com> To: "Daniel Ap. Martins Rosa" <daniel.mr@netsite.com.br> Cc: <xen-users@lists.xensource.com> Sent: Tuesday, August 21, 2007 3:52 PM Subject: Re: [Xen-users] RHEL5 Xen can''t allocate more then about 1.7GB RAMtoHVM guest On Tue, Aug 21, 2007 at 03:42:25PM -0300, Daniel Ap. Martins Rosa wrote:> I ´m with the same problem. > Exists solution for this in version 3.0.3 of Xen ?Upgrade. There were bugs in Xen 3.0.3 which prevented you assigning more than 2 GB ram to HVM guests. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: .cpan.org/~danberr/ -=| |=- Projects: at.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 505 -=| _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users