When I boot the ES7000 with 32 GB of RAM, dom0 is allocated almost all of it. I issued an xm mem-set 0 512 command to decrease dom0''s memory to 512 MB. Using xentop, I see the amount of memory allocated to dom0 slowly decrease. The kernel panic occurs when dom0''s memory is around 1 GB. The serial console output is attached. Using SLES10 Beta 11 upgraded to xen changeset 9903. <<serport2.txt>> Thanks,> Sue Krysan > Linux Systems Group > Unisys Corporation > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> When I boot the ES7000 with 32 GB of RAM, dom0 is allocated > almost all of it. I issued an xm mem-set 0 512 command to > decrease dom0''s memory to 512 MB. Using xentop, I see the > amount of memory allocated to dom0 slowly decrease. The > kernel panic occurs when dom0''s memory is around 1 GB. The > serial console output is attached. Using SLES10 Beta 11 > upgraded to xen changeset 9903.Can you repro this on -unstable? What about with a PAE hypervisor? Thanks, Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
We did run using unstable. We installed sles10 beta11 but we pulled and built xen-unstable changeset 9903 from xensource repository. We have not tried with PAE hypervisor. It is not our focus. How desperately would you need us to try PAE? Thanks,> Sue Krysan > Linux Systems Group > Unisys Corporation >-----Original Message----- From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk] Sent: Wednesday, May 03, 2006 4:19 PM To: Krysan, Susan; xen-devel@lists.xensource.com Cc: Subrahmanian, Raj; Vessey, Bruce A; Puthiyaparambil, Aravindh; Carb, Brian A; ian.pratt@cl.cam.ac.uk Subject: RE: [Xen-devel] xm mem-set causes kernel panic> When I boot the ES7000 with 32 GB of RAM, dom0 is allocated > almost all of it. I issued an xm mem-set 0 512 command to > decrease dom0''s memory to 512 MB. Using xentop, I see the > amount of memory allocated to dom0 slowly decrease. The > kernel panic occurs when dom0''s memory is around 1 GB. The > serial console output is attached. Using SLES10 Beta 11 > upgraded to xen changeset 9903.Can you repro this on -unstable? What about with a PAE hypervisor? Thanks, Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> We did run using unstable. We installed sles10 beta11 but we > pulled and built xen-unstable changeset 9903 from xensource > repository. We have not tried with PAE hypervisor. It is > not our focus. How desperately would you need us to try PAE?Interesting. If you boot with dom0_mem=512 and then start a 15GB guest, do you see problems shrinking the guest''s memory? I''d certainly be interested in hearing whether the problem exists on PAE. Ian> -----Original Message----- > From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk] > Sent: Wednesday, May 03, 2006 4:19 PM > To: Krysan, Susan; xen-devel@lists.xensource.com > Cc: Subrahmanian, Raj; Vessey, Bruce A; Puthiyaparambil, > Aravindh; Carb, Brian A; ian.pratt@cl.cam.ac.uk > Subject: RE: [Xen-devel] xm mem-set causes kernel panic > > > > When I boot the ES7000 with 32 GB of RAM, dom0 is allocated > almost all > > of it. I issued an xm mem-set 0 512 command to decrease > dom0''s memory > > to 512 MB. Using xentop, I see the amount of memory > allocated to dom0 > > slowly decrease. The kernel panic occurs when dom0''s > memory is around > > 1 GB. The serial console output is attached. Using SLES10 Beta 11 > > upgraded to xen changeset 9903. > > Can you repro this on -unstable? What about with a PAE hypervisor? > > Thanks, > Ian >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Krysan, Susan wrote:> We did run using unstable. We installed sles10 beta11 but we pulled and built xen-unstable changeset 9903 from xensource repository. We have not tried with PAE hypervisor. It is not our focus. How desperately would you need us to try PAE? >I have done this on 32GB x86_64 as well and have pretty much the same problem. I think the problem is that the static memory allocations in the kernel for things like mem_map, page_struct etc, are quite large for 32GB, and do not shrink (or do they?) when balooning to 512 MB. This leaves almost no usable memory when balooning down, and you get OOM. If you know you don''t need 32G for dom0, I would try setting dom0_mem to something much lower, like 1GB. -Andrew Theurer> Thanks, > > >> Sue Krysan >> Linux Systems Group >> Unisys Corporation >> >> > > > -----Original Message----- > From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk] > Sent: Wednesday, May 03, 2006 4:19 PM > To: Krysan, Susan; xen-devel@lists.xensource.com > Cc: Subrahmanian, Raj; Vessey, Bruce A; Puthiyaparambil, Aravindh; Carb, > Brian A; ian.pratt@cl.cam.ac.uk > Subject: RE: [Xen-devel] xm mem-set causes kernel panic > > > >> When I boot the ES7000 with 32 GB of RAM, dom0 is allocated >> almost all of it. I issued an xm mem-set 0 512 command to >> decrease dom0''s memory to 512 MB. Using xentop, I see the >> amount of memory allocated to dom0 slowly decrease. The >> kernel panic occurs when dom0''s memory is around 1 GB. The >> serial console output is attached. Using SLES10 Beta 11 >> upgraded to xen changeset 9903. >> > > Can you repro this on -unstable? What about with a PAE hypervisor? > > Thanks, > Ian > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3 May 2006, at 22:15, Ian Pratt wrote:>> We did run using unstable. We installed sles10 beta11 but we >> pulled and built xen-unstable changeset 9903 from xensource >> repository. We have not tried with PAE hypervisor. It is >> not our focus. How desperately would you need us to try PAE? > > Interesting. If you boot with dom0_mem=512 and then start a 15GB guest, > do you see problems shrinking the guest''s memory? > > I''d certainly be interested in hearing whether the problem exists on > PAE.I don''t think this behaviour is surprising. The kernel is probably taking about 1% of memory to track memory usage. So trying to balloon down to 1-2% of your initial memory reservation is rather harsh. The ''fix'' here is to stop users being able to shoot their feet off, by setting a lower bound on ballooning that is related to the initial memory size of the guest, and/or quiesce the balloon driver when low-memory and out-of-memory code paths are executed in the guest. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Puthiyaparambil, Aravindh
2006-May-04 15:49 UTC
RE: [Xen-devel] xm mem-set causes kernel panic
> I don''t think this behaviour is surprising. The kernel is probably > taking about 1% of memory to track memory usage. So trying to balloon > down to 1-2% of your initial memory reservation is rather harsh.I agree.> The ''fix'' here is to stop users being able to shoot their feet off, by > setting a lower bound on ballooning that is related to the initial > memory size of the guest, and/or quiesce the balloon driver when > low-memory and out-of-memory code paths are executed in the guest.We will look into this and send in a patch. Cheers, Aravindh _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel