Hi.
I have a machine with 8GB of memory and I''d like to create two domUs in
such a
way that each domain can use memory from about 160MB to 7800MB (set manually
using xm mem-set).
First, I create Domain-1 with memory=7800, than reduce its current memory
allocation to 160MB by xm mem-set 1 160. After that, the second domU is
started in the same way. Up to now, everything is ok.
After logging in the first domain, free tells me there is 62MB of free memory
out of total 160MB and 1GB of free swap. Then I start a process which consumes
more than 62 megabytes of memory (to be specific, view a 100+MB file with the
vim). After a while, the process is killed by OOM killer, even though there is
plenty of space available in swap. When I create the domain with memory=160,
everything works as expected, free space in swap is used and no process is
killed by OOM.
Do you have any idea where the problem could be?
Thanks a lot.
Jirka
--
In the beginning the Universe was created. This has made a lot of people very
angry and been widely regarded as a bad move.
-- Douglas Adams, "The Restaurant at the End of the Universe"
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Ballooning down to 160MB is not quite the same as creating a domain with 160MB. What''s really going on here, is that you create a domain with 7800MB, and then the balloon driver tells the kernel that it''s allocated (7800-160)MBs. The kernel really doesn''t know that the balloon driver gave that memory back to Xen--it just thinks it''s a greedy kernel module. At any rate, Linux tends to allocate certain structures as percentages. Furthermore, the swapping heuristics are going to be based on the assumption that the system really has 7800MB of memory. Since you only have a 1G swap, I imagine it will be very conservative about how it uses that swap. Regards, Anthony Liguori Jiri Denemark wrote:> Hi. > > I have a machine with 8GB of memory and I''d like to create two domUs in such a > way that each domain can use memory from about 160MB to 7800MB (set manually > using xm mem-set). > > First, I create Domain-1 with memory=7800, than reduce its current memory > allocation to 160MB by xm mem-set 1 160. After that, the second domU is > started in the same way. Up to now, everything is ok. > > After logging in the first domain, free tells me there is 62MB of free memory > out of total 160MB and 1GB of free swap. Then I start a process which consumes > more than 62 megabytes of memory (to be specific, view a 100+MB file with the > vim). After a while, the process is killed by OOM killer, even though there is > plenty of space available in swap. When I create the domain with memory=160, > everything works as expected, free space in swap is used and no process is > killed by OOM. > > Do you have any idea where the problem could be? > > Thanks a lot. > > Jirka >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thanks for explaining how the balloon driver actually works.> What''s really going on here, is that you create a domain with 7800MB, and > then the balloon driver tells the kernel that it''s allocated (7800-160)MBs. > The kernel really doesn''t know that the balloon driver gave that memory back > to Xen--it just thinks it''s a greedy kernel module.So the kernel still has 7800MB of memory, (7800-160)MBs eaten up by balloon driver and 160MBs available to userspace, right? In that case, it seems it''s rather an issue of VM subsystem in Linux. It''s strange the kernel is not willing to swap out a process and kills it even if there is enough free swap space. But I guess that''s somehow connected with those (7800-160)MBs "held" by ballon driver which kernel cannot swap out. Anyway it would be nice if the kernel could behave in exactly the same way regardless of whether a domain is started with limited memory or its memory is ballooned down. I wonder if playing with altering kernel''s swappiness would help in that... I tried to make 8GB swap but that didn''t help much. Jirka -- The reason computer chips are so small is computers don''t eat much. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> So the kernel still has 7800MB of memory, (7800-160)MBs eaten up by balloon > driver and 160MBs available to userspace, right? In that case, it seems > it''s rather an issue of VM subsystem in Linux.Well, the kernel is probably thinking "Arrrg I''ve almost used all physical memory! Must free some now in case I run out!". Seems odd that it''d do that rather than swapping - but how much free physical memory did you have? Is it possible that with other tasks running too you''d run out (or quite low) of physical memory? The OOM killer is presumably worried that the kernel will run out of ram and not be able to use it for essential kernel data structures... The easy fix for you is probably to either remove the OOM killer (I think it''s a compile-time option, and some people would like to see it removed completely) or to creat the domain smaller to start with.> I wonder if playing with altering kernel''s swappiness would help in that... > I tried to make 8GB swap but that didn''t help much.Hmmmm. Not sure, but I''d be more comfortable with reducing the "incentive" to the OOM killer in terms of physical memory set up, rather than trying to just increase swap and hope... Good luck! Btw, this behaviour could be thought of as a bug (in that we''re making Linux do silly things) so it''d be worth posting a bug on BugZilla and sending details to the xen-devel list - if you have time, that is. Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > Mark Williamson > Sent: 16 October 2006 17:21 > To: xen-users@lists.xensource.com > Cc: Anthony Liguori; Jiri Denemark > Subject: Re: [Xen-users] Re: Memory resizing > > > So the kernel still has 7800MB of memory, (7800-160)MBs > eaten up by balloon > > driver and 160MBs available to userspace, right? In that > case, it seems > > it''s rather an issue of VM subsystem in Linux. > > Well, the kernel is probably thinking "Arrrg I''ve almost used > all physical > memory! Must free some now in case I run out!". Seems odd > that it''d do that > rather than swapping - but how much free physical memory did > you have? Is it > possible that with other tasks running too you''d run out (or > quite low) of > physical memory? > > The OOM killer is presumably worried that the kernel will run > out of ram and > not be able to use it for essential kernel data structures... > > The easy fix for you is probably to either remove the OOM > killer (I think it''s > a compile-time option, and some people would like to see it removed > completely) or to creat the domain smaller to start with. > > > I wonder if playing with altering kernel''s swappiness would > help in that... > > I tried to make 8GB swap but that didn''t help much. > > Hmmmm. Not sure, but I''d be more comfortable with reducing > the "incentive" to > the OOM killer in terms of physical memory set up, rather > than trying to just > increase swap and hope...One way to (to some extent at least) avoid the OOM killer is to "echo 2 < /proc/sys/vm/overcommit_memory" as described in "man malloc" - it means that applications and kernel isn''t allowed to allocate memory that isn''t ACTUALLY there - so when an application happily says "give me 100MB of RAM", malloc checks to see if there REALLY is 100MB of RAM before it says "Yes", whilst in the "normal" case, it says "Sure, here''s a block of memory that could be made into 100MB at some point". The problem trying to be solved here is that many applications ask for a huge amount of memory even if they don''t need it [the app needs a 300 long and 300 wide array SOMETIMES, but most of the time, the app just needs 300 in one direction and 4 in the other, so a lot of memory is not ACTUALLY used]. So memory allocation is allowed to "Happy-go-lucky" and "overspend". -- Mats> > Good luck! > > Btw, this behaviour could be thought of as a bug (in that > we''re making Linux > do silly things) so it''d be worth posting a bug on BugZilla > and sending > details to the xen-devel list - if you have time, that is. > > Cheers, > Mark > > -- > Dave: Just a question. What use is a unicyle with no seat? > And no pedals! > Mark: To answer a question with a question: What use is a skateboard? > Dave: Skateboards have wheels. > Mark: My wheel has a wheel! > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users