Good Afternoon, Is it possible to limit a guest OS to a specific memory bandwidth allocation? The purpose is to stop one guest OS from saturating the memory bus. Thank you, Robert _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
You mean QoS on memory bandwidth? It''s an interesting question. (there are similar problems like QoS on shared L2) I think it''s hard to do. Even for a native OS, it may only control this indirectly by adjusting the CPU time slice. And I don''t think any commercial OS has implemented such feature yet. Weiming On Thu, Feb 14, 2008 at 9:27 PM, Robert Stober <rstober@platform.com> wrote:> Good Afternoon, > > Is it possible to limit a guest OS to a specific memory bandwidth > allocation? The purpose is to stop one guest OS from saturating the > memory bus. > > Thank you, > > Robert > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Weiming, I agree that it is very hard, and that no one has done it. But nevertheless I suggest the following question to the Xen developers: Given the fact that memory bandwidth is shared amongst multiple cores on a single die, assume that one VM is running on each core. What is to stop one VM from saturating the memory bus, causing reduced performance of all the other VMs? This is the general multi-core problem, not specific to Xen. But it affects Xen greatly. What use is it to allocate memory to a VM if it can''t use the memory because a process of another VM has saturated the memory bus? Thank you, Robert _____ From: weiming [mailto:zephyr.zhao@gmail.com] Sent: Thursday, February 14, 2008 7:02 PM To: Robert Stober Cc: xen-devel@lists.xensource.com Subject: Re: [Xen-devel] A question no one can answer You mean QoS on memory bandwidth? It''s an interesting question. (there are similar problems like QoS on shared L2) I think it''s hard to do. Even for a native OS, it may only control this indirectly by adjusting the CPU time slice. And I don''t think any commercial OS has implemented such feature yet. Weiming On Thu, Feb 14, 2008 at 9:27 PM, Robert Stober <rstober@platform.com> wrote: Good Afternoon, Is it possible to limit a guest OS to a specific memory bandwidth allocation? The purpose is to stop one guest OS from saturating the memory bus. Thank you, Robert _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Robert, The memory bus controller like Memory Controller Hub (MCH) on Intel''s north bridge should arbitrate the traffic from different cores/processors. I guess the hardware can guarantee some kind of fairness so that no core would be starving and the other one is busy. (just my guess, I don''t know the details) However, control the bandwidth with priorities is difficult. Any one has ideas to this problem? Weiming On Thu, Feb 14, 2008 at 10:24 PM, Robert Stober <rstober@platform.com> wrote:> Weiming, > > I agree that it is very hard, and that no one has done it. But nevertheless > I suggest the following question to the Xen developers: > > Given the fact that memory bandwidth is shared amongst multiple cores on a > single die, assume that one VM is running on each core. What is to stop one > VM from saturating the memory bus, causing reduced performance of all the > other VMs? This is the general multi-core problem, not specific to Xen. But > it affects Xen greatly. What use is it to allocate memory to a VM if it > can''t use the memory because a process of another VM has saturated the > memory bus? > > Thank you, > > Robert > > ------------------------------ > *From:* weiming [mailto:zephyr.zhao@gmail.com] > *Sent:* Thursday, February 14, 2008 7:02 PM > *To:* Robert Stober > *Cc:* xen-devel@lists.xensource.com > *Subject:* Re: [Xen-devel] A question no one can answer > > You mean QoS on memory bandwidth? It''s an interesting question. (there are > similar problems like QoS on shared L2) > > I think it''s hard to do. Even for a native OS, it may only control this > indirectly by adjusting the CPU time slice. And I don''t think any commercial > OS has implemented such feature yet. > > Weiming > > On Thu, Feb 14, 2008 at 9:27 PM, Robert Stober <rstober@platform.com> > wrote: > > > Good Afternoon, > > > > Is it possible to limit a guest OS to a specific memory bandwidth > > allocation? The purpose is to stop one guest OS from saturating the > > memory bus. > > > > Thank you, > > > > Robert > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Without hardware support to help with this (which does not exist) the hypervisor is restricted to crude methods like monitoring cache-miss performance counters and preempting an overly-aggressive guest. It is rather likely that to get smooth enough throttling of bandwidth you would incur significant monitoring and scheduling overhead. It''s not even certain that this approach would work at all! -- Keir On 15/2/08 02:27, "Robert Stober" <rstober@platform.com> wrote:> Good Afternoon, > > Is it possible to limit a guest OS to a specific memory bandwidth > allocation? The purpose is to stop one guest OS from saturating the > memory bus. > > Thank you, > > Robert > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 14 Feb 2008 22:24:18 -0500 "Robert Stober" <rstober@platform.com> wrote:> Weiming, > > I agree that it is very hard, and that no one has done it. But > nevertheless I suggest the following question to the Xen developers: > > Given the fact that memory bandwidth is shared amongst multiple cores on > a single die, assume that one VM is running on each core. What is to > stop one VM from saturating the memory bus, causing reduced performance > of all the other VMs? This is the general multi-core problem, not > specific to Xen. But it affects Xen greatly. What use is it to allocate > memory to a VM if it can''t use the memory because a process of another > VM has saturated the memory bus?Its perfectly doable on modern x86 - you use the profiling registers and set them up so you get a threshold interrupt when too much main memory traffic is counted (which you use to reschedule punishing the memory user). There are research papers on it from quite a long time back but afaik nobody ever implemented it in production although its not too hard to do so might be an interesting project. Alan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hello Robert and others, Below is a link to a presentation given at a conference last summer, where the authors used cache colour allocation schemes to limit L2 cache overuse by an application. If your goal is to create a fair memory share per-guest (bandwidth or otherwise), you may want to consider the possibility of managing it from a level closer to the CPU. http://www.ideal.ece.ufl.edu/workshops/wiosca07/P4Slides.pdf Dan. On Fri, Feb 15, 2008 at 5:05 AM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:> On Thu, 14 Feb 2008 22:24:18 -0500 > "Robert Stober" <rstober@platform.com> wrote: > > > Weiming, > > > > I agree that it is very hard, and that no one has done it. But > > nevertheless I suggest the following question to the Xen developers: > > > > Given the fact that memory bandwidth is shared amongst multiple cores on > > a single die, assume that one VM is running on each core. What is to > > stop one VM from saturating the memory bus, causing reduced performance > > of all the other VMs? This is the general multi-core problem, not > > specific to Xen. But it affects Xen greatly. What use is it to allocate > > memory to a VM if it can''t use the memory because a process of another > > VM has saturated the memory bus? > > Its perfectly doable on modern x86 - you use the profiling registers and > set them up so you get a threshold interrupt when too much main memory > traffic is counted (which you use to reschedule punishing the memory > user). There are research papers on it from quite a long time back but > afaik nobody ever implemented it in production although its not too hard > to do so might be an interesting project. > > Alan > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel