Roger Cruz
2010-Jan-25 15:36 UTC
[Xen-devel] Question on hyperthreading and Westmere processors.
Hi folks, I''m wondering if one of you could answer a couple of questions with regards to the new Intel processors and VM scheduling when hyperthreads are involved. 1) For Nahelem/Westmere processors, Intel has a feature where it can turbo the processor''s frequency. In an SMP environment, would this turbo boost affect Xen in any ways? For example, does the TSC offset computation between processor''s becomes invalid? Are there any other features or reasons why a Westmere processor would cause a problem for Xen? 2) These new processors have hyper threading enabled. Is Xen''s scheduling algorithm cognizant of threads so that it can most effectively schedule VMs on idle cores rather than use an idle thread on a busy core? Let me explain that better. For example, on 1 quad-core HT-enabled processor server, there are a total of 8 working units (4 cores + 4 threads). Assume we labelt them this way: [core#,thread#]: [0,0], [0,1], [1,0], [1,1], [2,0], [2,1], [3,0], [3,1]. If there are 3 VMs exist, does the scheduler arrange the VMs such that VM1 is on [0,0], VM2 is on [1,0], and VM3 on [2,0]? Or does it tread all the work units without regards to their locations and schedule them as [0,0], [0,1] and [1,0], for example? In the latter case, two VMs are schedule on the same core, which would be less effective than scheduled on the idle core. Thanks Roger _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Yu, Ke
2010-Jan-27 02:04 UTC
[Xen-devel] RE: Question on hyperthreading and Westmere processors.
Hi Roger, For the 1), the TSC will works as normal when turbo mode is enabled. in recent Intel processor since Pentium 4, TSC frequency is constant regardless the CPU frequency scaling. i.e. TSC will always run @ highest non-turbo frequency, so the TSC offset should be valid. You can search X86_FEATURE_CONSTANT_TSC for more detail. And I don''t see any point that Westmere could cause Xen problem. if you observe any problem, feel free to report in mailing list. For 2), Xen scheduler is HT aware, and will pick the idle cores first rather than an idle thread on a busy core. You can check xen/common/sched_credit.c:_csched_cpu_pick() for more detail. Regards Ke From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Roger Cruz Sent: Monday, January 25, 2010 11:37 PM To: xen-devel@lists.xensource.com Subject: [Xen-devel] Question on hyperthreading and Westmere processors. Hi folks, I''m wondering if one of you could answer a couple of questions with regards to the new Intel processors and VM scheduling when hyperthreads are involved. 1) For Nahelem/Westmere processors, Intel has a feature where it can turbo the processor''s frequency. In an SMP environment, would this turbo boost affect Xen in any ways? For example, does the TSC offset computation between processor''s becomes invalid? Are there any other features or reasons why a Westmere processor would cause a problem for Xen? 2) These new processors have hyper threading enabled. Is Xen''s scheduling algorithm cognizant of threads so that it can most effectively schedule VMs on idle cores rather than use an idle thread on a busy core? Let me explain that better. For example, on 1 quad-core HT-enabled processor server, there are a total of 8 working units (4 cores + 4 threads). Assume we labelt them this way: [core#,thread#]: [0,0], [0,1], [1,0], [1,1], [2,0], [2,1], [3,0], [3,1]. If there are 3 VMs exist, does the scheduler arrange the VMs such that VM1 is on [0,0], VM2 is on [1,0], and VM3 on [2,0]? Or does it tread all the work units without regards to their locations and schedule them as [0,0], [0,1] and [1,0], for example? In the latter case, two VMs are schedule on the same core, which would be less effective than scheduled on the idle core. Thanks Roger _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Roger Cruz
2010-Feb-01 15:20 UTC
RE: [Xen-devel] Question on hyperthreading and Westmere processors.
Hi folks, I was wondering if someone would be kind enough to look over my questions and provide an answer or some "gut" feeling on whether any issues may arise on Westmere processors. I don''t have one to try XenServer on myself so I''m looking for others who can help. Thanks Roger ________________________________ From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Roger Cruz Sent: Monday, January 25, 2010 10:37 AM To: xen-devel@lists.xensource.com Subject: [Xen-devel] Question on hyperthreading and Westmere processors. Hi folks, I''m wondering if one of you could answer a couple of questions with regards to the new Intel processors and VM scheduling when hyperthreads are involved. 1) For Nahelem/Westmere processors, Intel has a feature where it can turbo the processor''s frequency. In an SMP environment, would this turbo boost affect Xen in any ways? For example, does the TSC offset computation between processor''s becomes invalid? Are there any other features or reasons why a Westmere processor would cause a problem for Xen? 2) These new processors have hyper threading enabled. Is Xen''s scheduling algorithm cognizant of threads so that it can most effectively schedule VMs on idle cores rather than use an idle thread on a busy core? Let me explain that better. For example, on 1 quad-core HT-enabled processor server, there are a total of 8 working units (4 cores + 4 threads). Assume we labelt them this way: [core#,thread#]: [0,0], [0,1], [1,0], [1,1], [2,0], [2,1], [3,0], [3,1]. If there are 3 VMs exist, does the scheduler arrange the VMs such that VM1 is on [0,0], VM2 is on [1,0], and VM3 on [2,0]? Or does it tread all the work units without regards to their locations and schedule them as [0,0], [0,1] and [1,0], for example? In the latter case, two VMs are schedule on the same core, which would be less effective than scheduled on the idle core. Thanks Roger _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2010-Feb-01 15:37 UTC
Re: [Xen-devel] Question on hyperthreading and Westmere processors.
New Intel processors should all have constant-rate TSCs, regardless of P- and C-states, and regardless of things like turbo boosts. The credit scheduler likes to schedule across cores where possible, before making use of the second hyperthread of a pair. -- Keir On 01/02/2010 15:20, "Roger Cruz" <rcruz@marathontechnologies.com> wrote:> Hi folks, > > I was wondering if someone would be kind enough to look over my questions and > provide an answer or some ³gut² feeling on whether any issues may arise on > Westmere processors. I don¹t have one to try XenServer on myself so I¹m > looking for others who can help. > > Thanks > Roger > > > > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Roger Cruz > Sent: Monday, January 25, 2010 10:37 AM > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] Question on hyperthreading and Westmere processors. > > > Hi folks, > > I¹m wondering if one of you could answer a couple of questions with regards to > the new Intel processors and VM scheduling when hyperthreads are involved. > > 1) For Nahelem/Westmere processors, Intel has a feature where it can > turbo the processor¹s frequency. In an SMP environment, would this turbo > boost affect Xen in any ways? For example, does the TSC offset computation > between processor¹s becomes invalid? Are there any other features or reasons > why a Westmere processor would cause a problem for Xen? > > 2) These new processors have hyper threading enabled. Is Xen¹s > scheduling algorithm cognizant of threads so that it can most effectively > schedule VMs on idle cores rather than use an idle thread on a busy core? > > > > Let me explain that better. For example, on 1 quad-core HT-enabled processor > server, there are a total of 8 working units (4 cores + 4 threads). Assume > we labelt them this way: [core#,thread#]: [0,0], [0,1], [1,0], [1,1], [2,0], > [2,1], [3,0], [3,1]. If there are 3 VMs exist, does the scheduler arrange the > VMs such that VM1 is on [0,0], VM2 is on [1,0], and VM3 on [2,0]? Or does it > tread all the work units without regards to their locations and schedule them > as [0,0], [0,1] and [1,0], for example? In the latter case, two VMs are > schedule on the same core, which would be less effective than scheduled on the > idle core. > > > > Thanks > Roger >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel