hi all, I am using xen3.0.3 on dual core hyperthreaded processor (in all 4 cores). There are 2 VMs vm1,vm2 among which vm1 has a webserver running on it. While testing the performance of webserver, when I introduce some load on vm2 which involves some computations the webserver performance goes down. This happened even in the case where CPU usage by both of vm1,vm2 is restricted to 100% each. Is it expected behavior ? if yes then how does one can control addition of new virtual machines as adding every new VM will result in lowering performance of other VMs. Through scheduling parameters we can just specify amount of CPU to be used in relative sense (weight) & upper limit (cap). But how to tackle this point. I am new in this area & wanna set up a lab using virtualization, so want to find solution for this. thanks, Harry we always have a choice... _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
You should show detail configuration. Your information is too short. Anyway I guess each domain has one vcpu. If so, this is normal behavior. Because one vcpu cannot allocate two or more pcpu at once. Thanks Atsushi SAKAI "Harry Smith" <harry.smith272@gmail.com> wrote:> hi all, > > I am using xen3.0.3 on dual core hyperthreaded processor (in all 4 cores). > There are 2 VMs vm1,vm2 among which vm1 has a webserver running on it. > > While testing the performance of webserver, when I introduce some load on > vm2 which involves some computations the webserver performance goes down. > This happened even in the case where CPU usage by both of vm1,vm2 is > restricted to 100% each. > > Is it expected behavior ? if yes then how does one can control addition of > new virtual machines as adding every new VM will result in lowering > performance of other VMs. Through scheduling parameters we can just specify > amount of CPU to be used in relative sense (weight) & upper limit (cap). But > how to tackle this point. > > I am new in this area & wanna set up a lab using virtualization, so want to > find solution for this. > > thanks, > Harry > > we always have a choice..._______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Atsushi, On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote:> > > You should show detail configuration. > Your information is too short. > > Anyway I guess each domain has one vcpu. > If so, this is normal behavior. > Because one vcpu cannot allocate two or more pcpu at once.Right, but shouldn''t Xen hypervisor be capable of migrating the VCPU among the available PCPUs on a multiprocessor system, like in this case? And criteria should be the load on the PCPU or the idle PCPUs. yes/no? Am i missing something here? Thanks ~psr Thanks> Atsushi SAKAI > > "Harry Smith" <harry.smith272@gmail.com> wrote: > > > hi all, > > > > I am using xen3.0.3 on dual core hyperthreaded processor (in all 4 > cores). > > There are 2 VMs vm1,vm2 among which vm1 has a webserver running on it. > > > > While testing the performance of webserver, when I introduce some load > on > > vm2 which involves some computations the webserver performance goes > down. > > This happened even in the case where CPU usage by both of vm1,vm2 is > > restricted to 100% each. > > > > Is it expected behavior ? if yes then how does one can control addition > of > > new virtual machines as adding every new VM will result in lowering > > performance of other VMs. Through scheduling parameters we can just > specify > > amount of CPU to be used in relative sense (weight) & upper limit (cap). > But > > how to tackle this point. > > > > I am new in this area & wanna set up a lab using virtualization, so want > to > > find solution for this. > > > > thanks, > > Harry > > > > we always have a choice... > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- --- pradeep singh rautela "Genius is 1% inspiration, and 99% perspiration" - not me :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
One vcpu can use one pcpu at one time. It means 100% is maxium for one vcpu domain. If you want to use cpu resources, you should set more vcpu. Thanks Atsushi SAKAI "pradeep singh rautela" <rautelap@gmail.com> wrote:> Hi Atsushi, > > On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > > > > > > You should show detail configuration. > > Your information is too short. > > > > Anyway I guess each domain has one vcpu. > > If so, this is normal behavior. > > Because one vcpu cannot allocate two or more pcpu at once. > > > Right, but shouldn''t Xen hypervisor be capable of migrating the VCPU among > the available PCPUs on a multiprocessor system, like in this case? And > criteria should be the load on the PCPU or the idle PCPUs. > yes/no? > > Am i missing something here? > > Thanks > ~psr > > Thanks > > Atsushi SAKAI > > > > "Harry Smith" <harry.smith272@gmail.com> wrote: > > > > > hi all, > > > > > > I am using xen3.0.3 on dual core hyperthreaded processor (in all 4 > > cores). > > > There are 2 VMs vm1,vm2 among which vm1 has a webserver running on it. > > > > > > While testing the performance of webserver, when I introduce some load > > on > > > vm2 which involves some computations the webserver performance goes > > down. > > > This happened even in the case where CPU usage by both of vm1,vm2 is > > > restricted to 100% each. > > > > > > Is it expected behavior ? if yes then how does one can control addition > > of > > > new virtual machines as adding every new VM will result in lowering > > > performance of other VMs. Through scheduling parameters we can just > > specify > > > amount of CPU to be used in relative sense (weight) & upper limit (cap). > > But > > > how to tackle this point. > > > > > > I am new in this area & wanna set up a lab using virtualization, so want > > to > > > find solution for this. > > > > > > thanks, > > > Harry > > > > > > we always have a choice... > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > > > -- > --- > pradeep singh rautela > > "Genius is 1% inspiration, and 99% perspiration" - not me :)_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote:> > One vcpu can use one pcpu at one time. > It means 100% is maxium for one vcpu domain. > If you want to use cpu resources, you should set more vcpu.Ok, this explains a lot of things. As i understand this , more VCPUs means more freedom to hypervisor to migrate them among physical CPUs, depending on the free PCPUs available. In general domU1 / | \ vcpu1 vcpu2 vcpu3 pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 I mean ,domU1 can run on any vcpu , right? now vcpu1, vcpu2, vcpu3 share a one to many reationship between pcpus[1....6]. That is a vcpu can run on any of the pcus available to the Xen hypervisor(unless i explicitly pin it to ). Is my naive understanding of what you explained is correct? Thank you ~psr> Thanks > Atsushi SAKAI > > > "pradeep singh rautela" <rautelap@gmail.com> wrote: > > > Hi Atsushi, > > > > On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > > > > > > > > > You should show detail configuration. > > > Your information is too short. > > > > > > Anyway I guess each domain has one vcpu. > > > If so, this is normal behavior. > > > Because one vcpu cannot allocate two or more pcpu at once. > > > > > > Right, but shouldn''t Xen hypervisor be capable of migrating the VCPU > among > > the available PCPUs on a multiprocessor system, like in this case? And > > criteria should be the load on the PCPU or the idle PCPUs. > > yes/no? > > > > Am i missing something here? > > > > Thanks > > ~psr > > > > Thanks > > > Atsushi SAKAI > > > > > > "Harry Smith" <harry.smith272@gmail.com> wrote: > > > > > > > hi all, > > > > > > > > I am using xen3.0.3 on dual core hyperthreaded processor (in all 4 > > > cores). > > > > There are 2 VMs vm1,vm2 among which vm1 has a webserver running on > it. > > > > > > > > While testing the performance of webserver, when I introduce some > load > > > on > > > > vm2 which involves some computations the webserver performance goes > > > down. > > > > This happened even in the case where CPU usage by both of vm1,vm2 is > > > > restricted to 100% each. > > > > > > > > Is it expected behavior ? if yes then how does one can control > addition > > > of > > > > new virtual machines as adding every new VM will result in lowering > > > > performance of other VMs. Through scheduling parameters we can just > > > specify > > > > amount of CPU to be used in relative sense (weight) & upper limit > (cap). > > > But > > > > how to tackle this point. > > > > > > > > I am new in this area & wanna set up a lab using virtualization, so > want > > > to > > > > find solution for this. > > > > > > > > thanks, > > > > Harry > > > > > > > > we always have a choice... > > > > > > > > > > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.xensource.com > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > -- > > --- > > pradeep singh rautela > > > > "Genius is 1% inspiration, and 99% perspiration" - not me :) > > >-- --- pradeep singh rautela "Genius is 1% inspiration, and 99% perspiration" - not me :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Atsushi & Pradeep, thanks for replying back. I have 4 VCPUs for each of VM. But the point I wanted to stress upon is - "This happened even in the case where CPU usage by both of vm1,vm2 is restricted to 100% each. " I had pinned all 4 VCPUs of each VM to a single phys. CPU. & I have 4 phys. CPUs means my vm1 was using cpu1, vm2 using cpu2 & domain-0 using cpu0,cpu3 Problem is when there is no load on vm2, webserver performance of vm1 is better. But when vm2 has some compute-intense load then vm1 webserver performance goes down. Please note that CPU consumption of vm1 shown by xentop in both cases is 100%, still webserver performance goes down by around 15-20%. Even after trying to isolate two VMs, existence of load on one VM is affecting other. so is it expected behavior ? thanks, Harry On 5/10/07, pradeep singh rautela <rautelap@gmail.com> wrote:> > > > On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > > > > One vcpu can use one pcpu at one time. > > It means 100% is maxium for one vcpu domain. > > If you want to use cpu resources, you should set more vcpu. > > > Ok, this explains a lot of things. > As i understand this , more VCPUs means more freedom to hypervisor to > migrate them among physical CPUs, depending on the free PCPUs available. > > In general > > domU1 > / | \ > vcpu1 vcpu2 vcpu3 > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > I mean ,domU1 can run on any vcpu , right? now vcpu1, vcpu2, vcpu3 share a > one to many reationship between pcpus[1....6]. That is a vcpu can run on any > of the pcus available to the Xen hypervisor(unless i explicitly pin it to ). > > > Is my naive understanding of what you explained is correct? > > Thank you > ~psr > > > Thanks > > Atsushi SAKAI > > > > > > "pradeep singh rautela" <rautelap@gmail.com> wrote: > > > > > Hi Atsushi, > > > > > > On 5/10/07, Atsushi SAKAI < sakaia@jp.fujitsu.com> wrote: > > > > > > > > > > > > You should show detail configuration. > > > > Your information is too short. > > > > > > > > Anyway I guess each domain has one vcpu. > > > > If so, this is normal behavior. > > > > Because one vcpu cannot allocate two or more pcpu at once. > > > > > > > > > Right, but shouldn''t Xen hypervisor be capable of migrating the VCPU > > among > > > the available PCPUs on a multiprocessor system, like in this case? And > > > criteria should be the load on the PCPU or the idle PCPUs. > > > yes/no? > > > > > > Am i missing something here? > > > > > > Thanks > > > ~psr > > > > > > Thanks > > > > Atsushi SAKAI > > > > > > > > "Harry Smith" <harry.smith272@gmail.com> wrote: > > > > > > > > > hi all, > > > > > > > > > > I am using xen3.0.3 on dual core hyperthreaded processor (in all 4 > > > > cores). > > > > > There are 2 VMs vm1,vm2 among which vm1 has a webserver running on > > it. > > > > > > > > > > While testing the performance of webserver, when I introduce some > > load > > > > on > > > > > vm2 which involves some computations the webserver performance > > goes > > > > down. > > > > > This happened even in the case where CPU usage by both of vm1,vm2 > > is > > > > > restricted to 100% each. > > > > > > > > > > Is it expected behavior ? if yes then how does one can control > > addition > > > > of > > > > > new virtual machines as adding every new VM will result in > > lowering > > > > > performance of other VMs. Through scheduling parameters we can > > just > > > > specify > > > > > amount of CPU to be used in relative sense (weight) & upper limit > > (cap). > > > > But > > > > > how to tackle this point. > > > > > > > > > > I am new in this area & wanna set up a lab using virtualization, > > so want > > > > to > > > > > find solution for this. > > > > > > > > > > thanks, > > > > > Harry > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > _______________________________________________ > > > > Xen-devel mailing list > > > > Xen-devel@lists.xensource.com > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > -- > > > --- > > > pradeep singh rautela > > > > > > "Genius is 1% inspiration, and 99% perspiration" - not me :) > > > > > > > > > -- > --- > pradeep singh rautela > > "Genius is 1% inspiration, and 99% perspiration" - not me :) >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, You should check I/O behavior. If I/O occured, other domain(vm1, vm2) data is handled by dom0 as driver domain. Thanks Atsushi SAKAI "Harry Smith" <harry.smith272@gmail.com> wrote:> Hi Atsushi & Pradeep, > > thanks for replying back. > I have 4 VCPUs for each of VM. But the point I wanted to stress upon is - > "This happened even in the case where CPU usage by both of vm1,vm2 is > restricted to 100% each. " > I had pinned all 4 VCPUs of each VM to a single phys. CPU. & I have 4 phys. > CPUs > means my vm1 was using cpu1, vm2 using cpu2 & domain-0 using cpu0,cpu3 > > Problem is when there is no load on vm2, webserver performance of vm1 is > better. But when vm2 has some compute-intense load then vm1 webserver > performance goes down. > Please note that CPU consumption of vm1 shown by xentop in both cases is > 100%, still webserver performance goes down by around 15-20%. > Even after trying to isolate two VMs, existence of load on one VM is > affecting other. > > so is it expected behavior ? > > thanks, > Harry > > > > On 5/10/07, pradeep singh rautela <rautelap@gmail.com> wrote: > > > > > > > > On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > > > > > > One vcpu can use one pcpu at one time. > > > It means 100% is maxium for one vcpu domain. > > > If you want to use cpu resources, you should set more vcpu. > > > > > > Ok, this explains a lot of things. > > As i understand this , more VCPUs means more freedom to hypervisor to > > migrate them among physical CPUs, depending on the free PCPUs available. > > > > In general > > > > domU1 > > / | \ > > vcpu1 vcpu2 vcpu3 > > > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > > > I mean ,domU1 can run on any vcpu , right? now vcpu1, vcpu2, vcpu3 share a > > one to many reationship between pcpus[1....6]. That is a vcpu can run on any > > of the pcus available to the Xen hypervisor(unless i explicitly pin it to ). > > > > > > Is my naive understanding of what you explained is correct? > > > > Thank you > > ~psr > > > > > Thanks > > > Atsushi SAKAI > > > > > > > > > "pradeep singh rautela" <rautelap@gmail.com> wrote: > > > > > > > Hi Atsushi, > > > > > > > > On 5/10/07, Atsushi SAKAI < sakaia@jp.fujitsu.com> wrote: > > > > > > > > > > > > > > > You should show detail configuration. > > > > > Your information is too short. > > > > > > > > > > Anyway I guess each domain has one vcpu. > > > > > If so, this is normal behavior. > > > > > Because one vcpu cannot allocate two or more pcpu at once. > > > > > > > > > > > > Right, but shouldn''t Xen hypervisor be capable of migrating the VCPU > > > among > > > > the available PCPUs on a multiprocessor system, like in this case? And > > > > criteria should be the load on the PCPU or the idle PCPUs. > > > > yes/no? > > > > > > > > Am i missing something here? > > > > > > > > Thanks > > > > ~psr > > > > > > > > Thanks > > > > > Atsushi SAKAI > > > > > > > > > > "Harry Smith" <harry.smith272@gmail.com> wrote: > > > > > > > > > > > hi all, > > > > > > > > > > > > I am using xen3.0.3 on dual core hyperthreaded processor (in all 4 > > > > > cores). > > > > > > There are 2 VMs vm1,vm2 among which vm1 has a webserver running on > > > it. > > > > > > > > > > > > While testing the performance of webserver, when I introduce some > > > load > > > > > on > > > > > > vm2 which involves some computations the webserver performance > > > goes > > > > > down. > > > > > > This happened even in the case where CPU usage by both of vm1,vm2 > > > is > > > > > > restricted to 100% each. > > > > > > > > > > > > Is it expected behavior ? if yes then how does one can control > > > addition > > > > > of > > > > > > new virtual machines as adding every new VM will result in > > > lowering > > > > > > performance of other VMs. Through scheduling parameters we can > > > just > > > > > specify > > > > > > amount of CPU to be used in relative sense (weight) & upper limit > > > (cap). > > > > > But > > > > > > how to tackle this point. > > > > > > > > > > > > I am new in this area & wanna set up a lab using virtualization, > > > so want > > > > > to > > > > > > find solution for this. > > > > > > > > > > > > thanks, > > > > > > Harry > > > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Xen-devel mailing list > > > > > Xen-devel@lists.xensource.com > > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > > > > -- > > > > --- > > > > pradeep singh rautela > > > > > > > > "Genius is 1% inspiration, and 99% perspiration" - not me :) > > > > > > > > > > > > > > > -- > > --- > > pradeep singh rautela > > > > "Genius is 1% inspiration, and 99% perspiration" - not me :) > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, that''s true. But dom0 has 4 VCPUs mapped on 2 physical CPU. CPU usage --> case1) when vm2 doesn''t have any load dom0 20-25% vm1 100% vm2 0% case2) when vm2 has a compute-intense load dom0 20-25% vm1 100% vm2 100% So my question is that in this case there is 200% of CPU available to dom0, still it is using only 20-25%, but in case2 webserver throughput in vm1 goes down by 15-20%. Why this is so? Why dom0 can''t use more CPU to process vm1 & vm2 requests separately ? As we are trying to show that vm1, vm2 are two OS running independetly, why they affect each other''s performance ? thanks, Harry On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote:> > Hi, > > You should check I/O behavior. > > If I/O occured, > other domain(vm1, vm2) data is handled by dom0 as driver domain. > > Thanks > Atsushi SAKAI > > > "Harry Smith" <harry.smith272@gmail.com> wrote: > > > Hi Atsushi & Pradeep, > > > > thanks for replying back. > > I have 4 VCPUs for each of VM. But the point I wanted to stress upon is > - > > "This happened even in the case where CPU usage by both of vm1,vm2 is > > restricted to 100% each. " > > I had pinned all 4 VCPUs of each VM to a single phys. CPU. & I have 4 > phys. > > CPUs > > means my vm1 was using cpu1, vm2 using cpu2 & domain-0 using cpu0,cpu3 > > > > Problem is when there is no load on vm2, webserver performance of vm1 is > > better. But when vm2 has some compute-intense load then vm1 webserver > > performance goes down. > > Please note that CPU consumption of vm1 shown by xentop in both cases is > > 100%, still webserver performance goes down by around 15-20%. > > Even after trying to isolate two VMs, existence of load on one VM is > > affecting other. > > > > so is it expected behavior ? > > > > thanks, > > Harry > > > > > > > > On 5/10/07, pradeep singh rautela <rautelap@gmail.com> wrote: > > > > > > > > > > > > On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > > > > > > > > One vcpu can use one pcpu at one time. > > > > It means 100% is maxium for one vcpu domain. > > > > If you want to use cpu resources, you should set more vcpu. > > > > > > > > > Ok, this explains a lot of things. > > > As i understand this , more VCPUs means more freedom to hypervisor to > > > migrate them among physical CPUs, depending on the free PCPUs > available. > > > > > > In general > > > > > > domU1 > > > / | \ > > > vcpu1 vcpu2 vcpu3 > > > > > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > > > > > I mean ,domU1 can run on any vcpu , right? now vcpu1, vcpu2, vcpu3 > share a > > > one to many reationship between pcpus[1....6]. That is a vcpu can run > on any > > > of the pcus available to the Xen hypervisor(unless i explicitly pin it > to ). > > > > > > > > > Is my naive understanding of what you explained is correct? > > > > > > Thank you > > > ~psr > > > > > > > Thanks > > > > Atsushi SAKAI > > > > > > > > > > > > "pradeep singh rautela" <rautelap@gmail.com> wrote: > > > > > > > > > Hi Atsushi, > > > > > > > > > > On 5/10/07, Atsushi SAKAI < sakaia@jp.fujitsu.com> wrote: > > > > > > > > > > > > > > > > > > You should show detail configuration. > > > > > > Your information is too short. > > > > > > > > > > > > Anyway I guess each domain has one vcpu. > > > > > > If so, this is normal behavior. > > > > > > Because one vcpu cannot allocate two or more pcpu at once. > > > > > > > > > > > > > > > Right, but shouldn''t Xen hypervisor be capable of migrating the > VCPU > > > > among > > > > > the available PCPUs on a multiprocessor system, like in this case? > And > > > > > criteria should be the load on the PCPU or the idle PCPUs. > > > > > yes/no? > > > > > > > > > > Am i missing something here? > > > > > > > > > > Thanks > > > > > ~psr > > > > > > > > > > Thanks > > > > > > Atsushi SAKAI > > > > > > > > > > > > "Harry Smith" <harry.smith272@gmail.com> wrote: > > > > > > > > > > > > > hi all, > > > > > > > > > > > > > > I am using xen3.0.3 on dual core hyperthreaded processor (in > all 4 > > > > > > cores). > > > > > > > There are 2 VMs vm1,vm2 among which vm1 has a webserver > running on > > > > it. > > > > > > > > > > > > > > While testing the performance of webserver, when I introduce > some > > > > load > > > > > > on > > > > > > > vm2 which involves some computations the webserver performance > > > > goes > > > > > > down. > > > > > > > This happened even in the case where CPU usage by both of > vm1,vm2 > > > > is > > > > > > > restricted to 100% each. > > > > > > > > > > > > > > Is it expected behavior ? if yes then how does one can > control > > > > addition > > > > > > of > > > > > > > new virtual machines as adding every new VM will result in > > > > lowering > > > > > > > performance of other VMs. Through scheduling parameters we > can > > > > just > > > > > > specify > > > > > > > amount of CPU to be used in relative sense (weight) & upper > limit > > > > (cap). > > > > > > But > > > > > > > how to tackle this point. > > > > > > > > > > > > > > I am new in this area & wanna set up a lab using > virtualization, > > > > so want > > > > > > to > > > > > > > find solution for this. > > > > > > > > > > > > > > thanks, > > > > > > > Harry > > > > > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > Xen-devel mailing list > > > > > > Xen-devel@lists.xensource.com > > > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > --- > > > > > pradeep singh rautela > > > > > > > > > > "Genius is 1% inspiration, and 99% perspiration" - not me :) > > > > > > > > > > > > > > > > > > > > > -- > > > --- > > > pradeep singh rautela > > > > > > "Genius is 1% inspiration, and 99% perspiration" - not me :) > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > Harry Smith > Sent: 10 May 2007 09:24 > To: Atsushi SAKAI > Cc: xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] fair scheduling > > Hi, > > that''s true. But dom0 has 4 VCPUs mapped on 2 physical CPU. > CPU usage --> > case1) when vm2 doesn''t have any load > dom0 20-25% vm1 100% vm2 0% > > case2) when vm2 has a compute-intense load > dom0 20-25% vm1 100% vm2 100% > > So my question is that in this case there is 200% of CPU > available to dom0, still it is using only 20-25%, but in > case2 webserver throughput in vm1 goes down by 15-20%. Why > this is so?Here are some architectural features that may affect things: 1. Are PCPU''s "real" CPU cores or HyperThreaded "virtual" cores? If the latter, the answer to why performance goes down is obvious... [Sorry if you covered this in an earlier mail] 2. Is it possible that cache-contention is affecting the performance? 3. Is it possible that memory bandwidth is affecting the performance? 4. Is it possible that Page-table/TLB contention is affecting performance?> Why dom0 can''t use more CPU to process vm1 & vm2 requests > separately ? As we are trying to show that vm1, vm2 are two > OS running independetly, why they affect each other''s performance ?Because there are still running on shared hardware, so one OS''s behaviour will affect the overall hardware load, perhaps? I''m not saying it is so, but I suspect it''s at least part of the answer. -- Mats> > thanks, > Harry > > > > > On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > > Hi, > > You should check I/O behavior. > > If I/O occured, > other domain(vm1, vm2) data is handled by dom0 as driver domain. > > Thanks > Atsushi SAKAI > > > "Harry Smith" < harry.smith272@gmail.com > <mailto:harry.smith272@gmail.com> > wrote: > > > Hi Atsushi & Pradeep, > > > > thanks for replying back. > > I have 4 VCPUs for each of VM. But the point I > wanted to stress upon is - > > "This happened even in the case where CPU usage by > both of vm1,vm2 is > > restricted to 100% each. " > > I had pinned all 4 VCPUs of each VM to a single phys. > CPU. & I have 4 phys. > > CPUs > > means my vm1 was using cpu1, vm2 using cpu2 & > domain-0 using cpu0,cpu3 > > > > Problem is when there is no load on vm2, webserver > performance of vm1 is > > better. But when vm2 has some compute-intense load > then vm1 webserver > > performance goes down. > > Please note that CPU consumption of vm1 shown by > xentop in both cases is > > 100%, still webserver performance goes down by around 15-20%. > > Even after trying to isolate two VMs, existence of > load on one VM is > > affecting other. > > > > so is it expected behavior ? > > > > thanks, > > Harry > > > > > > > > On 5/10/07, pradeep singh rautela <rautelap@gmail.com> wrote: > > > > > > > > > > > > On 5/10/07, Atsushi SAKAI < sakaia@jp.fujitsu.com> wrote: > > > > > > > > One vcpu can use one pcpu at one time. > > > > It means 100% is maxium for one vcpu domain. > > > > If you want to use cpu resources, you should set > more vcpu. > > > > > > > > > Ok, this explains a lot of things. > > > As i understand this , more VCPUs means more > freedom to hypervisor to > > > migrate them among physical CPUs, depending on the > free PCPUs available. > > > > > > In general > > > > > > domU1 > > > / | \ > > > vcpu1 vcpu2 vcpu3 > > > > > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > > > > > I mean ,domU1 can run on any vcpu , right? now > vcpu1, vcpu2, vcpu3 share a > > > one to many reationship between pcpus[1....6]. That > is a vcpu can run on any > > > of the pcus available to the Xen hypervisor(unless > i explicitly pin it to ). > > > > > > > > > Is my naive understanding of what you explained is correct? > > > > > > Thank you > > > ~psr > > > > > > > Thanks > > > > Atsushi SAKAI > > > > > > > > > > > > "pradeep singh rautela" <rautelap@gmail.com> wrote: > > > > > > > > > Hi Atsushi, > > > > > > > > > > On 5/10/07, Atsushi SAKAI < > sakaia@jp.fujitsu.com> wrote: > > > > > > > > > > > > > > > > > > You should show detail configuration. > > > > > > Your information is too short. > > > > > > > > > > > > Anyway I guess each domain has one vcpu. > > > > > > If so, this is normal behavior. > > > > > > Because one vcpu cannot allocate two or more > pcpu at once. > > > > > > > > > > > > > > > Right, but shouldn''t Xen hypervisor be capable > of migrating the VCPU > > > > among > > > > > the available PCPUs on a multiprocessor system, > like in this case? And > > > > > criteria should be the load on the PCPU or the > idle PCPUs. > > > > > yes/no? > > > > > > > > > > Am i missing something here? > > > > > > > > > > Thanks > > > > > ~psr > > > > > > > > > > Thanks > > > > > > Atsushi SAKAI > > > > > > > > > > > > "Harry Smith" < harry.smith272@gmail.com > <mailto:harry.smith272@gmail.com> > wrote: > > > > > > > > > > > > > hi all, > > > > > > > > > > > > > > I am using xen3.0.3 on dual core > hyperthreaded processor (in all 4 > > > > > > cores). > > > > > > > There are 2 VMs vm1,vm2 among which vm1 has > a webserver running on > > > > it. > > > > > > > > > > > > > > While testing the performance of webserver, > when I introduce some > > > > load > > > > > > on > > > > > > > vm2 which involves some computations the > webserver performance > > > > goes > > > > > > down. > > > > > > > This happened even in the case where CPU > usage by both of vm1,vm2 > > > > is > > > > > > > restricted to 100% each. > > > > > > > > > > > > > > Is it expected behavior ? if yes then how > does one can control > > > > addition > > > > > > of > > > > > > > new virtual machines as adding every new VM > will result in > > > > lowering > > > > > > > performance of other VMs. Through > scheduling parameters we can > > > > just > > > > > > specify > > > > > > > amount of CPU to be used in relative sense > (weight) & upper limit > > > > (cap). > > > > > > But > > > > > > > how to tackle this point. > > > > > > > > > > > > > > I am new in this area & wanna set up a lab > using virtualization, > > > > so want > > > > > > to > > > > > > > find solution for this. > > > > > > > > > > > > > > thanks, > > > > > > > Harry > > > > > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > Xen-devel mailing list > > > > > > Xen-devel@lists.xensource.com > > > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > --- > > > > > pradeep singh rautela > > > > > > > > > > "Genius is 1% inspiration, and 99% > perspiration" - not me :) > > > > > > > > > > > > > > > > > > > > > -- > > > --- > > > pradeep singh rautela > > > > > > "Genius is 1% inspiration, and 99% perspiration" - > not me :) > > > > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 5/10/07, Petersson, Mats <Mats.Petersson@amd.com> wrote:> > > -----Original Message----- > > From: xen-devel-bounces@lists.xensource.com > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > > Harry Smith > > Sent: 10 May 2007 09:24 > > To: Atsushi SAKAI > > Cc: xen-devel@lists.xensource.com > > Subject: Re: [Xen-devel] fair scheduling > > > > Hi, > > > > that''s true. But dom0 has 4 VCPUs mapped on 2 physical CPU. > > CPU usage --> > > case1) when vm2 doesn''t have any load > > dom0 20-25% vm1 100% vm2 0% > > > > case2) when vm2 has a compute-intense load > > dom0 20-25% vm1 100% vm2 100% > > > > So my question is that in this case there is 200% of CPU > > available to dom0, still it is using only 20-25%, but in > > case2 webserver throughput in vm1 goes down by 15-20%. Why > > this is so? > > Here are some architectural features that may affect things: > > 1. Are PCPU''s "real" CPU cores or HyperThreaded "virtual" cores? If the > latter, the answer to why performance goes down is obvious... [Sorry if > you covered this in an earlier mail]thanks Mats, yeah, I have mentioned that, but I am quite curious about knowing how this affects the performance. So bigger issue comes in my mind is that if I want to manage some dynamic workload with such hyperthreaded dual core CPUs, then how should I estimate resource usage? Is there any tool which can take my high-level QoS requirements, process them and come up with some scheduling/ workload management scheme such that resource utilization is maximized. Not only resource utilization but throughput should be maximized because in this case utilization was same in both cases, but throughput affected largely. With every increase in number of virtual machine, performance of existing VMs will degrade. Then question is how does Virtualization helps in maximizing resource utilization & giving predictable throughput to applications running inside VMs? 2. Is it possible that cache-contention is affecting the performance?> > 3. Is it possible that memory bandwidth is affecting the performance? > > 4. Is it possible that Page-table/TLB contention is affecting > performance? > > > > Why dom0 can''t use more CPU to process vm1 & vm2 requests > > separately ? As we are trying to show that vm1, vm2 are two > > OS running independetly, why they affect each other''s performance ? > > Because there are still running on shared hardware, so one OS''s > behaviour will affect the overall hardware load, perhaps? I''m not saying > it is so, but I suspect it''s at least part of the answer. > > -- > Mats > > > > thanks, > > Harry > > > > > > > > > > On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > > > > Hi, > > > > You should check I/O behavior. > > > > If I/O occured, > > other domain(vm1, vm2) data is handled by dom0 as driver domain. > > > > Thanks > > Atsushi SAKAI > > > > > > "Harry Smith" < harry.smith272@gmail.com > > <mailto:harry.smith272@gmail.com> > wrote: > > > > > Hi Atsushi & Pradeep, > > > > > > thanks for replying back. > > > I have 4 VCPUs for each of VM. But the point I > > wanted to stress upon is - > > > "This happened even in the case where CPU usage by > > both of vm1,vm2 is > > > restricted to 100% each. " > > > I had pinned all 4 VCPUs of each VM to a single phys. > > CPU. & I have 4 phys. > > > CPUs > > > means my vm1 was using cpu1, vm2 using cpu2 & > > domain-0 using cpu0,cpu3 > > > > > > Problem is when there is no load on vm2, webserver > > performance of vm1 is > > > better. But when vm2 has some compute-intense load > > then vm1 webserver > > > performance goes down. > > > Please note that CPU consumption of vm1 shown by > > xentop in both cases is > > > 100%, still webserver performance goes down by around 15-20%. > > > Even after trying to isolate two VMs, existence of > > load on one VM is > > > affecting other. > > > > > > so is it expected behavior ? > > > > > > thanks, > > > Harry > > > > > > > > > > > > On 5/10/07, pradeep singh rautela <rautelap@gmail.com> wrote: > > > > > > > > > > > > > > > > On 5/10/07, Atsushi SAKAI < sakaia@jp.fujitsu.com> wrote: > > > > > > > > > > One vcpu can use one pcpu at one time. > > > > > It means 100% is maxium for one vcpu domain. > > > > > If you want to use cpu resources, you should set > > more vcpu. > > > > > > > > > > > > Ok, this explains a lot of things. > > > > As i understand this , more VCPUs means more > > freedom to hypervisor to > > > > migrate them among physical CPUs, depending on the > > free PCPUs available. > > > > > > > > In general > > > > > > > > domU1 > > > > / | \ > > > > vcpu1 vcpu2 vcpu3 > > > > > > > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > > > > > > > I mean ,domU1 can run on any vcpu , right? now > > vcpu1, vcpu2, vcpu3 share a > > > > one to many reationship between pcpus[1....6]. That > > is a vcpu can run on any > > > > of the pcus available to the Xen hypervisor(unless > > i explicitly pin it to ). > > > > > > > > > > > > Is my naive understanding of what you explained is correct? > > > > > > > > Thank you > > > > ~psr > > > > > > > > > Thanks > > > > > Atsushi SAKAI > > > > > > > > > > > > > > > "pradeep singh rautela" <rautelap@gmail.com> wrote: > > > > > > > > > > > Hi Atsushi, > > > > > > > > > > > > On 5/10/07, Atsushi SAKAI < > > sakaia@jp.fujitsu.com> wrote: > > > > > > > > > > > > > > > > > > > > > You should show detail configuration. > > > > > > > Your information is too short. > > > > > > > > > > > > > > Anyway I guess each domain has one vcpu. > > > > > > > If so, this is normal behavior. > > > > > > > Because one vcpu cannot allocate two or more > > pcpu at once. > > > > > > > > > > > > > > > > > > Right, but shouldn''t Xen hypervisor be capable > > of migrating the VCPU > > > > > among > > > > > > the available PCPUs on a multiprocessor system, > > like in this case? And > > > > > > criteria should be the load on the PCPU or the > > idle PCPUs. > > > > > > yes/no? > > > > > > > > > > > > Am i missing something here? > > > > > > > > > > > > Thanks > > > > > > ~psr > > > > > > > > > > > > Thanks > > > > > > > Atsushi SAKAI > > > > > > > > > > > > > > "Harry Smith" < harry.smith272@gmail.com > > <mailto:harry.smith272@gmail.com> > wrote: > > > > > > > > > > > > > > > hi all, > > > > > > > > > > > > > > > > I am using xen3.0.3 on dual core > > hyperthreaded processor (in all 4 > > > > > > > cores). > > > > > > > > There are 2 VMs vm1,vm2 among which vm1 has > > a webserver running on > > > > > it. > > > > > > > > > > > > > > > > While testing the performance of webserver, > > when I introduce some > > > > > load > > > > > > > on > > > > > > > > vm2 which involves some computations the > > webserver performance > > > > > goes > > > > > > > down. > > > > > > > > This happened even in the case where CPU > > usage by both of vm1,vm2 > > > > > is > > > > > > > > restricted to 100% each. > > > > > > > > > > > > > > > > Is it expected behavior ? if yes then how > > does one can control > > > > > addition > > > > > > > of > > > > > > > > new virtual machines as adding every new VM > > will result in > > > > > lowering > > > > > > > > performance of other VMs. Through > > scheduling parameters we can > > > > > just > > > > > > > specify > > > > > > > > amount of CPU to be used in relative sense > > (weight) & upper limit > > > > > (cap). > > > > > > > But > > > > > > > > how to tackle this point. > > > > > > > > > > > > > > > > I am new in this area & wanna set up a lab > > using virtualization, > > > > > so want > > > > > > > to > > > > > > > > find solution for this. > > > > > > > > > > > > > > > > thanks, > > > > > > > > Harry > > > > > > > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Xen-devel mailing list > > > > > > > Xen-devel@lists.xensource.com > > > > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > --- > > > > > > pradeep singh rautela > > > > > > > > > > > > "Genius is 1% inspiration, and 99% > > perspiration" - not me :) > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > --- > > > > pradeep singh rautela > > > > > > > > "Genius is 1% inspiration, and 99% perspiration" - > > not me :) > > > > > > > > > > > > > > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
hi everyone,
in given case, why performance of one vm affects other vm ? if VMs are said to
be independent, then why one VM''s load affects the performance of other
VM?
as shown by Harry, in both cases CPU consumption by VM is same then why
webserver performance degrades ?? Does xen doesn''t provide performance
isolation ?
regards,
-Rahul
Hi Atsushi & Pradeep,
thanks for replying back.
I have 4 VCPUs for each of VM. But the point I wanted to stress upon is -
"This happened even in the case where CPU usage by both of vm1,vm2 is
restricted to 100% each. "
I had pinned all 4 VCPUs of each VM to a single phys. CPU. & I have 4 phys.
CPUs
means my vm1 was using cpu1, vm2 using cpu2 & domain-0 using cpu0,cpu3
Problem is when there is no load on vm2, webserver performance of vm1 is better.
But when vm2 has some compute-intense load then vm1 webserver performance goes
down.
Please note that CPU consumption of vm1 shown by xentop in both cases is 100%,
still webserver performance goes down by around 15-20%.
Even after trying to isolate two VMs, existence of load on one VM is affecting
other.
so is it expected behavior ?
thanks,
Harry
On 5/10/07, pradeep singh rautela <rautelap@xxxxxxxxx > wrote:
On 5/10/07, Atsushi SAKAI <sakaia@xxxxxxxxxxxxxx> wrote: One vcpu can
use one pcpu at one time.
It means 100% is maxium for one vcpu domain.
If you want to use cpu resources, you should set more vcpu.
Ok, this explains a lot of things.
As i understand this , more VCPUs means more freedom to hypervisor to migrate
them among physical CPUs, depending on the free PCPUs available.
In general
domU1
/ | \
vcpu1 vcpu2 vcpu3
pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6
I mean ,domU1 can run on any vcpu , right? now vcpu1, vcpu2, vcpu3 share a one
to many reationship between pcpus[1....6]. That is a vcpu can run on any of the
pcus available to the Xen hypervisor(unless i explicitly pin it to ).
Is my naive understanding of what you explained is correct?
Thank you
~psr
Thanks
Atsushi SAKAI
"pradeep singh rautela" <rautelap@xxxxxxxxx> wrote:
> Hi Atsushi,>
> On 5/10/07, Atsushi SAKAI < sakaia@xxxxxxxxxxxxxx> wrote:
> >
> >
> > You should show detail configuration.
> > Your information is too short.
> >
> > Anyway I guess each domain has one vcpu.
> > If so, this is normal behavior.
> > Because one vcpu cannot allocate two or more pcpu at once.
>
>
> Right, but shouldn''t Xen hypervisor be capable of migrating the
VCPU among
> the available PCPUs on a multiprocessor system, like in this case?
And> criteria should be the load on the PCPU or the idle PCPUs.
> yes/no?
>
> Am i missing something here?
>
> Thanks
> ~psr
>
> Thanks
> > Atsushi SAKAI
> >
> > "Harry Smith" < harry.smith272@xxxxxxxxx> wrote:
> >
> > > hi all,
> > >
> > > I am using xen3.0.3 on dual core hyperthreaded processor (in all
4
> > cores).
> > > There are 2 VMs vm1,vm2 among which vm1 has a webserver running
on it.
> > >
> > > While testing the performance of webserver, when I introduce some
load
> > on
> > > vm2 which involves some computations the webserver performance
goes
> > down.
> > > This happened even in the case where CPU usage by both of vm1,vm2
is
> > > restricted to 100% each.
> > >
> > > Is it expected behavior ? if yes then how does one can control
addition
> > of
> > > new virtual machines as adding every new VM will result in
lowering
> > > performance of other VMs. Through scheduling parameters we can
just
> > specify
> > > amount of CPU to be used in relative sense (weight) & upper
limit (cap).
> > But
> > > how to tackle this point.> > >
> > > I am new in this area & wanna set up a lab using
virtualization, so want
> > to
> > > find solution for this.
> > >
> > > thanks,
> > > Harry
> > >
> > > we always have a choice...
> >
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> >
>
>
>
> --
> ---
> pradeep singh rautela
>
> "Genius is 1% inspiration, and 99% perspiration" - not me :)
--
---
pradeep singh rautela
"Genius is 1% inspiration, and 99% perspiration" - not me :)
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
---------------------------------
Office firewalls, cyber cafes, college labs, don''t allow you to
download CHAT? Here''s a solution!
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > Harry Smith > Sent: 10 May 2007 12:31 > To: Petersson, Mats > Cc: Atsushi SAKAI; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] fair scheduling > > > > On 5/10/07, Petersson, Mats <Mats.Petersson@amd.com> wrote: > > > -----Original Message----- > > From: xen-devel-bounces@lists.xensource.com > > [mailto:xen-devel-bounces@lists.xensource.com ] On Behalf Of > > Harry Smith > > Sent: 10 May 2007 09:24 > > To: Atsushi SAKAI > > Cc: xen-devel@lists.xensource.com > > Subject: Re: [Xen-devel] fair scheduling > > > > Hi, > > > > that''s true. But dom0 has 4 VCPUs mapped on 2 physical CPU. > > CPU usage --> > > case1) when vm2 doesn''t have any load > > dom0 20-25% vm1 100% vm2 0% > > > > case2) when vm2 has a compute-intense load > > dom0 20-25% vm1 100% vm2 100% > > > > So my question is that in this case there is 200% of CPU > > available to dom0, still it is using only 20-25%, but in > > case2 webserver throughput in vm1 goes down by 15-20%. Why > > this is so? > > Here are some architectural features that may affect things: > > 1. Are PCPU''s "real" CPU cores or HyperThreaded > "virtual" cores? If the > latter, the answer to why performance goes down is > obvious... [Sorry if > you covered this in an earlier mail] > > > thanks Mats, > yeah, I have mentioned that, but I am quite curious about > knowing how this affects the performance. > So bigger issue comes in my mind is that if I want to manage > some dynamic workload with such hyperthreaded dual core CPUs, > then how should I estimate resource usage? Is there any tool > which can take my high-level QoS requirements, process them > and come up with some scheduling/ workload management scheme > such that resource utilization is maximized. Not only > resource utilization but throughput should be maximized > because in this case utilization was same in both cases, but > throughput affected largely.Ok, let''s start with one important factor here: CPU-load and actual performance are two different things. CPU-load isn''t easy to measure (on any reasonably modern processor at least), because the processor, applicaton and OS interactions are very complex. The only thing we can reasonably accurately measure is CPU idle-time. So when you see 100% CPU-load, it actually means that it''s spending 0% of the time in idle. How it uses that time "not in idle" is not necessarily the same when you put the processor in different circumstances, e.g. the second core on the same socket being more or less used, memory or I/O operations being more or less congested, etc, etc. One of the real problems here is cache-utilization (and collisions in the cache). TLB collisions/pressure can also cause similar increases in latency for individual instructions - so when an instruction when there''s nothing else to run takes (on average) 4 clock-cycles, when there''s a competing process[1] running on the second core, the instruction takes 7 cycles, because the processor had to reload the cache and/or TLB every. Another factor is simply that the increased load on the memory bus, where the first and second processor(core) competes for a shared resource. If there is only one process running, then there is less congestion on the "road" to the processor, whilst if there is two processes both using the memory bus intensely, there will be heavy congestion (not helped much by memory devices being more likely to need full addressing, rather than "within page"[2] addressing, which increases the memory access time noticably for each of those accesses. I''m not aware of a specific tool that does load balancing and such automagically. I don''t really know how to solve the problem (other than the obvious suggestion of "underestimate what the processor can do at full load"), I''m only trying to explain that it''s not just a case of adjusting the scheduler. The scheduler (whether we talk of the current or some new fancy one) will not be able to determine how well or bad a particular domain will put pressure on the memory bus for example. And on the account of your specific processor model: you have a dual-core, dual hyperthread processor. This essentially means that most of the CPU resources when running two threads on the the same physical core are shared. This in turn means that Core0, HT(0,1) will be affected by each other (i.e. increase in load on HT0 will affect HT1 and vice versa. Core1, HT(0,1) will affect each other. All four hyperthreads share the same IO and memory bus, so if you have some load on the memory bus from the first core, the second core will be affected by this.> > With every increase in number of virtual machine, performance > of existing VMs will degrade. Then question is how does > Virtualization helps in maximizing resource utilization & > giving predictable throughput to applications running inside VMs?It helps in cases where you have, say, three different servers, that each have a longish peak CPU-load of 15%, and you combine all three into one physical machine. But you will use more than 3 x 15% CPU-load to achieve the same functionality, because the management of the virtual machines will take some effort from the CPU. If you try to squeeze three 100% loads from single CPU servers into a three virtual machines on a 4-CPU server, then you may not get the same level of performance - but it does depend A LOT on what the exactly what the two guests are doing. [1] I use the word process here quite loosely. It could be another thread of the same application just as much as another virtual machine. [2] Memories have a few small buffer, called "pages", which contain a (copy of) small amount of memory content. Each time a memory location not covered by a "currently open" page, the memory controller must issue a "full address", which means two sets of address cycles to the memory chips, rather than a single "address within page". -- Mats> > > > > > 2. Is it possible that cache-contention is affecting > the performance? > > 3. Is it possible that memory bandwidth is affecting > the performance? > > 4. Is it possible that Page-table/TLB contention is affecting > performance? > > > > Why dom0 can''t use more CPU to process vm1 & vm2 requests > > separately ? As we are trying to show that vm1, vm2 are two > > OS running independetly, why they affect each other''s > performance ? > > Because there are still running on shared hardware, so one OS''s > behaviour will affect the overall hardware load, > perhaps? I''m not saying > it is so, but I suspect it''s at least part of the answer. > > -- > Mats > > > > thanks, > > Harry > > > > > > > > > > On 5/10/07, Atsushi SAKAI <sakaia@jp.fujitsu.com> wrote: > > > > Hi, > > > > You should check I/O behavior. > > > > If I/O occured, > > other domain(vm1, vm2) data is handled by dom0 > as driver domain. > > > > Thanks > > Atsushi SAKAI > > > > > > "Harry Smith" < harry.smith272@gmail.com > <mailto:harry.smith272@gmail.com> > > <mailto:harry.smith272@gmail.com> > wrote: > > > > > Hi Atsushi & Pradeep, > > > > > > thanks for replying back. > > > I have 4 VCPUs for each of VM. But the point I > > wanted to stress upon is - > > > "This happened even in the case where CPU usage by > > both of vm1,vm2 is > > > restricted to 100% each. " > > > I had pinned all 4 VCPUs of each VM to a single phys. > > CPU. & I have 4 phys. > > > CPUs > > > means my vm1 was using cpu1, vm2 using cpu2 & > > domain-0 using cpu0,cpu3 > > > > > > Problem is when there is no load on vm2, webserver > > performance of vm1 is > > > better. But when vm2 has some compute-intense load > > then vm1 webserver > > > performance goes down. > > > Please note that CPU consumption of vm1 shown by > > xentop in both cases is > > > 100%, still webserver performance goes down > by around 15-20%. > > > Even after trying to isolate two VMs, existence of > > load on one VM is > > > affecting other. > > > > > > so is it expected behavior ? > > > > > > thanks, > > > Harry > > > > > > > > > > > > On 5/10/07, pradeep singh rautela > <rautelap@gmail.com> wrote: > > > > > > > > > > > > > > > > On 5/10/07, Atsushi SAKAI < > sakaia@jp.fujitsu.com> wrote: > > > > > > > > > > One vcpu can use one pcpu at one time. > > > > > It means 100% is maxium for one vcpu domain. > > > > > If you want to use cpu resources, you should set > > more vcpu. > > > > > > > > > > > > Ok, this explains a lot of things. > > > > As i understand this , more VCPUs means more > > freedom to hypervisor to > > > > migrate them among physical CPUs, depending on the > > free PCPUs available. > > > > > > > > In general > > > > > > > > domU1 > > > > / | \ > > > > vcpu1 vcpu2 vcpu3 > > > > > > > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > > > > > > > I mean ,domU1 can run on any vcpu , right? now > > vcpu1, vcpu2, vcpu3 share a > > > > one to many reationship between pcpus[1....6]. That > > is a vcpu can run on any > > > > of the pcus available to the Xen hypervisor(unless > > i explicitly pin it to ). > > > > > > > > > > > > Is my naive understanding of what you > explained is correct? > > > > > > > > Thank you > > > > ~psr > > > > > > > > > Thanks > > > > > Atsushi SAKAI > > > > > > > > > > > > > > > "pradeep singh rautela" > <rautelap@gmail.com> wrote: > > > > > > > > > > > Hi Atsushi, > > > > > > > > > > > > On 5/10/07, Atsushi SAKAI < > > sakaia@jp.fujitsu.com> wrote: > > > > > > > > > > > > > > > > > > > > > You should show detail configuration. > > > > > > > Your information is too short. > > > > > > > > > > > > > > Anyway I guess each domain has one vcpu. > > > > > > > If so, this is normal behavior. > > > > > > > Because one vcpu cannot allocate two or more > > pcpu at once. > > > > > > > > > > > > > > > > > > Right, but shouldn''t Xen hypervisor be capable > > of migrating the VCPU > > > > > among > > > > > > the available PCPUs on a multiprocessor system, > > like in this case? And > > > > > > criteria should be the load on the PCPU or the > > idle PCPUs. > > > > > > yes/no? > > > > > > > > > > > > Am i missing something here? > > > > > > > > > > > > Thanks > > > > > > ~psr > > > > > > > > > > > > Thanks > > > > > > > Atsushi SAKAI > > > > > > > > > > > > > > "Harry Smith" < > harry.smith272@gmail.com <mailto:harry.smith272@gmail.com> > > <mailto:harry.smith272@gmail.com> > wrote: > > > > > > > > > > > > > > > hi all, > > > > > > > > > > > > > > > > I am using xen3.0.3 on dual core > > hyperthreaded processor (in all 4 > > > > > > > cores). > > > > > > > > There are 2 VMs vm1,vm2 among which vm1 has > > a webserver running on > > > > > it. > > > > > > > > > > > > > > > > While testing the performance of webserver, > > when I introduce some > > > > > load > > > > > > > on > > > > > > > > vm2 which involves some computations the > > webserver performance > > > > > goes > > > > > > > down. > > > > > > > > This happened even in the case where CPU > > usage by both of vm1,vm2 > > > > > is > > > > > > > > restricted to 100% each. > > > > > > > > > > > > > > > > Is it expected behavior ? if yes then how > > does one can control > > > > > addition > > > > > > > of > > > > > > > > new virtual machines as adding every new VM > > will result in > > > > > lowering > > > > > > > > performance of other VMs. Through > > scheduling parameters we can > > > > > just > > > > > > > specify > > > > > > > > amount of CPU to be used in relative sense > > (weight) & upper limit > > > > > (cap). > > > > > > > But > > > > > > > > how to tackle this point. > > > > > > > > > > > > > > > > I am new in this area & wanna set up a lab > > using virtualization, > > > > > so want > > > > > > > to > > > > > > > > find solution for this. > > > > > > > > > > > > > > > > thanks, > > > > > > > > Harry > > > > > > > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Xen-devel mailing list > > > > > > > Xen-devel@lists.xensource.com > > > > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > --- > > > > > > pradeep singh rautela > > > > > > > > > > > > "Genius is 1% inspiration, and 99% > > perspiration" - not me :) > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > --- > > > > pradeep singh rautela > > > > > > > > "Genius is 1% inspiration, and 99% perspiration" - > > not me :) > > > > > > > > > > > > > > > > > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > rahul gundecha > Sent: 10 May 2007 13:00 > To: harry.smith272@gmail.com; xen devel > Subject: Re: [Xen-devel] fair scheduling > > hi everyone, > > in given case, why performance of one vm affects other vm ? > if VMs are said to be independent, then why one VM''s load > affects the performance of other VM? > > as shown by Harry, in both cases CPU consumption by VM is > same then why webserver performance degrades ?? Does xen > doesn''t provide performance isolation ?Well, it does have caps and weights to allow the scheduler to distribute the CPU performance evenly, but it is, as I explained in a different post just a few seconds ago, not easy to determine such things as the effect of memory, cache, TLB and I/O operations. If one CPU is more busy because it takes longer to get memory, how does the scheduler know this (and more importantly, what should the scheduler DO about it)? If VM''s are purely CPU-bound, such as some very simple calculation (small enough that both the code and data fits in cache nicely for all VM''s at the same time) should work out nicely fair. When it gets more complex, where memory, cache and IO operations get mixed into the equation, it''s so much more complex. In Harry''s case, the matter is further complicated by the fact that half of the "CPU''s" are virtual CPU''s (hyperthreads), which adds some benefit, but certainly not as much as a real core, so expecting to get 400% CPU performance out of 4 hyperthreads is far above what you might expect. Finally, [I should have thought this through earlier and added it to the previous post to Harry] part of the missing percentage in Harry''s case is probaly due to the fact that with one active VM and Dom0 able to run on two different cores, the load on Dom0 can wholly fit in Core1 without affecting Core0''s execution at all, where when the vm2 is using 100% cpu load, both Core0 and Core1 are fully loaded, which means that the 25% load on Dom0 will have to be shared out across Core0 and Core1. This is probably the biggest factor in this case - not memory or IO load. -- Mats> > regards, > -Rahul > > > > Hi Atsushi & Pradeep, > > thanks for replying back. > I have 4 VCPUs for each of VM. But the point I wanted > to stress upon is - > "This happened even in the case where CPU usage by both > of vm1,vm2 is restricted to 100% each. " > I had pinned all 4 VCPUs of each VM to a single phys. > CPU. & I have 4 phys. CPUs > means my vm1 was using cpu1, vm2 using cpu2 & domain-0 > using cpu0,cpu3 > > Problem is when there is no load on vm2, webserver > performance of vm1 is better. But when vm2 has some > compute-intense load then vm1 webserver performance goes down. > Please note that CPU consumption of vm1 shown by xentop > in both cases is 100%, still webserver performance goes down > by around 15-20%. > Even after trying to isolate two VMs, existence of load > on one VM is affecting other. > > so is it expected behavior ? > > thanks, > Harry > > > > > On 5/10/07, pradeep singh rautela <rautelap@xxxxxxxxx > wrote: > > > > On 5/10/07, Atsushi SAKAI <sakaia@xxxxxxxxxxxxxx> wrote: > > One vcpu can use one pcpu at one time. > It means 100% is maxium for one vcpu domain. > If you want to use cpu resources, you > should set more vcpu. > > > Ok, this explains a lot of things. > As i understand this , more VCPUs means more > freedom to hypervisor to migrate them among physical CPUs, > depending on the free PCPUs available. > > > In general > > domU1 > / | \ > vcpu1 vcpu2 vcpu3 > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > I mean ,domU1 can run on any vcpu , right? now > vcpu1, vcpu2, vcpu3 share a one to many reationship between > pcpus[1....6]. That is a vcpu can run on any of the pcus > available to the Xen hypervisor(unless i explicitly pin it to ). > > Is my naive understanding of what you explained > is correct? > > Thank you > ~psr > > > > Thanks > Atsushi SAKAI > > > "pradeep singh rautela" > <rautelap@xxxxxxxxx> wrote: > > > Hi Atsushi, > > > > On 5/10/07, Atsushi SAKAI < > sakaia@xxxxxxxxxxxxxx <mailto:sakaia@xxxxxxxxxxxxxx> > wrote: > > > > > > > > > You should show detail configuration. > > > Your information is too short. > > > > > > Anyway I guess each domain has one vcpu. > > > If so, this is normal behavior. > > > Because one vcpu cannot allocate > two or more pcpu at once. > > > > > > Right, but shouldn''t Xen hypervisor > be capable of migrating the VCPU among > > the available PCPUs on a > multiprocessor system, like in this case? And > > criteria should be the load on the > PCPU or the idle PCPUs. > > yes/no? > > > > Am i missing something here? > > > > Thanks > > ~psr > > > > Thanks > > > Atsushi SAKAI > > > > > > "Harry Smith" < > harry.smith272@xxxxxxxxx <mailto:harry.smith272@xxxxxxxxx> > wrote: > > > > > > > hi all, > > > > > > > > I am using xen3.0.3 on dual core > hyperthreaded processor (in all 4 > > > cores). > > > > There are 2 VMs vm1,vm2 among > which vm1 has a webserver running on it. > > > > > > > > While testing the performance of > webserver, when I introduce some load > > > on > > > > vm2 which involves some > computations the webserver performance goes > > > down. > > > > This happened even in the case > where CPU usage by both of vm1,vm2 is > > > > restricted to 100% each. > > > > > > > > Is it expected behavior ? if yes > then how does one can control addition > > > of > > > > new virtual machines as adding > every new VM will result in lowering > > > > performance of other VMs. > Through scheduling parameters we can just > > > specify > > > > amount of CPU to be used in > relative sense (weight) & upper limit (cap). > > > But > > > > how to tackle this point. > > > > > > > > I am new in this area & wanna set > up a lab using virtualization, so want > > > to > > > > find solution for this. > > > > > > > > thanks, > > > > Harry > > > > > > > > we always have a choice... > > > > > > > > > > > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@xxxxxxxxxxxxxxxxxxx > > > > http://lists.xensource.com/xen-devel > <http://lists.xensource.com/xen-devel> > > > > > > > > > > > -- > > --- > > pradeep singh rautela > > > > "Genius is 1% inspiration, and 99% > perspiration" - not me :) > > > > > > > > -- > --- > pradeep singh rautela > > "Genius is 1% inspiration, and 99% > perspiration" - not me :) > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@xxxxxxxxxxxxxxxxxxx > http://lists.xensource.com/xen-devel > > > > ________________________________ > > Office firewalls, cyber cafes, college labs, don''t allow you > to download CHAT? Here''s a solution! > <http://us.rd.yahoo.com/mail/in/ymessenger/*http://in.messenge > r.yahoo.com/webmessengerpromo.php> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
thanks alot mats for detailed clarification, it really helped me as well. coming to the issue of hyperthreded CPU, 1) if we can''t expect that they will fetch 200% of performance then whats the benefit from using them ? ( I dont know whats prime use of hyperthreding, if this is the real scenario... ) 2) However given some processors, how to go about designing workload management. Means how I can atleast estimate resource usage. If I need to know that how much load my infrastructure can support, while considering that load can be of mixed type. As you said " but it does depend A LOT on what the exactly what the two guests are doing " , how to do such analysis ? As it seems that processor architecture will be major player. What''s current approach regarding this. I read that data centres are increasingly using virtualization, in that case how above said process is carried out. "Petersson, Mats" <Mats.Petersson@amd.com> wrote:> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > rahul gundecha > Sent: 10 May 2007 13:00 > To: harry.smith272@gmail.com; xen devel > Subject: Re: [Xen-devel] fair scheduling > > hi everyone, > > in given case, why performance of one vm affects other vm ? > if VMs are said to be independent, then why one VM''s load > affects the performance of other VM? > > as shown by Harry, in both cases CPU consumption by VM is > same then why webserver performance degrades ?? Does xen > doesn''t provide performance isolation ?Well, it does have caps and weights to allow the scheduler to distribute the CPU performance evenly, but it is, as I explained in a different post just a few seconds ago, not easy to determine such things as the effect of memory, cache, TLB and I/O operations. If one CPU is more busy because it takes longer to get memory, how does the scheduler know this (and more importantly, what should the scheduler DO about it)? If VM''s are purely CPU-bound, such as some very simple calculation (small enough that both the code and data fits in cache nicely for all VM''s at the same time) should work out nicely fair. When it gets more complex, where memory, cache and IO operations get mixed into the equation, it''s so much more complex. In Harry''s case, the matter is further complicated by the fact that half of the "CPU''s" are virtual CPU''s (hyperthreads), which adds some benefit, but certainly not as much as a real core, so expecting to get 400% CPU performance out of 4 hyperthreads is far above what you might expect. Finally, [I should have thought this through earlier and added it to the previous post to Harry] part of the missing percentage in Harry''s case is probaly due to the fact that with one active VM and Dom0 able to run on two different cores, the load on Dom0 can wholly fit in Core1 without affecting Core0''s execution at all, where when the vm2 is using 100% cpu load, both Core0 and Core1 are fully loaded, which means that the 25% load on Dom0 will have to be shared out across Core0 and Core1. This is probably the biggest factor in this case - not memory or IO load. -- Mats> > regards, > -Rahul > > > > Hi Atsushi & Pradeep, > > thanks for replying back. > I have 4 VCPUs for each of VM. But the point I wanted > to stress upon is - > "This happened even in the case where CPU usage by both > of vm1,vm2 is restricted to 100% each. " > I had pinned all 4 VCPUs of each VM to a single phys. > CPU. & I have 4 phys. CPUs > means my vm1 was using cpu1, vm2 using cpu2 & domain-0 > using cpu0,cpu3 > > Problem is when there is no load on vm2, webserver > performance of vm1 is better. But when vm2 has some > compute-intense load then vm1 webserver performance goes down. > Please note that CPU consumption of vm1 shown by xentop > in both cases is 100%, still webserver performance goes down > by around 15-20%. > Even after trying to isolate two VMs, existence of load > on one VM is affecting other. > > so is it expected behavior ? > > thanks, > Harry > > > > > On 5/10/07, pradeep singh rautela wrote: > > > > On 5/10/07, Atsushi SAKAI wrote: > > One vcpu can use one pcpu at one time. > It means 100% is maxium for one vcpu domain. > If you want to use cpu resources, you > should set more vcpu. > > > Ok, this explains a lot of things. > As i understand this , more VCPUs means more > freedom to hypervisor to migrate them among physical CPUs, > depending on the free PCPUs available. > > > In general > > domU1 > / | \ > vcpu1 vcpu2 vcpu3 > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > I mean ,domU1 can run on any vcpu , right? now > vcpu1, vcpu2, vcpu3 share a one to many reationship between > pcpus[1....6]. That is a vcpu can run on any of the pcus > available to the Xen hypervisor(unless i explicitly pin it to ). > > Is my naive understanding of what you explained > is correct? > > Thank you > ~psr > > > > Thanks > Atsushi SAKAI > > > "pradeep singh rautela" > wrote: > > > Hi Atsushi, > > > > On 5/10/07, Atsushi SAKAI < > sakaia@xxxxxxxxxxxxxx > wrote: > > > > > > > > > You should show detail configuration. > > > Your information is too short. > > > > > > Anyway I guess each domain has one vcpu. > > > If so, this is normal behavior. > > > Because one vcpu cannot allocate > two or more pcpu at once. > > > > > > Right, but shouldn''t Xen hypervisor > be capable of migrating the VCPU among > > the available PCPUs on a > multiprocessor system, like in this case? And > > criteria should be the load on the > PCPU or the idle PCPUs. > > yes/no? > > > > Am i missing something here? > > > > Thanks > > ~psr > > > > Thanks > > > Atsushi SAKAI > > > > > > "Harry Smith" < > harry.smith272@xxxxxxxxx > wrote: > > > > > > > hi all, > > > > > > > > I am using xen3.0.3 on dual core > hyperthreaded processor (in all 4 > > > cores). > > > > There are 2 VMs vm1,vm2 among > which vm1 has a webserver running on it. > > > > > > > > While testing the performance of > webserver, when I introduce some load > > > on > > > > vm2 which involves some > computations the webserver performance goes > > > down. > > > > This happened even in the case > where CPU usage by both of vm1,vm2 is > > > > restricted to 100% each. > > > > > > > > Is it expected behavior ? if yes > then how does one can control addition > > > of > > > > new virtual machines as adding > every new VM will result in lowering > > > > performance of other VMs. > Through scheduling parameters we can just > > > specify > > > > amount of CPU to be used in > relative sense (weight) & upper limit (cap). > > > But > > > > how to tackle this point. > > > > > > > > I am new in this area & wanna set > up a lab using virtualization, so want > > > to > > > > find solution for this. > > > > > > > > thanks, > > > > Harry > > > > > > > > we always have a choice... > > > > > > > > > > > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@xxxxxxxxxxxxxxxxxxx > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > -- > > --- > > pradeep singh rautela > > > > "Genius is 1% inspiration, and 99% > perspiration" - not me :) > > > > > > > > -- > --- > pradeep singh rautela > > "Genius is 1% inspiration, and 99% > perspiration" - not me :) > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@xxxxxxxxxxxxxxxxxxx > http://lists.xensource.com/xen-devel > > > > ________________________________ > > Office firewalls, cyber cafes, college labs, don''t allow you > to download CHAT? Here''s a solution! > > r.yahoo.com/webmessengerpromo.php> >-Rahooooooooooooooool... exceptions are most common things to happen..... --------------------------------- Heres a new way to find what you''re looking for - Yahoo! Answers --0-1077727752-1178807910=:64904 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: 8bit thanks alot mats for detailed clarification, it really helped me as well.<br> <br> coming to the issue of hyperthreded CPU, <br> 1) if we can''t expect that they will fetch 200% of performance then whats the benefit from using them ? ( I dont know whats prime use of hyperthreding, if this is the real scenario... )<br> 2) However given some processors, how to go about designing workload management. Means how I can atleast estimate resource usage. If I need to know that how much load my infrastructure can support, while considering that load can be of mixed type. As you said " but it does depend A LOT on what the exactly what the two guests are doing " , how to do such analysis ? As it seems that processor architecture will be major player. What''s current approach regarding this.<br><br>I read that data centres are increasingly using virtualization, in that case how above said process is carried out. <br><br><b><i>"Petersson, Mats" <Mats.Petersson@amd.com></i></b> wrote:<blockquote class="replbq" style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"> <br><br>> -----Original Message-----<br>> From: xen-devel-bounces@lists.xensource.com <br>> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of <br>> rahul gundecha<br>> Sent: 10 May 2007 13:00<br>> To: harry.smith272@gmail.com; xen devel<br>> Subject: Re: [Xen-devel] fair scheduling<br>> <br>> hi everyone,<br>> <br>> in given case, why performance of one vm affects other vm ? <br>> if VMs are said to be independent, then why one VM''s load <br>> affects the performance of other VM?<br>> <br>> as shown by Harry, in both cases CPU consumption by VM is <br>> same then why webserver performance degrades ?? Does xen <br>> doesn''t provide performance isolation ?<br><br>Well, it does have caps and weights to allow the scheduler to distribute<br>the CPU performance evenly, but it is, as I explained in a different<br>post just a few seconds ago, not easy to determine such things as the<br>effect of memory, cache, TLB and I/O operations. If one CPU is more busy<br>because it takes longer to get memory, how does the scheduler know this<br>(and more importantly, what should the scheduler DO about it)?<br><br><br>If VM''s are purely CPU-bound, such as some very simple calculation<br>(small enough that both the code and data fits in cache nicely for all<br>VM''s at the same time) should work out nicely fair. When it gets more<br>complex, where memory, cache and IO operations get mixed into the<br>equation, it''s so much more complex. <br><br>In Harry''s case, the matter is further complicated by the fact that half<br>of the "CPU''s" are virtual CPU''s (hyperthreads), which adds some<br>benefit, but certainly not as much as a real core, so expecting to get<br>400% CPU performance out of 4 hyperthreads is far above what you might<br>expect. <br><br>Finally, [I should have thought this through earlier and added it to the<br>previous post to Harry] part of the missing percentage in Harry''s case<br>is probaly due to the fact that with one active VM and Dom0 able to run<br>on two different cores, the load on Dom0 can wholly fit in Core1 without<br>affecting Core0''s execution at all, where when the vm2 is using 100% cpu<br>load, both Core0 and Core1 are fully loaded, which means that the 25%<br>load on Dom0 will have to be shared out across Core0 and Core1. This is<br>probably the biggest factor in this case - not memory or IO load.<br><br>--<br>Mats<br>> <br>> regards,<br>> -Rahul<br>> <br>> <br>> <br>> Hi Atsushi & Pradeep,<br>> <br>> thanks for replying back.<br>> I have 4 VCPUs for each of VM. But the point I wanted <br>> to stress upon is - <br>> "This happened even in the case where CPU usage by both <br>> of vm1,vm2 is restricted to 100% each. "<br>> I had pinned all 4 VCPUs of each VM to a single phys. <br>> CPU. & I have 4 phys. CPUs<br>> means my vm1 was using cpu1, vm2 using cpu2 & domain-0 <br>> using cpu0,cpu3 <br>> <br>> Problem is when there is no load on vm2, webserver <br>> performance of vm1 is better. But when vm2 has some <br>> compute-intense load then vm1 webserver performance goes down.<br>> Please note that CPU consumption of vm1 shown by xentop <br>> in both cases is 100%, still webserver performance goes down <br>> by around 15-20%.<br>> Even after trying to isolate two VMs, existence of load <br>> on one VM is affecting other. <br>> <br>> so is it expected behavior ?<br>> <br>> thanks,<br>> Harry<br>> <br>> <br>> <br>> <br>> On 5/10/07, pradeep singh rautela <rautelap@xxxxxxxxx> wrote:<br>> <br>> <br>> <br>> On 5/10/07, Atsushi SAKAI <sakaia@xxxxxxxxxxxxxx> wrote:<br>> <br>> One vcpu can use one pcpu at one time.<br>> It means 100% is maxium for one vcpu domain.<br>> If you want to use cpu resources, you <br>> should set more vcpu.<br>> <br>> <br>> Ok, this explains a lot of things. <br>> As i understand this , more VCPUs means more <br>> freedom to hypervisor to migrate them among physical CPUs, <br>> depending on the free PCPUs available. <br>> <br>> <br>> In general <br>> <br>> domU1 <br>> / | \<br>> vcpu1 vcpu2 vcpu3 <br>> <br>> pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 <br>> <br>> I mean ,domU1 can run on any vcpu , right? now <br>> vcpu1, vcpu2, vcpu3 share a one to many reationship between <br>> pcpus[1....6]. That is a vcpu can run on any of the pcus <br>> available to the Xen hypervisor(unless i explicitly pin it to ). <br>> <br>> Is my naive understanding of what you explained <br>> is correct? <br>> <br>> Thank you <br>> ~psr<br>> <br>> <br>> <br>> Thanks<br>> Atsushi SAKAI<br>> <br>> <br>> "pradeep singh rautela" <br>> <rautelap@xxxxxxxxx> wrote:<br>> <br>> > Hi Atsushi,<br>> ><br>> > On 5/10/07, Atsushi SAKAI < <br>> sakaia@xxxxxxxxxxxxxx <mailto:sakaia@xxxxxxxxxxxxxx> > wrote:<br>> > ><br>> > ><br>> > > You should show detail configuration.<br>> > > Your information is too short.<br>> > ><br>> > > Anyway I guess each domain has one vcpu. <br>> > > If so, this is normal behavior.<br>> > > Because one vcpu cannot allocate <br>> two or more pcpu at once.<br>> ><br>> ><br>> > Right, but shouldn''t Xen hypervisor <br>> be capable of migrating the VCPU among <br>> > the available PCPUs on a <br>> multiprocessor system, like in this case? And<br>> > criteria should be the load on the <br>> PCPU or the idle PCPUs.<br>> > yes/no?<br>> ><br>> > Am i missing something here?<br>> ><br>> > Thanks <br>> > ~psr<br>> ><br>> > Thanks<br>> > > Atsushi SAKAI<br>> > ><br>> > > "Harry Smith" < <br>> harry.smith272@xxxxxxxxx <mailto:harry.smith272@xxxxxxxxx> > wrote:<br>> > ><br>> > > > hi all, <br>> > > ><br>> > > > I am using xen3.0.3 on dual core <br>> hyperthreaded processor (in all 4<br>> > > cores).<br>> > > > There are 2 VMs vm1,vm2 among <br>> which vm1 has a webserver running on it.<br>> > > > <br>> > > > While testing the performance of <br>> webserver, when I introduce some load<br>> > > on<br>> > > > vm2 which involves some <br>> computations the webserver performance goes<br>> > > down.<br>> > > > This happened even in the case <br>> where CPU usage by both of vm1,vm2 is <br>> > > > restricted to 100% each.<br>> > > ><br>> > > > Is it expected behavior ? if yes <br>> then how does one can control addition<br>> > > of<br>> > > > new virtual machines as adding <br>> every new VM will result in lowering <br>> > > > performance of other VMs. <br>> Through scheduling parameters we can just<br>> > > specify<br>> > > > amount of CPU to be used in <br>> relative sense (weight) & upper limit (cap).<br>> > > But <br>> > > > how to tackle this point.<br>> > > ><br>> > > > I am new in this area & wanna set <br>> up a lab using virtualization, so want<br>> > > to<br>> > > > find solution for this.<br>> > > > <br>> > > > thanks,<br>> > > > Harry<br>> > > ><br>> > > > we always have a choice...<br>> > ><br>> > ><br>> > ><br>> > > <br>> _______________________________________________<br>> > > Xen-devel mailing list <br>> > > Xen-devel@xxxxxxxxxxxxxxxxxxx<br>> > > <br>> http://lists.xensource.com/xen-devel <br>> <http: lists.xensource.com="" xen-devel=""> <br>> > ><br>> ><br>> > <br>> ><br>> > --<br>> > ---<br>> > pradeep singh rautela<br>> ><br>> > "Genius is 1% inspiration, and 99% <br>> perspiration" - not me :)<br>> <br>> <br>> <br>> <br>> <br>> <br>> <br>> -- <br>> ---<br>> pradeep singh rautela <br>> <br>> "Genius is 1% inspiration, and 99% <br>> perspiration" - not me :) <br>> <br>> <br>> _______________________________________________<br>> Xen-devel mailing list<br>> Xen-devel@xxxxxxxxxxxxxxxxxxx<br>> http://lists.xensource.com/xen-devel <br>> <br>> <br>> <br>> ________________________________<br>> <br>> Office firewalls, cyber cafes, college labs, don''t allow you <br>> to download CHAT? Here''s a solution! <br>> <http: us.rd.yahoo.com="" mail="" in="" ymessenger="" *http:="" in.messenge=""><br>> r.yahoo.com/webmessengerpromo.php> <br>> <br><br><br></http:></http:></mailto:harry.smith272@xxxxxxxxx></mailto:sakaia@xxxxxxxxxxxxxx></rautelap@xxxxxxxxx></sakaia@xxxxxxxxxxxxxx></rautelap@xxxxxxxxx></blockquote><br><BR><BR><span style="color: rgb(0, 0, 191);"></span><img src="file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot.jpg" alt=""><img src="file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot-1.jpg" alt=""><img src="file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot-2.jpg" alt=""><a target="_blank" href="http://mail.google.com/mail/?realattid=f_ez7y22zo&attid=0.1&disp=inline&view=att&th=11146fbd76f18c5f"><img class="thi" src="http://mail.google.com/mail/?realattid=f_ez7y22zo&attid=0.1&disp=thd&view=att&th=11146fbd76f18c5f"></a><br><span style="color: rgb(0, 0, 255); font-style: italic;"></span><font size="4">-<span style="font-style: italic; color: rgb(0, 0, 191);">Rahooooooooooooooool...</span></font><br> <span style="color: rgb(0, 0, 191);">exceptions are most common things to happen..... <img src="http://us.i1.yimg.com/us.yimg.com/i/mesg/tsmileys2/03.gif"><br></span><p>  <hr size=1></hr> Heres a new way to find what you''re looking for - <a href="http://us.rd.yahoo.com/mail/in/yanswers/*http://in.answers.yahoo.com/">Yahoo! Answers</a> --0-1077727752-1178807910=:64904-- --===============0898654332=Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============0898654332==--
> -----Original Message----- > From: rahul gundecha [mailto:rahoolgundecha@yahoo.co.in] > Sent: 10 May 2007 15:39 > To: Petersson, Mats; harry.smith272@gmail.com; xen devel > Subject: RE: [Xen-devel] fair scheduling > > thanks alot mats for detailed clarification, it really helped > me as well. > > coming to the issue of hyperthreded CPU, > 1) if we can''t expect that they will fetch 200% of > performance then whats the benefit from using them ? ( I dont > know whats prime use of hyperthreding, if this is the real > scenario... )Because it gives more than 100%? If you can improve the overall throughput of the processor above the 100% that a single core gives by adding a relatively small amount of extra logic (particularly if the amount of extra logic is less than the improved performance). There are several things that can give hyperthreading an advantage over non-hyperthreaded processors, but it''s a complex subject, and it''s got a lot to do wit pipeline-length, branch predictability and other not so trivial subjects. But if you do some research on hyperthreading (particularly in Intel''s implementation), you''ll find that on SOME workloads, the performance is actually decreasing. I can''t remember if it was Oracle or IBM that have a recommendation that Intel processors with Hyperthreading should be used with Hyperthreading turned off, because that gives better performance.> 2) However given some processors, how to go about designing > workload management. Means how I can atleast estimate > resource usage. If I need to know that how much load my > infrastructure can support, while considering that load can > be of mixed type. As you said " but it does depend A LOT on > what the exactly what the two guests are doing " , how to do > such analysis ? As it seems that processor architecture will > be major player. What''s current approach regarding this.I don''t have a concrete answer here, mostly because it is a VERY complex subject. But there are tools that can identify on a system-level what either a particular process, or the entire system, is doing - for example "oprofile". Analyzing for example how many TLB-misses, L1/L2 cache misses, "cycles waiting for memory access" are things that oprofile can tell the user. Given understanding of what type of operations each application (or set of applications intended for a VM) is performing, combined with some understanding of the whole system architecture and the cabalities of the architecture (such as memory bandwidth, I/O bandwidth, etc), can help give an indication how a set of VM''s will perform. This isn''t a precise science (unless you can predict/simulate the workload interaction very precisely), as there are things that get much worse if you have the right/wrong type of interaction (for example if you have a shared cache, and one application does a lot of memory activity at sequential locations, it will "destroy" the cache for the other processor(s) quicker than an application that doesn''t fill the cache so "effectively"). Of course, if we ignore the problem with Hyperthreading, a rough estimation of "cpu-load" will give you a pretty close to the "correct" answer. The problem with hyperthreading is that some of the CPU resources are shared (in particular the execution units), which means that two competing threads will have to "wait their turn", a bit like a busy road that has two lanes of traffic coming into one. It is hard to predict if you get 80, 100, 120 or 150% of the expected single processor performance, since it depends specifically on the "success" of the processors scheduling the two queues of instructions. Highly optimized code with few branches and few memory load-stalls will have lower performance than code that has lots of (non-predictable) branches or memory loads that the processor has to wait for (cache-misses). For "proper" multi-core processors, the per-core gain isn''t quite 100% (so two cores will not necessarily give 200% of a single core''s performance), but it''s much closer than the hyperthreaded example (excluding pathological cases such as cache-thrashing[1]). Also consider that in many cases, the idea behind using virtualization for example in server consolidation is to move servers that are severly under-utilized onto a single system that has similar performance of the original server, say for example we have three servers running at an average of 15%, peak 25%. Running those three in one machine would give loads around 45-75% on the guest-side, which leaves at least 25% for "overhead", which should be sufficient in most cases. [1] Cache-thrashing is where one processor has some data in cache, and the second processor tries to write to the cached location. This will (in a traditional multi-CPU scenario) force the data to be written to memory and then read into the cache in the second processor. Under the "rigth" circumstances, this can significantly lower the performance of multiprocessor architectures. [I remmember one case where I was analyzing a benchmark, where the 2P case got about 75% of the 1P case instead of the theoretical 200%. By re-arranging the data used by two threads of the same application so that the two data-structures were cache-line aligned, the overall performance went to 198% of the theoretical 200%, which was expected]. -- Mats> > I read that data centres are increasingly using > virtualization, in that case how above said process is carried out. > > "Petersson, Mats" <Mats.Petersson@amd.com> wrote: > > > > > -----Original Message----- > > From: xen-devel-bounces@lists.xensource.com > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > > rahul gundecha > > Sent: 10 May 2007 13:00 > > To: harry.smith272@gmail.com; xen devel > > Subject: Re: [Xen-devel] fair scheduling > > > > hi everyone, > > > > in given case, why performance of one vm affects other vm ? > > if VMs are said to be independent, then why one VM''s load > > affects the performance of other VM? > > > > as shown by Harry, in both cases CPU consumption by VM is > > same then why webserver performance degrades ?? Does xen > > doesn''t provide performance isolation ? > > Well, it does have caps and weights to allow the > scheduler to distribute > the CPU performance evenly, but it is, as I explained > in a different > post just a few seconds ago, not easy to determine such > things as the > effect of memory, cache, TLB and I/O operations. If one > CPU is more busy > because it takes longer to get memory, how does the > scheduler know this > (and more importantly, what should the scheduler DO about it)? > > > If VM''s are purely CPU-bound, such as some very simple > calculation > (small enough that both the code and data fits in cache > nicely for all > VM''s at the same time) should work out nicely fair. > When it gets more > complex, where memory, cache and IO operations get > mixed into the > equation, it''s so much more complex. > > In Harry''s case, the matter is further complicated by > the fact that half > of the "CPU''s" are virtual CPU''s (hyperthreads), which adds some > benefit, but certainly not as much as a real core, so > expecting to get > 400% CPU performance out of 4 hyperthreads is far above > what you might > expect. > > Finally, [I should have thought this through earlier > and added it to the > previous post to Harry] part of the missing percentage > in Harry''s case > is probaly due to the fact that with one active VM and > Dom0 able to run > on two different cores, the load on Dom0 can wholly fit > in Core1 without > affecting Core0''s execution at all, where when the vm2 > is using 100% cpu > load, both Core0 and Core1 are fully loaded, which > means that the 25% > load on Dom0 will have to be shared out across Core0 > and Core1. This is > probably the biggest factor in this case - not memory > or IO load. > > -- > Mats > > > > regards, > > -Rahul > > > > > > > > Hi Atsushi & Pradeep, > > > > thanks for replying back. > > I have 4 VCPUs for each of VM. But the point I wanted > > to stress upon is - > > "This happened even in the case where CPU usage by both > > of vm1,vm2 is restricted to 100% each. " > > I had pinned all 4 VCPUs of each VM to a single phys. > > CPU. & I have 4 phys. CPUs > > means my vm1 was using cpu1, vm2 using cpu2 & domain-0 > > using cpu0,cpu3 > > > > Problem is when there is no load on vm2, webserver > > performance of vm1 is better. But when vm2 has some > > compute-intense load then vm1 webserver performance goes down. > > Please note that CPU consumption of vm1 shown by xentop > > in both cases is 100%, still webserver performance goes down > > by around 15-20%. > > Even after trying to isolate two VMs, existence of load > > on one VM is affecting other. > > > > so is it expected behavior ? > > > > thanks, > > Harry > > > > > > > > > > On 5/10/07, pradeep singh rautela wrote: > > > > > > > > On 5/10/07, Atsushi SAKAI wrote: > > > > One vcpu can use one pcpu at one time. > > It means 100% is maxium for one vcpu domain. > > If you want to use cpu resources, you > > should set more vcpu. > > > > > > Ok, this explains a lot of things. > > As i understand this , more VCPUs means more > > freedom to hypervisor to migrate them among physical CPUs, > > depending on the free PCPUs available. > > > > > > In general > > > > domU1 > > / | \ > > vcpu1 vcpu2 vcpu3 > > > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > > > I mean ,domU1 can run on any vcpu , right? now > > vcpu1, vcpu2, vcpu3 share a one to many reationship between > > pcpus[1....6]. That is a vcpu can run on any of the pcus > > available to the Xen hypervisor(unless i explicitly > pin it to ). > > > > Is my naive understanding of what you explained > > is correct? > > > > Thank you > > ~psr > > > > > > > > Thanks > > Atsushi SAKAI > > > > > > "pradeep singh rautela" > > wrote: > > > > > Hi Atsushi, > > > > > > On 5/10/07, Atsushi SAKAI < > > sakaia@xxxxxxxxxxxxxx > wrote: > > > > > > > > > > > > You should show detail configuration. > > > > Your information is too short. > > > > > > > > Anyway I guess each domain has one vcpu. > > > > If so, this is normal behavior. > > > > Because one vcpu cannot allocate > > two or more pcpu at once. > > > > > > > > > Right, but shouldn''t Xen hypervisor > > be capable of migrating the VCPU among > > > the available PCPUs on a > > multiprocessor system, like in this case? And > > > criteria should be the load on the > > PCPU or the idle PCPUs. > > > yes/no? > > > > > > Am i missing something here? > > > > > > Thanks > > > ~psr > > > > > > Thanks > > > > Atsushi SAKAI > > > > > > > > "Harry Smith" < > > harry.smith272@xxxxxxxxx > wrote: > > > > > > > > > hi all, > > > > > > > > > > I am using xen3.0.3 on dual core > > hyperthreaded processor (in all 4 > > > > cores). > > > > > There are 2 VMs vm1,vm2 among > > which vm1 has a webserver running on it. > > > > > > > > > > While testing the performance of > > webserver, when I introduce some load > > > > on > > > > > vm2 which involves some > > computations the webserver performance goes > > > > down. > > > > > This happened even in the case > > where CPU usage by both of vm1,vm2 is > > > > > restricted to 100% each. > > > > > > > > > > Is it expected behavior ? if yes > > then how does one can control addition > > > > of > > > > > new virtual machines as adding > > every new VM will result in lowering > > > > > performance of other VMs. > > Through scheduling parameters we can just > > > > specify > > > > > amount of CPU to be used in > > relative sense (weight) & upper limit (cap). > > > > But > > > > > how to tackle this point. > > > > > > > > > > I am new in this area & wanna set > > up a lab using virtualization, so want > > > > to > > > > > find solution for this. > > > > > > > > > > thanks, > > > > > Harry > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Xen-devel mailing list > > > > Xen-devel@xxxxxxxxxxxxxxxxxxx > > > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > -- > > > --- > > > pradeep singh rautela > > > > > > "Genius is 1% inspiration, and 99% > > perspiration" - not me :) > > > > > > > > > > > > > > > > -- > > --- > > pradeep singh rautela > > > > "Genius is 1% inspiration, and 99% > > perspiration" - not me :) > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@xxxxxxxxxxxxxxxxxxx > > http://lists.xensource.com/xen-devel > > > > > > > > ________________________________ > > > > Office firewalls, cyber cafes, college labs, don''t allow you > > to download CHAT? Here''s a solution! > > > > r.yahoo.com/webmessengerpromo.php> > > > > > > > > > > > <file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot.j > pg> > <file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot-1 > .jpg> > <file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot-2 > .jpg> > <http://mail.google.com/mail/?realattid=f_ez7y22zo&attid=0.1&d > isp=inline&view=att&th=11146fbd76f18c5f> > -Rahooooooooooooooool... > exceptions are most common things to happen..... > <http://us.i1.yimg.com/us.yimg.com/i/mesg/tsmileys2/03.gif> > > > ________________________________ > > Here''s a new way to find what you''re looking for - Yahoo! > Answers > <http://us.rd.yahoo.com/mail/in/yanswers/*http://in.answers.ya > hoo.com/> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
thanks alot Mats for your kind reply... Sorry to bother you again with question, as you said this is very complex issue, then how does virtual data centers (virtual private servers) work ? how the data centers using xen for virtualization manage workload.. certainly they need some tool which may not be precise but atleast give an idea about infrastructure required. Because analysing these things for single time is ok, but if customer demands keeps on changing then it needs some tool. Or is it the case that there is always over-provisioning. "Petersson, Mats" <Mats.Petersson@amd.com> wrote:> -----Original Message----- > From: rahul gundecha [mailto:rahoolgundecha@yahoo.co.in] > Sent: 10 May 2007 15:39 > To: Petersson, Mats; harry.smith272@gmail.com; xen devel > Subject: RE: [Xen-devel] fair scheduling > > thanks alot mats for detailed clarification, it really helped > me as well. > > coming to the issue of hyperthreded CPU, > 1) if we can''t expect that they will fetch 200% of > performance then whats the benefit from using them ? ( I dont > know whats prime use of hyperthreding, if this is the real > scenario... )Because it gives more than 100%? If you can improve the overall throughput of the processor above the 100% that a single core gives by adding a relatively small amount of extra logic (particularly if the amount of extra logic is less than the improved performance). There are several things that can give hyperthreading an advantage over non-hyperthreaded processors, but it''s a complex subject, and it''s got a lot to do wit pipeline-length, branch predictability and other not so trivial subjects. But if you do some research on hyperthreading (particularly in Intel''s implementation), you''ll find that on SOME workloads, the performance is actually decreasing. I can''t remember if it was Oracle or IBM that have a recommendation that Intel processors with Hyperthreading should be used with Hyperthreading turned off, because that gives better performance.> 2) However given some processors, how to go about designing > workload management. Means how I can atleast estimate > resource usage. If I need to know that how much load my > infrastructure can support, while considering that load can > be of mixed type. As you said " but it does depend A LOT on > what the exactly what the two guests are doing " , how to do > such analysis ? As it seems that processor architecture will > be major player. What''s current approach regarding this.I don''t have a concrete answer here, mostly because it is a VERY complex subject. But there are tools that can identify on a system-level what either a particular process, or the entire system, is doing - for example "oprofile". Analyzing for example how many TLB-misses, L1/L2 cache misses, "cycles waiting for memory access" are things that oprofile can tell the user. Given understanding of what type of operations each application (or set of applications intended for a VM) is performing, combined with some understanding of the whole system architecture and the cabalities of the architecture (such as memory bandwidth, I/O bandwidth, etc), can help give an indication how a set of VM''s will perform. This isn''t a precise science (unless you can predict/simulate the workload interaction very precisely), as there are things that get much worse if you have the right/wrong type of interaction (for example if you have a shared cache, and one application does a lot of memory activity at sequential locations, it will "destroy" the cache for the other processor(s) quicker than an application that doesn''t fill the cache so "effectively"). Of course, if we ignore the problem with Hyperthreading, a rough estimation of "cpu-load" will give you a pretty close to the "correct" answer. The problem with hyperthreading is that some of the CPU resources are shared (in particular the execution units), which means that two competing threads will have to "wait their turn", a bit like a busy road that has two lanes of traffic coming into one. It is hard to predict if you get 80, 100, 120 or 150% of the expected single processor performance, since it depends specifically on the "success" of the processors scheduling the two queues of instructions. Highly optimized code with few branches and few memory load-stalls will have lower performance than code that has lots of (non-predictable) branches or memory loads that the processor has to wait for (cache-misses). For "proper" multi-core processors, the per-core gain isn''t quite 100% (so two cores will not necessarily give 200% of a single core''s performance), but it''s much closer than the hyperthreaded example (excluding pathological cases such as cache-thrashing[1]). Also consider that in many cases, the idea behind using virtualization for example in server consolidation is to move servers that are severly under-utilized onto a single system that has similar performance of the original server, say for example we have three servers running at an average of 15%, peak 25%. Running those three in one machine would give loads around 45-75% on the guest-side, which leaves at least 25% for "overhead", which should be sufficient in most cases. [1] Cache-thrashing is where one processor has some data in cache, and the second processor tries to write to the cached location. This will (in a traditional multi-CPU scenario) force the data to be written to memory and then read into the cache in the second processor. Under the "rigth" circumstances, this can significantly lower the performance of multiprocessor architectures. [I remmember one case where I was analyzing a benchmark, where the 2P case got about 75% of the 1P case instead of the theoretical 200%. By re-arranging the data used by two threads of the same application so that the two data-structures were cache-line aligned, the overall performance went to 198% of the theoretical 200%, which was expected]. -- Mats> > I read that data centres are increasingly using > virtualization, in that case how above said process is carried out. > > "Petersson, Mats" wrote: > > > > > -----Original Message----- > > From: xen-devel-bounces@lists.xensource.com > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > > rahul gundecha > > Sent: 10 May 2007 13:00 > > To: harry.smith272@gmail.com; xen devel > > Subject: Re: [Xen-devel] fair scheduling > > > > hi everyone, > > > > in given case, why performance of one vm affects other vm ? > > if VMs are said to be independent, then why one VM''s load > > affects the performance of other VM? > > > > as shown by Harry, in both cases CPU consumption by VM is > > same then why webserver performance degrades ?? Does xen > > doesn''t provide performance isolation ? > > Well, it does have caps and weights to allow the > scheduler to distribute > the CPU performance evenly, but it is, as I explained > in a different > post just a few seconds ago, not easy to determine such > things as the > effect of memory, cache, TLB and I/O operations. If one > CPU is more busy > because it takes longer to get memory, how does the > scheduler know this > (and more importantly, what should the scheduler DO about it)? > > > If VM''s are purely CPU-bound, such as some very simple > calculation > (small enough that both the code and data fits in cache > nicely for all > VM''s at the same time) should work out nicely fair. > When it gets more > complex, where memory, cache and IO operations get > mixed into the > equation, it''s so much more complex. > > In Harry''s case, the matter is further complicated by > the fact that half > of the "CPU''s" are virtual CPU''s (hyperthreads), which adds some > benefit, but certainly not as much as a real core, so > expecting to get > 400% CPU performance out of 4 hyperthreads is far above > what you might > expect. > > Finally, [I should have thought this through earlier > and added it to the > previous post to Harry] part of the missing percentage > in Harry''s case > is probaly due to the fact that with one active VM and > Dom0 able to run > on two different cores, the load on Dom0 can wholly fit > in Core1 without > affecting Core0''s execution at all, where when the vm2 > is using 100% cpu > load, both Core0 and Core1 are fully loaded, which > means that the 25% > load on Dom0 will have to be shared out across Core0 > and Core1. This is > probably the biggest factor in this case - not memory > or IO load. > > -- > Mats > > > > regards, > > -Rahul > > > > > > > > Hi Atsushi & Pradeep, > > > > thanks for replying back. > > I have 4 VCPUs for each of VM. But the point I wanted > > to stress upon is - > > "This happened even in the case where CPU usage by both > > of vm1,vm2 is restricted to 100% each. " > > I had pinned all 4 VCPUs of each VM to a single phys. > > CPU. & I have 4 phys. CPUs > > means my vm1 was using cpu1, vm2 using cpu2 & domain-0 > > using cpu0,cpu3 > > > > Problem is when there is no load on vm2, webserver > > performance of vm1 is better. But when vm2 has some > > compute-intense load then vm1 webserver performance goes down. > > Please note that CPU consumption of vm1 shown by xentop > > in both cases is 100%, still webserver performance goes down > > by around 15-20%. > > Even after trying to isolate two VMs, existence of load > > on one VM is affecting other. > > > > so is it expected behavior ? > > > > thanks, > > Harry > > > > > > > > > > On 5/10/07, pradeep singh rautela wrote: > > > > > > > > On 5/10/07, Atsushi SAKAI wrote: > > > > One vcpu can use one pcpu at one time. > > It means 100% is maxium for one vcpu domain. > > If you want to use cpu resources, you > > should set more vcpu. > > > > > > Ok, this explains a lot of things. > > As i understand this , more VCPUs means more > > freedom to hypervisor to migrate them among physical CPUs, > > depending on the free PCPUs available. > > > > > > In general > > > > domU1 > > / | \ > > vcpu1 vcpu2 vcpu3 > > > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > > > I mean ,domU1 can run on any vcpu , right? now > > vcpu1, vcpu2, vcpu3 share a one to many reationship between > > pcpus[1....6]. That is a vcpu can run on any of the pcus > > available to the Xen hypervisor(unless i explicitly > pin it to ). > > > > Is my naive understanding of what you explained > > is correct? > > > > Thank you > > ~psr > > > > > > > > Thanks > > Atsushi SAKAI > > > > > > "pradeep singh rautela" > > wrote: > > > > > Hi Atsushi, > > > > > > On 5/10/07, Atsushi SAKAI < > > sakaia@xxxxxxxxxxxxxx > wrote: > > > > > > > > > > > > You should show detail configuration. > > > > Your information is too short. > > > > > > > > Anyway I guess each domain has one vcpu. > > > > If so, this is normal behavior. > > > > Because one vcpu cannot allocate > > two or more pcpu at once. > > > > > > > > > Right, but shouldn''t Xen hypervisor > > be capable of migrating the VCPU among > > > the available PCPUs on a > > multiprocessor system, like in this case? And > > > criteria should be the load on the > > PCPU or the idle PCPUs. > > > yes/no? > > > > > > Am i missing something here? > > > > > > Thanks > > > ~psr > > > > > > Thanks > > > > Atsushi SAKAI > > > > > > > > "Harry Smith" < > > harry.smith272@xxxxxxxxx > wrote: > > > > > > > > > hi all, > > > > > > > > > > I am using xen3.0.3 on dual core > > hyperthreaded processor (in all 4 > > > > cores). > > > > > There are 2 VMs vm1,vm2 among > > which vm1 has a webserver running on it. > > > > > > > > > > While testing the performance of > > webserver, when I introduce some load > > > > on > > > > > vm2 which involves some > > computations the webserver performance goes > > > > down. > > > > > This happened even in the case > > where CPU usage by both of vm1,vm2 is > > > > > restricted to 100% each. > > > > > > > > > > Is it expected behavior ? if yes > > then how does one can control addition > > > > of > > > > > new virtual machines as adding > > every new VM will result in lowering > > > > > performance of other VMs. > > Through scheduling parameters we can just > > > > specify > > > > > amount of CPU to be used in > > relative sense (weight) & upper limit (cap). > > > > But > > > > > how to tackle this point. > > > > > > > > > > I am new in this area & wanna set > > up a lab using virtualization, so want > > > > to > > > > > find solution for this. > > > > > > > > > > thanks, > > > > > Harry > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Xen-devel mailing list > > > > Xen-devel@xxxxxxxxxxxxxxxxxxx > > > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > -- > > > --- > > > pradeep singh rautela > > > > > > "Genius is 1% inspiration, and 99% > > perspiration" - not me :) > > > > > > > > > > > > > > > > -- > > --- > > pradeep singh rautela > > > > "Genius is 1% inspiration, and 99% > > perspiration" - not me :) > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@xxxxxxxxxxxxxxxxxxx > > http://lists.xensource.com/xen-devel > > > > > > > > ________________________________ > > > > Office firewalls, cyber cafes, college labs, don''t allow you > > to download CHAT? Here''s a solution! > > > > r.yahoo.com/webmessengerpromo.php> > > > > > > > > > > > > pg> > > .jpg> > > .jpg> > > isp=inline&view=att&th=11146fbd76f18c5f> > -Rahooooooooooooooool... > exceptions are most common things to happen..... > > > > ________________________________ > > Here''s a new way to find what you''re looking for - Yahoo! > Answers > > hoo.com/> >-Rahooooooooooooooool... exceptions are most common things to happen..... --------------------------------- Heres a new way to find what you''re looking for - Yahoo! Answers --0-1952366782-1178878208=:46245 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: 8bit thanks alot Mats for your kind reply...<br><br>Sorry to bother you again with question, as you said this is very complex issue, then how does virtual data centers (virtual private servers) work ? <br>how the data centers using xen for virtualization manage workload.. certainly they need some tool which may not be precise but atleast give an idea about infrastructure required.<br>Because analysing these things for single time is ok, but if customer demands keeps on changing then it needs some tool. Or is it the case that there is always over-provisioning.<br><br><br><b><i>"Petersson, Mats" <Mats.Petersson@amd.com></i></b> wrote:<blockquote class="replbq" style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"> <br><br>> -----Original Message-----<br>> From: rahul gundecha [mailto:rahoolgundecha@yahoo.co.in] <br>> Sent: 10 May 2007 15:39<br>> To: Petersson, Mats; harry.smith272@gmail.com; xen devel<br>> Subject: RE: [Xen-devel] fair scheduling<br>> <br>> thanks alot mats for detailed clarification, it really helped <br>> me as well.<br>> <br>> coming to the issue of hyperthreded CPU, <br>> 1) if we can''t expect that they will fetch 200% of <br>> performance then whats the benefit from using them ? ( I dont <br>> know whats prime use of hyperthreding, if this is the real <br>> scenario... )<br><br>Because it gives more than 100%? If you can improve the overall<br>throughput of the processor above the 100% that a single core gives by<br>adding a relatively small amount of extra logic (particularly if the<br>amount of extra logic is less than the improved performance). <br><br>There are several things that can give hyperthreading an advantage over<br>non-hyperthreaded processors, but it''s a complex subject, and it''s got a<br>lot to do wit pipeline-length, branch predictability and other not so<br>trivial subjects.<br><br>But if you do some research on hyperthreading (particularly in Intel''s<br>implementation), you''ll find that on SOME workloads, the performance is<br>actually decreasing. I can''t remember if it was Oracle or IBM that have<br>a recommendation that Intel processors with Hyperthreading should be<br>used with Hyperthreading turned off, because that gives better<br>performance. <br><br>> 2) However given some processors, how to go about designing <br>> workload management. Means how I can atleast estimate <br>> resource usage. If I need to know that how much load my <br>> infrastructure can support, while considering that load can <br>> be of mixed type. As you said " but it does depend A LOT on <br>> what the exactly what the two guests are doing " , how to do <br>> such analysis ? As it seems that processor architecture will <br>> be major player. What''s current approach regarding this.<br><br>I don''t have a concrete answer here, mostly because it is a VERY complex<br>subject. But there are tools that can identify on a system-level what<br>either a particular process, or the entire system, is doing - for<br>example "oprofile". Analyzing for example how many TLB-misses, L1/L2<br>cache misses, "cycles waiting for memory access" are things that<br>oprofile can tell the user. <br><br>Given understanding of what type of operations each application (or set<br>of applications intended for a VM) is performing, combined with some<br>understanding of the whole system architecture and the cabalities of the<br>architecture (such as memory bandwidth, I/O bandwidth, etc), can help<br>give an indication how a set of VM''s will perform. This isn''t a precise<br>science (unless you can predict/simulate the workload interaction very<br>precisely), as there are things that get much worse if you have the<br>right/wrong type of interaction (for example if you have a shared cache,<br>and one application does a lot of memory activity at sequential<br>locations, it will "destroy" the cache for the other processor(s)<br>quicker than an application that doesn''t fill the cache so<br>"effectively"). <br><br>Of course, if we ignore the problem with Hyperthreading, a rough<br>estimation of "cpu-load" will give you a pretty close to the "correct"<br>answer. <br><br>The problem with hyperthreading is that some of the CPU resources are<br>shared (in particular the execution units), which means that two<br>competing threads will have to "wait their turn", a bit like a busy road<br>that has two lanes of traffic coming into one. It is hard to predict if<br>you get 80, 100, 120 or 150% of the expected single processor<br>performance, since it depends specifically on the "success" of the<br>processors scheduling the two queues of instructions. Highly optimized<br>code with few branches and few memory load-stalls will have lower<br>performance than code that has lots of (non-predictable) branches or<br>memory loads that the processor has to wait for (cache-misses). <br><br>For "proper" multi-core processors, the per-core gain isn''t quite 100%<br>(so two cores will not necessarily give 200% of a single core''s<br>performance), but it''s much closer than the hyperthreaded example<br>(excluding pathological cases such as cache-thrashing[1]). <br><br>Also consider that in many cases, the idea behind using virtualization<br>for example in server consolidation is to move servers that are severly<br>under-utilized onto a single system that has similar performance of the<br>original server, say for example we have three servers running at an<br>average of 15%, peak 25%. Running those three in one machine would give<br>loads around 45-75% on the guest-side, which leaves at least 25% for<br>"overhead", which should be sufficient in most cases.<br><br>[1] Cache-thrashing is where one processor has some data in cache, and<br>the second processor tries to write to the cached location. This will<br>(in a traditional multi-CPU scenario) force the data to be written to<br>memory and then read into the cache in the second processor. Under the<br>"rigth" circumstances, this can significantly lower the performance of<br>multiprocessor architectures. [I remmember one case where I was<br>analyzing a benchmark, where the 2P case got about 75% of the 1P case<br>instead of the theoretical 200%. By re-arranging the data used by two<br>threads of the same application so that the two data-structures were<br>cache-line aligned, the overall performance went to 198% of the<br>theoretical 200%, which was expected]. <br><br>--<br>Mats<br><br>> <br>> I read that data centres are increasingly using <br>> virtualization, in that case how above said process is carried out. <br>> <br>> "Petersson, Mats" <Mats.Petersson@amd.com> wrote:<br>> <br>> <br>> <br>> > -----Original Message-----<br>> > From: xen-devel-bounces@lists.xensource.com <br>> > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of <br>> > rahul gundecha<br>> > Sent: 10 May 2007 13:00<br>> > To: harry.smith272@gmail.com; xen devel<br>> > Subject: Re: [Xen-devel] fair scheduling<br>> > <br>> > hi everyone,<br>> > <br>> > in given case, why performance of one vm affects other vm ? <br>> > if VMs are said to be independent, then why one VM''s load <br>> > affects the performance of other VM?<br>> > <br>> > as shown by Harry, in both cases CPU consumption by VM is <br>> > same then why webserver performance degrades ?? Does xen <br>> > doesn''t provide performance isolation ?<br>> <br>> Well, it does have caps and weights to allow the <br>> scheduler to distribute<br>> the CPU performance evenly, but it is, as I explained <br>> in a different<br>> post just a few seconds ago, not easy to determine such <br>> things as the<br>> effect of memory, cache, TLB and I/O operations. If one <br>> CPU is more busy<br>> because it takes longer to get memory, how does the <br>> scheduler know this<br>> (and more importantly, what should the scheduler DO about it)?<br>> <br>> <br>> If VM''s are purely CPU-bound, such as some very simple <br>> calculation<br>> (small enough that both the code and data fits in cache <br>> nicely for all<br>> VM''s at the same time) should work out nicely fair. <br>> When it gets more<br>> complex, where memory, cache and IO operations get <br>> mixed into the<br>> equation, it''s so much more complex. <br>> <br>> In Harry''s case, the matter is further complicated by <br>> the fact that half<br>> of the "CPU''s" are virtual CPU''s (hyperthreads), which adds some<br>> benefit, but certainly not as much as a real core, so <br>> expecting to get<br>> 400% CPU performance out of 4 hyperthreads is far above <br>> what you might<br>> expect. <br>> <br>> Finally, [I should have thought this through earlier <br>> and added it to the<br>> previous post to Harry] part of the missing percentage <br>> in Harry''s case<br>> is probaly due to the fact that with one active VM and <br>> Dom0 able to run<br>> on two different cores, the load on Dom0 can wholly fit <br>> in Core1 without<br>> affecting Core0''s execution at all, where when the vm2 <br>> is using 100% cpu<br>> load, both Core0 and Core1 are fully loaded, which <br>> means that the 25%<br>> load on Dom0 will have to be shared out across Core0 <br>> and Core1. This is<br>> probably the biggest factor in this case - not memory <br>> or IO load.<br>> <br>> --<br>> Mats<br>> > <br>> > regards,<br>> > -Rahul<br>> > <br>> > <br>> > <br>> > Hi Atsushi & Pradeep,<br>> > <br>> > thanks for replying back.<br>> > I have 4 VCPUs for each of VM. But the point I wanted <br>> > to stress upon is - <br>> > "This happened even in the case where CPU usage by both <br>> > of vm1,vm2 is restricted to 100% each. "<br>> > I had pinned all 4 VCPUs of each VM to a single phys. <br>> > CPU. & I have 4 phys. CPUs<br>> > means my vm1 was using cpu1, vm2 using cpu2 & domain-0 <br>> > using cpu0,cpu3 <br>> > <br>> > Problem is when there is no load on vm2, webserver <br>> > performance of vm1 is better. But when vm2 has some <br>> > compute-intense load then vm1 webserver performance goes down.<br>> > Please note that CPU consumption of vm1 shown by xentop <br>> > in both cases is 100%, still webserver performance goes down <br>> > by around 15-20%.<br>> > Even after trying to isolate two VMs, existence of load <br>> > on one VM is affecting other. <br>> > <br>> > so is it expected behavior ?<br>> > <br>> > thanks,<br>> > Harry<br>> > <br>> > <br>> > <br>> > <br>> > On 5/10/07, pradeep singh rautela wrote:<br>> > <br>> > <br>> > <br>> > On 5/10/07, Atsushi SAKAI wrote:<br>> > <br>> > One vcpu can use one pcpu at one time.<br>> > It means 100% is maxium for one vcpu domain.<br>> > If you want to use cpu resources, you <br>> > should set more vcpu.<br>> > <br>> > <br>> > Ok, this explains a lot of things. <br>> > As i understand this , more VCPUs means more <br>> > freedom to hypervisor to migrate them among physical CPUs, <br>> > depending on the free PCPUs available. <br>> > <br>> > <br>> > In general <br>> > <br>> > domU1 <br>> > / | \<br>> > vcpu1 vcpu2 vcpu3 <br>> > <br>> > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 <br>> > <br>> > I mean ,domU1 can run on any vcpu , right? now <br>> > vcpu1, vcpu2, vcpu3 share a one to many reationship between <br>> > pcpus[1....6]. That is a vcpu can run on any of the pcus <br>> > available to the Xen hypervisor(unless i explicitly <br>> pin it to ). <br>> > <br>> > Is my naive understanding of what you explained <br>> > is correct? <br>> > <br>> > Thank you <br>> > ~psr<br>> > <br>> > <br>> > <br>> > Thanks<br>> > Atsushi SAKAI<br>> > <br>> > <br>> > "pradeep singh rautela" <br>> > wrote:<br>> > <br>> > > Hi Atsushi,<br>> > ><br>> > > On 5/10/07, Atsushi SAKAI < <br>> > sakaia@xxxxxxxxxxxxxx > wrote:<br>> > > ><br>> > > ><br>> > > > You should show detail configuration.<br>> > > > Your information is too short.<br>> > > ><br>> > > > Anyway I guess each domain has one vcpu. <br>> > > > If so, this is normal behavior.<br>> > > > Because one vcpu cannot allocate <br>> > two or more pcpu at once.<br>> > ><br>> > ><br>> > > Right, but shouldn''t Xen hypervisor <br>> > be capable of migrating the VCPU among <br>> > > the available PCPUs on a <br>> > multiprocessor system, like in this case? And<br>> > > criteria should be the load on the <br>> > PCPU or the idle PCPUs.<br>> > > yes/no?<br>> > ><br>> > > Am i missing something here?<br>> > ><br>> > > Thanks <br>> > > ~psr<br>> > ><br>> > > Thanks<br>> > > > Atsushi SAKAI<br>> > > ><br>> > > > "Harry Smith" < <br>> > harry.smith272@xxxxxxxxx > wrote:<br>> > > ><br>> > > > > hi all, <br>> > > > ><br>> > > > > I am using xen3.0.3 on dual core <br>> > hyperthreaded processor (in all 4<br>> > > > cores).<br>> > > > > There are 2 VMs vm1,vm2 among <br>> > which vm1 has a webserver running on it.<br>> > > > > <br>> > > > > While testing the performance of <br>> > webserver, when I introduce some load<br>> > > > on<br>> > > > > vm2 which involves some <br>> > computations the webserver performance goes<br>> > > > down.<br>> > > > > This happened even in the case <br>> > where CPU usage by both of vm1,vm2 is <br>> > > > > restricted to 100% each.<br>> > > > ><br>> > > > > Is it expected behavior ? if yes <br>> > then how does one can control addition<br>> > > > of<br>> > > > > new virtual machines as adding <br>> > every new VM will result in lowering <br>> > > > > performance of other VMs. <br>> > Through scheduling parameters we can just<br>> > > > specify<br>> > > > > amount of CPU to be used in <br>> > relative sense (weight) & upper limit (cap).<br>> > > > But <br>> > > > > how to tackle this point.<br>> > > > ><br>> > > > > I am new in this area & wanna set <br>> > up a lab using virtualization, so want<br>> > > > to<br>> > > > > find solution for this.<br>> > > > > <br>> > > > > thanks,<br>> > > > > Harry<br>> > > > ><br>> > > > > we always have a choice...<br>> > > ><br>> > > ><br>> > > ><br>> > > > <br>> > _______________________________________________<br>> > > > Xen-devel mailing list <br>> > > > Xen-devel@xxxxxxxxxxxxxxxxxxx<br>> > > > <br>> > http://lists.xensource.com/xen-devel <br>> > <br>> > > ><br>> > ><br>> > > <br>> > ><br>> > > --<br>> > > ---<br>> > > pradeep singh rautela<br>> > ><br>> > > "Genius is 1% inspiration, and 99% <br>> > perspiration" - not me :)<br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > <br>> > -- <br>> > ---<br>> > pradeep singh rautela <br>> > <br>> > "Genius is 1% inspiration, and 99% <br>> > perspiration" - not me :) <br>> > <br>> > <br>> > _______________________________________________<br>> > Xen-devel mailing list<br>> > Xen-devel@xxxxxxxxxxxxxxxxxxx<br>> > http://lists.xensource.com/xen-devel <br>> > <br>> > <br>> > <br>> > ________________________________<br>> > <br>> > Office firewalls, cyber cafes, college labs, don''t allow you <br>> > to download CHAT? Here''s a solution! <br>> > <br>> > r.yahoo.com/webmessengerpromo.php> <br>> > <br>> <br>> <br>> <br>> <br>> <br>> <br>> <br>> <br>> <file: c:="" docume%7e1="" rahool="" locals%7e1="" temp="" moz-screenshot.j=""><br>> pg> <br>> <file: c:="" docume%7e1="" rahool="" locals%7e1="" temp="" moz-screenshot-1=""><br>> .jpg> <br>> <file: c:="" docume%7e1="" rahool="" locals%7e1="" temp="" moz-screenshot-2=""><br>> .jpg> <br>> <http: mail.google.com="" mail="" ?realattid="f_ez7y22zo&attid=0.1&d<br">> isp=inline&view=att&th=11146fbd76f18c5f> <br>> -Rahooooooooooooooool...<br>> exceptions are most common things to happen..... <br>> <http: us.i1.yimg.com="" us.yimg.com="" i="" mesg="" tsmileys2="" 03.gif=""> <br>> <br>> <br>> ________________________________<br>> <br>> Here''s a new way to find what you''re looking for - Yahoo! <br>> Answers <br>> <http: us.rd.yahoo.com="" mail="" in="" yanswers="" *http:="" in.answers.ya=""><br>> hoo.com/> <br>> <br><br><br></http:></http:></http:></file:></file:></file:></Mats.Petersson@amd.com></blockquote><br><BR><BR><span style="color: rgb(0, 0, 191);"></span><img src="file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot.jpg" alt=""><img src="file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot-1.jpg" alt=""><img src="file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot-2.jpg" alt=""><a target="_blank" href="http://mail.google.com/mail/?realattid=f_ez7y22zo&attid=0.1&disp=inline&view=att&th=11146fbd76f18c5f"><img class="thi" src="http://mail.google.com/mail/?realattid=f_ez7y22zo&attid=0.1&disp=thd&view=att&th=11146fbd76f18c5f"></a><br><span style="color: rgb(0, 0, 255); font-style: italic;"></span><font size="4">-<span style="font-style: italic; color: rgb(0, 0, 191);">Rahooooooooooooooool...</span></font><br> <span style="color: rgb(0, 0, 191);">exceptions are most common things to happen..... <img src="http://us.i1.yimg.com/us.yimg.com/i/mesg/tsmileys2/03.gif"><br></span><p>  <hr size=1></hr> Heres a new way to find what you''re looking for - <a href="http://us.rd.yahoo.com/mail/in/yanswers/*http://in.answers.yahoo.com/">Yahoo! Answers</a> --0-1952366782-1178878208=:46245-- --===============0690223154=Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============0690223154==--
> -----Original Message----- > From: rahul gundecha [mailto:rahoolgundecha@yahoo.co.in] > Sent: 11 May 2007 11:10 > To: Petersson, Mats; harry.smith272@gmail.com; xen devel > Subject: RE: [Xen-devel] fair scheduling > > thanks alot Mats for your kind reply... > > Sorry to bother you again with question, as you said this is > very complex issue, then how does virtual data centers > (virtual private servers) work ?I suspect the answer is as below.> how the data centers using xen for virtualization manage > workload.. certainly they need some tool which may not be > precise but atleast give an idea about infrastructure required. > Because analysing these things for single time is ok, but if > customer demands keeps on changing then it needs some tool. > Or is it the case that there is always over-provisioning.Probably for the most cases, that''s what they do. You can''t run a server environment at near 100% constant load - that''s just not going to work long term. There are tools to measure the overall load on the system, and if that reaches close to 100% for any extended periods, you need a bigger server. Note however that I don''t work for a virtual server company, so I don''t really know. But I think it''s essentially the same problem as when you have real private servers: If the load is too high, the customer will have to get a bigger server [whether the service provider supplies this, or the customer has to pay more first is something that would be defined by the contract]. I do know that a website I''m connected with had an upgrade of the data-base server simply because the db-server wasn''t up to the job when the website increased the load on the database (due to changes in how it uses the database). Before this upgrade, the service of the site was a bit poor. -- Mats> > > "Petersson, Mats" <Mats.Petersson@amd.com> wrote: > > > > > -----Original Message----- > > From: rahul gundecha [mailto:rahoolgundecha@yahoo.co.in] > > Sent: 10 May 2007 15:39 > > To: Petersson, Mats; harry.smith272@gmail.com; xen devel > > Subject: RE: [Xen-devel] fair scheduling > > > > thanks alot mats for detailed clarification, it really helped > > me as well. > > > > coming to the issue of hyperthreded CPU, > > 1) if we can''t expect that they will fetch 200% of > > performance then whats the benefit from using them ? ( I dont > > know whats prime use of hyperthreding, if this is the real > > scenario... ) > > Because it gives more than 100%? If you can improve the overall > throughput of the processor above the 100% that a > single core gives by > adding a relatively small amount of extra logic > (particularly if the > amount of extra logic is less than the improved performance). > > There are several things that can give hyperthreading > an advantage over > non-hyperthreaded processors, but it''s a complex > subject, and it''s got a > lot to do wit pipeline-length, branch predictability > and other not so > trivial subjects. > > But if you do some research on hyperthreading > (particularly in Intel''s > implementation), you''ll find that on SOME workloads, > the performance is > actually decreasing. I can''t remember if it was Oracle > or IBM that have > a recommendation that Intel processors with > Hyperthreading should be > used with Hyperthreading turned off, because that gives better > performance. > > > 2) However given some processors, how to go about designing > > workload management. Means how I can atleast estimate > > resource usage. If I need to know that how much load my > > infrastructure can support, while considering that load can > > be of mixed type. As you said " but it does depend A LOT on > > what the exactly what the two guests are doing " , how to do > > such analysis ? As it seems that processor architecture will > > be major player. What''s current approach regarding this. > > I don''t have a concrete answer here, mostly because it > is a VERY complex > subject. But there are tools that can identify on a > system-level what > either a particular process, or the entire system, is > doing - for > example "oprofile". Analyzing for example how many > TLB-misses, L1/L2 > cache misses, "cycles waiting for memory access" are things that > oprofile can tell the user. > > Given understanding of what type of operations each > application (or set > of applications intended for a VM) is performing, > combined with some > understanding of the whole system architecture and the > cabalities of the > architecture (such as memory bandwidth, I/O bandwidth, > etc), can help > give an indication how a set of VM''s will perform. This > isn''t a precise > science (unless you can predict/simulate the workload > interaction very > precisely), as there are things that get much worse if > you have the > right/wrong type of interaction (for example if you > have a shared cache, > and one application does a lot of memory activity at sequential > locations, it will "destroy" the cache for the other > processor(s) > quicker than an application that doesn''t fill the cache so > "effectively"). > > Of course, if we ignore the problem with Hyperthreading, a rough > estimation of "cpu-load" will give you a pretty close > to the "correct" > answer. > > The problem with hyperthreading is that some of the CPU > resources are > shared (in particular the execution units), which means that two > competing threads will have to "wait their turn", a bit > like a busy road > that has two lanes of traffic coming into one. It is > hard to predict if > you get 80, 100, 120 or 150% of the expected single processor > performance, since it depends specifically on the > "success" of the > processors scheduling the two queues of instructions. > Highly optimized > code with few branches and few memory load-stalls will > have lower > performance than code that has lots of > (non-predictable) branches or > memory loads that the processor has to wait for (cache-misses). > > For "proper" multi-core processors, the per-core gain > isn''t quite 100% > (so two cores will not necessarily give 200% of a single core''s > performance), but it''s much closer than the > hyperthreaded example > (excluding pathological cases such as cache-thrashing[1]). > > Also consider that in many cases, the idea behind using > virtualization > for example in server consolidation is to move servers > that are severly > under-utilized onto a single system that has similar > performance of the > original server, say for example we have three servers > running at an > average of 15%, peak 25%. Running those three in one > machine would give > loads around 45-75% on the guest-side, which leaves at > least 25% for > "overhead", which should be sufficient in most cases. > > [1] Cache-thrashing is where one processor has some > data in cache, and > the second processor tries to write to the cached > location. This will > (in a traditional multi-CPU scenario) force the data to > be written to > memory and then read into the cache in the second > processor. Under the > "rigth" circumstances, this can significantly lower the > performance of > multiprocessor architectures. [I remmember one case where I was > analyzing a benchmark, where the 2P case got about 75% > of the 1P case > instead of the theoretical 200%. By re-arranging the > data used by two > threads of the same application so that the two > data-structures were > cache-line aligned, the overall performance went to 198% of the > theoretical 200%, which was expected]. > > -- > Mats > > > > > I read that data centres are increasingly using > > virtualization, in that case how above said process > is carried out. > > > > "Petersson, Mats" wrote: > > > > > > > > > -----Original Message----- > > > From: xen-devel-bounces@lists.xensource.com > > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > > > rahul gundecha > > > Sent: 10 May 2007 13:00 > > > To: harry.smith272@gmail.com; xen devel > > > Subject: Re: [Xen-devel] fair scheduling > > > > > > hi everyone, > > > > > > in given case, why performance of one vm affects other vm ? > > > if VMs are said to be independent, then why one VM''s load > > > affects the performance of other VM? > > > > > > as shown by Harry, in both cases CPU consumption by VM is > > > same then why webserver performance degrades ?? Does xen > > > doesn''t provide performance isolation ? > > > > Well, it does have caps and weights to allow the > > scheduler to distribute > > the CPU performance evenly, but it is, as I explained > > in a different > > post just a few seconds ago, not easy to determine such > > things as the > > effect of memory, cache, TLB and I/O operations. If one > > CPU is more busy > > because it takes longer to get memory, how does the > > scheduler know this > > (and more importantly, what should the scheduler DO about it)? > > > > > > If VM''s are purely CPU-bound, such as some very simple > > calculation > > (small enough that both the code and data fits in cache > > nicely for all > > VM''s at the same time) should work out nicely fair. > > When it gets more > > complex, where memory, cache and IO operations get > > mixed into the > > equation, it''s so much more complex. > > > > In Harry''s case, the matter is further complicated by > > the fact that half > > of the "CPU''s" are virtual CPU''s (hyperthreads), > which adds some > > benefit, but certainly not as much as a real core, so > > expecting to get > > 400% CPU performance out of 4 hyperthreads is far above > > what you might > > expect. > > > > Finally, [I should have thought this through earlier > > and added it to the > > previous post to Harry] part of the missing percentage > > in Harry''s case > > is probaly due to the fact that with one active VM and > > Dom0 able to run > > on two different cores, the load on Dom0 can wholly fit > > in Core1 without > > affecting Core0''s execution at all, where when the vm2 > > is using 100% cpu > > load, both Core0 and Core1 are fully loaded, which > > means that the 25% > > load on Dom0 will have to be shared out across Core0 > > and Core1. This is > > probably the biggest factor in this case - not memory > > or IO load. > > > > -- > > Mats > > > > > > regards, > > > -Rahul > > > > > > > > > > > > Hi Atsushi & Pradeep, > > > > > > thanks for replying back. > > > I have 4 VCPUs for each of VM. But the point I wanted > > > to stress upon is - > > > "This happened even in the case where CPU usage by both > > > of vm1,vm2 is restricted to 100% each. " > > > I had pinned all 4 VCPUs of each VM to a single phys. > > > CPU. & I have 4 phys. CPUs > > > means my vm1 was using cpu1, vm2 using cpu2 & domain-0 > > > using cpu0,cpu3 > > > > > > Problem is when there is no load on vm2, webserver > > > performance of vm1 is better. But when vm2 has some > > > compute-intense load then vm1 webserver performance > goes down. > > > Please note that CPU consumption of vm1 shown by xentop > > > in both cases is 100%, still webserver performance > goes down > > > by around 15-20%. > > > Even after trying to isolate two VMs, existence of load > > > on one VM is affecting other. > > > > > > so is it expected behavior ? > > > > > > thanks, > > > Harry > > > > > > > > > > > > > > > On 5/10/07, pradeep singh rautela wrote: > > > > > > > > > > > > On 5/10/07, Atsushi SAKAI wrote: > > > > > > One vcpu can use one pcpu at one time. > > > It means 100% is maxium for one vcpu domain. > > > If you want to use cpu resources, you > > > should set more vcpu. > > > > > > > > > Ok, this explains a lot of things. > > > As i understand this , more VCPUs means more > > > freedom to hypervisor to migrate them among physical CPUs, > > > depending on the free PCPUs available. > > > > > > > > > In general > > > > > > domU1 > > > / | \ > > > vcpu1 vcpu2 vcpu3 > > > > > > pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 > > > > > > I mean ,domU1 can run on any vcpu , right? now > > > vcpu1, vcpu2, vcpu3 share a one to many reationship between > > > pcpus[1....6]. That is a vcpu can run on any of the pcus > > > available to the Xen hypervisor(unless i explicitly > > pin it to ). > > > > > > Is my naive understanding of what you explained > > > is correct? > > > > > > Thank you > > > ~psr > > > > > > > > > > > > Thanks > > > Atsushi SAKAI > > > > > > > > > "pradeep singh rautela" > > > wrote: > > > > > > > Hi Atsushi, > > > > > > > > On 5/10/07, Atsushi SAKAI < > > > sakaia@xxxxxxxxxxxxxx > wrote: > > > > > > > > > > > > > > > You should show detail configuration. > > > > > Your information is too short. > > > > > > > > > > Anyway I guess each domain has one vcpu. > > > > > If so, this is normal behavior. > > > > > Because one vcpu cannot allocate > > > two or more pcpu at once. > > > > > > > > > > > > Right, but shouldn''t Xen hypervisor > > > be capable of migrating the VCPU among > > > > the available PCPUs on a > > > multiprocessor system, like in this case? And > > > > criteria should be the load on the > > > PCPU or the idle PCPUs. > > > > yes/no? > > > > > > > > Am i missing something here? > > > > > > > > Thanks > > > > ~psr > > > > > > > > Thanks > > > > > Atsushi SAKAI > > > > > > > > > > "Harry Smith" < > > > harry.smith272@xxxxxxxxx > wrote: > > > > > > > > > > > hi all, > > > > > > > > > > > > I am using xen3.0.3 on dual core > > > hyperthreaded processor (in all 4 > > > > > cores). > > > > > > There are 2 VMs vm1,vm2 among > > > which vm1 has a webserver running on it. > > > > > > > > > > > > While testing the performance of > > > webserver, when I introduce some load > > > > > on > > > > > > vm2 which involves some > > > computations the webserver performance goes > > > > > down. > > > > > > This happened even in the case > > > where CPU usage by both of vm1,vm2 is > > > > > > restricted to 100% each. > > > > > > > > > > > > Is it expected behavior ? if yes > > > then how does one can control addition > > > > > of > > > > > > new virtual machines as adding > > > every new VM will result in lowering > > > > > > performance of other VMs. > > > Through scheduling parameters we can just > > > > > specify > > > > > > amount of CPU to be used in > > > relative sense (weight) & upper limit (cap). > > > > > But > > > > > > how to tackle this point. > > > > > > > > > > > > I am new in this area & wanna set > > > up a lab using virtualization, so want > > > > > to > > > > > > find solution for this. > > > > > > > > > > > > thanks, > > > > > > Harry > > > > > > > > > > > > we always have a choice... > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Xen-devel mailing list > > > > > Xen-devel@xxxxxxxxxxxxxxxxxxx > > > > > > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > --- > > > > pradeep singh rautela > > > > > > > > "Genius is 1% inspiration, and 99% > > > perspiration" - not me :) > > > > > > > > > > > > > > > > > > > > > > > > -- > > > --- > > > pradeep singh rautela > > > > > > "Genius is 1% inspiration, and 99% > > > perspiration" - not me :) > > > > > > > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@xxxxxxxxxxxxxxxxxxx > > > http://lists.xensource.com/xen-devel > > > > > > > > > > > > ________________________________ > > > > > > Office firewalls, cyber cafes, college labs, don''t > allow you > > > to download CHAT? Here''s a solution! > > > > > > r.yahoo.com/webmessengerpromo.php> > > > > > > > > > > > > > > > > > > > > > > > pg> > > > > .jpg> > > > > .jpg> > > > isp=inline&view=att&th=11146fbd76f18c5f> > > -Rahooooooooooooooool... > > exceptions are most common things to happen..... > > > > > > > > ________________________________ > > > > Here''s a new way to find what you''re looking for - Yahoo! > > Answers > > > > hoo.com/> > > > > > > > > > > > <file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot.jpg> <file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-> screenshot-1.jpg>> <file:///C:/DOCUME%7E1/rahool/LOCALS%7E1/Temp/moz-screenshot-2.jpg> <http://mail.google.com/mail/?realattid=f_ez7y22zo&attid=> 0.1&disp=inline&view=att&th=11146fbd76f18c5f>> -Rahooooooooooooooool... > exceptions are most common things to happen..... > <http://us.i1.yimg.com/us.yimg.com/i/mesg/tsmileys2/03.gif> > > > ________________________________ > > Here''s a new way to find what you''re looking for - Yahoo! > Answers > <http://us.rd.yahoo.com/mail/in/yanswers/*http://in.answers.yahoo.com/>>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel