Hannes Kuehnemund
2007-May-24 15:38 UTC
[Xen-devel] host information in domU over /proc or /sys filesystem
Hi, I have a few questions. But lets start with the background: Several other virtualization technologies beside Xen do provide the possibility to get configuration information of the VM itself and the host it is running on through the /proc file system. These technologies are for example zLinux or Linux on ppc64. This feature is very important for quality of service aspects. How many physical CPU do I have? How many CPU time can I use? Other important information would the real memory and the hostname of the host machine. Basically all this information is provided by other virtualization technologies as read only values. Having a huge hosting solution with enterprise software running in virtual machines depends on these quality of service aspects. Having a black box makes debugging of performance issues impossible and thus the whole solution is not usable. Is it possible to provide such a solution with Xen (of course only if enabled in dom0 due to security reasons!) over the /proc or /sysfs? Thanks, Hannes -- * Hannes Kuehnemund * SAP Linuxlab * SAP AG * Dietmar Hopp Allee 16 * 69190 Walldorf, Germany T +49 6227 7-40615 F +49 6227 78-34584 mailto: hannes.kuehnemund@sap.com sap.com Sitz der Gesellschaft/Registered Office: Walldorf, Germany Vorstand/SAP Executive Board: Henning Kagermann (Sprecher/CEO), Léo Apotheker (stellvertretender Sprecher/Deputy CEO), Werner Brandt, Claus Heinrich, Gerhard Oswald, Peter Zencke Vorsitzender des Aufsichtsrats/Chairperson of the SAP Supervisory Board: Hasso Plattner Registergericht/Commercial Register Mannheim No HRB 350269 Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank. This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com lists.xensource.com/xen-devel
Mark Williamson
2007-May-28 19:11 UTC
Re: [Xen-devel] host information in domU over /proc or /sys filesystem
> This feature is very important for quality of service aspects. How many > physical CPU do I have? How many CPU time can I use?I''m not exactly sure whether a domU needs to know this? or do you mean dom0 specifically? The number of physical CPUs the domain''s VCPUs are being scheduled across might change dynamically. You can, however, change the number of VCPUs a domain is allocated as an indication of how many CPUs it should try to optimise for. There''s also /sys/hypervisor which contains some relevant information. dom0 is permitted to access more detailed information about the host machine through a hypercall (use "xm info" to see what''s available).> Other important > information would the real memory and the hostname of thehost machine. I''m not sure why a domU would need to know the hostname of the host machine? Also, this could change during migration, so it''s not a stable value. Cheers, Mark> Basically all this information is provided by other virtualization > technologies as read only values. Having a huge hosting solution with > enterprise software running in virtual machines depends on these quality > of service aspects. Having a black box makes debugging of performance > issues impossible and thus the whole solution is not usable. > > Is it possible to provide such a solution with Xen (of course only if > enabled in dom0 due to security reasons!) over the /proc or /sysfs? > > Thanks, > Hannes-- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com lists.xensource.com/xen-devel
Hannes Kuehnemund
2007-May-29 16:04 UTC
Re: [Xen-devel] host information in domU over /proc or /sys filesystem
Mark Williamson wrote: >> This feature is very important for quality of service aspects. How >> many physical CPU do I have? How many CPU time can I use?> I''m not exactly sure whether a domU needs to know this? or do you mean dom0 > specifically?Actually, the domU itself shouldn''t know this, but maybe an application running in domU. Imagine the following situation: Three party setup: 1) Software vendor 2) Hosting provider 3) Customer (using Software from 1) and a virtual machine from 2)) In case the hosting provider changes something in the setup, the performance decreases and Customer complains at the software vendor the really bad performance of the software sold. Having no chance to get at least number of physical cpu''s assigned to domU makes problem solving impossible. This theoretical setup would need the interaction of a third party, the Hosting provider. If there would be a possibility for the hosting provider to provide this information (physical CPU assigned) to a domU, reduces the costs and time spend for interaction getting a third party involved in a support call. The information which physical CPU''s are assigned to a domU does not affect security or encapsulation, because the domU does _not_ know how many physical CPU''s dom0 has at all or how many of them are assigned to other VM''s.> The number of physical CPUs the domain''s VCPUs are being scheduled across > might change dynamically. You can, however, change the number of VCPUs a > domain is allocated as an indication of how many CPUs it should try to > optimise for.That it changes dynamically is not a problem at all. We would collect the data, how many physical CPU''s are assigned on a 5 minute base in our application to keep track of it. We then can check if response times correlate with number of physical CPU''s.> There''s also /sys/hypervisor which contains some relevant > information.Yes, that for the hint. Unfortunately is data is not enough :(> dom0 is permitted to access more detailed information about the host machine > through a hypercall (use "xm info" to see what''s available). >> Other important > information would the real memory and the hostname of the > host machine. > I''m not sure why a domU would need to know the hostname of the host machine? > Also, this could change during migration, so it''s not a stable value.In case nothing like the physical CPU assignment will be implemented, domU must know, which CIM server to query to get this kind of information. I know that this value will change, and because of this we need the information. If the domO would stay the same all the time, I could set a parameter in my software to query the CIM server directly, which wouldn''t work after a migration. The third possibility is to setup a dedicated VM which interacts between all available dom0''s and all available domU''s. But this is the last possible solution I''d prefer, because a third instance to manage is a third source of errors.> Cheers, > MarkCheers, Hannes>> Basically all this information is provided by other virtualization >> technologies as read only values. Having a huge hosting solution with >> enterprise software running in virtual machines depends on these quality >> of service aspects. Having a black box makes debugging of performance >> issues impossible and thus the whole solution is not usable. >> >> Is it possible to provide such a solution with Xen (of course only if >> enabled in dom0 due to security reasons!) over the /proc or /sysfs? >> >> Thanks, >> Hannes-- * Hannes Kuehnemund * SAP Linuxlab * SAP AG * Dietmar Hopp Allee 16 * 69190 Walldorf, Germany T +49 6227 7-40615 F +49 6227 78-34584 mailto: hannes.kuehnemund@sap.com sap.com Sitz der Gesellschaft/Registered Office: Walldorf, Germany Vorstand/SAP Executive Board: Henning Kagermann (Sprecher/CEO), Léo Apotheker (stellvertretender Sprecher/Deputy CEO), Werner Brandt, Claus Heinrich, Gerhard Oswald, Peter Zencke Vorsitzender des Aufsichtsrats/Chairperson of the SAP Supervisory Board: Hasso Plattner Registergericht/Commercial Register Mannheim No HRB 350269 Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank. This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com lists.xensource.com/xen-devel
Mark Williamson
2007-May-29 17:42 UTC
Re: [Xen-devel] host information in domU over /proc or /sys filesystem
> > The number of physical CPUs the domain''s VCPUs are being scheduled across > > might change dynamically. You can, however, change the number of VCPUs a > > domain is allocated as an indication of how many CPUs it should try to > > optimise for. > > That it changes dynamically is not a problem at all. We would collect > the data, how many physical CPU''s are assigned on a 5 minute base in our > application to keep track of it. We then can check if response times > correlate with number of physical CPU''s.Hmmmm. Well, it''s perfectly possible for the hosting provider to assign an approximate minimum share of the machine to the domU using the credit scheduler. This is specified effectively in terms of "percentage of one physical CPU in a period"... So for a domain with 90% and one vCPU it would try to schedule it 90% of the time on whatever PCPU is free. For a domain with 90% and 2 vCPUs it''ll try and make sure that the time the two vCPUs get in total is the same amount. Domains with > 1 vCPU can use > 100% by this metric. Anyhow, there are a number of parameters involved here: how much time in total the domain should get, how many vCPUs the domain has, and how the time gets shared between them. Rather than trying to sample how many pCPUs the domain gets to run on, perhaps it would be better to monitor the total time its vCPUs are being scheduled *somewhere*. Then you can just compare whether your pCPU time allowance corresponds to your SLA. This way the domain still doesn''t need to know which specific CPUs it gets scheduled on, but you can get all the information you need?> > dom0 is permitted to access more detailed information about the host > > machine through a hypercall (use "xm info" to see what''s available). > > > >> Other important > information would the real memory and the hostname of > >> the > > > > host machine. > > I''m not sure why a domU would need to know the hostname of the host > > machine? Also, this could change during migration, so it''s not a stable > > value. > > In case nothing like the physical CPU assignment will be implemented, > domU must know, which CIM server to query to get this kind of > information. I know that this value will change, and because of this we > need the information. If the domO would stay the same all the time, I > could set a parameter in my software to query the CIM server directly, > which wouldn''t work after a migration. > > The third possibility is to setup a dedicated VM which interacts between > all available dom0''s and all available domU''s. But this is the last > possible solution I''d prefer, because a third instance to manage is a > third source of errors.I''m still not sure why the domU would need to know the host''s hostname though? Or is it just as a fallback if the CPU data isn''t directly available? Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com lists.xensource.com/xen-devel
Hannes Kuehnemund
2007-Jun-19 14:51 UTC
Re: [Xen-devel] host information in domU over /proc or /sys filesystem
Mark Williamson wrote:> Rather than trying to sample how many pCPUs the domain gets to run on, perhaps > it would be better to monitor the total time its vCPUs are being scheduled > *somewhere*. Then you can just compare whether your pCPU time allowance > corresponds to your SLA. > > This way the domain still doesn''t need to know which specific CPUs it gets > scheduled on, but you can get all the information you need?VMware for example handles this information in a similar way. Having the total consumed CPU time (cpu_time) for a VM inside the VM (which is only available in dom0 at the moment) one can calculate how many computing power is available. Lets consider, that my VM has 2 VCPUs configured. The VM furthermore is configured to run on 2 physical CPUs, and this is also the SLA between the hosting provider and the customer. But, another VM is also running on these 2 physical CPU''s. Having the possibility to check the cpu_time on a frequent base, e.g. seconds, one would be able to check, if the guaranteed phsical CPUs are available. To check this, simply generate a high load inside the VM, and if the cpu_time increases every second by the value of 2, both physical CPUs are available. In case the cpu_time only increases by the value on 1 or 1,5 it is clear, that the SLA is not fullfilled. Is it possbile to provide this information to the domU (if explicitly enabled in dom0)?>> The third possibility is to setup a dedicated VM which interacts between >> all available dom0''s and all available domU''s. But this is the last >> possible solution I''d prefer, because a third instance to manage is a >> third source of errors. > I''m still not sure why the domU would need to know the host''s hostname though? > Or is it just as a fallback if the CPU data isn''t directly available?This is meant to be the fallback, in case it is not possible to get any data from the hypervisor. I''m not sure, but having HVM setups running, getting data might be very difficult. Therefore, having a fallback (which can be turned on, but will be turned off by default) can be very convenient.> Cheers, > MarkCheers, Hannes -- * Hannes Kuehnemund * SAP Linuxlab * SAP AG * Dietmar Hopp Allee 16 * 69190 Walldorf, Germany T +49 6227 7-40615 F +49 6227 78-34584 mailto: hannes.kuehnemund@sap.com sap.com Sitz der Gesellschaft/Registered Office: Walldorf, Germany Vorstand/SAP Executive Board: Henning Kagermann (Sprecher/CEO), Léo Apotheker (stellvertretender Sprecher/Deputy CEO), Werner Brandt, Claus Heinrich, Gerhard Oswald, Peter Zencke Vorsitzender des Aufsichtsrats/Chairperson of the SAP Supervisory Board: Hasso Plattner Registergericht/Commercial Register Mannheim No HRB 350269 Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank. This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com lists.xensource.com/xen-devel
Reasonably Related Threads
- [LLVMdev] llvm-gcc-4.2 RELEASE_26 bootstrap failure on Solaris/SPARC - unhandled REAL_TYPE during compilation of '__powitf2'
- [LLVMdev] llvm-gcc-4.2 RELEASE_26 bootstrap failure on Solaris/SPARC - unhandled REAL_TYPE during compilation of '__powitf2'
- Installation of rJava package fails
- [LLVMdev] llvm-gcc-4.2 RELEASE_26 bootstrap failure on Solaris/SPARC - unhandled REAL_TYPE during compilation of '__powitf2'
- [LLVMdev] llvm-gcc-4.2 RELEASE_26 bootstrap failure on Solaris/SPARC - unhandled REAL_TYPE during compilation of '__powitf2'