Hi Folks, I''ve been looking a bit at various cluster file systems, and I keep coming back to their qemu interfaces. But... I really don''t understand where qemu fits into xen. Can anybody explain where and how qemu is utilized within xen - for paravirtualized VMs vs. VMs that use hardware virtualization extensions vs. paravirtualized VMs that take advantage of some hardware virtualization extensions? (For context, I''m trying to explore how one might use Sheepdog with Xen.) Thanks very much, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
Miles Fidelman wrote:> Hi Folks, > > I''ve been looking a bit at various cluster file systems, and I keep > coming back to their qemu interfaces. But... I really don''t > understand where qemu fits into xen. > > Can anybody explain where and how qemu is utilized within xen - for > paravirtualized VMs vs. VMs that use hardware virtualization > extensions vs. paravirtualized VMs that take advantage of some > hardware virtualization extensions? > > (For context, I''m trying to explore how one might use Sheepdog with Xen.)and more precisely, qemu includes support for sheepdog - can a paravirtualized xen vm, use qemu to access an image file stored in sheepdog (and if so, how) -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Sun, 2012-12-30 at 20:14 +0000, Miles Fidelman wrote:> Hi Folks, > > I''ve been looking a bit at various cluster file systems, and I keep > coming back to their qemu interfaces. But... I really don''t understand > where qemu fits into xen. > > Can anybody explain where and how qemu is utilized within xen - for > paravirtualized VMs vs. VMs that use hardware virtualization extensions > vs. paravirtualized VMs that take advantage of some hardware > virtualization extensions?You might find that http://wiki.xen.org/wiki/Xen_Overview answers some of your question. The main use of QEMU in Xen is to provide emulation of the PC platform (i.e. the random devices in a typical PC, like key board controller, north/south bridge, PCI bus etc for fully virtualised (HVM) guests. PV guests have no emulated devices and therefore QEMU is not use in this capacity there. The other use is that qemu can optionally be used to provide certain PV device backends. Most commonly these are provided by the dom0 or driver domain kernel drivers but for example the "xenfb" (framebuffer PV backend) lives in qemu. You can also (optionally) use the "qdisk" backend for the PV disk backend, the other options are the in-kernel blkback or blktap2. If you have any of these configured then you might get a qemu process for a PV guest too. Ian.
Ian, Thanks! Can you elaborate just a bit (see below): Ian Campbell wrote:> On Sun, 2012-12-30 at 20:14 +0000, Miles Fidelman wrote: >> Hi Folks, >> >> I''ve been looking a bit at various cluster file systems, and I keep >> coming back to their qemu interfaces. But... I really don''t understand >> where qemu fits into xen. >> >> Can anybody explain where and how qemu is utilized within xen - for >> paravirtualized VMs vs. VMs that use hardware virtualization extensions >> vs. paravirtualized VMs that take advantage of some hardware >> virtualization extensions? > You might find that http://wiki.xen.org/wiki/Xen_Overview answers some > of your question.Knew most of that... though the page has been greatly expanded since the last time I looked at it. It''s the gory details I''m looking for...> The main use of QEMU in Xen is to provide emulation of the PC platform > (i.e. the random devices in a typical PC, like key board controller, > north/south bridge, PCI bus etc for fully virtualised (HVM) guests. PV > guests have no emulated devices and therefore QEMU is not use in this > capacity there. > > The other use is that qemu can optionally be used to provide certain PV > device backends. Most commonly these are provided by the dom0 or driver > domain kernel drivers but for example the "xenfb" (framebuffer PV > backend) lives in qemu. You can also (optionally) use the "qdisk" > backend for the PV disk backend, the other options are the in-kernel > blkback or blktap2. If you have any of these configured then you might > get a qemu process for a PV guest too. >Can you elaborate on this just a bit? It sounds like one can use qemu drivers from an HVM, to access the PV drivers. But what about the other way around? I''m running PV guests, so normally qemu doesn''t apply, but... what I''m trying to figure out is whether there''s some way to use the qemu drivers to access the sheepdog cluster file system - which only provides a qemu interface. Thanks again, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Wed, 2013-01-02 at 16:35 +0000, Miles Fidelman wrote:> > The main use of QEMU in Xen is to provide emulation of the PC platform > > (i.e. the random devices in a typical PC, like key board controller, > > north/south bridge, PCI bus etc for fully virtualised (HVM) guests. PV > > guests have no emulated devices and therefore QEMU is not use in this > > capacity there. > > > > The other use is that qemu can optionally be used to provide certain PV > > device backends. Most commonly these are provided by the dom0 or driver > > domain kernel drivers but for example the "xenfb" (framebuffer PV > > backend) lives in qemu. You can also (optionally) use the "qdisk" > > backend for the PV disk backend, the other options are the in-kernel > > blkback or blktap2. If you have any of these configured then you might > > get a qemu process for a PV guest too. > > > > Can you elaborate on this just a bit? It sounds like one can use qemu > drivers from an HVM, to access the PV drivers. But what about the other > way around? I''m running PV guests, so normally qemu doesn''t apply, > but... what I''m trying to figure out is whether there''s some way to use > the qemu drivers to access the sheepdog cluster file system - which only > provides a qemu interface.I''m not sure what you mean by "qemu drivers". Do you mean virtio? virtio and the Xen PV drivers fill the same niche for KVM and Xen respectively, but there is no concept of using virtio to access the Xen PV drivers (that doesn''t really make conceptual sense since they are the same thing) There was a summer of code project last year to implement virtio support for Xen directly. I think it was working for HVM but I don''t think the PV version got completed and since the qemu folks weren''t keen on a new virtio "bus" it was never pursued further. If you meant emulated devices rather than virtio then there is no way to have a PV guest use an emulated device. Looking (very briefly) at the sheepdog page it looks KVM specific to me, I wouldn''t expect it to Just Work on Xen -- someone would need to do a porting job. Ian.
On Wed, 2013-01-02 at 16:53 +0000, Ian Campbell wrote:> On Wed, 2013-01-02 at 16:35 +0000, Miles Fidelman wrote: > > > > The main use of QEMU in Xen is to provide emulation of the PC platform > > > (i.e. the random devices in a typical PC, like key board controller, > > > north/south bridge, PCI bus etc for fully virtualised (HVM) guests. PV > > > guests have no emulated devices and therefore QEMU is not use in this > > > capacity there. > > > > > > The other use is that qemu can optionally be used to provide certain PV > > > device backends. Most commonly these are provided by the dom0 or driver > > > domain kernel drivers but for example the "xenfb" (framebuffer PV > > > backend) lives in qemu. You can also (optionally) use the "qdisk" > > > backend for the PV disk backend, the other options are the in-kernel > > > blkback or blktap2. If you have any of these configured then you might > > > get a qemu process for a PV guest too. > > > > > > > Can you elaborate on this just a bit? It sounds like one can use qemu > > drivers from an HVM, to access the PV drivers. But what about the other > > way around? I''m running PV guests, so normally qemu doesn''t apply, > > but... what I''m trying to figure out is whether there''s some way to use > > the qemu drivers to access the sheepdog cluster file system - which only > > provides a qemu interface. > > I''m not sure what you mean by "qemu drivers". Do you mean virtio? > > virtio and the Xen PV drivers fill the same niche for KVM and Xen > respectively, but there is no concept of using virtio to access the Xen > PV drivers (that doesn''t really make conceptual sense since they are the > same thing) > > There was a summer of code project last year to implement virtio support > for Xen directly. I think it was working for HVM but I don''t think the > PV version got completed and since the qemu folks weren''t keen on a new > virtio "bus" it was never pursued further. >Should be "the year before last year". Tt is 2013 now. ;-) HVM should work. But PV transport layer was prototype and the "bus" implementation was not elegant.> If you meant emulated devices rather than virtio then there is no way to > have a PV guest use an emulated device. > > Looking (very briefly) at the sheepdog page it looks KVM specific to me, > I wouldn''t expect it to Just Work on Xen -- someone would need to do a > porting job. >I happen to know a Sheepdog committer. I can ask him about the portability of Sheepdog. Probably we can make this a GSoC project this year? Wei.
Ian Campbell wrote:> On Wed, 2013-01-02 at 16:35 +0000, Miles Fidelman wrote: > > >> Can you elaborate on this just a bit? It sounds like one can use qemu >> drivers from an HVM, to access the PV drivers. But what about the other >> way around? I''m running PV guests, so normally qemu doesn''t apply, >> but... what I''m trying to figure out is whether there''s some way to use >> the qemu drivers to access the sheepdog cluster file system - which only >> provides a qemu interface. > I''m not sure what you mean by "qemu drivers". Do you mean virtio?qemu-dm> Looking (very briefly) at the sheepdog page it looks KVM specific to me, > I wouldn''t expect it to Just Work on Xen -- someone would need to do a > porting job.There''s a good diagram on slide 5 of http://www.osrg.net/sheepdog/_static/lca2010_miniconf.pdf It looks like sheepdog''s servers (which talk to disks) run directly on linux, while VMs access virtual block devices via qemu. Performance may be better w/ KVM, but sheepdog doesn''t seem to depend on it. There''s been some discussion on the sheepdog development lists that sheepdog vdi devices (virtual disk images) are compatible with qemu-dm - which suggests that one could mount a sheepdog device from a HVM. I''m wondering if there''s a way to use qemu-dm from a PVM. Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
Wei Liu wrote:> I happen to know a Sheepdog committer. I can ask him about the > portability of Sheepdog. Probably we can make this a GSoC project this > year? Wei.Funny thing, this seems to have been on the GSoC 2012 list of ideas for Sheepdog (https://github.com/collie/sheepdog/wiki/Google-summer-of-code-2012): _____ Create sheepdog client library * *Brief explanation:* The goal of this task is to implement libsheepdog, and enables other clients than QEMU to use Sheepdog block-level volumes. * *Expected results:* libsheepdog provides --------- Be interesting to know what happened to it. And there does seem to be some work going on on "sheepdev" as a linux kernel module - a couple of references to it on the sheepdog dev list - but not really clear what''s happening. Thanks, Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Wed, 2013-01-02 at 17:35 +0000, Miles Fidelman wrote:> Wei Liu wrote: > > I happen to know a Sheepdog committer. I can ask him about the > > portability of Sheepdog. Probably we can make this a GSoC project this > > year? Wei. > > Funny thing, this seems to have been on the GSoC 2012 list of ideas for > Sheepdog > (https://github.com/collie/sheepdog/wiki/Google-summer-of-code-2012): > > _____ > > Create sheepdog client library > > * *Brief explanation:* The goal of this task is to implement > libsheepdog, and enables other clients than QEMU to use Sheepdog > block-level volumes. > * *Expected results:* libsheepdog provides > > --------- >Unfortunately they didn''t make it to GSoC 2012... Judging from the description above, it looks like that sheepdog will be QEMU independent in the future? But such project is not Xen dependent either, so it is hard to put it under Xen umbrella. The best bet now is to have libsheepdog ready and we do some plumber job to make it Xen available (but I don''t know how much work ahead). This can be a potential GSoC project for Xen. Wei.
MAYAP KAMGA Christine larissa
2013-Jan-03 09:12 UTC
[Beginners question]- Compilation error while trying user program from domU
Hello
I'm having almost the same problem as below.
In my case, i want to use "cpufreq.h" and percpu.h libraries to get
current processor frequency.
**This is my sample program**
#include <acpi/cpufreq/cpufreq.h>
#include <asm/percpu.h>
static unsigned int get_my_cur_freq(int cpuid)
{
struct cpufreq_policy *policy;
policy = per_cpu(cpufreq_cpu_policy, cpuid);
if (!policy)
return 0;
else
return policy->cur;
}
I linked the necesary librairy, but while executing my makefile, i
having the followinf error:
XenRoot/xen/include/acpi/cpufreq/processor_perf.h:61: error: expected
declaration specifiers or ‘...’ before ‘cpufreq_statistic_data’
XenRoot/xen/include/acpi/cpufreq/processor_perf.h:61: warning: data
definition has no type or storage class
XenRoot/xen/include/acpi/cpufreq/processor_perf.h:61: warning: type
defaults to ‘int’ in declaration of ‘DECLARE_PER_CPU’
In file included from readCurrentCpuCounters.c:6:
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:23: warning: data definition
has no type or storage class
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:23: warning: type defaults
to ‘int’ in declaration of ‘DECLARE_PER_CPU’
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:23: warning: parameter names
(without types) in function declaration
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:25: error: expected ‘=’,
‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘cpufreq_verbose’
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:62: error: expected
declaration specifiers or ‘...’ before ‘cpufreq_cpu_policy’
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:62: warning: data definition
has no type or storage class
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:62: warning: type defaults
to ‘int’ in declaration of ‘DECLARE_PER_CPU’
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:62: error: conflicting types
for ‘DECLARE_PER_CPU’
XenRoot/xen/include/acpi/cpufreq/processor_perf.h:61: note: previous
declaration of ‘DECLARE_PER_CPU’ was here
XenRoot/xen/include/acpi/cpufreq/cpufreq.h: In function
‘cpufreq_register_driver’:
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:164: error: ‘EINVAL’
undeclared (first use in this function)
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:164: error: (Each undeclared
identifier is reported only once
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:164: error: for each
function it appears in.)
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:167: error: ‘EBUSY’
undeclared (first use in this function)
XenRoot/xen/include/acpi/cpufreq/cpufreq.h: In function
‘cpufreq_unregister_driver’:
XenRoot/xen/include/acpi/cpufreq/cpufreq.h:177: error: ‘EINVAL’
undeclared (first use in this function)
readCurrentCpuCounters.c: In function ‘get_cur_freq_on_cpu’:
readCurrentCpuCounters.c:26: error: ‘per_cpu__cpufreq_cpu_policy’
undeclared (first use in this function)
readCurrentCpuCounters.c:26: warning: type defaults to ‘int’ in
declaration of ‘type name’
readCurrentCpuCounters.c:26: error: invalid type argument of ‘unary *’
(have ‘int’)
make: *** [readCurrentCpuCounters.o] Erreur 1
Please, what can i do to solve this issue.
PS: i'm using Xen in his 4.1.2 version.
Thanks in advance
On Wed, 2 Jan 2013 22:02:05 -0800 (PST), Lakshman Jayaraju
wrote:> Hi all,
>
> My aim is to invoke hypercalls from programs inside guest domains. So
> in
> this regard I wrote a sample program as below that invokes a
> function(xc_domain_getinfo) declared in xenctrl.h.
>
> *Sample Program:*
> #include<stdio.h>
> #include<xenctrl.h>
> int main()
> {
> printf("Attempt to invoke hypercall!!!\n");
> int handle,rc;
> xc_dominfo_t *info;
> handle = xc_interface_open();
> printf(Äcquired handle :%d\n",handle);
> rc = xc_domain_getinfo(handle,3,1,info);
> printf(" The returned value is :%d\n",rc);
> return 0;
> }
>
> I linked xenctrl library from dom0, but I am getting compilation
> error
> saying *xenctrl.h: No such file or directory*. Then as per the error
> message
> I tried copying(by downloading source) required header files as and
> when it
> says missing header files name on consecutive compilation. But I
> ended up
> with path conflicts and couldn't succeed in compiling.
>
> *My doubts:*
> * Do I need to copy header files manually into guest domain? - I
> tried
> copying header files required as per compilation error msgs but I
> ended up
> with conflicts.
> * Any pointers to detailed guide to invoke hypercalls from guest
> domains??
>
>
> Thanks,
> Lakshman
>
>
>
>
> --
> View this message in context:
>
>
http://xen.1045712.n5.nabble.com/Beginners-question-Compilation-error-while-trying-user-program-from-domU-tp5713299.html
> Sent from the Xen - User mailing list archive at Nabble.com.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users
On Wed, 2013-01-02 at 17:30 +0000, Miles Fidelman wrote:> Ian Campbell wrote: > > On Wed, 2013-01-02 at 16:35 +0000, Miles Fidelman wrote: > > > > > >> Can you elaborate on this just a bit? It sounds like one can use qemu > >> drivers from an HVM, to access the PV drivers. But what about the other > >> way around? I''m running PV guests, so normally qemu doesn''t apply, > >> but... what I''m trying to figure out is whether there''s some way to use > >> the qemu drivers to access the sheepdog cluster file system - which only > >> provides a qemu interface. > > I''m not sure what you mean by "qemu drivers". Do you mean virtio? > qemu-dmI still don''t really know what you mean. Apart from being a Xen specific term qemu-dm provides emulation of physical devices and Xen PV devices. Drivers are typically things in the guest which talk to those.> > Looking (very briefly) at the sheepdog page it looks KVM specific to me, > > I wouldn''t expect it to Just Work on Xen -- someone would need to do a > > porting job. > > There''s a good diagram on slide 5 of > http://www.osrg.net/sheepdog/_static/lca2010_miniconf.pdf > > It looks like sheepdog''s servers (which talk to disks) run directly on > linux, while VMs access virtual block devices via qemu. Performance may > be better w/ KVM, but sheepdog doesn''t seem to depend on it.> > There''s been some discussion on the sheepdog development lists that > sheepdog vdi devices (virtual disk images) are compatible with qemu-dm - > which suggests that one could mount a sheepdog device from a HVM. I''m > wondering if there''s a way to use qemu-dm from a PVM.So it looks like sheepdog can contain disk images which are exposed to the guest as either an emulated IDE device or a virtio block device (although those slides mention neither explicitly) and the clustered filesystem aspect is not actually exposed to the guest. I suppose it ought to be possible to have the qdisk backend in Qemu use the sheepdog backend instead of a file or block device or whatever. It would be worth trying a vbd configured with "backendtype=qdisk,target=sheepdog:<foo>" since sheepdog:<foo> seems to be the syntax (judging from the "Start the VM" section of https://github.com/collie/sheepdog/wiki/Getting-Started). I don''t know if the Xen toolstack supports passing arbitrary disk configurations to qemu though, someone (like you) would need to investigate and possibly do some plumbing on the Xen toolstack side (I''d be happy to advise on list if it comes to this). The same sort of thing ought to apply to using the Ceph backend in qemu. It would be worth googling to see if anyone has done this since I''d expect the approach ought to carry across. Ian.
Ian Campbell wrote:> On Wed, 2013-01-02 at 17:30 +0000, Miles Fidelman wrote: >> Ian Campbell wrote: >>> On Wed, 2013-01-02 at 16:35 +0000, Miles Fidelman wrote: >>> >>> >>>> Can you elaborate on this just a bit? It sounds like one can use qemu >>>> drivers from an HVM, to access the PV drivers. But what about the other >>>> way around? I'm running PV guests, so normally qemu doesn't apply, >>>> but... what I'm trying to figure out is whether there's some way to use >>>> the qemu drivers to access the sheepdog cluster file system - which only >>>> provides a qemu interface. >>> I'm not sure what you mean by "qemu drivers". Do you mean virtio? >> qemu-dm > I still don't really know what you mean. Apart from being a Xen specific > term qemu-dm provides emulation of physical devices and Xen PV devices. > Drivers are typically things in the guest which talk to those.Well, maybe I'm using the term a bit loosely, to refer to the processing chain between an o/s call from userspace to a specific hardware or virtual device. It looks like "virtio on xen" is one implementation (per http://wiki.xen.org/wiki/Virtio_On_Xen) - though perhaps not yet integrated into the main distribution (the page seems a little dated). Meanwhile, it looks like Qemu is the basis for Xen's native virtual device support for HVMs, but it's not clear what support is available to PVMs. (The description of virtio on xen describes both virtio for HVM guests, and PV guests). Still trying to understand how all the pieces fit together, particularly when running without hardware virtualization support (thinking about integration with a couple of older boxes).> So it looks like sheepdog can contain disk images which are exposed to > the guest as either an emulated IDE device or a virtio block device > (although those slides mention neither explicitly) and the clustered > filesystem aspect is not actually exposed to the guest. > > I suppose it ought to be possible to have the qdisk backend in Qemu use > the sheepdog backend instead of a file or block device or whatever. > > It would be worth trying a vbd configured with > "backendtype=qdisk,target=sheepdog:<foo>" since sheepdog:<foo> seems to > be the syntax (judging from the "Start the VM" section of > https://github.com/collie/sheepdog/wiki/Getting-Started).This does seem to be the experiment to try (project for the weekend, I think). Thanks for "backendtype=qdisk,target=sheepdog:<foo>"- that's given me a good hook for searching through documentation.> > I don't know if the Xen toolstack supports passing arbitrary disk > configurations to qemu though, someone (like you) would need to > investigate and possibly do some plumbing on the Xen toolstack side (I'd > be happy to advise on list if it comes to this).Ahh, and this gets to the heart of the question, and I think I found an answer in the Xen 4.2 release notes (http://wiki.xen.org/wiki/Xen_4.2_Feature_List): " • Support for upstream qemu * Used by default when required for PV guests (e.g. qdisk backend or VFB support)" Which seems to imply that a PV guest CAN access a qdisk, and since upstream qemu supports sheepdog, this might just all work without having to assemble a custom build. (Though I'm still trying to find an architectural diagram to help me visualize all the interfaces and data flows).> The same sort of thing ought to apply to using the Ceph backend in qemu. > It would be worth googling to see if anyone has done this since I'd > expect the approach ought to carry across.Looks like Ceph is a lot easier to integrate - since it provides a kernel-based block device interface (/dev/rbd). From the ceph documentation: "QEMU can pass a block device from the host on to a guest, but since QEMU 0.15, there’s no need to map an image as a block device on the host. Instead, QEMU can access an image as a virtual block device directly via librbd." Thanks again! Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users