hi, with Xen increasingly depending on Linux for bootstrap, drivers, packet filtering etc., would it make sense to have the option of compiling Xen as a Linux kernel module, like in VMWare or coLinux? It seems this would give similar performance to Xen 1.2, while retaining most of the benefits of the NGIO model (i.e. not having to port drivers). The only downside would be the lack of driver isolation, but most people would be willing to live with that is my guess (plus as long as there is no IO-MMU a bad driver is still able to take down the complete system anyhow). I imagine this could be done in a way that would also work under other host-OSes, like *BSD or Windows. Any comments? Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
I''m obviously partisan, but I think that would largely defeat the privilege de-coupling that they''re trying to achieve. Not to mention nullifying the notion of a driver domain. -Kip On Tue, 25 Jan 2005, Jacob Gorm Hansen wrote:> hi, > > with Xen increasingly depending on Linux for bootstrap, drivers, packet > filtering etc., would it make sense to have the option of compiling Xen > as a Linux kernel module, like in VMWare or coLinux? > > It seems this would give similar performance to Xen 1.2, while retaining > most of the benefits of the NGIO model (i.e. not having to port > drivers). The only downside would be the lack of driver isolation, but > most people would be willing to live with that is my guess (plus as long > as there is no IO-MMU a bad driver is still able to take down the > complete system anyhow). > > I imagine this could be done in a way that would also work under other > host-OSes, like *BSD or Windows. > > Any comments? > > Jacob > > > ------------------------------------------------------- > This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting > Tool for open source databases. Create drag-&-drop reports. Save time > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. > Download a FREE copy at http://www.intelliview.com/go/osdn_nl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel >------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Kip Macy wrote:> I''m obviously partisan, but I think that would largely defeat the > privilege de-coupling that they''re trying to achieve. Not to mention > nullifying the notion of a driver domain.Is anyone actually using driver domains, other than dom0? It would be really cool if hardware vendors started shipping drivers in Xen-VM format, but until that happens I do not see much motivation for having multiple full Linuxes around just to achieve a limited amount of isolation. It is no secret that I much preferred the old and simpler model, but I understand that noone likes having to port drivers, and I guess this is one reason for the current state of things. With Xen inside the host OS kernel, you would get the performance of the old model, and the ease of development of the new model, plus potentially the option of running Xen on millions of Windows-desktops (I am sure the Silicon Valley VCs would love to hear that ;-)). With all the OSes currently ported to Xen, the Xen hypercall interface could become a de-facto standard for paravirtualized OSes, especially if Xen were able to run on top of Windows as well as on top of Linux. Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> -----Original Message----- > From: xen-devel-admin@lists.sourceforge.net [mailto:xen-devel- > admin@lists.sourceforge.net] On Behalf Of Jacob Gorm Hansen > Sent: 26 January 2005 02:00 > To: Kip Macy > Cc: Xen-devel@lists.sourceforge.net > Subject: Re: [Xen-devel] Xen as a kernel module > > Kip Macy wrote: > > I''m obviously partisan, but I think that would largely defeat the > > privilege de-coupling that they''re trying to achieve. Not to mention > > nullifying the notion of a driver domain. > > Is anyone actually using driver domains, other than dom0? It would be > really cool if hardware vendors started shipping drivers in Xen-VM > format, but until that happens I do not see much motivation for having > multiple full Linuxes around just to achieve a limited amount of > isolation.I don''t think many people currently use separate driver domains and most people run all their device drivers in dom0 and export them from there. However, I think it would be foolish to remove the ability to run separate driver domains from Xen as it doesn''t seem to add significant overhead (over running them in Dom0) and it has the potential of being extremely useful. You already mentioned the shipping of device drivers, there might also some interesting security applications. We can currently already provide some degree of isolation against device driver bugs and for the DMA issue there are a number of known and existing solutions (see eg the L4 OSDI device driver paper). The grant-table design (unfortunately not fully implemented yet) with some form of IO-MMU should provide the rest of the isolation. The interfaces exported by Xen (for PCI config space, IO register access, and interrupts) should be generic enough to support other OSes as driver domains and the virtual device interfaces should be easily implementable in other OSes too. Again, I think it would be foolish to limit ourselves to Linux in this regard as your original post seems to suggest. I think it would be great to see other OSes being used as driver domains.> It is no secret that I much preferred the old and simpler model, but I > understand that noone likes having to port drivers, and I guess thisis> one reason for the current state of things. With Xen inside the hostOS> kernel, you would get the performance of the old model, and the easeof> development of the new model, plus potentially the option of runningXen> on millions of Windows-desktops (I am sure the Silicon Valley VCswould> love to hear that ;-)). > > With all the OSes currently ported to Xen, the Xen hypercall interface > could become a de-facto standard for paravirtualized OSes, especiallyif> Xen were able to run on top of Windows as well as on top of Linux.I think it would be harder to come up with OS agnostic/generic interfaces for Xen to be used inside a host OS (essentially sliding Xen underneath a OS) than with generic abstractions which a OS can use...plus there is the privilege decoupling issue Kip already raised. The current plan to move some of the platform init code out of Xen into a VM will certainly make this slightly more complicated (eg having the IOAPIC initialized by a VM and then handing control back to Xen) but I hope we will come up with a generic interface for Xen to deal with possibly different OSes to perform this task. At least that''s something I am advocating. Rolf> Jacob > > > ------------------------------------------------------- > This SF.Net email is sponsored by: IntelliVIEW -- InteractiveReporting> Tool for open source databases. Create drag-&-drop reports. Save time > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. > Download a FREE copy at http://www.intelliview.com/go/osdn_nl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Neugebauer, Rolf wrote:> However, I think it would be foolish to remove the ability to run > separate driver domains from Xen as it doesn''t seem to add significant > overhead (over running them in Dom0) and it has the potential of being > extremely useful. You already mentioned the shipping of device drivers, > there might also some interesting security applications. We can > currently already provide some degree of isolation against device driver > bugs and for the DMA issue there are a number of known and existing > solutions (see eg the L4 OSDI device driver paper). The grant-table > design (unfortunately not fully implemented yet) with some form of > IO-MMU should provide the rest of the isolation.First of all, my suggestion was to have this as a configuration option, or even a separate implementation, providing just the ability to host domUs. I am sure this is doable, as it seems to be what coLinux is doing with some success. I was not suggestion throwing away existing functionality, but to optimize for the common case, i.e. a single Linux providing all drivers. Secondly, there have been repeated reports on this list of people having problems with lower performance in domU than in dom0, perhaps due to cheap hardware, perhaps just due to misconfiguration, and the figures on the Xen website have not been updated to reflect what the actual situation is, so I guess nobody knows what the overhead will look like for a specific type of application. I would worry about MPI-style jobs, where you need both low latency and high bandwidth networking, and where you are likely to fully utilize the TLBs as well. I cannot see how performance does not get hurt in this situation, when Xen needs to flush the TLB for every interrupt, or alternatively needs to bundle interrupts, thus increasing latency. Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
well, I vote for Xen 2.0, i.e. keep dom0 and xen separate. My laptop runs xen as the default and I don''t see a huge hit performance wise. Plus, I get to dream that someday plan9 will be dom0. ron ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Tue, 25 Jan 2005, Jacob Gorm Hansen wrote:> with Xen increasingly depending on Linux for bootstrap, drivers, packet > filtering etc., would it make sense to have the option of compiling Xen as a > Linux kernel module, like in VMWare or coLinux?that''s weird, I just got this message a second time, but I still don''t think it is a good idea :-) ah, sourceforge. Anyway, I think the hypervisor should stand alone. Just my feeling about it. ron ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I would worry about MPI-style jobs, > where you need both low latency and high bandwidth networking, and where > you are likely to fully utilize the TLBs as well. I cannot see how > performance does not get hurt in this situation, when Xen needs to flush > the TLB for every interrupt, or alternatively needs to bundle > interrupts, thus increasing latency.This kind of workload needs measuring to find out just how bad things are -- it''s interesting if only because it will certainly represent a worst case scenario for us. Ultimately this should be fixed with smarter NICs. I am optimistic suitable support may end up in high-end NICs: manufacturers are always looking at how to differentiate their product, and we have a few contacts we would like to follow up when we have more time. :-) -- Keir ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
>with Xen increasingly depending on Linux for bootstrap, drivers, packet >filtering etc., would it make sense to have the option of compiling Xen >as a Linux kernel module, like in VMWare or coLinux?Huh? Which kind of vmware? Afaik the hosted (type II) versions of vmware when running on a linux host have some modules which get installed but the VMM itself is not a module. And coLinux is basically a windows device driver which does task switching - a very clever and useful piece of software but not really a Linux kernel module. Perhaps you mean: would it make sense to have a type II version of Xen? I certainly see no particular reason to do this, but no particular reason not to either. I''m not sure how much code similarity there would be though...>It seems this would give similar performance to Xen 1.2, while retaining >most of the benefits of the NGIO model (i.e. not having to port >drivers).Maybe - I guess it depends on what you mean. If you have: [ VM1 ] [ VM2 ] .... [ VMN ] [ new type II version of Xen ] [ linux kernel ] [ hardware ] then you require a way for VMx to communicate the new Xen thing, which then needs to syscall into the linux kernel. I''m not sure what VMx<->Xen comms would look like, or how it would perform. If you retain safety it seems like you might end up with the performance of UML, which if you go for ''high performance'' then you may need to turn off the safety catch. How did you see this working? What aspects of performance under Xen are you finding unacceptable? There will always be an overhead involved in running N operating systems and apps on a machine compared with just 1. Indeed, if you really want ''blistering performance'' you may find that even the overhead of a general purpose OS is too much. Application-specific OSes can increase performance (as can dynamic application-specific code path optimization, see e.g. synthesis, wiggin-redstone, etc).>The only downside would be the lack of driver isolation, but >most people would be willing to live with that is my guess (plus as long >as there is no IO-MMU a bad driver is still able to take down the >complete system anyhow).Well isolation (both security and performance) are two explicit design goals of Xen. If you want to have the illusion of multiple kernels without these properties, you can use linux vservers or BSD jail. We''re quite keen to evolve Xen in a way which makes it easier to run multiple configurations, but mainly in the space that /increases/ isolation (e.g. driver domains) rather than the other way.>I imagine this could be done in a way that would also work under other >host-OSes, like *BSD or Windows.Again, I''m not sure how much code-base similarity there would be with either current Xen or the type-II variant that you propose above. One nice thing about a type-I (unhosted) hypervisor is that you are - in principle at least - independent of the OSes you host. This means one can have a dom0 running Linux, BSD, Plan9 or even Windows. With type-II hypervisors you effectively need a new hypervisor for every hosted OS type (e.g. VMware workstation).>Any comments?It might be useful if you were to state what precisely the problem is that you wish to solve, why existing solutions are insufficient, and how your proposed solution would solve the problem. I''m not sure I really understand the answers to any of these at the moment. cheers, S. ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> Secondly, there have been repeated reports on this list of > people having > problems with lower performance in domU than in dom0, perhaps due to > cheap hardware, perhaps just due to misconfiguration,I think most of these problems were down to debugging accidently getting enabled in the stable build. We seem to be back to normal performance now. I think there are still some issues with particular ioapic''s, but this code is about to get rewritten and moved into dom0 anyhow.> and the > figures on > the Xen website have not been updated to reflect what the actual > situation is, so I guess nobody knows what the overhead will > look like > for a specific type of application.The performance figures for SPECCPU, OSDB/PostgreSQl, SPECWeb99/Apache, Postmark etc are pretty much identical between 2.0 and 1.2 --- see the "Safe Hardware Access with the Xen Virtual Machine Monitor" paper. The new IO model tends to burn more CPU for a given IO rate, but under high load things pipeline reasonably nicely and the privilege transitions are amortized. Latency does suffer, but it was never that great under 1.2 anyhow. Smarter NICs will help this a lot (e.g. see the Arsenic paper by myself and Keir). Ian ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Steven Hand wrote:> Maybe - I guess it depends on what you mean. If you have: > > [ VM1 ] [ VM2 ] .... [ VMN ] > > [ new type II version of Xen ] > > [ linux kernel ] > > [ hardware ]Sorry if this came out sounding as a bit of a troll, anyway, my suggested setup would look like this: [ VM1 ] [ VM2 ] .... [ VMN ] [ Xen + linux kernel ] [ hardware ]> then you require a way for VMx to communicate the new Xen thing, > which then needs to syscall into the linux kernel. I''m not sure > what VMx<->Xen comms would look like, or how it would perform. If > you retain safety it seems like you might end up with the performance > of UML, which if you go for ''high performance'' then you may need to > turn off the safety catch.Right now Xen is mapped somewhere in top of memory, I am not sure how domains are kept out of there, but I suppose it has to do with segments. The good thing about that is that hypercalls are cheap, and in Xen1.x I/O was cheap as well. My suggestion/question was a) why don''t we just put a full Linux up there, including drivers, and b) can we provide the Xen hypercall interface on top of other OSes as well?> How did you see this working?For Linux, I would relocate it to the top X megs of memory, and I would merge the Xen and Linux syscall handlers, essentially supporting two process models under the same OS. I would not map all of memory to Linux, just the pages it needs for its own stuff. For a driver OS, this would be fine, if you want to run applications as well there would be a tradeoff between how much you map to Linux and how much to Xen domains.> What aspects of performance under Xen are you finding unacceptable?I generally find performance acceptable, but as I said there are cases where there appears to be some friction against the goals of Xen (driver isolation) and the goals of the application (throughput, low latency).> Well isolation (both security and performance) are two explicit > design goals of Xen. If you want to have the illusion of multiple > kernels without these properties, you can use linux vservers or > BSD jail.I would argue that you could get the same level of isolation (except from driver isolation) if you merge the two, while achieving the same IO performance as the monolithic model, and still be able to reuse existing driver code.>>I imagine this could be done in a way that would also work under other >>host-OSes, like *BSD or Windows. > > > Again, I''m not sure how much code-base similarity there would be with > either current Xen or the type-II variant that you propose above.It would still be interesting to reuse existing Xen guestOS ports on top of different hypervisor implementations. Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> for a specific type of application. I would worry about MPI-style jobs, > where you need both low latency and high bandwidth networking, and whereWon''t many people running this sort of workload be using specialised network hardware anyhow? A modern cluster interconnect (Infiniband, for instance) will likely support direct IO to user level applications - this could still be done under Xen. Cheers, Mark> you are likely to fully utilize the TLBs as well. I cannot see how > performance does not get hurt in this situation, when Xen needs to flush > the TLB for every interrupt, or alternatively needs to bundle > interrupts, thus increasing latency. > > Jacob > > > ------------------------------------------------------- > This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting > Tool for open source databases. Create drag-&-drop reports. Save time > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. > Download a FREE copy at http://www.intelliview.com/go/osdn_nl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Wednesday 26 January 2005 08:41, Steven Hand wrote:> >with Xen increasingly depending on Linux for bootstrap, drivers, packet > >filtering etc., would it make sense to have the option of compiling Xen > >as a Linux kernel module, like in VMWare or coLinux? > > Huh? Which kind of vmware? Afaik the hosted (type II) versions of > vmware when running on a linux host have some modules which get installed > but the VMM itself is not a module. And coLinux is basically a windows > device driver which does task switching - a very clever and useful piece > of software but not really a Linux kernel module.IIRC, the coLinux device driver is available both for Windows and as a Linux kernel module. The same Linux kernel can then run in either environment.> Maybe - I guess it depends on what you mean. If you have: > > [ VM1 ] [ VM2 ] .... [ VMN ] > > [ new type II version of Xen ] > > [ linux kernel ] > > [ hardware ] > > then you require a way for VMx to communicate the new Xen thing, > which then needs to syscall into the linux kernel. I''m not sure > what VMx<->Xen comms would look like, or how it would perform. If > you retain safety it seems like you might end up with the performance > of UML, which if you go for ''high performance'' then you may need to > turn off the safety catch.Couldn''t we have the "Xen module" hijack the interrupt-handling of the host kernel (like the VMWare and coLinux modules themselves do AFAIK) in order to handle hypercalls directly? Device communications could be handled by backend / frontend drivers essentially the same as the ones we use for vanilla Xen. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Thursday 27 January 2005 00:19, Jacob Gorm Hansen wrote:> Sorry if this came out sounding as a bit of a troll, anyway, my > suggested setup would look like this: > > [ VM1 ] [ VM2 ] .... [ VMN ] > > [ Xen + linux kernel ] > > [ hardware ]I am interested in fast VMs. Assuming that you want dom0 to be ''[ Xen + linux kernel ]'' I fail to see how your proposed architecture helps there.> Right now Xen is mapped somewhere in top of memory, I am not sure how > domains are kept out of there, but I suppose it has to do with segments.As I understand this xen runs in ring0 and pushes the guests kernels one ring up into ring1 and then uses traps to allow the guest OSes to trap into the hypervisor as necessary.> The good thing about that is that hypercalls are cheap, and in Xen1.x > I/O was cheap as well.Cheap where? In dom0 or the VMs?> My suggestion/question was a) why don''t we just put a full Linux up > there, including drivers, and b) can we provide the Xen hypercall > interface on top of other OSes as well?a) Because that impacts security and robustness a lot. Security and robustness are the two attributes I want in software, especially a kernel. This is even more true for a hypervisor. b) I don''t understand what you are going at in b).> > What aspects of performance under Xen are you finding unacceptable? > > I generally find performance acceptable, but as I said there are cases > where there appears to be some friction against the goals of Xen (driver > isolation) and the goals of the application (throughput, low latency).Hmmm.... adding a layer of abstraction rarely improves throughput/latency. You do it anyway to gain flexibility. I do not like you proposal as it sacrifices flexibility I want for throughput/latency in a place I don''t care about.> > Well isolation (both security and performance) are two explicit > > design goals of Xen. If you want to have the illusion of multiple > > kernels without these properties, you can use linux vservers or > > BSD jail.Please keep those goals!> I would argue that you could get the same level of isolation (except > from driver isolation) if you merge the two, while achieving the same IO > performance as the monolithic model, and still be able to reuse existing > driver code.I fail to see where monolithic kernel comes into this... I assume you are referring to kernel running on a real machine instead of a virtual one. Your proposal would force me to have all network traffic pass through dom0, the system able to halt all VMs on the machine! I''d feel extremely nervous with such a setup (OK, I am paranoid;-). You do that to improve IO performance in dom0, which is the one virtual machine that I do not need IO performance: dom0 is meant on my systems to be able to setup VMs and nothing more (currently I use 16MiB of RAM for that domain). All work is done in other domains. Those can no longer access hardware directly with your proposal, thus it would even hurt IO performance for me.> It would still be interesting to reuse existing Xen guestOS ports on top > of different hypervisor implementations.The critical part of the hypervisor (from the perspective of a guest OS) is the interface. That seems well defined and wouldn''t need to change with your proposal. So you should not need to modify domU OSes. Your needs seem to be to have a fast dom0 and only spin of small sessions occasionally, doing most of the work in dom0. Mine is to have several servers sharing a piece of hardware. Dom0 is mostly idling while the other domains do the heavy lifting. My hope is that I will see more proposals about reducing the coupling of xen and dom0 (like being able to reboot dom0 without effecting the other domains). I have no relation with the xen project apart from using it, so I may only give my feelings. -- Gruss, Tobias ------------------------------------------------------------ Tobias Hunger The box said: ''Windows 95 or better'' tobias@aquazul.com So I installed Linux. ------------------------------------------------------------
> > Right now Xen is mapped somewhere in top of memory, I am not sure how > > domains are kept out of there, but I suppose it has to do with segments. > > As I understand this xen runs in ring0 and pushes the guests kernels one > ring up into ring1 and then uses traps to allow the guest OSes to trap into > the hypervisor as necessary.Yup. Xen is mapped into the top 64MB of virtual address space of every process (just above the Linux kernel). Segmentation is indeed used to ensure that: * user processes can only see their own memory * guest kernels can see their own memory and user process memory * Xen can see guest kernel memory and user process memory within the guest This is a straightforward extension of the way segments are used in vanilla Linux on x86. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Tobias Hunger wrote:> >>Right now Xen is mapped somewhere in top of memory, I am not sure how >>domains are kept out of there, but I suppose it has to do with segments. > > > As I understand this xen runs in ring0 and pushes the guests kernels one ring > up into ring1 and then uses traps to allow the guest OSes to trap into the > hypervisor as necessary. > > >>The good thing about that is that hypercalls are cheap, and in Xen1.x >>I/O was cheap as well. > > > Cheap where? In dom0 or the VMs?Both. Assuming that context switching is an expensive operation, and that most of the cost comes from flushing the TLBs, the current model has more overhead than a model where the drivers are mapped into the same address space (though still protected from domU access other that through hypercalls) as the currently running domain. Xen already contains magic that allows it to be permanently mapped into running domains, while still being protected from accidential/malicious access. I am calling that monolithic, because that is what standard linux does, and what the old Xen 1.x did. The current model is more like a microkernel, such as L4 or Mach (though of course a lot more clever, no offense people ;-)) Apart from perhaps increasing performance, I was saying it would be cool if the Xen-interface could be provided on platforms such as Windows or native Linux. People have done similar stuff before (e.g. VMWare vmmon, the Adeos nanokernel, coLinux), so it should be possible. It seems Xen would have better chances of world domination that way, if that is a goal. It would also save some duplication of effort in porting guestOSes to the different models out there. Anyway, it was just an idea, and it is likely that with future hardware the Xen NGIO/microkernel model will finally be overhead-free, vendors will start writing Xen-VM drivers, and we will all live happily ever after :-). Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Thu, 27 Jan 2005, Mark Williamson wrote:> Won''t many people running this sort of workload be using specialised > network hardware anyhow? A modern cluster interconnect (Infiniband, for > instance) will likely support direct IO to user level applications - > this could still be done under Xen.that''s been our assumption. ron ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ronald G. Minnich wrote:> > On Thu, 27 Jan 2005, Mark Williamson wrote: > > >>Won''t many people running this sort of workload be using specialised >>network hardware anyhow? A modern cluster interconnect (Infiniband, for >>instance) will likely support direct IO to user level applications - >>this could still be done under Xen. > > > that''s been our assumption.I guess so, though the clusters I have seen have all been Ethernet-only, due to the lower cost. Has anyone had any experience using Xen with this kind of hardware? Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Fri, 28 Jan 2005, Jacob Gorm Hansen wrote:> I guess so, though the clusters I have seen have all been Ethernet-only, due > to the lower cost. Has anyone had any experience using Xen with this kind of > hardware?Adam sulmicki is doing testing of that type of thing now, and we should have some results in a few weeks on a dual-xeon cluster of about 128 nodes.Part of that will involve a virtual Plan 9 cluster as well on the same hardware. You just gotta love Xen. ron ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ronald G. Minnich wrote:> > Adam sulmicki is doing testing of that type of thing now, and we should > have some results in a few weeks on a dual-xeon cluster of about 128 > nodes.Part of that will involve a virtual Plan 9 cluster as well on the > same hardware. You just gotta love Xen.I do love Xen. Not to worry ;-) Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
>> I guess so, though the clusters I have seen have all been Ethernet-only, due >> to the lower cost. Has anyone had any experience using Xen with this kind of >> hardware? > > Adam sulmicki is doing testing of that type of thing now, and we should > have some results in a few weeks on a dual-xeon cluster of about 128 > nodes.> Part of that will involve a virtual Plan 9 cluster as wellah, learn new stuff from the list :-)> on the same hardware. You just gotta love Xen.------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel