Hello: I''m wondering how guest domains are provided access to disks. I suppose Domain 0 has direct access to the actual hardware devices, and other domains simply see a virtualized block device. However, when a guest domain wants to make a request to the disk, how does it get that request to Domain 0 and how does Domain 0 actually receive those requests? There appears to be a virtual block device driver in drivers/xen/blkfront & blkback. Is this the driver used by the guest Domains to access the virtualized devices? My other question actually pertains to CoW support for disks. I noticed that there was some work done on making a CoW driver that lived in the XenoLinux kernels. Has this been made public? Have there been any attempts to make one that provides that functionality in Xen itself? Thanks, David Lie ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I''m wondering how guest domains are provided access to disks. I suppose > Domain 0 has direct access to the actual hardware devices, and other domains > simply see a virtualized block device. However, when a guest domain wants to > make a request to the disk, how does it get that request to Domain 0 and how > does Domain 0 actually receive those requests? There appears to be a virtual > block device driver in drivers/xen/blkfront & blkback. Is this the driver > used by the guest Domains to access the virtualized devices?Yes. The blkback driver goes in dom0 (or any other suitably privileged domain) and is able to export any block device Linux knows about (e.g. physical partition, LVM volume, loopback file etc) to it''s peer blkfront driver in the guest domain.> My other question actually pertains to CoW support for disks. > I noticed that there was some work done on making a CoW driver > that lived in the XenoLinux kernels. Has this been made > public? Have there been any attempts to make one that provides > that functionality in Xen itself?There are a bunch of CoW options: There''s Bin Ren''s CoW driver for Linux 2.4, or you can just use the standard LVM2 dm-snap stuff in Linux 2.6. The latter currently doesn''t deal well with having many CoW devices, but it shouldn''t be too hard to fix up. (Michael Vrable started looking at this.) Since disk space is cheap, it might be better to write a CoW driver that just uses CoW while it clones the actual disk content in the background. If you want CoW at the file system layer, there are a bunch of different union/stackable/overlay file system kernel drivers for Linux, but I''m not sure I could actually recommend any of them. One approach that we''ve used is a user-space NFS server that implements CoW semantics. It works OK, but performance isn''t as good as we''d like. One advantage of CoW schemes is that it should make it easy to implement a shared buffer cache (as its easy to know when blocks are identical). This is something we''re actively looking in to. Ian ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
For this e-mail, I''m assuming we''re talking about Xen 2.0...> I''m wondering how guest domains are provided access to disks. I suppose > Domain 0 has direct access to the actual hardware devices, and other > domains simply see a virtualized block device.All correct.> However, when a guest > domain wants to make a request to the disk, how does it get that request to > Domain 0 and how does Domain 0 actually receive those requests?When it boots, the unpriv domain sets up sharing of a page of memory between its frontend driver and the backend driver running in domain 0. The front and back end drivers do this by sending control messages via Xend. Xend also binds an interdomain "event channel" between the two domains. This allows the back and front end drivers to send each other "events" (virtual interrupts). Once the shared memory page is set up and the event channel bound, the two are used for direct communication between the back and front end drivers. Xend does not have to get involved anymore. When the frontend wants to request data, it places a descriptor containng details of the request into the shared memory region. It then sends an "event" to the backend driver (remember, this is running in dom0), which checks the shared memory region in response. Assuming it decides the requests are valid, it will issue requests within the domain 0 kernel to perform the IO directly into the memory of the unpriv guest. When the IO has finished, the backend puts a response into the shared memory page and sends an event to the frontend. The frontend responds to the virtual interrupt by checking which IO completed and calling the appropriate completion handlers. All this happens without Xen knowing about the virtual devices. Xen operates at the level of shared memory and event channels - it doesn''t know or care that domains are using these as virtual devices.> There > appears to be a virtual block device driver in drivers/xen/blkfront & > blkback. Is this the driver used by the guest Domains to access the > virtualized devices?Correct again. blkfront is the unpriv domain''s part of the driver, blkback is the portion that runs in domain 0 and sends requests to the disk where appropriate. Note that these are not pretending to be any specific model of real-world device - they''re purely designed for high-performance virtual machine IO.> My other question actually pertains to CoW support for disks. I noticed > that there was some work done on making a CoW driver that lived in the > XenoLinux kernels. Has this been made public?Since device virtualisation moved into the dom0 kernel and out of Xen, it''s possible to use your favourite CoW driver for vanilla Linux to achieve this. There''s the LVM snapshot facility and, I think, the csnap driver and various other patches that are out there...> Have there been any > attempts to make one that provides that functionality in Xen itself?The functionality wouldn''t be provided in Xen, since Xen itself doesn''t manage devices (that''s done by dom0 in Xen 2.0). It wouldn''t be a good idea to bloat the XenLinux backend driver with it anyway - the best idea is for a generic solution (like LVM, csnap, etc.). Unfortunately LVM and csnap both have their drawbacks, which have previously been discussed on this list... HTH, Mark ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > I''m wondering how guest domains are provided access to disks. I suppose > > Domain 0 has direct access to the actual hardware devices, and other > domains > > simply see a virtualized block device. However, when a guest domain > wants to > > make a request to the disk, how does it get that request to Domain 0 and > how > > does Domain 0 actually receive those requests? There appears to be a > virtual > > block device driver in drivers/xen/blkfront & blkback. Is this the > driver > > used by the guest Domains to access the virtualized devices? > > Yes. The blkback driver goes in dom0 (or any other suitably > privileged domain) and is able to export any block device Linux > knows about (e.g. physical partition, LVM volume, loopback file > etc) to it''s peer blkfront driver in the guest domain. >Thanks for the prompt reply. I''m a bit confused on this point then since the blkfront driver doesn''t appear to make any hypervisor calls to Xen. If it doesn''t do this, how does it communicate requests to the blkback driver that''s actually managing the block device? Thanks, David ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I''m a bit confused on this point then since the blkfront driver doesn''t > appear to make any hypervisor calls to Xen. If it doesn''t do this, how > does it communicate requests to the blkback driver that''s actually managing > the block device?The block frontend only makes hypercalls in order to send "events" to the backend driver when new requests are waiting in the shared memory region (the shared memory region is updated by just assigning to structs in a ring buffer and incrementing an integer index). In the network frontend, hypercalls are also used to relinquish buffer pages to the backend driver but this is not necessary for block devices. HTH, Mark ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Thu, 4 Nov 2004, Mark A. Williamson wrote:> The block frontend only makes hypercalls in order to send "events" to the > backend driver when new requests are waiting in the shared memory region (the > shared memory region is updated by just assigning to structs in a ring buffer > and incrementing an integer index).I have to say, now that my block frontend is working, that it is a very easy thing to deal with. Nice design! And, yes, I''m now booting CPU servers in Plan 9 with network and disk. Still having some weird Plan 9 issues, but getting there. ron ------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader''s Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel