Dutch Meyer
2009-Feb-18 22:49 UTC
[Xen-devel] [5 Patches] Synchronize blktap with citrix blktap2.
This is an updated patch set to the one I posted a few months back. These patches contain a completely rewritten blktap implementation and are an open source release of what Citrix intends to use in future releases of XenServer. They contain a number of bug fixes from my previous patchset. They also contain Citrix''s implementation of the VHD image format. VHD is what XenServer uses to store file-based images, and this code is considerably more robust and efficient than the qcow implementation that is in the tree today. Benefits to blktap2 over the old version of blktap: * Isolation from xenstore - Blktap devices are now created directly on the linux dom0 command line, rather than being spawned in response to XenStore events. This is handy for debugging, makes blktap generally easier to work with, and is a step toward a generic user-level block device implementation that is not Xen-specific. * Improved tapdisk infrastructure: simpler request forwarding, new request scheduler, request merging, more efficient use of AIO. * Improved tapdisk error handling and memory management. No allocations on the block data path, IO retry logic to protect guests transient block device failures. This has been tested and is known to work on weird environments such as NFS soft mounts. * Pause and snapshot of live virtual disks (see xmsnap script). * VHD support. The VHD code in this release has been rigorously tested, and represents a very mature implementation of the VHD image format. * No more duplication of mechanism with blkback. The blktap kernel module has changed dramatically from the original blktap. Blkback is now always used to talk to Xen guests, blktap just presents a Linux gendisk that blkback can export. This is done while preserving the zero-copy data path from domU to physical device. These patches deprecate the old blktap code, which can hopefully be removed from the tree completely at some point in the future. Signed-off-by: Jake Wires <jake.wires@citrix.com>, Dutch Meyer <dmeyer@cs.ubc.ca> Two blktap Patches (the xen tree): http://www.cs.ubc.ca/~dmeyer/blktap2/01-deprecate-blktap http://www.cs.ubc.ca/~dmeyer/blktap2/02-add-blktap2 And 3 kernel module patches (the linux tree): http://www.cs.ubc.ca/~dmeyer/blktap2/01-blktap-patch http://www.cs.ubc.ca/~dmeyer/blktap2/02-blkback-pagemap-patch http://www.cs.ubc.ca/~dmeyer/blktap2/03-blktap2-patch In addition to the above patches, the (external) ioemu git repository currently has some very heavy dependencies on blktap, and does not like the change of the blktap directory name to deprecate that code. The following patch needs to be applied to fix ioemu: http://www.cs.ubc.ca/~dmeyer/blktap2/tools_ioemu_dir_use_blktap_old.patch Ideally we can come up with a better integration between blktap2 and ioemu than what is currently in place with the old tapdisk code. For instance, blktap2 makes it possible to just treat the tapdisk-based devices as raw devices for qemu, severing these (now unnecessary) dependencies. --Dutch _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dutch Meyer
2009-Mar-27 06:09 UTC
[Xen-devel] Re: [5 Patches] Synchronize blktap with citrix blktap2.
Fixed a few issues, particularly on 64bit builds. Updated to build with most recent xen-unstable (19426). Kernel Changes: www.cs.ubc.ca/~dmeyer/blktap2/kernel_patches.tgz Dom0 userspace Changes: www.cs.ubc.ca/~dmeyer/blktap2/userspace_patches.tgz --Dutch On Wed, 18 Feb 2009, Dutch Meyer wrote:> > This is an updated patch set to the one I posted a few months back. > > These patches contain a completely rewritten blktap implementation > and are an open source release of what Citrix intends to use in future > releases of XenServer. They contain a number of bug fixes from my > previous patchset. > > They also contain Citrix''s implementation of the VHD image format. VHD is > what XenServer uses to store file-based images, and this code is considerably > more robust and efficient than the qcow implementation that is in the tree > today. > > Benefits to blktap2 over the old version of blktap: > > * Isolation from xenstore - Blktap devices are now created directly on > the linux dom0 command line, rather than being spawned in response > to XenStore events. This is handy for debugging, makes blktap > generally easier to work with, and is a step toward a generic > user-level block device implementation that is not Xen-specific. > > * Improved tapdisk infrastructure: simpler request forwarding, new > request scheduler, request merging, more efficient use of AIO. > > * Improved tapdisk error handling and memory management. No > allocations on the block data path, IO retry logic to protect guests > transient block device failures. This has been tested and is known > to work on weird environments such as NFS soft mounts. > > * Pause and snapshot of live virtual disks (see xmsnap script). > > * VHD support. The VHD code in this release has been rigorously > tested, and represents a very mature implementation of the VHD image > format. > > * No more duplication of mechanism with blkback. The blktap kernel > module has changed dramatically from the original blktap. Blkback > is now always used to talk to Xen guests, blktap just presents a > Linux gendisk that blkback can export. This is done while > preserving the zero-copy data path from domU to physical device. > > These patches deprecate the old blktap code, which can hopefully be > removed from the tree completely at some point in the future. > > Signed-off-by: Jake Wires <jake.wires@citrix.com>, Dutch Meyer > <dmeyer@cs.ubc.ca> > > Two blktap Patches (the xen tree): > http://www.cs.ubc.ca/~dmeyer/blktap2/01-deprecate-blktap > http://www.cs.ubc.ca/~dmeyer/blktap2/02-add-blktap2 > > And 3 kernel module patches (the linux tree): > http://www.cs.ubc.ca/~dmeyer/blktap2/01-blktap-patch > http://www.cs.ubc.ca/~dmeyer/blktap2/02-blkback-pagemap-patch > http://www.cs.ubc.ca/~dmeyer/blktap2/03-blktap2-patch > > In addition to the above patches, the (external) ioemu git repository > currently has some very heavy dependencies on blktap, and does not like the > change of the blktap directory name to deprecate that code. The following > patch needs to be applied to fix ioemu: > > http://www.cs.ubc.ca/~dmeyer/blktap2/tools_ioemu_dir_use_blktap_old.patch > > Ideally we can come up with a better integration between blktap2 and ioemu > than what is currently in place with the old tapdisk code. For instance, > blktap2 makes it possible to just treat the tapdisk-based devices as raw > devices for qemu, severing these (now unnecessary) dependencies. > > --Dutch >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dutch Meyer
2009-Apr-03 06:15 UTC
[Xen-devel] Re: [5 Patches] Synchronize blktap with citrix blktap2.
I''ve been reminded that the ioemu patch to update the blktap dependencies in that repo is still necessary. It is available here: http://www.cs.ubc.ca/~dmeyer/blktap2/tools_ioemu_dir_use_blktap_old.patch --Dutch On Thu, 26 Mar 2009, Dutch Meyer wrote:> Fixed a few issues, particularly on 64bit builds. Updated to build with most > recent xen-unstable (19426). > > Kernel Changes: > www.cs.ubc.ca/~dmeyer/blktap2/kernel_patches.tgz > > Dom0 userspace Changes: > www.cs.ubc.ca/~dmeyer/blktap2/userspace_patches.tgz > > --Dutch > > On Wed, 18 Feb 2009, Dutch Meyer wrote: > >> >> This is an updated patch set to the one I posted a few months back. >> >> These patches contain a completely rewritten blktap implementation >> and are an open source release of what Citrix intends to use in future >> releases of XenServer. They contain a number of bug fixes from my >> previous patchset. >> >> They also contain Citrix''s implementation of the VHD image format. VHD is >> what XenServer uses to store file-based images, and this code is >> considerably more robust and efficient than the qcow implementation that >> is in the tree today. >> >> Benefits to blktap2 over the old version of blktap: >> >> * Isolation from xenstore - Blktap devices are now created directly on >> the linux dom0 command line, rather than being spawned in response >> to XenStore events. This is handy for debugging, makes blktap >> generally easier to work with, and is a step toward a generic >> user-level block device implementation that is not Xen-specific. >> >> * Improved tapdisk infrastructure: simpler request forwarding, new >> request scheduler, request merging, more efficient use of AIO. >> >> * Improved tapdisk error handling and memory management. No >> allocations on the block data path, IO retry logic to protect guests >> transient block device failures. This has been tested and is known >> to work on weird environments such as NFS soft mounts. >> >> * Pause and snapshot of live virtual disks (see xmsnap script). >> >> * VHD support. The VHD code in this release has been rigorously >> tested, and represents a very mature implementation of the VHD image >> format. >> >> * No more duplication of mechanism with blkback. The blktap kernel >> module has changed dramatically from the original blktap. Blkback >> is now always used to talk to Xen guests, blktap just presents a >> Linux gendisk that blkback can export. This is done while >> preserving the zero-copy data path from domU to physical device. >> >> These patches deprecate the old blktap code, which can hopefully be >> removed from the tree completely at some point in the future. >> >> Signed-off-by: Jake Wires <jake.wires@citrix.com>, Dutch Meyer >> <dmeyer@cs.ubc.ca> >> >> Two blktap Patches (the xen tree): >> http://www.cs.ubc.ca/~dmeyer/blktap2/01-deprecate-blktap >> http://www.cs.ubc.ca/~dmeyer/blktap2/02-add-blktap2 >> >> And 3 kernel module patches (the linux tree): >> http://www.cs.ubc.ca/~dmeyer/blktap2/01-blktap-patch >> http://www.cs.ubc.ca/~dmeyer/blktap2/02-blkback-pagemap-patch >> http://www.cs.ubc.ca/~dmeyer/blktap2/03-blktap2-patch >> >> In addition to the above patches, the (external) ioemu git repository >> currently has some very heavy dependencies on blktap, and does not like >> the change of the blktap directory name to deprecate that code. The >> following patch needs to be applied to fix ioemu: >> >> http://www.cs.ubc.ca/~dmeyer/blktap2/tools_ioemu_dir_use_blktap_old.patch >> >> Ideally we can come up with a better integration between blktap2 and ioemu >> than what is currently in place with the old tapdisk code. For instance, >> blktap2 makes it possible to just treat the tapdisk-based devices as raw >> devices for qemu, severing these (now unnecessary) dependencies. >> >> --Dutch >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2009-May-19 01:26 UTC
[Xen-devel] Re: [5 Patches] Synchronize blktap with citrix blktap2.
On 27/03/2009 07:09, "Dutch Meyer" <dmeyer@cs.ubc.ca> wrote:> Fixed a few issues, particularly on 64bit builds. Updated to build with > most recent xen-unstable (19426). > > Kernel Changes: > www.cs.ubc.ca/~dmeyer/blktap2/kernel_patches.tgz > > Dom0 userspace Changes: > www.cs.ubc.ca/~dmeyer/blktap2/userspace_patches.tgzDutch, Can you please refresh these patches, and the ioemu patch also, for current xen-unstable now that 3.4 is released? If you do that I can check them straight in now. Thanks, Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dutch Meyer
2009-May-22 18:47 UTC
Re: [Xen-devel] Re: [5 Patches] Synchronize blktap with citrix blktap2.
Done. Patches are here: http://www.cs.ubc.ca/~dmeyer/blktap2/kernel_patches.tgz http://www.cs.ubc.ca/~dmeyer/blktap2/userspace_patches.tgz http://www.cs.ubc.ca/~dmeyer/blktap2/tools_ioemu_dir_use_blktap_old.patch Original checkin rational copied below your email, the same applies here. --Dutch On Mon, 18 May 2009, Keir Fraser wrote:> On 27/03/2009 07:09, "Dutch Meyer" <dmeyer@cs.ubc.ca> wrote: > >> Fixed a few issues, particularly on 64bit builds. Updated to build with >> most recent xen-unstable (19426). >> >> Kernel Changes: >> www.cs.ubc.ca/~dmeyer/blktap2/kernel_patches.tgz >> >> Dom0 userspace Changes: >> www.cs.ubc.ca/~dmeyer/blktap2/userspace_patches.tgz > > Dutch, > > Can you please refresh these patches, and the ioemu patch also, for current > xen-unstable now that 3.4 is released? If you do that I can check them > straight in now. > > Thanks, > Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >This is an updated patch set to the one I posted a few months back. These patches contain a completely rewritten blktap implementation and are an open source release of what Citrix intends to use in future releases of XenServer. They contain a number of bug fixes from my previous patchset. They also contain Citrix''s implementation of the VHD image format. VHD is what XenServer uses to store file-based images, and this code is considerably more robust and efficient than the qcow implementation that is in the tree today. Benefits to blktap2 over the old version of blktap: * Isolation from xenstore - Blktap devices are now created directly on the linux dom0 command line, rather than being spawned in response to XenStore events. This is handy for debugging, makes blktap generally easier to work with, and is a step toward a generic user-level block device implementation that is not Xen-specific. * Improved tapdisk infrastructure: simpler request forwarding, new request scheduler, request merging, more efficient use of AIO. * Improved tapdisk error handling and memory management. No allocations on the block data path, IO retry logic to protect guests transient block device failures. This has been tested and is known to work on weird environments such as NFS soft mounts. * Pause and snapshot of live virtual disks (see xmsnap script). * VHD support. The VHD code in this release has been rigorously tested, and represents a very mature implementation of the VHD image format. * No more duplication of mechanism with blkback. The blktap kernel module has changed dramatically from the original blktap. Blkback is now always used to talk to Xen guests, blktap just presents a Linux gendisk that blkback can export. This is done while preserving the zero-copy data path from domU to physical device. These patches deprecate the old blktap code, which can hopefully be removed from the tree completely at some point in the future. Signed-off-by: Jake Wires <jake.wires@citrix.com>, Dutch Meyer <dmeyer@cs.ubc.ca> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel