Hi, I am working as a system administrator at an internet platform service provider, and I am currently seeking to re-new our Xen virtualization infrastructure for which I am mostly responsible for. Currently, we run Xen 3.4.2/3.4.3 on RHEL/CentOS 5.x (5.7) as Dom0 with CentOS 5.x pv-guests. Based on my experiments, I am currently looking into Xen 4.1.2 on RHEL/CentOS 6.x (6.2), with a vanilla 3.2 (3.2.0/3.2.6) kernel as Dom0 and stock RHEL/CentOS 6.x guests as HVM/pv-ops DomUs. Our DomU''s have their disk-devices on iscsi-storage backends (EMC, netapp, Linux IET), we attach the disks to Dom0 via open-iscsi initiator and pass them as ''phy''-type blockdevices, using blkfront/back. As this is the time for a major overhaul anyway, I looked at ''pvscsi'', which is very attractive to me for a bunch of (administrative) reasons. I have created a working pvscsi-setup with the following steps: 1. Use CentOS 6.2 as base system 2. Add Xen hypervisor from Michael Young''s repository [1]. This is stock 4.1.2 with a patch from 4.1-testing pulled-in and various patches fixing compile-time issues, toolchain problems and system service management on RHEL-alike systems. (Nice work, btw!) 3. Use a vanilla 3.2.0 kernel from kernel.org and a .config based on a Fedora 16 stock-kernel config, tweaked to use pvscsi. As you know, Fedora 16 supports Dom0 operation and the only changes to the config are non-Xen related or add the Xen pvscsi config. I refrained from attaching the config to avoid mailinglist-clutter, but can do so on demand. 4. I pulled and applied the pvscsi-patch found in this post [2] from Konrad Rzeszutek Wilk. Putting it all together yielded a "working setup" in which I am able to pass an Dom0-attached iscsi target to a DomU as a vscsi-device. In DomU context, I could partition the device, create a filesystem on it and copy/read/verify some 10gig of data to it. Very basic testing until now, essentially. However, the whole issue raised some questions: - I pvscsi actively maintained? Is someone working on this or even prepare it for upstream inclusion in the pv-ops framework of the 3.x series? - Has anybody started on adding the vscsi-semantics to the xl toolstack? - Using ''xm'', I was only able to add a device via its SCSI tupel (HCTL, e.g: vscsi = [ ''7:0:0:0, 0:0:0:1'' ]) but not by any other device-node, such as ''/dev/sde'' or the ''/dev/disk/by-path/<iscsi-iqn>'' link. For these invocations, an "Error: Cannot find device <device>" message is given. I have not yet looked at the issue in detail (in /usr/lib64/python2.6/site-packages/xen/util/vscsi_util.py). Is this a known issue? Is there any developer(s) working on getting pvscsi ready for kernel.org 3.x upstream inclusion? Should i better retool my efforts using pv-ops xen/stable 2.6.32 series? If applicable, I''d be more than happy to provide assistance to move forward pvscsi. Maybe, as a start, provide a short, practical howto about what I did above to get pvscsi working and reference it by the Xen PVSCSI wiki page [3]? Regards, Thomas Weyergraf [1] http://xenbits.xen.org/people/mayoung/EL6.xen/ [2] http://lists.xen.org/archives/html/xen-devel/2011-11/msg02268.html [3] http://wiki.xen.org/xenwiki/XenPVSCSI
On Sun, Feb 19, 2012 at 03:22:08AM +0100, Thomas Weyergraf wrote:> Hi, I am working as a system administrator at an internet platform > service provider, and I am currently seeking to re-new our Xen > virtualization infrastructure for which I am mostly responsible for. > Currently, we run Xen 3.4.2/3.4.3 on RHEL/CentOS 5.x (5.7) as Dom0 > with CentOS 5.x pv-guests. > > Based on my experiments, I am currently looking into Xen 4.1.2 on > RHEL/CentOS 6.x (6.2), with a vanilla 3.2 (3.2.0/3.2.6) kernel as > Dom0 and stock RHEL/CentOS 6.x guests as HVM/pv-ops DomUs.Sweet.> > Our DomU''s have their disk-devices on iscsi-storage backends (EMC, > netapp, Linux IET), we attach the disks to Dom0 via open-iscsi > initiator and pass them as ''phy''-type blockdevices, using > blkfront/back.OK.> > As this is the time for a major overhaul anyway, I looked at > ''pvscsi'', which is very attractive to me for a bunch of > (administrative) reasons. I have created a working pvscsi-setup with > the following steps: > 1. Use CentOS 6.2 as base system > 2. Add Xen hypervisor from Michael Young''s repository [1]. This is > stock 4.1.2 with a patch from 4.1-testing pulled-in and various > patches fixing compile-time issues, toolchain problems and system > service management on RHEL-alike systems. (Nice work, btw!) > 3. Use a vanilla 3.2.0 kernel from kernel.org and a .config based on > a Fedora 16 stock-kernel config, tweaked to use pvscsi. As you know, > Fedora 16 supports Dom0 operation and the only changes to the config > are non-Xen related or add the Xen pvscsi config. I refrained from > attaching the config to avoid mailinglist-clutter, but can do so on > demand.Awesome.> 4. I pulled and applied the pvscsi-patch found in this post [2] from > Konrad Rzeszutek Wilk. > > Putting it all together yielded a "working setup" in which I am able > to pass an Dom0-attached iscsi target to a DomU as a vscsi-device. > In DomU context, I could partition the device, create a filesystem > on it and copy/read/verify some 10gig of data to it. Very basic > testing until now, essentially. However, the whole issue raised some > questions: > > - I pvscsi actively maintained? Is someone working on this or even > prepare it for upstream inclusion in the pv-ops framework of the 3.x > series?So nobody has stepped up to do the upstream task and it is on my todo list but mostly at the end.> - Has anybody started on adding the vscsi-semantics to the xl toolstack?Not that I know of.> - Using ''xm'', I was only able to add a device via its SCSI tupel > (HCTL, e.g: vscsi = [ ''7:0:0:0, 0:0:0:1'' ]) but not by any other > device-node, such as ''/dev/sde'' or the > ''/dev/disk/by-path/<iscsi-iqn>'' link. For these invocations, an > "Error: Cannot find device <device>" message is given. I have not > yet looked at the issue in detail (in > /usr/lib64/python2.6/site-packages/xen/util/vscsi_util.py). Is this > a known issue?First time I hear of it. But then I didn''t even think to use /dev/sde to pass in a device since SCSI devices aren''t neccessarily block ones. They can be scanners, medical devices, tape drivers, etc - which might not have a /dev/sdX (thought they will have an /dev/sgX).> > Is there any developer(s) working on getting pvscsi ready for > kernel.org 3.x upstream inclusion? Should i better retool my efforts > using pv-ops xen/stable 2.6.32 series?For 3.x upstream inclusion - not that I know off. Why would you want to retool your effort in using the 2.6.32 series as opposed to 3.x? What other items is the 3.x missing compared to 2.6.32? Now the good news is that I am happy to keep this going (pv-scsi) and fixing it, and having incremental updates to make it better.> > If applicable, I''d be more than happy to provide assistance to move > forward pvscsi. Maybe, as a start, provide a short, practical howto > about what I did above to get pvscsi working and reference it by the > Xen PVSCSI wiki page [3]?Sure. Also, did you use the git tree to pull the patches or just the big patch file ? The git tree has fixes to make it work under 3.2 as well.> > Regards, > Thomas Weyergraf > > [1] http://xenbits.xen.org/people/mayoung/EL6.xen/ > [2] http://lists.xen.org/archives/html/xen-devel/2011-11/msg02268.html > [3] http://wiki.xen.org/xenwiki/XenPVSCSI > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
I had just compiled pvscsi into 3.3.0-rc1 based on Konrad Rzeszutek Wilk git. I had extracted patch from pvscssi-01 and applied it back to 3.3.0-rc1. I am able to load both xen-scsiback and xen-scsifront modules but I don''t see anything happening. does xl supports pvscsi or it needs to be patched. I am running 4.2 unstable. On Tue, Feb 21, 2012 at 7:55 AM, Konrad Rzeszutek Wilk < konrad.wilk@oracle.com> wrote:> On Sun, Feb 19, 2012 at 03:22:08AM +0100, Thomas Weyergraf wrote: > > Hi, I am working as a system administrator at an internet platform > > service provider, and I am currently seeking to re-new our Xen > > virtualization infrastructure for which I am mostly responsible for. > > Currently, we run Xen 3.4.2/3.4.3 on RHEL/CentOS 5.x (5.7) as Dom0 > > with CentOS 5.x pv-guests. > > > > Based on my experiments, I am currently looking into Xen 4.1.2 on > > RHEL/CentOS 6.x (6.2), with a vanilla 3.2 (3.2.0/3.2.6) kernel as > > Dom0 and stock RHEL/CentOS 6.x guests as HVM/pv-ops DomUs. > > Sweet. > > > > Our DomU''s have their disk-devices on iscsi-storage backends (EMC, > > netapp, Linux IET), we attach the disks to Dom0 via open-iscsi > > initiator and pass them as ''phy''-type blockdevices, using > > blkfront/back. > > OK. > > > > As this is the time for a major overhaul anyway, I looked at > > ''pvscsi'', which is very attractive to me for a bunch of > > (administrative) reasons. I have created a working pvscsi-setup with > > the following steps: > > 1. Use CentOS 6.2 as base system > > 2. Add Xen hypervisor from Michael Young''s repository [1]. This is > > stock 4.1.2 with a patch from 4.1-testing pulled-in and various > > patches fixing compile-time issues, toolchain problems and system > > service management on RHEL-alike systems. (Nice work, btw!) > > 3. Use a vanilla 3.2.0 kernel from kernel.org and a .config based on > > a Fedora 16 stock-kernel config, tweaked to use pvscsi. As you know, > > Fedora 16 supports Dom0 operation and the only changes to the config > > are non-Xen related or add the Xen pvscsi config. I refrained from > > attaching the config to avoid mailinglist-clutter, but can do so on > > demand. > > Awesome. > > 4. I pulled and applied the pvscsi-patch found in this post [2] from > > Konrad Rzeszutek Wilk. > > > > Putting it all together yielded a "working setup" in which I am able > > to pass an Dom0-attached iscsi target to a DomU as a vscsi-device. > > In DomU context, I could partition the device, create a filesystem > > on it and copy/read/verify some 10gig of data to it. Very basic > > testing until now, essentially. However, the whole issue raised some > > questions: > > > > - I pvscsi actively maintained? Is someone working on this or even > > prepare it for upstream inclusion in the pv-ops framework of the 3.x > > series? > > So nobody has stepped up to do the upstream task and it is on my todo list > but mostly at the end. > > > - Has anybody started on adding the vscsi-semantics to the xl toolstack? > > Not that I know of. > > > - Using ''xm'', I was only able to add a device via its SCSI tupel > > (HCTL, e.g: vscsi = [ ''7:0:0:0, 0:0:0:1'' ]) but not by any other > > device-node, such as ''/dev/sde'' or the > > ''/dev/disk/by-path/<iscsi-iqn>'' link. For these invocations, an > > "Error: Cannot find device <device>" message is given. I have not > > yet looked at the issue in detail (in > > /usr/lib64/python2.6/site-packages/xen/util/vscsi_util.py). Is this > > a known issue? > > First time I hear of it. But then I didn''t even think to use /dev/sde > to pass in a device since SCSI devices aren''t neccessarily block ones. > They can be scanners, medical devices, tape drivers, etc - which > might not have a /dev/sdX (thought they will have an /dev/sgX). > > > > > Is there any developer(s) working on getting pvscsi ready for > > kernel.org 3.x upstream inclusion? Should i better retool my efforts > > using pv-ops xen/stable 2.6.32 series? > > For 3.x upstream inclusion - not that I know off. > Why would you want to retool your effort in using the 2.6.32 series > as opposed to 3.x? What other items is the 3.x missing compared to 2.6.32? > > Now the good news is that I am happy to keep this going (pv-scsi) > and fixing it, and having incremental updates to make it better. > > > > > If applicable, I''d be more than happy to provide assistance to move > > forward pvscsi. Maybe, as a start, provide a short, practical howto > > about what I did above to get pvscsi working and reference it by the > > Xen PVSCSI wiki page [3]? > > Sure. Also, did you use the git tree to pull the patches or just > the big patch file ? The git tree has fixes to make it work under 3.2 as > well. > > > > > Regards, > > Thomas Weyergraf > > > > [1] http://xenbits.xen.org/people/mayoung/EL6.xen/ > > [2] http://lists.xen.org/archives/html/xen-devel/2011-11/msg02268.html > > [3] http://wiki.xen.org/xenwiki/XenPVSCSI > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Thu, Feb 23, 2012 at 05:19:00PM -0700, jacek burghardt wrote:> I had just compiled pvscsi into 3.3.0-rc1 based on Konrad Rzeszutek Wilk > git. I had extracted patch from pvscssi-01 and applied it back to > 3.3.0-rc1. I am able to load both xen-scsiback and xen-scsifront modules > but I don''t see anything happening. > does xl supports pvscsi or it needs to be patched. I am running 4.2 > unstable.Did you try to use xm?
On Thu, Feb 23, 2012 at 05:19:00PM -0700, jacek burghardt wrote:> I had just compiled pvscsi into 3.3.0-rc1 based on Konrad Rzeszutek Wilk > git. I had extracted patch from pvscssi-01 and applied it back to > 3.3.0-rc1. I am able to load both xen-scsiback and xen-scsifront modules > but I don''t see anything happening. > does xl supports pvscsi or it needs to be patched. I am running 4.2 > unstable.I don''t think there is currently any PVSCSI support in xl/libxl. -- Pasi> > On Tue, Feb 21, 2012 at 7:55 AM, Konrad Rzeszutek Wilk > <[1]konrad.wilk@oracle.com> wrote: > > On Sun, Feb 19, 2012 at 03:22:08AM +0100, Thomas Weyergraf wrote: > > Hi, I am working as a system administrator at an internet platform > > service provider, and I am currently seeking to re-new our Xen > > virtualization infrastructure for which I am mostly responsible for. > > Currently, we run Xen 3.4.2/3.4.3 on RHEL/CentOS 5.x (5.7) as Dom0 > > with CentOS 5.x pv-guests. > > > > Based on my experiments, I am currently looking into Xen 4.1.2 on > > RHEL/CentOS 6.x (6.2), with a vanilla 3.2 (3.2.0/3.2.6) kernel as > > Dom0 and stock RHEL/CentOS 6.x guests as HVM/pv-ops DomUs. > > Sweet. > > > > Our DomU''s have their disk-devices on iscsi-storage backends (EMC, > > netapp, Linux IET), we attach the disks to Dom0 via open-iscsi > > initiator and pass them as ''phy''-type blockdevices, using > > blkfront/back. > > OK. > > > > As this is the time for a major overhaul anyway, I looked at > > ''pvscsi'', which is very attractive to me for a bunch of > > (administrative) reasons. I have created a working pvscsi-setup with > > the following steps: > > 1. Use CentOS 6.2 as base system > > 2. Add Xen hypervisor from Michael Young''s repository [1]. This is > > stock 4.1.2 with a patch from 4.1-testing pulled-in and various > > patches fixing compile-time issues, toolchain problems and system > > service management on RHEL-alike systems. (Nice work, btw!) > > 3. Use a vanilla 3.2.0 kernel from [2]kernel.org and a .config based > on > > a Fedora 16 stock-kernel config, tweaked to use pvscsi. As you know, > > Fedora 16 supports Dom0 operation and the only changes to the config > > are non-Xen related or add the Xen pvscsi config. I refrained from > > attaching the config to avoid mailinglist-clutter, but can do so on > > demand. > > Awesome. > > 4. I pulled and applied the pvscsi-patch found in this post [2] from > > Konrad Rzeszutek Wilk. > > > > Putting it all together yielded a "working setup" in which I am able > > to pass an Dom0-attached iscsi target to a DomU as a vscsi-device. > > In DomU context, I could partition the device, create a filesystem > > on it and copy/read/verify some 10gig of data to it. Very basic > > testing until now, essentially. However, the whole issue raised some > > questions: > > > > - I pvscsi actively maintained? Is someone working on this or even > > prepare it for upstream inclusion in the pv-ops framework of the 3.x > > series? > > So nobody has stepped up to do the upstream task and it is on my todo > list > but mostly at the end. > > - Has anybody started on adding the vscsi-semantics to the xl > toolstack? > > Not that I know of. > > - Using ''xm'', I was only able to add a device via its SCSI tupel > > (HCTL, e.g: vscsi = [ ''7:0:0:0, 0:0:0:1'' ]) but not by any other > > device-node, such as ''/dev/sde'' or the > > ''/dev/disk/by-path/<iscsi-iqn>'' link. For these invocations, an > > "Error: Cannot find device <device>" message is given. I have not > > yet looked at the issue in detail (in > > /usr/lib64/python2.6/site-packages/xen/util/vscsi_util.py). Is this > > a known issue? > > First time I hear of it. But then I didn''t even think to use /dev/sde > to pass in a device since SCSI devices aren''t neccessarily block ones. > They can be scanners, medical devices, tape drivers, etc - which > might not have a /dev/sdX (thought they will have an /dev/sgX). > > > > Is there any developer(s) working on getting pvscsi ready for > > [3]kernel.org 3.x upstream inclusion? Should i better retool my > efforts > > using pv-ops xen/stable 2.6.32 series? > > For 3.x upstream inclusion - not that I know off. > Why would you want to retool your effort in using the 2.6.32 series > as opposed to 3.x? What other items is the 3.x missing compared to > 2.6.32? > > Now the good news is that I am happy to keep this going (pv-scsi) > and fixing it, and having incremental updates to make it better. > > > > If applicable, I''d be more than happy to provide assistance to move > > forward pvscsi. Maybe, as a start, provide a short, practical howto > > about what I did above to get pvscsi working and reference it by the > > Xen PVSCSI wiki page [3]? > > Sure. Also, did you use the git tree to pull the patches or just > the big patch file ? The git tree has fixes to make it work under 3.2 as > well. > > > > Regards, > > Thomas Weyergraf > > > > [1] [4]http://xenbits.xen.org/people/mayoung/EL6.xen/ > > [2] > [5]http://lists.xen.org/archives/html/xen-devel/2011-11/msg02268.html > > [3] [6]http://wiki.xen.org/xenwiki/XenPVSCSI > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > [7]Xen-devel@lists.xen.org > > [8]http://lists.xen.org/xen-devel > > _______________________________________________ > Xen-devel mailing list > [9]Xen-devel@lists.xen.org > [10]http://lists.xen.org/xen-devel > > References > > Visible links > 1. mailto:konrad.wilk@oracle.com > 2. http://kernel.org/ > 3. http://kernel.org/ > 4. http://xenbits.xen.org/people/mayoung/EL6.xen/ > 5. http://lists.xen.org/archives/html/xen-devel/2011-11/msg02268.html > 6. http://wiki.xen.org/xenwiki/XenPVSCSI > 7. mailto:Xen-devel@lists.xen.org > 8. http://lists.xen.org/xen-devel > 9. mailto:Xen-devel@lists.xen.org > 10. http://lists.xen.org/xen-devel> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel