I wonder if someone can point me in right directions. I have two dell servers I setup iscsi so I have four 2 tb hard drives and i had used lvm to create one big partiton and share it using iscsi. How I go about assigning sections of iscsi for virtual hard drives . should go about assigning Should I export the whole 8TB as one iscsi and then use lvm to create smaller virtual disk. Or should I export smaller portions of lvm as separate iscsi. Right now I use lvm and this is my disk = [ ''format=raw, vdev=xvda, access=rw, target=/dev/xen/win2012b. Do I need to install and use oopen-iscsi ? Any pointers to documentation _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
jacek burghardt wrote: I wonder if someone can point me in right directions. I have two dell servers I setup iscsi so I have four 2 tb hard drives and i had used lvm to create one big partiton and share it using iscsi. How I go about assigning sections of iscsi for virtual hard drives . should go about assigning Should I export the whole 8TB as one iscsi and then use lvm to create smaller virtual disk. Or should I export smaller portions of lvm as separate iscsi. I think general advice tends to be to create a separate iSCSI volume for each virtual disk, or at the very least per virtual machine. That way it''s easy to resize a volume at the iSCSI level, and to move a guest by disconnecting the volume from one host and connect it to a different one. You could use just one big volume, put LVM on it, and slice it up that way - but then the entire volume (and all the guest volumes within it) are tied to just one machine - unless you go for a clustered LVM setup. Also, you need to do the iSCSI connections in Dom0 - because fo the way Xen networking works (single thread in Dom0), iSCSI performance in a DomU is "rather poor".
jacek burghardt wrote:> I had created 30 300GB lvms and exported them as luvs on iscsi. Now how I tell xen to use iscsi luv for virtual hadr drive ?Exactly the same way you''d make any other disk available. You just need to substitute the appropriate path, which is likely to be something like "disk/by-path/ip-...isci..."You will of course have to setup the appropriate iSCSI connections first. However, I''ve had trouble with this and on the one iSCSI volume I''m using I have to use /dev/sdc for some reason I''ve not had time to look into. I don''t know whether it''s a common problem or specific to my setup. PS - it''s bad etiquette to take a conversation off-list without being prompted first. Please keep replies/further questions to the list.
http://backdrift.org/xen-block-iscsi-script-with-multipath-support Read all comments!!!!!! 2013/1/21 Simon Hobson <linux@thehobsons.co.uk>> jacek burghardt wrote: > > I had created 30 300GB lvms and exported them as luvs on iscsi. Now how > I tell xen to use iscsi luv for virtual hadr drive ? > > Exactly the same way you''d make any other disk available. You just need to > substitute the appropriate path, which is likely to be something like > "disk/by-path/ip-...isci..."You will of course have to setup the > appropriate iSCSI connections first. > > However, I''ve had trouble with this and on the one iSCSI volume I''m using > I have to use /dev/sdc for some reason I''ve not had time to look into. I > don''t know whether it''s a common problem or specific to my setup. > > > PS - it''s bad etiquette to take a conversation off-list without being > prompted first. Please keep replies/further questions to the list. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Carlos wrote: http://backdrift.org/xen-block-iscsi-script-with-multipath-support Read all comments!!!!!! Looks interesting. I''ll have to look at it when I have time to read it properly.
Hey, Guys: While the discussion on Backdraft (http://backdrift.org/xen-block-iscsi-script-with-multipath-support) is very powerful it seems to focus exclusively on the use of multipath. I think that Jacek is asking whether to: 1. export the logical volumes (i.e., LV''s) from his LVM volume group (VG) as individual iSCSI LUN''s or to 2. export the underlying partitions and then import them to his host system as physical volumes (PV''s) and then parition create the VG and the LV''s on the host system. Eric P.>________________________________ > From: jacek burghardt <jaceksburghardt@gmail.com> >To: xen-users <xen-users@lists.xen.org> >Sent: Saturday, January 19, 2013 6:31 PM >Subject: [Xen-users] iscsi on xen > > >I wonder if someone can point me in right directions. I have two dell servers I setup iscsi so I have four 2 tb hard drives and i had used lvm to create one big partiton and share it using iscsi. How I go about assigning sections of iscsi for virtual hard drives . should go about assigning Should I export the whole 8TB as one iscsi and then use lvm to create smaller virtual disk. Or should I export smaller portions of lvm as separate iscsi. Right now I use lvm and this is my disk = [ ''format=raw, vdev=xvda, access=rw, target=/dev/xen/win2012b. Do I need to install and use oopen-iscsi ? Any pointers to documentation > >_______________________________________________ >Xen-users mailing list >Xen-users@lists.xen.org >http://lists.xen.org/xen-users > >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On 01/22/2013 02:01 PM, Eric wrote:> Hey, Guys: > > While the discussion on Backdraft (http://backdrift.org/xen-block-iscsi-script-with-multipath-support) is very powerful it seems to focus exclusively on the use of multipath. I think that Jacek is asking whether to: > > 1. export the logical volumes (i.e., LV''s) from his LVM volume group (VG) as individual iSCSI LUN''s or to > 2. export the underlying partitions and then import them to his host system as physical volumes (PV''s) and then parition create the VG and the LV''s on the host system. > > Eric P. > > > -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! -------> *From:* jacek burghardt <jaceksburghardt@gmail.com> > *To:* xen-users <xen-users@lists.xen.org> > *Sent:* Saturday, January 19, 2013 6:31 PM > *Subject:* [Xen-users] iscsi on xen > > I wonder if someone can point me in right directions. I have two dell servers I setup iscsi so I have four 2 tb hard drives and i had used lvm to create one big partiton and share it using iscsi. How I go about assigning sections of iscsi for virtual hard drives . should go about assigning Should I export the whole 8TB as one iscsi and then use lvm to create smaller virtual disk. Or should I export smaller portions of lvm as separate iscsi. Right now I use lvm and this is my disk = [ ''format=raw, vdev=xvda, access=rw, target=/dev/xen/win2012b. Do I need to install and use oopen-iscsi ? Any pointers to documentation > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org <mailto:Xen-users@lists.xen.org> > http://lists.xen.org/xen-users > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-usersWe export our logical volumes (LVM2) via iscsi LUNs to our xen boxes, one advantage of this scheme is you can connect more than one xen box to the iscsi targets to provide live migration support. Our system looks like this.. PV --> VG --> LVM --> drbd --> iscsi-target =====LAN===== xen + iscsi-client This is a production system with 8 xen boxes and 60 VMs and works very well. Regards. Steve.
Thanks for sharing, Steve: I noticed that you''re using DRBD to replicate the LV''s and then exporting the individual DRBD resources via iSCSI (instead of using DRBD to replicate the entire PV and then exporting the underlying block device to the iSCSI client [for cLVM]). This seems cumbersome as each new LV requires that a new DRBD resource and iSCSI LUN be defined. Have you tried using DRBD resources as whole PV''s [and then exporting LV''s]? Check out the Linbit Tech Guide "Highly available iSCSI with DRBD and Pacemaker" (http://www.linbit.com/en/downloads/tech-guides?download=9:highly-available-iscsi-with-drbd-and-pacemaker) Eric Pretorious Truckee, CA>________________________________ > From: Steve Dawson <sdawson@pa.metrocast.net> >To: xen-users@lists.xen.org >Sent: Tuesday, January 22, 2013 2:15 PM >Subject: Re: [Xen-users] iscsi on xen > >We export our logical volumes (LVM2) via iscsi LUNs to our xen boxes, one advantage of this scheme is you can connect more than one xen box to the iscsi targets to provide live migration support. > >Our system looks like this.. > >PV --> VG --> LVM --> drbd --> iscsi-target =====LAN===== xen + iscsi-client > >This is a production system with 8 xen boxes and 60 VMs and works very well. > >Regards. > >Steve. > >_______________________________________________ >Xen-users mailing list >Xen-users@lists.xen.org >http://lists.xen.org/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Btw: we are using iscsi with xen for about 5 years. One lun and several lvm partitions above it to host domU. Recently we had power problem in DC and we came into problem that initializing LVM disk is very very slow when LVM snapshots are in use. 2013/1/23 Eric <epretorious@yahoo.com>:> Thanks for sharing, Steve: > > I noticed that you''re using DRBD to replicate the LV''s and then exporting > the individual DRBD resources via iSCSI (instead of using DRBD to replicate > the entire PV and then exporting the underlying block device to the iSCSI > client [for cLVM]). This seems cumbersome as each new LV requires that a new > DRBD resource and iSCSI LUN be defined. Have you tried using DRBD resources > as whole PV''s [and then exporting LV''s]? > > Check out the Linbit Tech Guide "Highly available iSCSI with DRBD and > Pacemaker" > (http://www.linbit.com/en/downloads/tech-guides?download=9:highly-available-iscsi-with-drbd-and-pacemaker) > > Eric Pretorious > Truckee, CA > > > ________________________________ > From: Steve Dawson <sdawson@pa.metrocast.net> > To: xen-users@lists.xen.org > Sent: Tuesday, January 22, 2013 2:15 PM > Subject: Re: [Xen-users] iscsi on xen > > We export our logical volumes (LVM2) via iscsi LUNs to our xen boxes, one > advantage of this scheme is you can connect more than one xen box to the > iscsi targets to provide live migration support. > > Our system looks like this.. > > PV --> VG --> LVM --> drbd --> iscsi-target =====LAN===== xen + iscsi-client > > This is a production system with 8 xen boxes and 60 VMs and works very well. > > Regards. > > Steve. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users
Eric wrote:> I think that Jacek is asking whether to:> 1. export the logical volumes (i.e., LV''s) from his LVM volume group (VG) > as individual iSCSI LUN''s or to > 2. export the underlying partitions and then import them to his host > system as physical volumes (PV''s) and then parition create the VG and > the LV''s on the host system.Indeed. But there is no correct answer - it depends on various factors, not least your requirements and how you run your systems. If you have one host, and no requirement to move any guests around, then mounting one iSCSI volume in the host and putting LVM on top of that will be the easiest way to go. If you have multiple hosts and expect to need to move guests around, then one iSCSI volume/guest or one iSCSI volume/guest volume is the way to go. Alternatively, you either have to move guests by physically shutting it down and copying it''s files (which has disk space management issues), or you have to run a clustered LVM on top of your single iSCSI volume. The scripts mentioned by Carlos are designed to automate some of the headache of managing large numbers of iSCSI volumes - ie make option 2 easier to manage.
On 01/22/2013 07:15 PM, Eric wrote:> Thanks for sharing, Steve: > > I noticed that you''re using DRBD to replicate the LV''s and then exporting the individual DRBD resources via iSCSI (instead of using DRBD to replicate the entire PV and then exporting the underlying block device to the iSCSI client [for cLVM]). This seems cumbersome as each new LV requires that a new DRBD resource and iSCSI LUN be defined. Have you tried using DRBD resources as whole PV''s [and then exporting LV''s]? > > Check out the Linbit Tech Guide "Highly available iSCSI with DRBD and Pacemaker" (http://www.linbit.com/en/downloads/tech-guides?download=9:highly-available-iscsi-with-drbd-and-pacemaker) > > Eric Pretorious > Truckee, CA > > > -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! -------> *From:* Steve Dawson <sdawson@pa.metrocast.net> > *To:* xen-users@lists.xen.org > *Sent:* Tuesday, January 22, 2013 2:15 PM > *Subject:* Re: [Xen-users] iscsi on xen > > We export our logical volumes (LVM2) via iscsi LUNs to our xen boxes, one advantage of this scheme is you can connect more than one xen box to the iscsi targets to provide live migration support. > > Our system looks like this.. > > PV --> VG --> LVM --> drbd --> iscsi-target =====LAN===== xen + iscsi-client > > This is a production system with 8 xen boxes and 60 VMs and works very well. > > Regards. > > Steve. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org <mailto:Xen-users@lists.xen.org> > http://lists.xen.org/xen-users > >The main reasons we don''t replicate the entire PV is we have live and test versions of most of our VMs, our replication partner is remote and has limited bandwidth. None of our test systems are replicated. Limiting the number of replicated LVs keeps our bandwidth usage within limits. Regards, Steve.