I want to build on Open Source tools a high availabitiy server to store my images, someone with suggestions? experiences plz. really aprreciate. Started to try Gluster but yesterday discussion pointed me very important things. thx _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Alejandro Weintz Aguilar schreef:> I want to build on Open Source tools a high availabitiy server to store > my images, someone with suggestions? experiences plz. really aprreciate. > > Started to try Gluster but yesterday discussion pointed me very > important things.In my opinion, take Open Solaris, the new version has improved iSCSI support. In that way you have ''safe storage'' and you can run your images over iSCSI. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHl1/bYH1+F2Rqwn0RCrnxAKCJ8n2uObpMy5podPrfxFETX4WKTgCcCOQ5 juSncCbWnLH11y07RA0Uinc=JxE3 -----END PGP SIGNATURE----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I have my XEN servers on a SAN and I use OCFS2 and EVMS to create storage areas for my servers. This also requires that you configure heartbeat, as the cluster part of EVMS uses heartbeat. I''ve run into a couple of issues with this method, though - first, I''m not able to resize volumes on EVMS. Second, I''ve had some trouble getting heartbeat to work correctly, especially when adding nodes to it. That could just be my lack of experience with the tool, though. Based on my experience, I''d suggest something like this: 1) If you don''t have a SAN, get a machine that has a good deal of built-in redundancy (redundant power supplies, HARDWARE RAID unit, etc.) and set it up as an iSCSI server (you can use iSCSI Enterprise Target - ietf - for this). Use dual network cards and some sort of redundant Ethernet setup (bonding, for example) to connect this machine to a switch. Even the cheapest of "smart switches" (NetGear GS108T, for example) support link aggregation. 2) Use either iSCSI HBAs (Qlogic has some nice ones) or the iSCSI software initiator and connect your XEN servers to the iSCSI Target. 3) Use LVM2 with clvmd (Cluster LVM Daemon) to manage volumes. This will give you the flexibility to choose whether you''d like to create one big storage area and store the disk images as files in that folder or configure each VM to have its own LVM volume. 4) If you choose to create a large file share and mount it, use OCFS2 or GFS on the share. Configuration of either of these filesystems is relatively easy. GFS has the plus of supporting POSIX ACLs, but I''m not sure that really matters on XEN servers, so compare them and pick the ones you like. -Nick>>> On Wed, Jan 23, 2008 at 8:04 AM, Alejandro Weintz Aguilar <alejandro@pixelestudios.com> wrote:I want to build on Open Source tools a high availabitiy server to store my images, someone with suggestions? experiences plz. really aprreciate. Started to try Gluster but yesterday discussion pointed me very important things. thx _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
A setup with debian + GB network + Open-Iscsi +heartbeat would work? thx On Wed, 2008-01-23 at 08:50 -0700, Nick Couchman wrote:> I have my XEN servers on a SAN and I use OCFS2 and EVMS to create > storage areas for my servers. This also requires that you configure > heartbeat, as the cluster part of EVMS uses heartbeat. I''ve run into > a couple of issues with this method, though - first, I''m not able to > resize volumes on EVMS. Second, I''ve had some trouble getting > heartbeat to work correctly, especially when adding nodes to it. That > could just be my lack of experience with the tool, though. > > > Based on my experience, I''d suggest something like this: > > 1) If you don''t have a SAN, get a machine that has a good deal of > built-in redundancy (redundant power supplies, HARDWARE RAID unit, > etc.) and set it up as an iSCSI server (you can use iSCSI Enterprise > Target - ietf - for this). Use dual network cards and some sort of > redundant Ethernet setup (bonding, for example) to connect this > machine to a switch. Even the cheapest of "smart switches" (NetGear > GS108T, for example) support link aggregation. > > 2) Use either iSCSI HBAs (Qlogic has some nice ones) or the iSCSI > software initiator and connect your XEN servers to the iSCSI Target. > > 3) Use LVM2 with clvmd (Cluster LVM Daemon) to manage volumes. This > will give you the flexibility to choose whether you''d like to create > one big storage area and store the disk images as files in that folder > or configure each VM to have its own LVM volume. > > 4) If you choose to create a large file share and mount it, use OCFS2 > or GFS on the share. Configuration of either of these filesystems is > relatively easy. GFS has the plus of supporting POSIX ACLs, but I''m > not sure that really matters on XEN servers, so compare them and pick > the ones you like. > > > -Nick > > >>> On Wed, Jan 23, 2008 at 8:04 AM, Alejandro Weintz Aguilar > <alejandro@pixelestudios.com> wrote: > > > I want to build on Open Source tools a high availabitiy server to > store > my images, someone with suggestions? experiences plz. really > aprreciate. > > Started to try Gluster but yesterday discussion pointed me very > important things. > > thx > > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Yes, that would work. You only need heartbeat if you choose to use EVMS. If you don''t use EVMS but want to use LVM2, you need clvmd which works with the RedHat Cluster tools. If you don''t want to use either of those, you don''t have to, and you can just pick a cluster-enabled filesystem to use on the nodes. -Nick>>> On Wed, Jan 23, 2008 at 10:08 AM, Alejandro Weintz Aguilar <alejandro@pixelestudios.com> wrote:A setup with debian + GB network + Open-Iscsi +heartbeat would work? thx On Wed, 2008-01-23 at 08:50 -0700, Nick Couchman wrote:> I have my XEN servers on a SAN and I use OCFS2 and EVMS to create > storage areas for my servers. This also requires that you configure > heartbeat, as the cluster part of EVMS uses heartbeat. I''ve run into > a couple of issues with this method, though - first, I''m not able to > resize volumes on EVMS. Second, I''ve had some trouble getting > heartbeat to work correctly, especially when adding nodes to it. That > could just be my lack of experience with the tool, though. > > > Based on my experience, I''d suggest something like this: > > 1) If you don''t have a SAN, get a machine that has a good deal of > built-in redundancy (redundant power supplies, HARDWARE RAID unit, > etc.) and set it up as an iSCSI server (you can use iSCSI Enterprise > Target - ietf - for this). Use dual network cards and some sort of > redundant Ethernet setup (bonding, for example) to connect this > machine to a switch. Even the cheapest of "smart switches" (NetGear > GS108T, for example) support link aggregation. > > 2) Use either iSCSI HBAs (Qlogic has some nice ones) or the iSCSI > software initiator and connect your XEN servers to the iSCSI Target. > > 3) Use LVM2 with clvmd (Cluster LVM Daemon) to manage volumes. This > will give you the flexibility to choose whether you''d like to create > one big storage area and store the disk images as files in that folder > or configure each VM to have its own LVM volume. > > 4) If you choose to create a large file share and mount it, use OCFS2 > or GFS on the share. Configuration of either of these filesystems is > relatively easy. GFS has the plus of supporting POSIX ACLs, but I''m > not sure that really matters on XEN servers, so compare them and pick > the ones you like. > > > -Nick > > >>> On Wed, Jan 23, 2008 at 8:04 AM, Alejandro Weintz Aguilar > <alejandro@pixelestudios.com> wrote: > > > I want to build on Open Source tools a high availabitiy server to > store > my images, someone with suggestions? experiences plz. really > aprreciate. > > Started to try Gluster but yesterday discussion pointed me very > important things. > > thx > > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> I want to build on Open Source tools a high availabitiy server to store > my images, someone with suggestions? experiences plz. really aprreciate. > > Started to try Gluster but yesterday discussion pointed me very > important things.For Image storage you don''t need a Cluster Filesystem. It even maybe a performance bottleneck, as any Clusterfilesystem will need write locking over the net. What you really need is some kind of shared storage. That could be Fibrechannel, Iscsi or (today not any more common) SCSI. The shared storage must have redundant paths to be really high availability. So that means: Redundant paths to the storage as well as redundant raid controllers. (For the FC example: Dual HBA''s, Dual FC Switch, Storage with Dual Controller). For the path discovery use dm-multipath. What I''m doing in the front: Three server (all with dual FC HBA) with Scientific Linux 5.1 (a RHEL clone like CentOS). The servers are members of a cluster (of course you need servers with a BMC and IPMI over Lan for the fencing). On the clusternodes I run clvmd (Cluster Logical Volume Manager) and rgmanager (Resource Manager). The images are just Logical Volumes, they will not be mounted on the hosts. The cluster locking is only needed for the lvm commands, so its no bottleneck. Any node sees the same images, so on failover another node could immediately restart the failed domU. rgmanager handles nicely the failover of domU''s. If it''s enough availability when you''re domU''s are up again after a host failure after a few 10''s of seconds, then that''s it. If you need a little bit more high availability, you could think of clustering also domU''s and define failover services on that cluster. There are rgmanager scripts already in the distribution for a bunch of common services like nfs, samba, http and so on. Any software used in that setup is opensource. You could of course use other distributions which support Redhat Clustering, but easiest way is to use CentOS or Scientific Linux. Both have long term support and are available without license costs. Sincerly, Klaus -- Klaus Steinberger Beschleunigerlaboratorium Phone: (+49 89)289 14287 Am Coulombwall 6, D-85748 Garching, Germany FAX: (+49 89)289 14280 EMail: Klaus.Steinberger@Physik.Uni-Muenchen.DE URL: http://www.physik.uni-muenchen.de/~Klaus.Steinberger/ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Have a look at using DRBD, Heartbeat and XEN with two physical machines. That is the setup I''m currently testing and so far it works great. At the moment I use old hardware(from the P3 era) with old discs and only 100Mbps NICs, but still a live migration of a DomU is hardly noticeable. I had a go first with the Redhat cluster suite, but it doesn''t run well with XEN and DRBD. I had to go back in versions to get it running but then I lost important functionality. With version 0.8 of DRBD there is support for XEN live migration. I really like the way XEN takes care of the my DRBD resources, I have no script for this, it is all up to XEN. I then use Heartbeat to monitor the status of each host that will automatically migrate DomUs to the other host if needed. Another recommended thing is to use a STONITH device so Heartbeat can reset of the other host in case of a failure, have not tried that yet. Cheers, Daniel> > > I want to build on Open Source tools a high availabitiy server to store > > my images, someone with suggestions? experiences plz. really aprreciate. > > > > Started to try Gluster but yesterday discussion pointed me very > > important things. > > For Image storage you don''t need a Cluster Filesystem. It even maybe a > performance bottleneck, as any Clusterfilesystem will need write locking over > the net. > > What you really need is some kind of shared storage. That could be > Fibrechannel, Iscsi or (today not any more common) SCSI. > > The shared storage must have redundant paths to be really high availability. > So that means: Redundant paths to the storage as well as redundant raid > controllers. (For the FC example: Dual HBA''s, Dual FC Switch, Storage with > Dual Controller). For the path discovery use dm-multipath. > > What I''m doing in the front: > > Three server (all with dual FC HBA) with Scientific Linux 5.1 (a RHEL clone > like CentOS). The servers are members of a cluster (of course you need > servers with a BMC and IPMI over Lan for the fencing). On the clusternodes I > run clvmd (Cluster Logical Volume Manager) and rgmanager (Resource Manager). > > The images are just Logical Volumes, they will not be mounted on the hosts. > The cluster locking is only needed for the lvm commands, so its no > bottleneck. Any node sees the same images, so on failover another node could > immediately restart the failed domU. > > rgmanager handles nicely the failover of domU''s. > > If it''s enough availability when you''re domU''s are up again after a host > failure after a few 10''s of seconds, then that''s it._______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users