Hi, I was just wondering following idea, I guess somebody mentioned something similar and I''d like some thoughts on this. 1. create iscsi volume on Node-A and mount it locally with iscsiadm 2. create pool with this local iscsi-share 3. create iscsi volume on Node-B and share it to Node-A 4. create mirror from both disks on Node-A; zpool attach foopool localiscsivolume remotevolume Why not? After quick test it seems to fail and resilver like it should when nodes fail. Actual failover needs to be done manually though, but am I missing something relevant here? Yours Markus Kovero -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090911/329603ad/attachment.html>
This method also allows one to nest mirroring or some RAID-z level with mirroring. When I tested it with a older build a while back, I found performance really poor, about 1-2 MB/second, but my environment was also constrained. A major showstopper had been the infamous 3 minute iSCSI timeout, which was recently fixed, http://bugs.opensolaris.org/view_bug.do?bug_id=6497777. How is your performance? Also, why do you think failover has to be done manually? -- This message posted from opensolaris.org
On Sep 11, 2009, at 5:05 AM, Markus Kovero wrote:> Hi, I was just wondering following idea, I guess somebody mentioned > something similar and I?d like some thoughts on this. > > 1. create iscsi volume on Node-A and mount it locally with > iscsiadm > 2. create pool with this local iscsi-share > 3. create iscsi volume on Node-B and share it to Node-A > 4. create mirror from both disks on Node-A; zpool attach > foopool localiscsivolume remotevolume > > Why not? After quick test it seems to fail and resilver like it > should when nodes fail. Actual failover needs to be done manually > though, but am I missing something relevant here?This is more complicated than the more commonly used, simpler method: 1. create iscsi volume on Node-B, share to Node-A 2. zpool create mypool mirror local-vdev iscsi-vdev -- richard
This also makes failover more easy, as volumes are already shared via iscsi on both nodes. I have to poke it next week to see performance numbers, I could imagine it plays within expected iscsi performance, or it should atleast. Yours Markus Kovero -----Original Message----- From: Richard Elling [mailto:richard.elling at gmail.com] Sent: 11. syyskuuta 2009 19:53 To: Markus Kovero Cc: zfs-discuss at opensolaris.org Subject: Re: [zfs-discuss] sync replication easy way? On Sep 11, 2009, at 5:05 AM, Markus Kovero wrote:> Hi, I was just wondering following idea, I guess somebody mentioned > something similar and I''d like some thoughts on this. > > 1. create iscsi volume on Node-A and mount it locally with > iscsiadm > 2. create pool with this local iscsi-share > 3. create iscsi volume on Node-B and share it to Node-A > 4. create mirror from both disks on Node-A; zpool attach > foopool localiscsivolume remotevolume > > Why not? After quick test it seems to fail and resilver like it > should when nodes fail. Actual failover needs to be done manually > though, but am I missing something relevant here?This is more complicated than the more commonly used, simpler method: 1. create iscsi volume on Node-B, share to Node-A 2. zpool create mypool mirror local-vdev iscsi-vdev -- richard
I believe failover is best to be done manually just to be sure active node is really dead before importing it on another node, otherwise there could be serious issues I think. Yours Markus Kovero -----Original Message----- From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Maurice Volaski Sent: 11. syyskuuta 2009 19:24 To: zfs-discuss at opensolaris.org Subject: Re: [zfs-discuss] sync replication easy way? This method also allows one to nest mirroring or some RAID-z level with mirroring. When I tested it with a older build a while back, I found performance really poor, about 1-2 MB/second, but my environment was also constrained. A major showstopper had been the infamous 3 minute iSCSI timeout, which was recently fixed, http://bugs.opensolaris.org/view_bug.do?bug_id=6497777. How is your performance? Also, why do you think failover has to be done manually? -- This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss at opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
At 8:25 PM +0300 9/11/09, Markus Kovero wrote:>I believe failover is best to be done manually just to be sure >active node is really dead before importing it on another node, >otherwise there could be serious issues I think. >I believe there are many users of Linux-HA, aka heartbeat, who do failover automatically on Linux systems. You can configure a stonith device to "shoot the other node in the head." I had heartbeat running on OpenSolaris, though I never tested failover. Did you get decent performance when you tested? -- Maurice Volaski, maurice.volaski at einstein.yu.edu Computing Support, Rose F. Kennedy Center Albert Einstein College of Medicine of Yeshiva University
On Fri, Sep 11, 2009 at 12:53 PM, Richard Elling <richard.elling at gmail.com> wrote:> On Sep 11, 2009, at 5:05 AM, Markus Kovero wrote: > >> Hi, I was just wondering following idea, I guess somebody mentioned >> something similar and I?d like some thoughts on this. >> >> 1. ? ? ? create iscsi volume on Node-A and mount it locally with iscsiadm >> 2. ? ? ? create pool with this local iscsi-share >> 3. ? ? ? create iscsi volume on Node-B and share it to Node-A >> 4. ? ? ? create mirror from both disks on Node-A; zpool attach foopool >> localiscsivolume remotevolume >> >> Why not? After quick test it seems to fail and resilver like it should >> when nodes fail. Actual failover needs to be done manually though, but am I >> missing something relevant here? > > This is more complicated than the more commonly used, simpler method: > ? ? ? ?1. create iscsi volume on Node-B, share to Node-A > ? ? ? ?2. zpool create mypool mirror local-vdev iscsi-vdevSimpler, yes, but Markus'' method has the advantage that all devices in the pool will be numbered under the iSCSI controller. Question though, what happens in an active-active setup with pools on each side? Pin the iSCSI device names in path_to_inst to prevent overlap? In that case it may be easier to use local devs on each side (easier to pin), which also has the benefit of not putting all the eggs in the iSCSI initiator basket. -Ross
On Sep 11, 2009, at 13:40, Maurice Volaski wrote:> At 8:25 PM +0300 9/11/09, Markus Kovero wrote: >> I believe failover is best to be done manually just to be sure >> active node is really dead before importing it on another node, >> otherwise there could be serious issues I think. > > I believe there are many users of Linux-HA, aka heartbeat, who do > failover automatically on Linux systems. You can configure a stonith > device to "shoot the other node in the head." I had heartbeat > running on OpenSolaris, though I never tested failover. > > Did you get decent performance when you tested?Solaris Cluster is a free download I believe (even in commercial production): http://www.sun.com/software/solaris/cluster/ For OpenSolaris there''s also OpenHA: http://www.opensolaris.com/learn/features/availability/ http://opensolaris.org/os/community/ha-clusters/ Need to cut a cheque for support though (for both/either).
Hi, I managed to test this out, it seems iscsitgt performance is suboptimal with this setup but somehow comstar maxes out gige easily, no performance issues there. Yours Markus Kovero -----Original Message----- From: Maurice Volaski [mailto:maurice.volaski at einstein.yu.edu] Sent: 11. syyskuuta 2009 20:40 To: Markus Kovero; zfs-discuss at opensolaris.org Subject: RE: [zfs-discuss] sync replication easy way? At 8:25 PM +0300 9/11/09, Markus Kovero wrote:>I believe failover is best to be done manually just to be sure >active node is really dead before importing it on another node, >otherwise there could be serious issues I think. >I believe there are many users of Linux-HA, aka heartbeat, who do failover automatically on Linux systems. You can configure a stonith device to "shoot the other node in the head." I had heartbeat running on OpenSolaris, though I never tested failover. Did you get decent performance when you tested? -- Maurice Volaski, maurice.volaski at einstein.yu.edu Computing Support, Rose F. Kennedy Center Albert Einstein College of Medicine of Yeshiva University