zfs discuss - May 2007 - ZFS Support for remote mirroring

Does ZFS support any type of remote mirroring? It seems at present my only two
options to achieve this would be Sun Cluster or Availability Suite. I thought
that this functionality was in the works, but I haven''t heard anything
lately.

Thanks!
Aaron Newcomb
http://opennewsshow.org
http://thesourceshow.org
 
 
This message posted from opensolaris.org

Aaron Newcomb wrote:
> Does ZFS support any type of remote mirroring? It seems at present my only
two options to achieve this would be Sun Cluster or Availability Suite. I
thought that this functionality was in the works, but I haven''t heard
anything lately.
>   
AVS is working today. (See Jim Dunham''s frequent posts.) Are you
looking
for something tied directly into ZFS or ???

Terry,

Yes. AVS is pretty expensive. If ZFS did this out of the box it would be a huge
differentiator. I know ZFS does snapshots today, but if we could extend this
functionality to work across distance then we would have something that could
compete with expensive solutions from EMC, HP, IBM, NetApp, etc. And to do it
with open source software ... even better.

Aaron Newcomb
http://opennewsshow.org
http://thesourceshow.org
 
 
This message posted from opensolaris.org

Aaron Newcomb wrote:
> Terry,
>
> Yes. AVS is pretty expensive. If ZFS did this out of the box it would be a
huge differentiator. I know ZFS does snapshots today, but if we could extend
this functionality to work across distance then we would have something that
could compete with expensive solutions from EMC, HP, IBM, NetApp, etc. And to do
it with open source software ... even better.
AVS is already open-sourced. Not sure as to the free part but given the 
code is out there ...

http://www.opensolaris.org/os/project/avs/
http://www.opensolaris.org/os/project/avs/files/ for the files.

Aaron Newcomb wrote:
> Does ZFS support any type of remote mirroring? It seems at present my
> only two options to achieve this would be Sun Cluster or Availability
> Suite. I thought that this functionality was in the works, but I
haven''t
> heard anything lately.
You could put something together using iSCSI, or zfs send/recv.

--matt

ZFS send/receive?? I am not familiar with this feature. Is there a doc I 
can reference?

Thanks,

Aaron Newcomb
Sr. Systems Engineer
Sun Microsystems
aaron.newcomb at sun.com
Cell: 513-238-9511
Office: 513-562-4409




Matthew Ahrens wrote:
> Aaron Newcomb wrote:
>> Does ZFS support any type of remote mirroring? It seems at present my
>> only two options to achieve this would be Sun Cluster or Availability
>> Suite. I thought that this functionality was in the works, but I
haven''t
>> heard anything lately.
>
> You could put something together using iSCSI, or zfs send/recv.
>
> --matt

Matthew Ahrens wrote:
> Aaron Newcomb wrote:
>> Does ZFS support any type of remote mirroring? It seems at present my
>> only two options to achieve this would be Sun Cluster or Availability
>> Suite. I thought that this functionality was in the works, but I
haven''t
>> heard anything lately.
>
> You could put something together using iSCSI, or zfs send/recv.
I think the definition of remote mirror is up for grabs here but in my 
mind remote mirror means the remote node has a always up to date copy of 
the primary data set modulo any transactions in flight. AVS, aka remote 
mirror, aka sndr, is usually used for this kind of work on the host. 
Storage arrays have things like, ahem, remote mirror, truecopy, srdf, etc.

I guess when we are defining a mirror, are you talking about a synchronous
mirror or an asynchronous mirror?

As stated earlier,  if you are looking for an asynchronous mirror and do not
want to use AVS, you can use zfs send and receive and craft a fairly simple
script that runs constantly and updates a remote filesystem.

zfs send takes a snapshot and turns it into a datastream to standard out while
zfs receive takes a stdin datastream and outputs it to a zfs filesystem. The zfs
send and receive structures are only limited to your creativity.

one example use might be the following

[i]zfs send pool/fs1 at snap1 | ssh remote_hostname zfs receive
remotepool/fs2[/i]

That would get you your initial copy, then you would have to take a snap and do
incrementals from there on in with something like

[i]zfs send -i pool/fs1 at snap1 pool/fs1 at snap2 | ssh remote_hostname zfs
receive remotepool/fs2[/i]

note that the filesystem at the other end (fs2 in thsi case) will be a live
filesystem that you can use anytime.

Now with that incremental commandline, you might run into a bug that is well
known and you can find a work around in these forums, so I won''t get
into it, but your script would have to incorporate the workaround which would
basically run a "[i]zfs rollback[/i]" command on the remote host
before you propagate the incremental changes.

~Bryan
 
 
This message posted from opensolaris.org

Torrey,

Yes. I am used to dealing with the array based software you mention in your post
as well as filesystem based products like AVS and Veritas Volume Replicator.

After reading some documentation send to me by Cindy Swearingen (Thanks Cindy!)
about zfs send and receive it seems that it is not quite what I am looking for,
but very close. I would need something more like a synchronous stream or an
asynchronous stream with write order preservation.
 
 
This message posted from opensolaris.org

Bryan,

Thanks for your suggestion. I am looking at this as more of a DR solution.
However, I might be able to use your method if my data can be a little old.
Perhaps this way I could sync the data nightly with a remote site to make sure
that I am no more than 24 hours behind in the case of a disaster.
 
 
This message posted from opensolaris.org

For whatever reason EMC notes (on PowerLink) suggest that ZFS is not supported
on their arrays. If one is going to use a ZFS filesystem on top of a EMC array
be warned about support issues.
 
 
This message posted from opensolaris.org

Hello Anantha,

Wednesday, May 9, 2007, 4:45:10 PM, you wrote:

ANS> For whatever reason EMC notes (on PowerLink) suggest that ZFS is
ANS> not supported on their arrays. If one is going to use a ZFS
ANS> filesystem on top of a EMC array be warned about support issues.

Nope. For a couple of months they actually do support ZFS.

See http://milek.blogspot.com/2007/01/is-emc-affraid-of-zfs_06.html


-- 
Best regards,
 Robert                            mailto:rmilkowski at task.gda.pl
                                       http://milek.blogspot.com

Anantha N. Srirama wrote:
> For whatever reason EMC notes (on PowerLink) suggest that ZFS is not
supported on their arrays. If one is going to use a ZFS filesystem on top of a
EMC array be warned about support issues.
They should have fixed that in their matrices. It should say something 
like, "EMC supports service LUNs to ZFS."

Folks,
We''re following up with EMC on this.  We''ll post something on
the alias
when we get it.

Please note that EMC would probably never say anything about 
OpenSolaris, but they''ll talk about Solaris ZFS....
Bev.

Torrey McMahon wrote:
> Anantha N. Srirama wrote:
>> For whatever reason EMC notes (on PowerLink) suggest that ZFS is not 
>> supported on their arrays. If one is going to use a ZFS filesystem on 
>> top of a EMC array be warned about support issues.
>
> They should have fixed that in their matrices. It should say something 
> like, "EMC supports service LUNs to ZFS."
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

To clarify further; EMC note "EMC Host Connectivity Guide for Solaris"
indicates that ZFS is supported on 11/06 (aka Update 3) and onwards. However,
they sneak in a cautionary disclaimer that snapshot and clone features are
supported by Sun. If one reads it carefully it appears that they do support ZFS
(not that they should care what FS or not is on their disks) but want to make a
big deal about ZFS by inserting this superfluous comment. Jealousy I suppose, I
don''t see a comparable disclaimer against VxFS for example.
 
 
This message posted from opensolaris.org

> To clarify further; EMC note "EMC Host Connectivity
> Guide for Solaris" indicates that ZFS is supported on
> 11/06 (aka Update 3) and onwards. However, they sneak
> in a cautionary disclaimer that snapshot and clone
> features are supported by Sun. If one reads it
> carefully it appears that they do support ZFS (not
> that they should care what FS or not is on their
> disks) but want to make a big deal about ZFS by
> inserting this superfluous comment. Jealousy I
> suppose, I don''t see a comparable disclaimer against
> VxFS for example.
Well, I think they are protecting themselves. Consider how array based software
deals with snapshots and replication at the LUN level.  Since ZFS filesystems
are logical entities in a pool, there''s no way for an array to
replicate or snapshot a single ZFS filesystem.

So perhaps they are saying that if you want to snapshot or clone a filesystem,
it is only support by the internal mechanisms of ZFS and not the array based
software which could only clone and snapshot an entire pool.
 
 
This message posted from opensolaris.org

> Bryan,
> 
> Thanks for your suggestion. I am looking at this as
> more of a DR solution. However, I might be able to
> use your method if my data can be a little old.
> Perhaps this way I could sync the data nightly with a
> remote site to make sure that I am no more than 24
> hours behind in the case of a disaster.
Yeah, totally understood.

However, if your RPO can be a little bit old, you gain a lot in bandwidth
conservation if you have a slow link between the two sites by replicating
incrementals on a schedule. Let me give a simplified example.

Consider for a moment replicating a single database that is 5MBs in size.  What
if that same database had a "rate of change" of a gigabyte per
hour(maybe all the data is constantly changing within the database).
It''s not growing in size, but it is being heavily modified.  If you use
a constant replication strategy with in order delivery, suddenly you have to
have enough buffer and bandwidth to transfer 24GBs a day in this example for a
5MB database. However, if you replicate once a night instead, you just have to
have enough buffer and bandwidth to replicate 5MBs.

My example is a bit extreme, but I wanted to illustrate the difference between
the bandwidth and resource conservation of a small RPO vs. a longer RPO.

I''m afraid another prob you might run into with trying to replicate ZFS
with AVS or array based replication is that you might have to replicate entire
Pools instead of single filesystems. That may or may not be a problem depending
on your filesystem architecture or DR needs. Maybe you need all the filesystems
anyway. :)

~Bryan
 
 
This message posted from opensolaris.org