zfs discuss - Jul 2010 - Is there any support for bi-directional synchronization in zfs?

Folks,

This is probably a very naive question.

Is it possible to set zfs for bi-directional synchronization of data across two
locations? I am thinking this is almost impossible. Consider two files A and B
at two different sites. There are three possible cases that require
synchronization:

   1. A is changed. B is unchanged.
   2. B is changed. A is unchanged.
   3. A is changed. B is changed.

While it is possible to achieve synchronization for the first two cases, case 3
requires special merging and is almost impossible.

I am thinking it is the same problem even at the block level.

Even to achieve 1 and 2 is a bit tricky given the latency between the two sites.
Is there anything in zfs that makes it easier?

Thank you in advance for sharing your thoughts.

Regards,
Peter
-- 
This message posted from opensolaris.org

On 07/14/10 07:10 PM, Peter Taps wrote:
> Folks,
>
> This is probably a very naive question.
>
> Is it possible to set zfs for bi-directional synchronization of data across
two locations? I am thinking this is almost impossible. Consider two files A and
B at two different sites. There are three possible cases that require
synchronization:
>
>     1. A is changed. B is unchanged.
>     2. B is changed. A is unchanged.
>     3. A is changed. B is changed.
>
> While it is possible to achieve synchronization for the first two cases,
case 3 requires special merging and is almost impossible.
>
> I am thinking it is the same problem even at the block level.
>
> Even to achieve 1 and 2 is a bit tricky given the latency between the two
sites. Is there anything in zfs that makes it easier?
>
> Thank you in advance for sharing your thoughts.
>
>    
Use a version control tool like hg or svn!

-- 
Ian.

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Peter Taps
> 
> Is it possible to set zfs for bi-directional synchronization of data
> across two locations? I am thinking this is almost impossible. Consider
You are probably looking for lustre, or gluster, or gfs, or afs, or
something like that.  No, ZFS doesn''t do this.

----- Original Message -----
> > From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> > bounces at opensolaris.org] On Behalf Of Peter Taps
> >
> > Is it possible to set zfs for bi-directional synchronization of data
> > across two locations? I am thinking this is almost impossible.
> > Consider
> 
> You are probably looking for lustre, or gluster, or gfs, or afs, or
> something like that. No, ZFS doesn''t do this.
I thought it was possible to run glusterfs over ZFS these days...

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy at karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er
et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av
idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og
relevante synonymer p? norsk.

On Jul 14, 2010, at 05:15, Ian Collins wrote:
> Use a version control tool like hg or svn!
Or Unison:

http://en.wikipedia.org/wiki/Unison_(file_synchronizer)

----- Original Message -----
> On Jul 14, 2010, at 05:15, Ian Collins wrote:
> 
> > Use a version control tool like hg or svn!
> 
> Or Unison:
> 
> http://en.wikipedia.org/wiki/Unison_(file_synchronizer)
Unison is slow, very slow on large datasets, so in a multiterabyte setup, it
will probably be quite useless. That''s from my personal testing,
though, others may have gained better results.

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy at karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er
et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av
idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og
relevante synonymer p? norsk.

On Jul 14, 2010, at 9:10 AM, Peter Taps wrote:
> Folks,
> 
> This is probably a very naive question.
> 
> Is it possible to set zfs for bi-directional synchronization of data across
two locations? I am thinking this is almost impossible. Consider two files A and
B at two different sites. There are three possible cases that require
synchronization:
> 
>   1. A is changed. B is unchanged.
>   2. B is changed. A is unchanged.
>   3. A is changed. B is changed.
> 
> While it is possible to achieve synchronization for the first two cases,
case 3 requires special merging and is almost impossible.
It is certainly not impossible, people do this every day.
> I am thinking it is the same problem even at the block level.
No, it is just much more difficult at the block level because blocks do not have
context.  Your view of A and B requires some level of context above the block
level. So you must do the reconciliation at that level, not below.  Hence the
recommendations to use unison, hg, svn, or even OpenOffice which have the
tools at the contextual level of the data to reconcile differences between two
objects.
> Even to achieve 1 and 2 is a bit tricky given the latency between the two
sites. Is there anything in zfs that makes it easier?
Don''t try to solve this problem by removing data contextual knowledge,
try to
solve it by increasing data context. 
 -- richard


-- 
Richard Elling
richard at nexenta.com   +1-760-896-4422