> EE> My main question is: does anyone have experience doing this in > EE> production? It looks good on html and man pages, but I would like to > EE> know if there are any caveats I should be aware of. Various threads > EE> I''ve read in the alias archives do not really seem to talk about > EE> people''s experiences with implementing it.There are a number of issues with ''zfs recv'' which were fixed in Nevada build 53, so I would recommend that you use that or later. (see bugs 6468748 6490104 6490105 6490829.) There''s also a number of areas where performance could be improved, which hopefully I''ll be able to get to soon.> btw: I haven''t looked at zfs send/recv code (yet) but it looks like > zfs send is traversing meta-data - it probably would be much faster if > it would send/read data linearly - perhaps upto some specified buffer > size, I wouldn''t mind to design 2GB of RAM just for replication if it > would speed it up considerably.Yes, zfs send traverses metadata, and this is exactly what makes it so fast. It isn''t possible to know what''s changed (or, in fact, read anything from a ZFS storage pool) without traversing the metadata. When doing remote replication, more memory will help, because then we will be able to keep more of the recent changes cached in memory and not have to read them off disk.> EE> Additionally, are there any plans for tools to facilitate such a > EE> system? Something along the lines of a zfsreplicated service, which > EE> could be more robust than a cron job?I''m working on ''zfs send -r'', which will allow you to easily send all the changes in a whole tree of filesystems (including properties, clones, ''zfs rename''s, etc). We''re thinking about how to make a better replicated "service" than making you roll your own shell scripts / cron jobs, but I''m not sure when that will be ready.> I''m also thinking about writing simple daemon which would make use of > libzfs and send/recv streams between hosts instead of scripts and > command line tools. But it''s after I''ll have first conclusions about > send/recv approach.Have fun, but keep in mind that libzfs is a private interface, so it can change at any time, thus breaking your daemon. zfs(1m) is the supported interface. --matt
Hello Eric,
Saturday, December 23, 2006, 6:26:22 AM, you wrote:
EE> Hi all,
EE> I''m currently investigating solutions for disaster recovery, and
would
EE> like to go with a zfs-based solution. From what I understand, there
EE> are two possible methods of achieving this: an iscsi mirror over a WAN
EE> link, and remote replication with incremental zfs send/recv. Due to
EE> performance considerations with the former, I''m looking mostly
at
EE> incremental replication over set intervals.
EE> My main question is: does anyone have experience doing this in
EE> production? It looks good on html and man pages, but I would like to
EE> know if there are any caveats I should be aware of. Various threads
EE> I''ve read in the alias archives do not really seem to talk about
EE> people''s experiences with implementing it.
Lately I''ve started to investigate it in a production.
My first observation is that first full synchronization isn''t
that fast - at least on RAID-Z2. We''ll see about incremental.
Earlier tests are very promising for incremental synchronizations.
Have you also looked at SNDR? Especially when it comes to Open Solaris
next month.
btw: I haven''t looked at zfs send/recv code (yet) but it looks like
zfs send is traversing meta-data - it probably would be much faster if
it would send/read data linearly - perhaps upto some specified buffer
size, I wouldn''t mind to design 2GB of RAM just for replication if it
would speed it up considerably.
EE> Additionally, are there any plans for tools to facilitate such a
EE> system? Something along the lines of a zfsreplicated service, which
EE> could be more robust than a cron job?
I don''t know any.
There''s still open question for me - if I do 30 zfs sends in parallel
or one-by-one the total time will be quicker with ...?
I''ll have to check it later.
I''m also thinking about writing simple daemon which would make use of
libzfs and send/recv streams between hosts instead of scripts and
command line tools. But it''s after I''ll have first conclusions
about
send/recv approach.
--
Best regards,
Robert mailto:rmilkowski@task.gda.pl
http://milek.blogspot.com
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Hi all, I''m currently investigating solutions for disaster recovery, and would like to go with a zfs-based solution. From what I understand, there are two possible methods of achieving this: an iscsi mirror over a WAN link, and remote replication with incremental zfs send/recv. Due to performance considerations with the former, I''m looking mostly at incremental replication over set intervals. My main question is: does anyone have experience doing this in production? It looks good on html and man pages, but I would like to know if there are any caveats I should be aware of. Various threads I''ve read in the alias archives do not really seem to talk about people''s experiences with implementing it. Additionally, are there any plans for tools to facilitate such a system? Something along the lines of a zfsreplicated service, which could be more robust than a cron job? Regards, Eric _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Hello Matthew,
Thursday, January 4, 2007, 12:11:26 AM, you wrote:
MA> There''s also a number of areas where performance could be
improved, which
MA> hopefully I''ll be able to get to soon.
Any update? I would be definitely interested in speeding up zfs
send/recv process.
MA> When doing remote replication, more memory will help, because then we
will
MA> be able to keep more of the recent changes cached in memory and not have
to
MA> read them off disk.
I was looking at the code some time ago and it looks like implementing
asynchronous and continuos replication would be quite easy to
implement. Actually one of my developers is looking into it right now.
We''re thinking about creating another ioctl and then similar to how
it''s done right now we would like to send transactions to specified
file descriptor - loop in a kernel with some sleep (2x txg_time?) and
send all transactions. Maybe we miss something but if we''re not it
should be really easy to implement. That way one could really easily
setup continuos replication between two file systems and do the
snapshoting separately to get in point copies. If we get there then we
would like to create userland tool to actually manage all
replications, etc.
That way we should get most (all) transactions from memory once we''re
synced up on the source host and really recent backup.
--
Best regards,
Robert mailto:rmilkowski at task.gda.pl
http://milek.blogspot.com