zfs discuss - Mar 2007 - ZFS filesystem online backup question

I have to backup many filesystems, which are changing and machines are heavy
loaded.
The idea is to backup online - this should avoid I/O read operations from disks,
data should go from cache.

Now I''m using script that does snapshot and zfs send.
I want to automate this operation and add new option to zfs send 

     zfs send [-w sec ] [-i <snapshot>] <snapshot>

for example

    zfs send -w 10 pool/fs at replicate

zfs send then would:
1. create replicate snapshot if it does not exist
2. send data
3. wait 10 seconds
4. rename snapshot to replicate_previous ( destroy previous if exists )
5. goto 1.

All snapshot operations are done in kernel - it works faster then.
I have implemented this mechanism and it works.

Do you think this change will be integrated to opensolaris ?
Is there chance this option will be available in Solaris update 4 ?

Maybe there is other way to backup filesystem online ?
I tried to traverse changing filesystem, but it does not work.
 
 
This message posted from opensolaris.org

On Tue, 27 Mar 2007, [UTF-8] ?^Aukasz wrote:
> zfs send then would:
> 1. create replicate snapshot if it does not exist
> 2. send data
> 3. wait 10 seconds
> 4. rename snapshot to replicate_previous ( destroy previous if exists )
> 5. goto 1.
>
> All snapshot operations are done in kernel - it works faster then. I
> have implemented this mechanism and it works.
Out of curiosity, what is the timing difference between a userland script
and performing the operations in the kernel?

Which of the steps are you attempting to speed up?  What''s the
bottleneck?


Regards,
markm

>Out of curiosity, what is the timing difference between a userland script
>and performing the operations in the kernel?
[root at zfs ~]# time zfs destroy solaris/d101 at replicate_previous ; time zfs
rename solaris/d101 at replicate_latest solaris/d101 at replicate_previous; time
zfs snapshot solaris/d101 at replicate_latest

real    0m5.220s
user    0m0.010s
sys     0m0.023s

real    0m5.856s
user    0m0.010s
sys     0m0.023s

real    0m7.620s
user    0m0.009s
sys     0m0.029s
[root at zfs ~]# time zfs destroy solaris/d101 at replicate_previous ; time zfs
rename solaris/d101 at replicate_latest solaris/d101 at replicate_previous; time
zfs snapshot solaris/d101 at replicate_latest

real    0m7.363s
user    0m0.010s
sys     0m0.031s

real    0m5.107s
user    0m0.010s
sys     0m0.022s

real    0m7.888s
user    0m0.009s
sys     0m0.024s

Operation takes 15 - 20 seconds

In kernel it takes ( time in ms ):
  0  42867       dmu_objset_snapshot:return time 2471
  1  42867       dmu_objset_snapshot:return time 10803
  1  42867       dmu_objset_snapshot:return time 7968
  0  42867       dmu_objset_snapshot:return time 14139
  0  42867       dmu_objset_snapshot:return time 14405
  1  42867       dmu_objset_snapshot:return time 8883
  0  42867       dmu_objset_snapshot:return time 4960

Now the code in kernel is without optimalization

zfs_unmount_snap(snap_previous, NULL);
dmu_objset_destroy(snap_previous);
zfs_unmount_snap(zc->zc_value, NULL);
dmu_objset_rename(zc->zc_value, snap_previous);		
error = dmu_objset_snapshot(zc->zc_name,
	    						REPLICATE_SNAPSHOT_LATEST, 0);

In kernel operation can be optimized and done in one dsl_sync_task_do call.
 
 
This message posted from opensolaris.org

?ukasz wrote:
> All snapshot operations are done in kernel - it works faster then.
> I have implemented this mechanism and it works.
Cool!
> Do you think this change will be integrated to opensolaris ?
It''s possible, but I''d prefer to first exhaust all options for
improving
performance of the base operations.
> Is there chance this option will be available in Solaris update 4 ?
No, it''s too late to integrate features into update 4.

--matt

On Tue, 27 Mar 2007, [UTF-8] ?^Aukasz wrote:
> >Out of curiosity, what is the timing difference between a userland
script
> >and performing the operations in the kernel?
>
> Operation takes 15 - 20 seconds
>
> In kernel it takes ( time in ms ):
  [between 2.5 and 14.5 seconds]

Very nice improvement.
> In kernel operation can be optimized and done in one dsl_sync_task_do
> call.
Is this where the speed-up is, or is it that libzfs has a lot of overhead
for the three operations (destroy, snapshot, rename)?

Currently, destroy and snapshot have got a -r option for performing
recursive operations on snapshots, and rename is getting one soon.  Will
your changes handle recursive sends too?  Or do they require a separate
zfs send per filesystem?


Regards,
markm