zfs discuss - Jun 2009 - zfs send -R gets blocked by auto-snapshot service

Hi,

I''m trying to send a pool (its filesystems) from a pc to another, so I
first created a recursive snapshot:

# zfs list
NAME                         USED  AVAIL  REFER  MOUNTPOINT
nas                          840G   301G  3,28G  /nas
nas/drivers                 12,6G   301G  12,6G  /nas/drivers
nas/nwserv                   110G   301G  49,0G  /nas/nwserv
nas/rsync_clienti            166G   301G  17,5G  /nas/rsync_clienti
nas/rsync_privati            560M   301G  36,5K  /nas/rsync_privati
nas/rsync_privati/gianluca   560M   301G   560M  /nas/rsync_privati/gianluca
nas/samba                    486G   301G   486G  /nas/samba
nas/sviluppo-bak            1,91G   301G  1,60G  /nas/sviluppo-bak
nas/winsrv                  60,0G   301G  55,1G  /nas/winsrv

# zfs snapshot -r nas at T1

zfs list -t snapshot -r nas | grep T1
nas at T1                                                                  0    
-  3,28G  -
nas/drivers at T1                                                          0    
-  12,6G  -
nas/nwserv at T1                                                           0    
-  49,0G  -
nas/rsync_clienti at T1                                                 101K    
-  17,5G  -
nas/rsync_privati at T1                                                    0    
-  36,5K  -
nas/rsync_privati/gianluca at T1                                           0    
-   560M  -
nas/samba at T1                                                            0    
-   486G  -
nas/sviluppo-bak at T1                                                     0    
-  1,60G  -
nas/winsrv at T1                                                           0    
-  55,1G  -

So, now I have a recursive snapshot encompassing all my nas pool and its
filesystems.

Then I issued a:

pfexec /sbin/zfs send -R nas at T1 | ssh biscotto pfexec /sbin/zfs recv -dF
iscsi/nasone

this one from an user account which has auto login on the receiving box; the
copy started ok and kept running for more than an hour, but stopped with this
error:

warning: cannot send ''nas/nwserv at
zfs-auto-snap.frequent-2009-06-08-11.00'': no such pool or dataset
warning: cannot send ''nas/nwserv at
zfs-auto-snap.frequent-2009-06-08-11.15'': no such pool or dataset
warning: cannot send ''nas/nwserv at T1'': incremental source
(@zfs-auto-snap.frequent-2009-06-08-11.15) does not exist

Now, from the error it seems that T1 needs all the snapshots which were active
at the time it was created, which is not what I would expect from a snapshot.

I''ve now stopped the auto-snapshot service and restarted the send, but
I''m wondering if this is the correct and expected behaviour or If
I''m making something wrong or maybe I did not fully understand what
send -R is supposed to do.

Maurilio
-- 
This message posted from opensolaris.org

On Mon, 2009-06-08 at 05:38 -0700, Maurilio Longo wrote:
> Now, from the error it seems that T1 needs all the snapshots which
> were active at the time it was created, which is not what I would
> expect from a snapshot.
>From the man page, -R tries to replicate everything, including any
existing snapshots to the send stream.

What''s probably happened, is that the auto-snapshot service has
dutifully destroyed snapshots that have reached their expiration, and
this has broken -R, which is trying to send those now-non-existent
snapshots.

I bet what you really want is ''zfs send -r fs at snap'' which
would send a
snapshot of all descendant datasets with an ''@snap'' snapshot,
rather
than all snapshots of all descendant datasets up to ''@snap'':
unfortunately that command argument doesn''t exist.

You could easily script this with:

 # zfs snapshot -r mydataset at T1
 # for ds in $(zfs list -t filesystem,volume -o name -r mydataset)
 > do
 > echo sending $ds at T1
 > zfs send $ds at T1 | ssh remote-host zfs recv -d foo
 > done

	cheers,
			tim

Tim,

I really was trying to have a full copy of my pool onto a different pc, so I
think that I have to use -R otherwise I would loose all the history (monthly and
weekly and daily snapshots) of my data which is valuable for me.

That said, I fear that during a send -R the autosnapshot service should be
disabled on the receiving end as well or  it could start to change the incoming
filesystem; am I right?

And if it is so, it means that zfs send/receive cannot be used to really have
backups of a heavily used system; apart from the autosnapshot service a user
could be creating/deleting snapshot breaking the send/receive process.

Anyway thanks a lot for you help!

Maurilio.
-- 
This message posted from opensolaris.org

On Tue, 2009-06-09 at 06:52 -0700, Maurilio Longo wrote:
> I really was trying to have a full copy of my pool onto a different
> pc, so I think that I have to use -R otherwise I would loose all the
> history (monthly and weekly and daily snapshots) of my data which is
> valuable for me.
Fair enough, just disable auto-snapshots while you''re doing the send -R
and you''ll be ok.
> That said, I fear that during a send -R the autosnapshot service
> should be disabled on the receiving end as well or  it could start to
> change the incoming filesystem; am I right?
Yes, I think so.
> And if it is so, it means that zfs send/receive cannot be used to
> really have backups of a heavily used system; apart from the
> autosnapshot service a user could be creating/deleting snapshot
> breaking the send/receive process.
Hm, that would appear to be true - it''d be worth testing it, logging a
bug against "send -R" if that''s the case.

	cheers,
			tim