Btrfs devel - Dec 2013 - btrfs-send-receive vs rsync for (incremental/full) backups

OK... So for backing up across a local network to a second physical host...

Is btrfs-send-receive stable enough now to be used?

How does send-receive compare to using rsync for backups?


Any comments please from those using such things?

Thanks,
Martin

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Fri, Dec 13, 2013 at 07:44:16PM +0000, Martin wrote:
> OK... So for backing up across a local network to a second physical host...
> 
> Is btrfs-send-receive stable enough now to be used?
   It seems to be OK for me at the moment -- I''m not running it across
a network (yet), but I am using it between different filesystems on
the same host, using pipes.
> How does send-receive compare to using rsync for backups?
   It''s more fiddly to get right, and needs a bit more thought about
how to set it up sanely. I have a python library that I use to write
my backup scripts. I''ll have to write it up one of these days...
> Any comments please from those using such things?
   Works for me, so far, as far as I know. Although we''ve got recent
reports of *something* still going wrong with systemd journal files.
No investigation of that yet, though.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- In one respect at least, the Martians are a happy people: ---    
                          they have no lawyers.

On Dec 13, 2013, at 1:35 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
> On Fri, Dec 13, 2013 at 07:44:16PM +0000, Martin wrote:
>> OK... So for backing up across a local network to a second physical
host...
>> 
>> Is btrfs-send-receive stable enough now to be used?
> 
>   It seems to be OK for me at the moment -- I''m not running it
across
> a network (yet), but I am using it between different filesystems on
> the same host, using pipes.
Question. Are the source file system''s checksums (and metadata)
preserved in the send file? Or does the send file just contain data, and
checksums are recomputed on receive? I''m curious if the send file can
be stored on a non-checksumming file system, and yet have a means to verify the
integrity of the files once received. Either the checksums in the send file
being check during receive, or subsequent to receive completion by using scrub?

And yes I can confirm that send outputting to a file on a 2nd drive, and
receiving it to yet another drive does work, and the source and destination
subvolumes are the same in seemingly every way. I didn''t separately
rsync checksum compare the two subvolumes, however. That seems like possibly a
good test. I also haven''t tried doing it over a network.


Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Fri, Dec 13, 2013 at 02:30:26PM -0700, Chris Murphy
wrote:
> 
> On Dec 13, 2013, at 1:35 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
> 
> > On Fri, Dec 13, 2013 at 07:44:16PM +0000, Martin wrote:
> >> OK... So for backing up across a local network to a second
physical host...
> >> 
> >> Is btrfs-send-receive stable enough now to be used?
> > 
> >   It seems to be OK for me at the moment -- I''m not running
it across
> > a network (yet), but I am using it between different filesystems on
> > the same host, using pipes.
> Question. Are the source file system''s checksums (and metadata)
> preserved in the send file?
   I don''t think checksums are transported (from what I remember of
the FAR format).
> Or does the send file just contain data, and checksums are
> recomputed on receive? I''m curious if the send file can be stored
on
> a non-checksumming file system, and yet have a means to verify the
> integrity of the files once received. Either the checksums in the
> send file being check during receive, or subsequent to receive
> completion by using scrub?
   The FAR format contains a sequence of what amount to FS commands:
there are equivalents of cp, rm, chmod, chown, setfattr, cat (of data
inline in the FAR file). There''s also a data type that clones an
extent from a file in another subvolume with a given UUID. Assuming
you have some way of keeping track of subvolume UUIDs on an FS that
doesn''t support them, there''s no reason you can''t
generate or unpack
FAR files on a non-checksumming (and non-CoW) FS.
> And yes I can confirm that send outputting to a file on a 2nd drive,
> and receiving it to yet another drive does work, and the source and
> destination subvolumes are the same in seemingly every way. I
didn''t
> separately rsync checksum compare the two subvolumes, however. That
> seems like possibly a good test. I also haven''t tried doing it
over
> a network.
   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Once is happenstance; twice is coincidence; three times ---     
                            is enemy action.