zfs discuss - Feb 2009 - zfs send over sbmfs results in invalid record type on zfs receive

Hi,

I redirected a zfs send stream to a file on a smbfs mount (Windows share):
zfs send snapshot > /mnt/win/snapshot.zfs

When zfs receiving this file, the process aborts due to an ''invalid
backup stream'' error.   zfs receive -nv gives a little bit more
information:

Assertion failed: !"invalid record type", file
../common/libzfs_sendrecv.c, line 1542, function recv_skip

Is it unsafe to redirect a zfs send stream over the network?  Can I restore the
snapshot in any other way?  I''m using the send and receive commands on
b105.

Thanks,
-- 
This message posted from opensolaris.org

Niels,

The zfs man page has an ssh example.

Regards,

..Remco

Niels Van Hee wrote:
> Hi,
> 
> I redirected a zfs send stream to a file on a smbfs mount (Windows share):
> zfs send snapshot > /mnt/win/snapshot.zfs
> 
> When zfs receiving this file, the process aborts due to an
''invalid backup stream'' error.   zfs receive -nv gives a
little bit more information:
> 
> Assertion failed: !"invalid record type", file
../common/libzfs_sendrecv.c, line 1542, function recv_skip
> 
> Is it unsafe to redirect a zfs send stream over the network?  Can I restore
the snapshot in any other way?  I''m using the send and receive commands
on b105.
> 
> Thanks,

Hi Remco,

isn''t the ssh example used in a context where you immediately receive
the stream (on another solaris system)?  What I''m trying to do is
storing a snapshot on another system or media (tape, dvd,...) and restoring it
later.  I know zfs send and receive are not ideal as a backup solution, but I
would have thought that remotely storing snapshots wouldn''t have been a
problem.

Regards,
Niels
-- 
This message posted from opensolaris.org

Niels,

I do this all the time, to a USB connected external drive. And I have 
managed to restore a snapshot too :-)
I don''t see why you could mount an NFS mount from the service where you
want to backup too. I wonder if windows does something bad to the file, 
is it larger then 2 Gb?

..Remco

Niels Van Hee wrote:
> Hi Remco,
> 
> isn''t the ssh example used in a context where you immediately
receive the stream (on another solaris system)?  What I''m trying to do
is storing a snapshot on another system or media (tape, dvd,...) and restoring
it later.  I know zfs send and receive are not ideal as a backup solution, but I
would have thought that remotely storing snapshots wouldn''t have been a
problem.
> 
> Regards,
> Niels

Hi Remco,

the snapshots are indeed large files (just over 4gb).  I haven''t had
problems copying tarballs over 2gb to this share however.  I''ll tar my
home directory and try to copy that too to find out what happens :-)
-- 
This message posted from opensolaris.org

On Tue, Feb 3, 2009 at 8:00 PM, Niels Van Hee <nvanhee at gmail.com>
wrote:
> Hi Remco,
>
> the snapshots are indeed large files (just over 4gb).
Just wondering, why didn''t you compress it first? something like

zfs send | gzip > backup.zfs.gz

It should save lots of network bandwitdh. Plus, IF it does have
something to do with file too large, compressing should help as a
workaround.

Regards,

Fajar

Hi,
> Just wondering, why didn''t you compress it first?
I''m just trying out the basic zfs features on my home pc.  I
don''t pay a lot of attentention to network bandwidth or compression at
the moment..

And afaik, there''s no issue with large files on OpenSolaris, I can copy
a tarball of my home dir (over 4gb as well) to a windows share and untar it
later without any problem.  Or is a zfs send redirect that different?
-- 
This message posted from opensolaris.org

On Tue, 3 Feb 2009, Niels Van Hee wrote:
>
> the snapshots are indeed large files (just over 4gb).  I haven''t
had
> problems copying tarballs over 2gb to this share however.  I''ll
tar
> my home directory and try to copy that too to find out what happens
Make sure that the unix shell you are using supports large files.

Bob
=====================================Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

On Tue, 3 Feb 2009, Fajar A. Nugraha wrote:
>
> Just wondering, why didn''t you compress it first? something like
>
> zfs send | gzip > backup.zfs.gz
>
> It should save lots of network bandwitdh. Plus, IF it does have
> something to do with file too large, compressing should help as a
> workaround.
The ''lzop'' compressor is much better for this since it is
*much*
faster.

Bob
=====================================Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

On Wed, Feb 4, 2009 at 1:28 AM, Bob Friesenhahn
<bfriesen at simple.dallas.tx.us> wrote:
> On Tue, 3 Feb 2009, Fajar A. Nugraha wrote:
>>
>> Just wondering, why didn''t you compress it first? something
like
>>
>> zfs send | gzip > backup.zfs.gz
>>
>
> The ''lzop'' compressor is much better for this since it is
*much* faster.
Sure, when enabling compression on zfs fs.

I''m talking about compressing the zfs send stream. Last time I test,
zfs send is always uncompressed regardless of fs compression setting.
Thus, compressing it before writing to a file might be beneficial on
certain situation.

Regards,

Fajar

On Wed, Feb 4, 2009 at 10:41 AM, Fajar A. Nugraha <fajar at fajar.net>
wrote:
>>> Just wondering, why didn''t you compress it first?
something like
>>>
>>> zfs send | gzip > backup.zfs.gz
>>>
>>
>> The ''lzop'' compressor is much better for this since
it is *much* faster.
>
> Sure, when enabling compression on zfs fs.
I thought you''re refering to zfs compression algorithm where LZJB
(default) is much faster.
Seems that you''re refereing to lzop to compress the zfs send stream.

Sorry for the confusion.

On Wed, 4 Feb 2009, Fajar A. Nugraha wrote:
> On Wed, Feb 4, 2009 at 1:28 AM, Bob Friesenhahn
> <bfriesen at simple.dallas.tx.us> wrote:
>> On Tue, 3 Feb 2009, Fajar A. Nugraha wrote:
>>>
>>> Just wondering, why didn''t you compress it first?
something like
>>>
>>> zfs send | gzip > backup.zfs.gz
>>>
>>
>> The ''lzop'' compressor is much better for this since
it is *much* faster.
>
> Sure, when enabling compression on zfs fs.
Note that in the user''s command line, the ''gzip'' is
changed to ''lzop''.
It has nothing to do with ZFS.  Of course change the file extention to 
something like .lzo.  If the ''lzop'' utility is not available,
then it
is necessary to install it.

In my testing I found that lzop was *much* faster than gzip and the 
result was maybe 15% less compressed.  It makes a huge difference when 
you are talking about a lot of data.

Bob
=====================================Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/