zfs discuss - Feb 2007 - ZFS checksums - block or file level

I am trying to understand if zfs checksums apply at a file or a block level.  We
know that zfs provides end to end checksum integrity, and I assumed that when I
write a file to a zfs filesystem, the checksum was calculated at a file level,
as opposed to say, a block level.  However, I have noticed that when I create an
emulated volume, that volume has a checksum property, set to the same default as
a normal zfs filesystem.  I can even change the checksum value as normal, see
below:

# /usr/sbin/zfs create -V 50GB -b 128KB mypool/myvol

# /usr/sbin/zfs set checksum=sha256 mypool/myvol

Now on this emulated volume, I could place any number of structures that are not
zfs filesystems, say raw database volumes, or ufs, qfs, etc.  Since these do not
perform end to end checksums, can someone explain to me what the zfs checksum
would be doing at this point?
 
 
This message posted from opensolaris.org

On 2/1/07, Nathan Essex <nathan.essex at gmail.com>
wrote:
> I am trying to understand if zfs checksums apply at a file or a block
level.  We know that zfs provides end to end checksum integrity, and I assumed
that when I write a file to a zfs filesystem, the checksum was calculated at a
file level, as opposed to say, a block level.
ZFS checksums are done at the block level. End to end checksum
integrity means that when the actual data reaches the application from
the platter, we can guarantee to a very high certainty that the data
is uncorrupted. Either a block level checksum or a file level checksum
will suffice.
-- 
Regards,
Jeremy

ZFS checksums are at the block level.

Nathan Essex wrote On 02/01/07 08:27,:
> I am trying to understand if zfs checksums apply at a file or a block
level.  We know that zfs provides end to end checksum integrity, and I assumed
that when I write a file to a zfs filesystem, the checksum was calculated at a
file level, as opposed to say, a block level.  However, I have noticed that when
I create an emulated volume, that volume has a checksum property, set to the
same default as a normal zfs filesystem.  I can even change the checksum value
as normal, see below:
> 
> # /usr/sbin/zfs create -V 50GB -b 128KB mypool/myvol
> 
> # /usr/sbin/zfs set checksum=sha256 mypool/myvol
> 
> Now on this emulated volume, I could place any number of structures that
are not zfs filesystems, say raw database volumes, or ufs, qfs, etc.  Since
these do not perform end to end checksums, can someone explain to me what the
zfs checksum would be doing at this point?
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Thank You, so that means that even if I use something that writes raw i/o to a
zfs emulated volume, I still get the checksum protection, and hence data
corruption protection.
 
 
This message posted from opensolaris.org

Nathan Essex wrote:
> Thank You, so that means that even if I use something that writes raw i/o
to a zfs emulated volume, I still get the checksum protection, and hence data
corruption protection.
yes it does.

Also consider how BAD performance could be if it were actually 
calculated on a per file basis.

For example a 1 bit write on to the end of a 5G file would require 
reading and checksuming 5G of data to calculate the new checksum.

Block level is the only sensible way to do this IMO.

-- 
Darren J Moffat

Neil Perrin wrote:
> ZFS checksums are at the block level.
This has been causing some confusion lately, so perhaps we could say:
ZFS checksums are at the file system block level, not to be confused with
the disk block level or transport block level.
  -- richard

Richard Elling wrote:
> Neil Perrin wrote:
>> ZFS checksums are at the block level.
> 
> This has been causing some confusion lately, so perhaps we could say:
> ZFS checksums are at the file system block level, not to be confused with
> the disk block level or transport block level.
Saying that ZFS checksums are at the file system block level is also 
confusing since zvols have checksums too. May be it is better to say 
that ZFS checksums are at the zpool block level because zpool is the 
place where all blocks either from file system or zvol are stored.

Victor

Victor Latushkin wrote:
> 
> Richard Elling wrote:
>> Neil Perrin wrote:
>>> ZFS checksums are at the block level.
>>
>> This has been causing some confusion lately, so perhaps we could say:
>> ZFS checksums are at the file system block level, not to be confused
with
>> the disk block level or transport block level.
> 
> Saying that ZFS checksums are at the file system block level is also 
> confusing since zvols have checksums too. May be it is better to say 
> that ZFS checksums are at the zpool block level because zpool is the 
> place where all blocks either from file system or zvol are stored.
> 
> Victor
zpool block level makes the most sense to me, as a new ZFS user (long 
time list lurker).

--
Dave