zfs discuss - Oct 2010 - When is it okay to turn off the "verify" option.

Folks,

As I understand, the hash generated by sha256 is "almost" guaranteed
not to collide. I am thinking it is okay to turn off "verify" property
on the zpool. However, if there is indeed a collision, we lose data.
"Scrub" cannot recover such lost data.

I am wondering in real life when is it okay to turn off "verify"
option? I guess for storing business critical data (HR, finance, etc.), you
cannot afford to turn this option off.

Thank you in advance for your help.

Regards,
Peter
-- 
This message posted from opensolaris.org

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Peter Taps
> 
> As I understand, the hash generated by sha256 is "almost"
guaranteed
> not to collide. I am thinking it is okay to turn off "verify"
property
> on the zpool. However, if there is indeed a collision, we lose data.
> "Scrub" cannot recover such lost data.
> 
> I am wondering in real life when is it okay to turn off "verify"
> option? I guess for storing business critical data (HR, finance, etc.),
> you cannot afford to turn this option off.
Right on all points.  It''s a calculated risk.  If you have a hash
collision,
you will lose data undetected, and backups won''t save you unless *you*
are
the backup.  That is, if the good data, before it got corrupted by your
system, happens to be saved somewhere else before it reached your system.

Why do you want to turn verify off? If performance is the reason, is it
significant, on and off?

On Oct 4, 2010, at 2:28 PM, Edward Ned Harvey wrote:
>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
>> bounces at opensolaris.org] On Behalf Of Peter Taps
>> 
>> As I understand, the hash generated by sha256 is "almost"
guaranteed
>> not to collide. I am thinking it is okay to turn off "verify"
property
>> on the zpool. However, if there is indeed a collision, we lose data.
>> "Scrub" cannot recover such lost data.
>> 
>> I am wondering in real life when is it okay to turn off
"verify"
>> option? I guess for storing business critical data (HR, finance, etc.),
>> you cannot afford to turn this option off.
> 
> Right on all points.  It''s a calculated risk.  If you have a hash
collision,
> you will lose data undetected, and backups won''t save you unless
*you* are
> the backup.  That is, if the good data, before it got corrupted by your
> system, happens to be saved somewhere else before it reached your system.
> 
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Scott Meilicke

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Scott Meilicke
> 
> Why do you want to turn verify off? If performance is the reason, is it
> significant, on and off?
Under most circumstances, verify won''t hurt performance.  It
won''t hurt
reads of any kind, and it won''t hurt writes when you''re
writing unique data,
or if you''re writing duplicate data which is "warm" in the
read cache.

It will basically hurt write performance if you are writing duplicate data,
which was not read recently.  This might be the case, for example, if this
machine is the target for some remote machine to backup onto.

The problem doesn''t exist if you''re copying local data,
because you first
read the data (now it''s warm in cache) before writing it.  So the
verify
operation is essentially zero time in that case.