I have some questions from a customer about zfs checksums. Could anyone answer some of these? Thanks. Brian Customer: I would like to know more about zfs''s checksum feature. I''m guessing it is something that is applied to the data and not the disks (as in raid-5). For performance reasons, I turned off checksum on our zfs filesystem (along with atime updates). Because of a concern for possible data corruption (silent data corruption), I''m interested in turning checksum back on. When I do so, will it create checksums for existing files or will they need to be rewritten? And can you tell me the overhead involved with having checksum active (CPU time, additional space)? -- Brian McBride System Support Engineer Sun Microsystems Cell: 206-851-1028 Email: brian.mcbride at sun.com
On Tue, 1 Jul 2008, Brian McBride wrote:> > Customer: > I would like to know more about zfs''s checksum feature. I''m guessing > it is something that is applied to the data and not the disks (as in > raid-5).Data and metadata.> For performance reasons, I turned off checksum on our zfs filesystem > (along with atime updates). Because of a concern for possible data > corruption (silent data corruption), I''m interested in turning checksum > back on. When I do so, will it create checksums for existing files or > will they need to be rewritten? And can you tell me the overhead > involved with having checksum active (CPU time, additional space)?Turning the checksums off only disables them for user data. They are still enabled for filesystem metadata. I doubt that checksums will be computed for existing files until a block is copied/modified, but perhaps scrub can do that (I don''t know). On modern AMD Opteron hardware it seems that CPU overhead for checksums is very low (e.g. < 5%). I don''t see much value from disabling both atime and checksums in the fileysystem. There is more value to disabling atime updates in NFS mounts. Zfs is pretty lazy about updates so atime just adds slightly to total I/O but without noticeably increasing latency. In a benchmark I did using iozone with 8k I/O blocks in ZFS filesystems with 128K block size, I see that with atime the random witers test results in 834.79 ops/sec but without it increases to 853.56 ops/sec. This is a very small performance improvement. Likewise, with checksums disabled (but atime enabled) I see 839.78 ops/sec. Using 8K I/O blocks in a filesystem with 8K block size resulted in a huge performance difference but unfortunately I failed to record the result. Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Brian McBride wrote:> I have some questions from a customer about zfs checksums. > Could anyone answer some of these? Thanks. > > Brian > > Customer: > I would like to know more about zfs''s checksum feature. I''m guessing > it is something that is applied to the data and not the disks (as in > raid-5). >RAID-5 does not do checksumming. It does a parity calculation, but many RAID-5 implementations do not actually check the parity unless a disk reports an error. ZFS always checks the checksum, unless you disable it. At this point, I usually explain how people find faults in their SAN because ZFS''s checksum works end-to-end.> For performance reasons, I turned off checksum on our zfs filesystem > (along with atime updates). Because of a concern for possible data > corruption (silent data corruption), I''m interested in turning checksum > back on. When I do so, will it create checksums for existing files or > will they need to be rewritten? And can you tell me the overhead > involved with having checksum active (CPU time, additional space)? > >To put this in perspective, in general, the time it takes to read the data from disk is much larger than the time required to calculate the checksum. But, you can also use different checksum algorithms, with varying strength and computational requirements. By default, ZFS uses a Fletcher-2 algorithm, but you can enable Fletcher-4 or SHA-256. If you are planning to characterize the computational cost of checksums, please add these to your test plan and report back to us :-) -- richard