zfs code - Mar 2007 - Codereview ZFS using KCF/libmd for SHA256

This has been on my todo list for quite some time.  With the integration 
of support for the Niagara 2 crypto unit this filtered up to be come 
more important since ideally ZFS should be able to use the processors 
onboard SHA256 when available.

http://cr.grommit.com/~darrenm/zfs-sha256/

Some of you may have seen a previous version of this where the non 
_KERNEL variant used PKCS#11 APIs.  Since we now have libmd in userland 
we don''t need to do that anymore so there won''t be the
performance
regression caused by the PKCS#11 layer over the raw SHA256 API.

What performance & functional testing do I need to do before integration 
of this ?

-- 
Darren J Moffat

Bill Moore wrote:
> For testing, I would recommend running "zloop", a shell script
> (attached) that drives ztest through a bunch of different configs.  Let
> it run for a day or so, and make sure everything is kosher.
> 
> Also, I''d be interested in knowing how these changes help
performance on
> Niagara 2.
Thanks.  Since N2 isn''t released yet I won''t be able to post
the numbers
here but I''ll let you know how it goes.  If for some reason using the 
hardware doesn''t get a noticeable performance improvement (we
don''t go
to hardware for very small data size operations) then I may remove the 
KCF crypto_digest calls and just use SHA256 directly.

-- 
Darren J Moffat

Jeff Bonwick wrote:
> For the benefit of our friends at Apple, BSD, etc -- it would be
> good to keep the super-simple software implementation of SHA-256
> available as an #ifdef.
Sure, I''ll arrange the #ifdef that way.

This is however going to be a big problem for Apple and BSD when we add 
crypto support though because they don''t have the same crypto APIs that
we have (since there is no standard API for in kernel crypto).

-- 
Darren J Moffat

On Wed, Mar 07, 2007 at 10:36:15AM +0000, Darren J Moffat
wrote:
> Jeff Bonwick wrote:
> >For the benefit of our friends at Apple, BSD, etc -- it would be
> >good to keep the super-simple software implementation of SHA-256
> >available as an #ifdef.
> 
> Sure, I''ll arrange the #ifdef that way.
> 
> This is however going to be a big problem for Apple and BSD when we add
crypto support though because they don''t have the same crypto APIs that
we have (since there is no
> standard API for in kernel crypto).
Will it be possible to design ZFS crypto API which will hide internals
of OS-specific in-kernel crypto API? There is opencrypto API in
FreeBSD''s kernel on which I worked quite a bit, so I could help with
design. You can find more info about opencrypto here:

	http://www.freebsd.org/cgi/man.cgi?query=crypto&sektion=9&format=html

The good thing is that we probably need very simple API for ZFS.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-code/attachments/20070308/22281e82/attachment.bin>

Pawel Jakub Dawidek wrote:
> On Wed, Mar 07, 2007 at 10:36:15AM +0000, Darren J Moffat wrote:
>> Jeff Bonwick wrote:
>>> For the benefit of our friends at Apple, BSD, etc -- it would be
>>> good to keep the super-simple software implementation of SHA-256
>>> available as an #ifdef.
>> Sure, I''ll arrange the #ifdef that way.
>>
>> This is however going to be a big problem for Apple and BSD when we add
crypto support though because they don''t have the same crypto APIs that
we have (since there is no
>> standard API for in kernel crypto).
> 
> Will it be possible to design ZFS crypto API which will hide internals
> of OS-specific in-kernel crypto API? There is opencrypto API in
> FreeBSD''s kernel on which I worked quite a bit, so I could help
with
> design. You can find more info about opencrypto here:
My prototype bits has all the actually calls to the OpenSolaris KCF API 
(things like crypto_encrypt()) isolated into a single file.  The calls 
to do the encryption/decryption in places like ZIO are all generic - 
that was done partly for this reason and also because I was mirroring 
what compression/checksum already did anyway.

So yes as much as possible it will be isolated to one place.

We might (not yet decided) make some use of PKCS#11 in userland if we 
need to do stuff there.  Thats an even better choice because it is a 
standards based API.
> 
http://www.freebsd.org/cgi/man.cgi?query=crypto&sektion=9&format=html
> 
> The good thing is that we probably need very simple API for ZFS.
For the actual crypto yes the API is simple, the key management is the 
more complex part - as always :-)

-- 
Darren J Moffat