zfs code - Apr 2007 - ZFS boot: 3 smaller glitches with console, /etc/dfs/sharetab and /dev/random

Hi,

I''ve now gone through both the opensolaris instructions:

  http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/

and Tim Foster''s script:

  http://blogs.sun.com/timf/entry/zfs_bootable_datasets_happily_rumbling

for making my laptop ZFS bootable.


Both work well and here''s a big THANK YOU to the ZFS boot team!


There seem to be 3 smaller glitches with these approaches:

1. The instructions on opensolaris.org assume that one wants console output
   to show up in /dev/tty. This may be true for a server, but it isn''t
for a
   laptop or workstation user. Therefore, I suggest someone explains them to be
   optional as not everybody knows that these can be left out.

2. After going through the zfs-bootification, Solaris complains on reboot that
   /etc/dfs/sharetab is missing. Somehow this seems to have been fallen through
   the cracks of the find command. Well, touching /etc/dfs/sharetab just fixes
   the issue.

3. But here''s a more serious one: While booting, Solaris complains:

   Apr 19 15:00:37 foeni kcf: [ID 415456 kern.warning] WARNING: No randomness
   provider enabled for /dev/random. Use cryptoadm(1M) to enable a provider.

   Somehow, /dev/random and/or it''s counterpart in /devices seems to
have
   suffered from the migration procedure.

Does anybody know how to fix the /dev/random issue? I''m not very fluent
in
cryptoadm(1M) and some superficial reading of it''s manpage did not
enlighten
me too much (cryptoadm list -p claims all is well...).

Best regards and again, congratulations to the ZFS boot team!

   Constantin


-- 
Constantin Gonzalez                            Sun Microsystems GmbH, Germany
Platform Technology Group, Global Systems Engineering      http://www.sun.de/
Tel.: +49 89/4 60 08-25 91                   http://blogs.sun.com/constantin/

Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering

Constantin Gonzalez Schmitz wrote:
> 2. After going through the zfs-bootification, Solaris complains on reboot
that
>    /etc/dfs/sharetab is missing. Somehow this seems to have been fallen
through
>    the cracks of the find command. Well, touching /etc/dfs/sharetab just
fixes
>    the issue.
sharetab is now an in kernel filesystem.

> 3. But here''s a more serious one: While booting, Solaris
complains:
> 
>    Apr 19 15:00:37 foeni kcf: [ID 415456 kern.warning] WARNING: No
randomness
>    provider enabled for /dev/random. Use cryptoadm(1M) to enable a
provider.
> 
>    Somehow, /dev/random and/or it''s counterpart in /devices seems
to have
>    suffered from the migration procedure.
> 
> Does anybody know how to fix the /dev/random issue? I''m not very
fluent in
> cryptoadm(1M) and some superficial reading of it''s manpage did not
enlighten
> me too much (cryptoadm list -p claims all is well...).
Send the output of the following:

cryptoadm list -p
cat /etc/crypto/kcf.conf
dd if=/dev/random bs=10 count=1 | od -x  (basically is it working now)

Can you also send the messages file so we can see when during boot this 
happened since it might indicate a race condition with the availability 
of /dev/random that only shows up with a ZFS root.


-- 
Darren J Moffat

Hi,
> sharetab is now an in kernel filesystem.
ah, ok, so it falls under the "recreate missed mountpoints by hand"
rule :).
> cryptoadm list -p
# cryptoadm list -p

User-level providers:
====================/usr/lib/security/$ISA/pkcs11_kernel.so: all mechanisms are
enabled.
/usr/lib/security/$ISA/pkcs11_softtoken.so: all mechanisms are enabled. random
is enabled.

Kernel software providers:
=========================des: all mechanisms are enabled.
aes: all mechanisms are enabled.
arcfour: all mechanisms are enabled.
blowfish: all mechanisms are enabled.
sha1: all mechanisms are enabled.
md5: all mechanisms are enabled.
rsa: all mechanisms are enabled.
swrand: random is enabled.
sha2: all mechanisms are enabled.

Kernel hardware providers:
=========================
> cat /etc/crypto/kcf.conf
(Header omitted)

# Start SUNWcsr
des:supportedlist=CKM_DES_CBC,CKM_DES_ECB,CKM_DES3_CBC,CKM_DES3_ECB
aes:supportedlist=CKM_AES_ECB,CKM_AES_CBC
arcfour:supportedlist=CKM_RC4
blowfish:supportedlist=CKM_BLOWFISH_ECB,CKM_BLOWFISH_CBC
sha1:supportedlist=CKM_SHA_1,CKM_SHA_1_HMAC_GENERAL,CKM_SHA_1_HMAC
md5:supportedlist=CKM_MD5,CKM_MD5_HMAC_GENERAL,CKM_MD5_HMAC
rsa:supportedlist=CKM_RSA_PKCS,CKM_RSA_X_509,CKM_MD5_RSA_PKCS,CKM_SHA1_RSA_PKCS
swrand:supportedlist=random
sha2:supportedlist=CKM_SHA256,CKM_SHA256_HMAC,CKM_SHA256_HMAC_GENERAL,CKM_SHA384,CKM_SHA384_HMAC,CKM_SHA384_HMAC_GENERAL,CKM_SHA512,CKM_SHA512_HMAC,CKM_SHA512_HMAC_GENERAL
# End SUNWcsr
# Start SUNWdcar driver_names=dca
# End SUNWdcar
> dd if=/dev/random bs=10 count=1 | od -x  (basically is it working now)
# dd if=/dev/random bs=10 count=1 | od -x
1+0 records in
1+0 records out
0000000 68e0 8673 74df 22f4 638d
0000012

Looks reasonable.
> Can you also send the messages file so we can see when during boot this
> happened since it might indicate a race condition with the availability
> of /dev/random that only shows up with a ZFS root.
Is attached.

Hope this helps,
   Constantin

-- 
Constantin Gonzalez                            Sun Microsystems GmbH, Germany
Platform Technology Group, Global Systems Engineering      http://www.sun.de/
Tel.: +49 89/4 60 08-25 91                   http://blogs.sun.com/constantin/

Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: messages
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070419/524b76f6/attachment.ksh>

Constantin Gonzalez Schmitz wrote:
> 2. After going through the zfs-bootification, Solaris complains on reboot
that
>    /etc/dfs/sharetab is missing. Somehow this seems to have been fallen
through
>    the cracks of the find command. Well, touching /etc/dfs/sharetab just
fixes
>    the issue.
This is unrelated to ZFS boot issues, and sounds like this bug:

6542481 No sharetab after BFU from snv_55

It''s fixed in build 62.

Rob T

Hi,
>> 2. After going through the zfs-bootification, Solaris complains on
>> reboot that
>>    /etc/dfs/sharetab is missing. Somehow this seems to have been
>> fallen through
>>    the cracks of the find command. Well, touching /etc/dfs/sharetab
>> just fixes
>>    the issue.
> 
> This is unrelated to ZFS boot issues, and sounds like this bug:
> 
> 6542481 No sharetab after BFU from snv_55
> 
> It''s fixed in build 62.
hmm, that doesn''t fit what I saw:

- Upgraded from snv_61 to snv_62
- snv_62 booted with not problems (other than the t_optmgmt bug)
- Then migrated to ZFS boot
- Now the sharetab issues shows up.

So why did the sharetab issue only show up after the ZFSification of the
boot process?

Best regards,
    Constantin

-- 
Constantin Gonzalez                            Sun Microsystems GmbH, Germany
Platform Technology Group, Global Systems Engineering      http://www.sun.de/
Tel.: +49 89/4 60 08-25 91                   http://blogs.sun.com/constantin/

Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering

> Constantin Gonzalez Schmitz wrote:
> 
> > 2. After going through the zfs-bootification,
> Solaris complains on reboot that
> >    /etc/dfs/sharetab is missing. Somehow this seems
> to have been fallen through
> >    the cracks of the find command. Well, touching
> /etc/dfs/sharetab just fixes
> >    the issue.
> 
> sharetab is now an in kernel filesystem.
> 
> 
> > 3. But here''s a more serious one: While booting,
> Solaris complains:
> > 
> >    Apr 19 15:00:37 foeni kcf: [ID 415456
> kern.warning] WARNING: No randomness
> >    provider enabled for /dev/random. Use
> cryptoadm(1M) to enable a provider.
> > 
> >    Somehow, /dev/random and/or it''s counterpart in
> /devices seems to have
> >    suffered from the migration procedure.
> > 
> > Does anybody know how to fix the /dev/random issue?
> I''m not very fluent in
> > cryptoadm(1M) and some superficial reading of it''s
> manpage did not enlighten
> > me too much (cryptoadm list -p claims all is
> well...).
> 
> Send the output of the following:
> 
> cryptoadm list -p
> cat /etc/crypto/kcf.conf
> dd if=/dev/random bs=10 count=1 | od -x  (basically
> is it working now)
***************************************************************

# cryptoadm list -p                      

Provider at user level:
====================/usr/lib/security/$ISA/pkcs11_kernel.so: All Mechanisms are
activated.
/usr/lib/security/$ISA/pkcs11_softtoken.so: All Mechanisms are activated. random
is activated.

Kernel Software provider:
=========================des: All Mechanisms are activated..
aes: All Mechanisms are activated..
arcfour: All Mechanisms are activated..
blowfish: All Mechanisms are activated..
sha1: All Mechanisms are activated..
sha2: All Mechanisms are activated..
md4: All Mechanisms are activated..
md5: All Mechanisms are activated..
rsa: All Mechanisms are activated..
swrand: random is activated.

Kernel-Hardware provider:
=========================
****************************************************************

# cat /etc/crypto/kcf.conf
#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
#
# Copyright 2007 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)kcf.conf   1.10    07/04/10 SMI"
#
# /etc/crypto/kcf.conf
#
# Do NOT edit this file by hand.   An administrator should use cryptoadm(1m)
# to administer the cryptographic framework.  A developer for a kernel software
# provider package or a cryptographic provider device driver(s) package should
# provide an input file and use the {i,r}.kcfconf class action scripts to
# update this file during the installation and removal of the package.
#
# This document does not constitute an API.  The /etc/crypto/kcf.conf file may
# not exist or may have a different content or interpretation in a future 
# release.  The existence of this notice does not imply that any other 
# documentation that lacks this notice constitutes an API.
#
#  
#
# Start SUNWcsr
des:supportedlist=CKM_DES_CBC,CKM_DES_ECB,CKM_DES3_CBC,CKM_DES3_ECB
aes:supportedlist=CKM_AES_ECB,CKM_AES_CBC,CKM_AES_CTR
arcfour:supportedlist=CKM_RC4
blowfish:supportedlist=CKM_BLOWFISH_ECB,CKM_BLOWFISH_CBC
sha1:supportedlist=CKM_SHA_1,CKM_SHA_1_HMAC_GENERAL,CKM_SHA_1_HMAC
sha2:supportedlist=CKM_SHA256,CKM_SHA256_HMAC,CKM_SHA256_HMAC_GENERAL,CKM_SHA384,CKM_SHA384_HMAC,CKM_SHA384_HMAC_GENERAL,CKM_SHA512,CKM_SHA512_HMAC,CKM_SHA512_HMAC_GENERAL
md4:supportedlist=CKM_MD4
md5:supportedlist=CKM_MD5,CKM_MD5_HMAC_GENERAL,CKM_MD5_HMAC
rsa:supportedlist=CKM_RSA_PKCS,CKM_RSA_X_509,CKM_MD5_RSA_PKCS,CKM_SHA1_RSA_PKCS,CKM_SHA256_RSA_PKCS,CKM_SHA384_RSA_PKCS,CKM_SHA512_RSA_PKCS
swrand:supportedlist=random
# End SUNWcsr
# Start SUNWdcar driver_names=dca
# End SUNWdcar

******************************************************************************************

# dd if=/dev/random bs=10 count=1 | od -x
1+0 datasets in
1+0 datasets out
0000000 285a 16c1 e017 b6ee 05eb
0000012

*******************************************************************************************
 
 
This message posted from opensolaris.org

Hello

it seems i have the same problem after zfs boot installation (following this
setup on a snv_69 release
http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ). The outputs
from the requested command are similar to the outputs posted by dev2006.

Reading this page, i found no solution concerning the /dev/random problem. Is
there somewhere a procedure to repair my install ?

Thanks in advance
 
 
This message posted from opensolaris.org

forgot to specify some details : 

in my setup i do not install the ufsroot.

i have 2 disks 
-c0d0 for the ufs install 
-c1d0s0 which is my zfs root i want to exploit

my idea is to remove the c0d0 disk when the system will be ok
 
 
This message posted from opensolaris.org

Yannick Robert wrote:
> Hello
> 
> it seems i have the same problem after zfs boot installation (following
this setup on a snv_69 release
http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ). The outputs
from the requested command are similar to the outputs posted by dev2006.
> 
> Reading this page, i found no solution concerning the /dev/random problem.
Is there somewhere a procedure to repair my install ?
I''ve been thinking about the /dev/random problem recently and I think I
know what the problem is, but not yet how to fix it.  I''ve
cc''d the rest
of the crypto team.

With a UFS boot there is no attempted use of randomness until after 
svc://system/cryptosvc has run.  That service does two important things, 
first it starts kcfd to put in place the kernel thread pool for async 
crypto (not relevant to randomness) and secondly it runs ''cryptoadm 
refresh'' which pushes the (private) /etc/crypto/kcf.conf into the
kernel.

When /dev/random was initially integrated it was monolithic, that is the 
randomness pool and the entropy gatherer we combined.  Later on when KCF 
came along we split appart the pool (drv/random) from the software 
entropy provider (crypto/swrand).

Unlike UFS when we do a ZFS boot we do use the in kernel interface to 
/dev/random (random_get_bytes) before svc://system/cryptosvc has run. 
The message you are seeing is from KCF saying that it has a random pool 
but nothing providing entropy to it.  This is because swrand hasn''t yet
registered with kcf.

Now this was all done prior to newboot and SMF and part of the goal of 
why KCF works this way with software providers is was to ensure no boot 
time performance regressions by doing load on demand rather than forcing 
the loading of all modules at boot time.  With newboot on x86, and soon 
on SPARC, the swrand module will be in the boot archive anyway.


-- 
Darren J Moffat

> it seems i have the same problem after zfs boot
> installation (following this setup on a snv_69 release
> http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ).
Hmm, in step 4., wouldn''t it be better to use ufsdump / ufsrestore
instead of find / cpio to clone the ufs root into the zfs root pool?

    cd /zfsroot
    ufsdump 0f - / | ufsrestore -xf -


Advantages:

- it copies the mountpoint for the /etc/dfs/dfstab filesystem
  (and all the other mountpoints, like /tmp, /proc, /etc/mnttab, ...)


- it does not mess up the /lib/libc.so.1 shared library

  I think the procedure at the above url could copy the wrong
  version of the shared libc.so.1 into the zfsroot /lib/libc.so.1;
  this might explain bugs like 6423745,
  Synopsis: zfs root pool created while booted 64 bit can not be booted 32 bit

  
- the files hidden by the /devices mount are copied,too

> The outputs from the requested command
> are similar to the outputs posted by dev2006.
> 
> Reading this page, i found no solution concerning the
> /dev/random problem. Is there somewhere a procedure
> to repair my install ?

AFAICT, there''s nothing you can do to avoid the
"WARNING: No randomness provider enabled for /dev/random."
message with zfs root at this time.  It seems that zfs mountroot
needs some random numbers for mounting the zfs root filesystem,
and at that point early during the bootstrap there isn''t a fully
initialized
random device available.  This fact is remembered by the random
device and is reported later on, when the system is fully booted.

I think when the system is fully booted from zfs root, the random
device should work just fine.
 
 
This message posted from opensolaris.org

> in my setup i do not install the ufsroot.
> 
> i have 2 disks 
> -c0d0 for the ufs install 
> -c1d0s0 which is my zfs root i want to exploit
> 
> my idea is to remove the c0d0 disk when the system will be ok
Btw. if you''re trying to pull the ufs disk c0d0 from the system, and
physically move the zfs root disk from c1d0 -> c0d0 and use that as
the only disk (= boot disk) in the system, you''ll probably run into the
problem that zfs root becomes unbootable, because in the
etc/zfs/zpool.cache file the c1d0 name is still recorded for the
zpool containing the rootfs.

To fix it you probably have to boot a failsafe kernel from somewhere,
zpool import the pool from the disk''s new location, and copy the
updated /etc/zfs/zpool.cache into the zfs root filesystem and build
new boot archives there...
 
 
This message posted from opensolaris.org

Darren J Moffat wrote:
> Yannick Robert wrote:
>   
>> Hello
>>
>> it seems i have the same problem after zfs boot installation (following
this setup on a snv_69 release
http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ). The outputs
from the requested command are similar to the outputs posted by dev2006.
>>
>> Reading this page, i found no solution concerning the /dev/random
problem. Is there somewhere a procedure to repair my install ?
>>     
 To answer Yannick''s question, the /dev/random warning message does not
indicate
 any problem with the install and can be ignored.
> ...
>
> Unlike UFS when we do a ZFS boot we do use the in kernel interface to 
> /dev/random (random_get_bytes) before svc://system/cryptosvc has run.
>   
 To be exact, the API used by ZFS kernel module  is 
random_get_pseudo_bytes().
> The message you are seeing is from KCF saying that it has a random pool 
> but nothing providing entropy to it.  This is because swrand
hasn''t yet
> registered with kcf.
>   
 We had a similar issue with SCTP where in it uses the kernel API
 random_get_pseudo_bytes() before swrand could register.

 The solution we had there was to load swrand directly. From 
uts/sparc/ip/Makefile
    78  #
    79  # Depends on md5 and swrand (for SCTP). SCTP needs to depend on
    80  # swrand as it needs random numbers early on during boot before
    81  # kCF subsystem can load swrand.
    82  #
    83  LDFLAGS         += -dy -Nmisc/md5 -Ncrypto/swrand -Nmisc/hook 
-Nmisc/neti


 I think we can do a similar thing here. The zfs (or is it zfs-root ?) 
kernel module
 can have crypto/swrand as a dependency. I see that uts/sparc/zfs/Makefile
 lists drv/random as a dependency. This is not needed because the
 API is in modstubs now and it is not implemented in drv/random any more.
 That can be replaced with crypto/swrand.

 swrand does  not need any crypto signature verification. So, it can 
safely be loaded
 early on during boot.
> Now this was all done prior to newboot and SMF and part of the goal of 
> why KCF works this way with software providers is was to ensure no boot 
> time performance regressions by doing load on demand rather than forcing 
> the loading of all modules at boot time.
 
Yes. This requirement added a lot of complexity to KCF.
> With newboot on x86, and soon 
> on SPARC, the swrand module will be in the boot archive anyway.
>   
 That would be great. It is cleaner and will remove the need for ad hoc
 solutions like above.

-Krishna