thr3ads.net - zfs discuss - [zfs-discuss] Cannot mount encrypted filesystems. [Feb 2012]

If this information is useful, please help other people find it:
Share via:

Roberto Waltman

2012-Feb-18 05:12 UTC

[zfs-discuss] Cannot mount encrypted filesystems.

Looking for help regaining access to
encrypted ZFS file systems that
stopped accepting the encryption key.

I have a file server with a setup
as follows:

Solaris 11 Express 1010.11/snv_151a
8 x 2-TB disks, each one divided
into three equal size partitions,
three raidz3 pools built from a
"slice" across matching partitions:


  Disk 1  Disk 8  zpools
  +--+    +--+
  |p1| .. |p1| <- slice_0
  +--+    +--+
  |p2| .. |p2| <- slice_1
  +--+    +--+
  |p3| .. |p3| <- slice_2
  +--+    +--+

zpool status shows:

  ...
  NAME          STATE
  slice_0       ONLINE
    raidz3-0    ONLINE
      c7t0d0s0  ONLINE
      c7t1d0s0  ONLINE
      c7t2d0s0  ONLINE
      c7t3d0s0  ONLINE
      c7t4d0s0  ONLINE
      c7t5d0s0  ONLINE
      c7t6d0s0  ONLINE
      c7t7d0s0  ONLINE
  ...

And several file systems on each pool:
zfs list shows:

  rpool
  ...
  rpool/export
  rpool/export/home
  rpool/export/home/user1
  ...
  slice_0
  slice_0/base
  slice_0/base/fsys_0_1
  ...
  slice_0/base/fsys_0_last
  slice_1
  slice_1/base
  slice_1/base/fsys_1_1
  ...
  slice_1/base/fsys_1_last
  ...
etc.

The intermediate "base" file systems
are there only to set attributes
to be inherited by all other file
systems in the same pool.

They were created with encryption
on, forcing all others to be encrypted.

The keysource for slice_?/base
was set to
   "passphrase,prompt"
while creating the file systems.

Then I stored the keys (one key per
pool) in files in a subdirectory
of home/user1, and set keysource for
slice_0/base to
   "passphrase,file:///export/home/user1/keys/key_0"
(Similarly for the other two pools)

So far so good.
Several weeks and several terabytes
of data later, I decided to relocate
the files with the encryption keys
from a subdir of user1 to a subdir
of root. Copied the files and set
slice_0/base keysource to
   "passphrase,file:///root/keys/key_0", etc.

That broke it. After doing that, the base
file systems (that contain no data files)
can be mounted, but trying to mount any
other fs fails with the message:
"cannot load key for ''slice_?/base/fsys_?_?'': incorrect
key.

Using "zfs set" I can set the keysource
back and forth to the original directory
and the new one, or to prompt, etc.
I can change the "canmount" attribute,
etc., but not actually mount anything.

Tried changing the files attributes
to readable by all or only by owner.
Tried setting the keysource locally for
each fs with no success (other than not
being able to set it back to inherited
from base.)

Any other thing I can do? Most of the
data is either old junk or things I can
rip again or download again, but there
are some files I can not recover from
anywhere else.

Thanks,

--
Roberto Waltman

Darren J Moffat

2012-Feb-20 17:54 UTC

head link

[zfs-discuss] Cannot mount encrypted filesystems.

On 02/18/12 05:12, Roberto Waltman wrote:> Solaris 11 Express 1010.11/snv_151a
I strongly suggest upgrading to Solaris 11 there have been some 
important ZFS and specifically ZFS encryption related bug fixes.
> They were created with encryption
> on, forcing all others to be encrypted.
>
> The keysource for slice_?/base
> was set to
> "passphrase,prompt"
> while creating the file systems.
>
> Then I stored the keys (one key per
> pool) in files in a subdirectory
> of home/user1, and set keysource for
> slice_0/base to
> "passphrase,file:///export/home/user1/keys/key_0"
> (Similarly for the other two pools)
Did you ever export the slice_0 pool and reimport it or reboot the 
server ?  Basically are you and ZFS both 100% sure you had the correct 
passphrases stored in those files ?
> So far so good.
> Several weeks and several terabytes
> of data later, I decided to relocate
> the files with the encryption keys
> from a subdir of user1 to a subdir
> of root. Copied the files and set
> slice_0/base keysource to
> "passphrase,file:///root/keys/key_0", etc.
Exactly how did you do that ?

zfs key -c -o keysource=passphrase,file:///root/keys/key_0

or

zfs set keysource=passphrase,file:///root/keys/key_0

The first does a key change and actually reencryptes the on disk data 
encryption keys using the newly generated AES wrapping key that is 
derived from the passphrase. The second only change where to find the 
passphrase.
> That broke it. After doing that, the base
> file systems (that contain no data files)
> can be mounted, but trying to mount any
> other fs fails with the message:
> "cannot load key for ''slice_?/base/fsys_?_?'':
incorrect key.
Can post some sample output of:

zfs get -r encryption,keysource slice_0

In particular include a few examples of the filesystems you call
''base''
and the fsys ones.

What is important here is understanding where the encryption and 
keysource properties are set and where they are inherited.


-- 
Darren J Moffat

Roberto Waltman

2012-Feb-21 01:58 UTC

head link

[zfs-discuss] Cannot mount encrypted filesystems.

Darren J Moffat wrote:

Thanks for the reply,

 > I strongly suggest upgrading to Solaris 11 there have been some
 > important ZFS and specifically ZFS encryption related bug fixes.

Will do. (At least temporarily, until this
problem is solved. Long term plan is
switching to FreeNAS, even if that means
running an older version of ZFS)

 >> They were created with encryption
 >> on, forcing all others to be encrypted.
 >>
 >> The keysource for slice_?/base
 >> was set to
 >> "passphrase,prompt"
 >> while creating the file systems.
 >>
 >> Then I stored the keys (one key per
 >> pool) in files in a subdirectory
 >> of home/user1, and set keysource for
 >> slice_0/base to
 >> "passphrase,file:///export/home/user1/keys/key_0"
 >> (Similarly for the other two pools)
 >
 > Did you ever export the slice_0 pool and reimport it or reboot the
 > server ?  Basically are you and ZFS both 100% sure you had the correct
 > passphrases stored in those files ?

The system was rebooted many times
during a period of weeks,without
any problems.
(This is a home file server,
powered up on only on evenings and
weekends, when I am at home.)

The trouble began when I tried to
change the passphrase location as
described below.

 >> So far so good.
 >> Several weeks and several terabytes
 >> of data later, I decided to relocate
 >> the files with the encryption keys
 >> from a subdir of user1 to a subdir
 >> of root. Copied the files and set
 >> slice_0/base keysource to
 >> "passphrase,file:///root/keys/key_0", etc.
 >

 > Exactly how did you do that ?
 >
 > zfs key -c -o keysource=passphrase,file:///root/keys/key_0
 >
 > or
 >
 > zfs set keysource=passphrase,file:///root/keys/key_0
 >
 > The first does a key change and actually reencryptes the on disk data
 > encryption keys using the newly generated AES wrapping key that is
 > derived from the passphrase. The second only change where to find the
 > passphrase.

First, I did the 2nd. (Change location only)
I believe I tried the first form also *after*
things were already broken, but I''m sure the
passphrases were identical: slice_08, slice_18
and slice_28 for each pools 0/1/2. - The ''8''
to bring the length to the minimal
requirement of 8 characters.
( My goal for using encryption was just to
obfuscate the contents if, for example, I
send a disk out for repair; not to hide
anything from the NSA )

Question: I believed the keys generated from a
passphrase depend only on the passphrase, and
not on  how it is provided or where it is stored.
Is this a true statement?

 >> That broke it. After doing that, the base
 >> file systems (that contain no data files)
 >> can be mounted, but trying to mount any
 >> other fs fails with the message:
 >> "cannot load key for ''slice_?/base/fsys_?_?'':
incorrect key.
 >
 > Can post some sample output of:
 >
 > zfs get -r encryption,keysource slice_0

Actual commands output for slice_2/... pasted at the end.
The keysource was originally, (in the still working
system,) inherited from slice_2/base. It is now set
locally. At this moment some filesystems use "prompt",
others use "file", neither works.

I did try setting the keysource to "prompt,
to "file" at the original location [ actual path:
file:///export/home/trouser/passphrases/slice_2_passphrase ]
to "file" at the new location [ actual path:
file:///root/passphrases/slice_2_passphrase ]
for all file systems. It always failed.

 > In particular include a few examples of the filesystems you call
''base''
 > and the fsys ones.
 >
 > What is important here is understanding where the encryption and
 > keysource properties are set and where they are inherited.



==========================================  zfs get -r encryption,keysource
slice_2
==========================================
NAME                          PROPERTY    VALUE 
                                          SOURCE
slice_2                       encryption  on 
                                          local
slice_2                       keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base                  encryption  on 
                                          local
slice_2/base                  keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/bitsavers        encryption  on 
                                          inherited from slice_2/base
slice_2/base/bitsavers        keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/codesourcery     encryption  on 
                                          inherited from slice_2/base
slice_2/base/codesourcery     keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/embedded         encryption  on 
                                          inherited from slice_2/base
slice_2/base/embedded         keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/flightgear       encryption  on 
                                          inherited from slice_2/base
slice_2/base/flightgear       keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/forth            encryption  on 
                                          inherited from slice_2/base
slice_2/base/forth            keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/functional       encryption  on 
                                          inherited from slice_2/base
slice_2/base/functional       keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/linux            encryption  on 
                                          inherited from slice_2/base
slice_2/base/linux            keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/minix            encryption  on 
                                          inherited from slice_2/base
slice_2/base/minix            keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/modula_n_oberon  encryption  on 
                                          inherited from slice_2/base
slice_2/base/modula_n_oberon  keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/trimslice        encryption  on 
                                          inherited from slice_2/base
slice_2/base/trimslice        keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/ubuntu           encryption  on 
                                          inherited from slice_2/base
slice_2/base/ubuntu           keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/unclassified_2   encryption  on 
                                          inherited from slice_2/base
slice_2/base/unclassified_2   keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/x-plane          encryption  on 
                                          inherited from slice_2/base
slice_2/base/x-plane          keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local

=======================================  zfs get all slice_2
=======================================
NAME     PROPERTY              VALUE 
                               SOURCE
slice_2  type                  filesystem 
                               -
slice_2  creation              Sun Aug 14  0:16 2011 
                               -
slice_2  used                  136G 
                               -
slice_2  available             2.83T 
                               -
slice_2  referenced            74.5K 
                               -
slice_2  compressratio         1.00x 
                               -
slice_2  mounted               yes 
                               -
slice_2  quota                 none 
                               default
slice_2  reservation           none 
                               default
slice_2  recordsize            128K 
                               default
slice_2  mountpoint            /slices/slice_2 
                               local
slice_2  sharenfs              off 
                               local
slice_2  checksum              sha256-mac 
                               local
slice_2  compression           off 
                               local
slice_2  atime                 off 
                               local
slice_2  devices               off 
                               local
slice_2  exec                  on 
                               default
slice_2  setuid                on 
                               default
slice_2  readonly              off 
                               default
slice_2  zoned                 off 
                               default
slice_2  snapdir               hidden 
                               default
slice_2  aclinherit            restricted 
                               default
slice_2  canmount              on 
                               default
slice_2  xattr                 on 
                               default
slice_2  copies                1 
                               default
slice_2  version               5 
                               -
slice_2  utf8only              on 
                               -
slice_2  normalization         none 
                               -
slice_2  casesensitivity       sensitive 
                               -
slice_2  vscan                 off 
                               default
slice_2  nbmand                off 
                               default
slice_2  sharesmb              off 
                               default
slice_2  refquota              none 
                               default
slice_2  refreservation        none 
                               default
slice_2  primarycache          all 
                               default
slice_2  secondarycache        all 
                               default
slice_2  usedbysnapshots       0 
                               -
slice_2  usedbydataset         74.5K 
                               -
slice_2  usedbychildren        136G 
                               -
slice_2  usedbyrefreservation  0 
                               -
slice_2  logbias               latency 
                               default
slice_2  dedup                 off 
                               default
slice_2  mlslabel              none 
                               default
slice_2  sync                  standard 
                               default
slice_2  encryption            on 
                               local
slice_2  keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2  keystatus             available 
                               -
slice_2  rekeydate             Sun Aug 14  0:16 2011 
                               local
slice_2  rstchown              on 
                               default

=======================================  zfs get all slice_2/base
=======================================
NAME          PROPERTY              VALUE 
                                    SOURCE
slice_2/base  type                  filesystem 
                                    -
slice_2/base  creation              Sun Aug 14  0:16 2011 
                                    -
slice_2/base  used                  136G 
                                    -
slice_2/base  available             2.83T 
                                    -
slice_2/base  referenced            77.0K 
                                    -
slice_2/base  compressratio         1.00x 
                                    -
slice_2/base  mounted               yes 
                                    -
slice_2/base  quota                 none 
                                    default
slice_2/base  reservation           none 
                                    default
slice_2/base  recordsize            128K 
                                    default
slice_2/base  mountpoint            /slices/slice_2/base 
                                    local
slice_2/base  sharenfs              off 
                                    local
slice_2/base  checksum              sha256-mac 
                                    local
slice_2/base  compression           off 
                                    local
slice_2/base  atime                 off 
                                    local
slice_2/base  devices               off 
                                    local
slice_2/base  exec                  on 
                                    default
slice_2/base  setuid                on 
                                    default
slice_2/base  readonly              off 
                                    default
slice_2/base  zoned                 off 
                                    default
slice_2/base  snapdir               hidden 
                                    default
slice_2/base  aclinherit            restricted 
                                    default
slice_2/base  canmount              on 
                                    default
slice_2/base  xattr                 on 
                                    default
slice_2/base  copies                1 
                                    default
slice_2/base  version               5 
                                    -
slice_2/base  utf8only              on 
                                    -
slice_2/base  normalization         none 
                                    -
slice_2/base  casesensitivity       sensitive 
                                    -
slice_2/base  vscan                 off 
                                    default
slice_2/base  nbmand                off 
                                    default
slice_2/base  sharesmb              off 
                                    local
slice_2/base  refquota              none 
                                    default
slice_2/base  refreservation        none 
                                    default
slice_2/base  primarycache          all 
                                    default
slice_2/base  secondarycache        all 
                                    default
slice_2/base  usedbysnapshots       0 
                                    -
slice_2/base  usedbydataset         77.0K 
                                    -
slice_2/base  usedbychildren        136G 
                                    -
slice_2/base  usedbyrefreservation  0 
                                    -
slice_2/base  logbias               latency 
                                    default
slice_2/base  dedup                 off 
                                    default
slice_2/base  mlslabel              none 
                                    default
slice_2/base  sync                  standard 
                                    default
slice_2/base  encryption            on 
                                    local
slice_2/base  keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base  keystatus             available 
                                    -
slice_2/base  rekeydate             Sun Aug 14  0:16 2011 
                                    local
slice_2/base  rstchown              on 
                                    default

=======================================  zfs get all slice_2/base/bitsavers
=======================================
NAME                    PROPERTY              VALUE 
                                              SOURCE
slice_2/base/bitsavers  type                  filesystem 
                                              -
slice_2/base/bitsavers  creation              Sun Aug 14  0:16 2011 
                                              -
slice_2/base/bitsavers  used                  79.5K 
                                              -
slice_2/base/bitsavers  available             2.83T 
                                              -
slice_2/base/bitsavers  referenced            79.5K 
                                              -
slice_2/base/bitsavers  compressratio         1.00x 
                                              -
slice_2/base/bitsavers  mounted               no 
                                              -
slice_2/base/bitsavers  quota                 none 
                                              default
slice_2/base/bitsavers  reservation           none 
                                              default
slice_2/base/bitsavers  recordsize            128K 
                                              default
slice_2/base/bitsavers  mountpoint            /global/bitsavers/ 
                                              local
slice_2/base/bitsavers  sharenfs              off 
                                              local
slice_2/base/bitsavers  checksum              sha256-mac 
                                              inherited from slice_2/base
slice_2/base/bitsavers  compression           off 
                                              inherited from slice_2/base
slice_2/base/bitsavers  atime                 off 
                                              inherited from slice_2/base
slice_2/base/bitsavers  devices               off 
                                              inherited from slice_2/base
slice_2/base/bitsavers  exec                  on 
                                              default
slice_2/base/bitsavers  setuid                on 
                                              default
slice_2/base/bitsavers  readonly              off 
                                              default
slice_2/base/bitsavers  zoned                 off 
                                              default
slice_2/base/bitsavers  snapdir               hidden 
                                              default
slice_2/base/bitsavers  aclinherit            restricted 
                                              default
slice_2/base/bitsavers  canmount              on 
                                              default
slice_2/base/bitsavers  xattr                 on 
                                              default
slice_2/base/bitsavers  copies                1 
                                              default
slice_2/base/bitsavers  vscan                 off 
                                              default
slice_2/base/bitsavers  nbmand                off 
                                              default
slice_2/base/bitsavers  sharesmb              name=bitsavers 
                                              local
slice_2/base/bitsavers  refquota              none 
                                              default
slice_2/base/bitsavers  refreservation        none 
                                              default
slice_2/base/bitsavers  primarycache          all 
                                              default
slice_2/base/bitsavers  secondarycache        all 
                                              default
slice_2/base/bitsavers  usedbysnapshots       0 
                                              -
slice_2/base/bitsavers  usedbydataset         79.5K 
                                              -
slice_2/base/bitsavers  usedbychildren        0 
                                              -
slice_2/base/bitsavers  usedbyrefreservation  0 
                                              -
slice_2/base/bitsavers  logbias               latency 
                                              default
slice_2/base/bitsavers  dedup                 off 
                                              default
slice_2/base/bitsavers  mlslabel              none 
                                              default
slice_2/base/bitsavers  sync                  standard 
                                              default
slice_2/base/bitsavers  encryption            on 
                                              inherited from slice_2/base
slice_2/base/bitsavers  keysource 
passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase  local
slice_2/base/bitsavers  keystatus             unavailable 
                                              -
slice_2/base/bitsavers  rekeydate             Sun Aug 14  0:16 2011 
                                              local
slice_2/base/bitsavers  rstchown              on 
                                              default


=======================================  trying to mount
=======================================
root at turbofan:~# cat /export/home/trouser/passphrases/slice_2_passphrase
slice_28
root at turbofan:~# zfs mount slice_2/base/bitsavers
cannot load key for ''slice_2/base/bitsavers'': incorrect key.
root at turbofan:~# zfs set keysource=passphrase,prompt slice_2/base/bitsavers
root at turbofan:~# zfs mount slice_2/base/bitsavers
Enter passphrase for ''slice_2/base/bitsavers'': [ entering
slice_28 ]
cannot load key for ''slice_2/base/bitsavers'': incorrect key.
root at turbofan:~#

Darren J Moffat

2012-Feb-21 11:12 UTC

head link

[zfs-discuss] Cannot mount encrypted filesystems.

On 02/21/12 01:58, Roberto Waltman wrote:> First, I did the 2nd. (Change location only)
> I believe I tried the first form also *after*
> things were already broken, but I''m sure the
> passphrases were identical: slice_08, slice_18
> and slice_28 for each pools 0/1/2. - The ''8''
> to bring the length to the minimal
> requirement of 8 characters.
A ''zfs key -c'' won''t work unless a ''zfs key
-l'' or ''zfs mount'' has
successfully loaded the key first.

Can you send the ''zpool history slice_2'' output so I can see
what
commands have been run.
> ( My goal for using encryption was just to
> obfuscate the contents if, for example, I
> send a disk out for repair; not to hide
> anything from the NSA )
>
> Question: I believed the keys generated from a
> passphrase depend only on the passphrase, and
> not on how it is provided or where it is stored.
> Is this a true statement?
Almost, the passphrase case also depends on a hidden property called 
"salt" that is updated only when you do ''zfs key -c''
and was set to a
random value at the time the dataset was created.

Did you ever do a send|recv of these filesystems ?  There was a bug with 
send|recv in 151a that has since been fixed that could cause the salt to 
be zero''d out in some cases.
> slice_2/base/bitsavers keysource
> passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase local
This is the interesting part you have set the keysource explicitly on 
every leaf dataset - you didn''t need to do that it would have been 
inherited.

What this means is that even though you have the same passphrase for 
each dataset the actual data encryption key is different because the 
passphrase value plus the hidden salt property are used together to 
generated the wrapping key.

-- 
Darren J Moffat

Daniel Carosone

2012-Feb-21 23:33 UTC

head link

[zfs-discuss] Cannot mount encrypted filesystems.

On Tue, Feb 21, 2012 at 11:12:14AM +0000, Darren J Moffat
wrote:> Did you ever do a send|recv of these filesystems ?  There was a bug with  
> send|recv in 151a that has since been fixed that could cause the salt to  
> be zero''d out in some cases.
Ah, so that''s what that was.

I hit this problem some time ago, as was discussed here.
Unfortunately, I also wrote more data into the recv''d filesystem
before the next reboot, and only after that did the new fs become
eunmountable.

So, now that the bug is understood and fixed, if i still have the
original dataset (source of the send), can I use that to recover the
salt and thus the keys and new data? 

--
Dan.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20120222/74cebd05/attachment.bin>

Roberto Waltman

2012-Feb-22 06:10 UTC

head link

[zfs-discuss] Cannot mount encrypted filesystems.

Darren J Moffat wrote:>... 
> Can you send the ''zpool history slice_2'' output so I can
see what
> commands have been run.
> Pasted at the end - My recollection of the events was
wrong, I started with "keysource=passphrase,file..."
not prompt. (Used prompt in earlier experiments and
mixed things up)

  > Did you ever do a send|recv of these filesystems ?  There was a bug 
with> send|recv in 151a that has since been fixed that could cause the salt to 
> be zero''d out in some cases.
No.
> This is the interesting part you have set the keysource explicitly on 
> every leaf dataset - you didn''t need to do that it would have been
> inherited.
> 
> What this means is that even though you have the same passphrase for 
> each dataset the actual data encryption key is different because the 
> passphrase value plus the hidden salt property are used together to 
> generated the wrapping key.
Are you saying "Abandon all hope?"


#################################
    slice_2 history
#################################

##### Notes:
#####
##### (a) Some groups of commands are repeated because
#####     I was (re)running them from a shell script.
#####
##### (b) A couple filesystems were created, destroyed, then
#####     recreated. I copied some data later I did not want
#####     there, and/or was  checking how long the operation
#####     took.
#####
##### (c) I deleted a bunch (100+) of set sharesmb and set
#####     sharenfs commands that just added clutter
#####
##### (d) Have a symbolic link /home pointing to /export/home
#####

History for ''slice_2'':
2011-08-14.00:16:41 zpool create -O atime=off -O compression=off -O 
devices=off -O encryption=on -O 
keysource=passphrase,file:///home/trouser/slice_2_passphrase -O 
mountpoint=none -O utf8only=on slice_2 raidz3 c7t0d0s2 c7t1d0s2 c7t2d0s2 
c7t3d0s2 c7t4d0s2 c7t5d0s2 c7t6d0s2 c7t7d0s2
2011-08-14.00:16:41 zfs create -o atime=off -o compression=off -o 
devices=off -o encryption=on -o 
keysource=passphrase,file:///home/trouser/slice_2_passphrase -o 
mountpoint=none -o utf8only=on slice_2/base
2011-08-14.00:16:43 zfs create -o mountpoint=/global/codesourcery 
slice_2/base/codesourcery
2011-08-14.00:16:44 zfs create -o mountpoint=/global/x-plane 
slice_2/base/x-plane
2011-08-14.00:16:46 zfs create -o mountpoint=/global/modula_n_oberon 
slice_2/base/modula_n_oberon
2011-08-14.00:16:47 zfs create -o mountpoint=/global/filer_2 
slice_2/base/filer_2
2011-08-14.00:16:48 zfs create -o mountpoint=/global/forth 
slice_2/base/forth
2011-08-14.00:16:50 zfs create -o mountpoint=/global/functional 
slice_2/base/functional
2011-08-14.00:16:51 zfs create -o mountpoint=/global/bitsavers 
slice_2/base/bitsavers
2011-08-14.00:16:56 zfs create -o mountpoint=/global/flightgear 
slice_2/base/flightgear
2011-08-14.11:54:31 zfs set mountpoint=/slices/slice_2/base slice_2/base
2011-08-14.11:56:51 zfs mount slice_2/base/bitsavers
2011-08-14.11:56:51 zfs mount slice_2/base/codesourcery
2011-08-14.11:56:51 zfs mount slice_2/base/filer_2
2011-08-14.11:56:51 zfs mount slice_2/base/flightgear
2011-08-14.11:56:51 zfs mount slice_2/base/forth
2011-08-14.11:56:51 zfs mount slice_2/base/functional
2011-08-14.11:56:51 zfs mount slice_2/base/modula_n_oberon
2011-08-14.11:56:51 zfs mount slice_2/base/x-plane
2011-08-16.20:41:11 zfs mount slice_2/base
2011-08-16.20:41:11 zfs mount slice_2/base/bitsavers
2011-08-16.20:41:11 zfs mount slice_2/base/codesourcery
2011-08-16.20:41:11 zfs mount slice_2/base/filer_2
2011-08-16.20:41:11 zfs mount slice_2/base/flightgear
2011-08-16.20:41:11 zfs mount slice_2/base/forth
2011-08-16.20:41:11 zfs mount slice_2/base/functional
2011-08-16.20:41:11 zfs mount slice_2/base/modula_n_oberon
2011-08-16.20:41:11 zfs mount slice_2/base/x-plane
2011-08-16.21:47:37 zfs create -o mountpoint=/global/embedded 
slice_2/base/embedded
2011-08-18.00:00:53 zfs mount slice_2/base
2011-08-18.00:00:53 zfs mount slice_2/base/bitsavers
2011-08-18.00:00:58 zfs mount slice_2/base/codesourcery
2011-08-18.00:00:58 zfs mount slice_2/base/embedded
2011-08-18.00:00:58 zfs mount slice_2/base/filer_2
2011-08-18.00:00:58 zfs mount slice_2/base/flightgear
2011-08-18.00:00:58 zfs mount slice_2/base/forth
2011-08-18.00:00:58 zfs mount slice_2/base/functional
2011-08-18.00:00:58 zfs mount slice_2/base/modula_n_oberon
2011-08-18.00:00:58 zfs mount slice_2/base/x-plane
2011-08-18.21:21:27 zfs mount slice_2/base
2011-08-18.21:21:27 zfs mount slice_2/base/bitsavers
2011-08-18.21:21:27 zfs mount slice_2/base/codesourcery
2011-08-18.21:21:27 zfs mount slice_2/base/embedded
2011-08-18.21:21:27 zfs mount slice_2/base/filer_2
2011-08-18.21:21:27 zfs mount slice_2/base/flightgear
2011-08-18.21:21:27 zfs mount slice_2/base/forth
2011-08-18.21:21:27 zfs mount slice_2/base/functional
2011-08-18.21:21:27 zfs mount slice_2/base/modula_n_oberon
2011-08-18.21:21:27 zfs mount slice_2/base/x-plane
2011-08-19.22:06:33 zfs mount slice_2/base
2011-08-19.22:06:33 zfs mount slice_2/base/bitsavers
2011-08-19.22:06:33 zfs mount slice_2/base/codesourcery
2011-08-19.22:06:33 zfs mount slice_2/base/embedded
2011-08-19.22:06:33 zfs mount slice_2/base/filer_2
2011-08-19.22:06:33 zfs mount slice_2/base/flightgear
2011-08-19.22:06:33 zfs mount slice_2/base/forth
2011-08-19.22:06:33 zfs mount slice_2/base/functional
2011-08-19.22:06:33 zfs mount slice_2/base/modula_n_oberon
2011-08-19.22:06:33 zfs mount slice_2/base/x-plane
2011-08-19.23:52:32 zfs destroy slice_2/base/filer_2
2011-08-20.13:59:26 zfs mount slice_2/base
2011-08-20.13:59:26 zfs mount slice_2/base/bitsavers
2011-08-20.13:59:26 zfs mount slice_2/base/codesourcery
2011-08-20.13:59:26 zfs mount slice_2/base/embedded
2011-08-20.13:59:26 zfs mount slice_2/base/flightgear
2011-08-20.13:59:26 zfs mount slice_2/base/forth
2011-08-20.13:59:26 zfs mount slice_2/base/functional
2011-08-20.13:59:26 zfs mount slice_2/base/modula_n_oberon
2011-08-20.13:59:26 zfs mount slice_2/base/x-plane
2011-08-20.14:17:08 zfs create -o mountpoint=/global/minix 
slice_2/base/minix
2011-08-20.15:30:28 zfs create -o mountpoint=/global/unclassified_2 
slice_2/base/unclassified_2
2011-08-20.15:37:47 zfs create -o mountpoint=/global/trimslice 
slice_2/base/trimslice
2011-08-20.15:39:01 zfs destroy slice_2/base/trimslice
2011-08-20.15:39:12 zfs create -o mountpoint=/global/trimslice 
slice_2/base/trimslice
2011-08-20.20:24:44 zfs create -o mountpoint=/global/linux 
slice_2/base/linux
2011-08-20.20:25:01 zfs create -o mountpoint=/global/ubuntu 
slice_2/base/ubuntu

##### everything works ok up to this point, the following commands break it.

2011-08-23.23:48:35 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base
2011-08-23.23:48:35 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-23.23:48:35 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/codesourcery
2011-08-23.23:48:35 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/embedded
2011-08-23.23:48:36 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/flightgear
2011-08-23.23:48:36 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/forth
2011-08-23.23:48:36 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/functional
2011-08-23.23:48:36 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/linux
2011-08-23.23:48:37 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/minix
2011-08-23.23:48:37 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/modula_n_oberon
2011-08-23.23:48:37 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/trimslice
2011-08-23.23:48:37 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/ubuntu
2011-08-23.23:48:37 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/unclassified_2
2011-08-23.23:48:42 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/x-plane

##### Things broken already. got the first "incorrect key" messages - 
running the shell script again.
##### Don''t know why I don''t see the
''slice_2'' by itself in the previous
group

2011-08-23.23:50:04 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase slice_2
2011-08-23.23:50:04 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base
2011-08-23.23:50:04 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-23.23:50:04 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/codesourcery
2011-08-23.23:50:04 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/embedded
2011-08-23.23:50:05 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/flightgear
2011-08-23.23:50:05 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/forth
2011-08-23.23:50:05 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/functional
2011-08-23.23:50:05 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/linux
2011-08-23.23:50:05 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/minix
2011-08-23.23:50:06 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/modula_n_oberon
2011-08-23.23:50:06 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/trimslice
2011-08-23.23:50:06 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/ubuntu
2011-08-23.23:50:06 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/unclassified_2
2011-08-23.23:50:11 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/x-plane

#### slice_2/base can be mounted, all other filesystems cannot.
#### the following commands are futile attempts to get things back in 
working order,
####    all respond with "cannot load key... incorrect key"
####    (except, again, slice_2/base )

2011-08-23.23:50:51 zfs mount slice_2/base
2011-08-23.23:52:31 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase slice_2
2011-08-23.23:52:31 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base
2011-08-23.23:52:31 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-23.23:52:32 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/codesourcery
2011-08-23.23:52:32 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/embedded
2011-08-23.23:52:32 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/flightgear
2011-08-23.23:52:32 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/forth
2011-08-23.23:52:32 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/functional
2011-08-23.23:52:32 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/linux
2011-08-23.23:52:33 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/minix
2011-08-23.23:52:33 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/modula_n_oberon
2011-08-23.23:52:33 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/trimslice
2011-08-23.23:52:33 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/ubuntu
2011-08-23.23:52:33 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/unclassified_2
2011-08-23.23:52:38 zfs set 
keysource=passphrase,file:///root/passphrases/slice_2_passphrase 
slice_2/base/x-plane
2011-08-24.00:05:00 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2
2011-08-24.00:05:00 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base
2011-08-24.00:05:00 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-24.00:05:00 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/codesourcery
2011-08-24.00:05:00 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/embedded
2011-08-24.00:05:00 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/flightgear
2011-08-24.00:05:01 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/forth
2011-08-24.00:05:01 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/functional
2011-08-24.00:05:01 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/linux
2011-08-24.00:05:01 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/minix
2011-08-24.00:05:01 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/modula_n_oberon
2011-08-24.00:05:01 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/trimslice
2011-08-24.00:05:02 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/ubuntu
2011-08-24.00:05:02 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/unclassified_2
2011-08-24.00:05:07 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/x-plane
2011-08-24.22:31:49 zfs mount slice_2/base
2011-08-24.22:41:47 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-24.22:42:03 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_1_passphrase 
slice_2/base/bitsavers
2011-08-24.22:42:14 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_0_passphrase 
slice_2/base/bitsavers
2011-08-24.22:42:23 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-24.22:44:02 zfs set keysource=passphrase,prompt 
slice_2/base/bitsavers
2011-08-24.22:49:01 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2
2011-08-24.22:49:02 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base
2011-08-24.22:49:02 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-24.22:49:02 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/codesourcery
2011-08-24.22:49:02 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/embedded
2011-08-24.22:49:02 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/flightgear
2011-08-24.22:49:03 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/forth
2011-08-24.22:49:03 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/functional
2011-08-24.22:49:03 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/linux
2011-08-24.22:49:03 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/minix
2011-08-24.22:49:03 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/modula_n_oberon
2011-08-24.22:49:03 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/trimslice
2011-08-24.22:49:04 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/ubuntu
2011-08-24.22:49:04 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/unclassified_2
2011-08-24.22:49:09 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/x-plane
2011-08-24.22:49:53 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2
2011-08-24.22:49:54 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base
2011-08-24.22:49:54 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-24.22:49:54 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/codesourcery
2011-08-24.22:49:54 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/embedded
2011-08-24.22:49:54 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/flightgear
2011-08-24.22:49:54 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/forth
2011-08-24.22:49:55 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/functional
2011-08-24.22:49:55 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/linux
2011-08-24.22:49:55 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/minix
2011-08-24.22:49:55 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/modula_n_oberon
2011-08-24.22:49:55 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/trimslice
2011-08-24.22:49:55 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/ubuntu
2011-08-24.22:49:56 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/unclassified_2
2011-08-24.22:50:01 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/x-plane
2011-08-24.22:51:28 zfs set mountpoint=/slices/slice_2 slice_2
2011-08-24.22:52:30 zfs set mountpoint=/slices/slice_2 slice_2
2011-08-24.22:52:42 zfs set mountpoint=/slices/slice_2/base slice_2/base
2011-08-24.22:53:44 zfs set mountpoint=/global/bitsavers/ 
slice_2/base/bitsavers
2011-08-24.22:54:54 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/bitsavers
2011-08-24.22:55:02 zfs set mountpoint=/global/bitsavers/ 
slice_2/base/bitsavers
2011-09-05.21:41:36 zfs set keysource=passphrase,prompt 
slice_2/base/trimslice
2011-09-05.21:42:17 zfs set mountpoint=/global/trimslice/ 
slice_2/base/trimslice
2011-09-05.21:42:33 zfs set mountpoint=/global/trimslice/ 
slice_2/base/trimslice
2011-09-05.21:42:40 zfs set mountpoint=/global/trimslice/ 
slice_2/base/trimslice
2011-09-05.21:43:18 zfs set mountpoint=/global/trimslice/ 
slice_2/base/trimslice
2011-09-05.21:44:16 zfs set 
keysource=passphrase,file:///home/trouser/passphrases/slice_2_passphrase 
slice_2/base/trimslice
2012-02-05.22:09:06 zpool export slice_2
2012-02-20.20:01:28 zpool import slice_2
2012-02-20.20:35:11 zfs set 
keysource=passphrase,file:///export/home/trouser/passphrases/slice_2_passphrase 
slice_2/base/trimslice
2012-02-20.20:42:41 zfs set keysource=passphrase,prompt 
slice_2/base/bitsavers

#################################
    slice_2 status
    I just noticed the following:
#################################

root at turbofan:~#
root at turbofan:~# zpool status slice_2
   pool: slice_2
  state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
    see: http://www.sun.com/msg/ZFS-8000-8A
  scan: none requested
config:

	NAME          STATE     READ WRITE CKSUM
	slice_2       ONLINE      20     0     0
	  raidz3-0    ONLINE       0     0     0
	    c7t0d0s2  ONLINE       0     0     0
	    c7t1d0s2  ONLINE       0     0     0
	    c7t2d0s2  ONLINE       0     0     0
	    c7t3d0s2  ONLINE       0     0     0
	    c7t4d0s2  ONLINE       0     0     0
	    c7t5d0s2  ONLINE       0     0     0
	    c7t6d0s2  ONLINE       0     0     0
	    c7t7d0s2  ONLINE       0     0     0

errors: 11 data errors, use ''-v'' for a list
root at turbofan:~#
root at turbofan:~# zpool status -v slice_2
   pool: slice_2
  state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
    see: http://www.sun.com/msg/ZFS-8000-8A
  scan: none requested
config:

	NAME          STATE     READ WRITE CKSUM
	slice_2       ONLINE      20     0     0
	  raidz3-0    ONLINE       0     0     0
	    c7t0d0s2  ONLINE       0     0     0
	    c7t1d0s2  ONLINE       0     0     0
	    c7t2d0s2  ONLINE       0     0     0
	    c7t3d0s2  ONLINE       0     0     0
	    c7t4d0s2  ONLINE       0     0     0
	    c7t5d0s2  ONLINE       0     0     0
	    c7t6d0s2  ONLINE       0     0     0
	    c7t7d0s2  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

         slice_2/base/trimslice:<0x1>
         slice_2/base:<0x1>
         slice_2/base/codesourcery:<0x1>
         slice_2/base/x-plane:<0x1>
         slice_2/base/modula_n_oberon:<0x1>
         slice_2/base/minix:<0x1>
         slice_2/base/forth:<0x1>
         slice_2/base/functional:<0x1>
         slice_2/base/bitsavers:<0x1>
         slice_2/base/flightgear:<0x1>
         slice_2/base/embedded:<0x1>
root at turbofan:~#

#################################

Darren J Moffat

2012-Feb-22 16:28 UTC

head link

[zfs-discuss] Cannot mount encrypted filesystems.

On 02/22/12 06:10, Roberto Waltman wrote:> 2011-08-23.23:48:35 zfs set
> keysource=passphrase,file:///root/passphrases/slice_2_passphrase
> slice_2/base/bitsavers
That should have failed because the keysource property is inherited from 
slice_2/base. So you have found a bug and I can reproduce it.

The reason that should have failed is the source of where the keysource 
comes from is used to determine which dataset to look at for the hidden 
salt property.  We know what that salt property should actually be in 
your case because it is set on slice_2/base.

Unfortunately ''zfs set salt'' won''t work because salt
is read-only from
userland (so it doesn''t accidentally get overridden and cause the very 
same symptoms you have!).

In theory you would assume that you could go back to having the 
keysource inherited by running:
  ''zfs inherit keysource slice_2/base/bitsavers''

However that won''t work because of a protection we have in place to 
again avoid yet another route into these same symptoms. It will fail 
with an error message something like this:

cannot inherit keysource for ''slice_2/base/bitsavers'': use
''zfs key -c
-o keysource=...''

Using a hacked up libzfs that removes the check that ''zfs
inherit'' does
so I can get out of the situation and make the datasets accessible 
again.  So this is fixable so don''t abandon hope yet.

-- 
Darren J Moffat

Reasonably Related Threads

Search for more maybe matching threads

zfs discuss - Feb 2012 - Cannot mount encrypted filesystems.

[zfs-discuss] Cannot mount encrypted filesystems.

[zfs-discuss] Cannot mount encrypted filesystems.

[zfs-discuss] Cannot mount encrypted filesystems.

[zfs-discuss] Cannot mount encrypted filesystems.

[zfs-discuss] Cannot mount encrypted filesystems.

[zfs-discuss] Cannot mount encrypted filesystems.

[zfs-discuss] Cannot mount encrypted filesystems.

Reasonably Related Threads