freebsd stable - May 2017 - zpool imported twice with different names (was Re: Fwd: ZFS)

> On May 17, 2017, at 5:12 PM, Eric A. Borisch <eborisch at gmail.com>
wrote:
> 
> Short version:
> 
> It's not the -f causing the problem, it's the parsing and (double)
> importing via /boot/zfs/zpool.cache that is.
I was unclear? _how_ do you get the double entry in zpool.cache _except_ by
using the -f option with a zpool that is already imported ?
> Longer story:
<snip>
> So it is not the import with -f that is causing the problem here, it is the
> following reboot when the zpool.cache file is parsed and ensuing
> double-import.
I would maintain that it is the combination of the import of an already imported
zpool which causes the double entry in the zpool.cache in combination with a
recent rename of the zpool.

Would (did) the import without the force fail ? The last host this zpool was
imported on was _this_ host, so the ?is this zpool used by another host? check
would not stop such an import.

Or, is the zpool rename not updating the name in the zpool.cache file ?

I don?t currently have a test system available or I would experiment... 
> (As an aside, using the -f is required to import a pool last
> used and not exported by another system, and is a common and documented use
> case. If you look at the code, it turns on a the ZFS_IMPORT_ANY_HOST flag,
> not an 'ignore all errors' state. There is a separate -F which is
more
> 'force'-ish (turns on rewinding).)
There are times when it is necessary to force an operation. One of the
underlying assumptions is that the person using the force option _knows_ with a
very high degree of certainty that the zpool is not in use, either by another
system or the same system.

On Wed, May 17, 2017 at 8:27 PM Paul Kraus <paul at kraus-haus.org> wrote:
>
> > On May 17, 2017, at 5:12 PM, Eric A. Borisch <eborisch at
gmail.com> wrote:
> >
> > Short version:
> >
> > It's not the -f causing the problem, it's the parsing and
(double)
> > importing via /boot/zfs/zpool.cache that is.
>
> I was unclear? _how_ do you get the double entry in zpool.cache _except_
> by using the -f option with a zpool that is already imported ?
>
To be clear, I'm using "already imported" to me == shows up in
'zpool list'
of some running OS, not described in a config file somewhere. The Oracle
description [1] that goes along with 'import -f' is: "Do not
attempt to
import a pool that is active on one system to another system. ZFS is not a
native cluster, distributed, or parallel file system and cannot provide
concurrent access from multiple, different hosts." This sounds much more
like like my description (shows in a running zpool list) than described by
a config/cache file.

Steps to recreate: (I did it in < 5 minutes in a VM.) Arabic numerals are
user actions, Roman numerals are zpool state.

1) Install FreeBSD root-on-zfs onto 'Alpha'

I. Alpha exists on disk, and is active whenever your computer is running

1a) Decide at some point later you would like to rename your root pool...
so you

2) Boot off live CD

II. Alpha exists on disk, but is not active (imported)

3) zpool import alpha beta (because that's how you rename pools)
3a) zpool helpfully points out that it *may be in use* on another system,
and use -f to import it anyway. Realize you are at the system that was
using it, but in a completely different environment, and decide that it is
safe to import.

III. Alpha still exists on disk, and is still not active

4) zpool import -f alpha beta succeeds without any complaints.

IV. Alpha no longer exists, Beta exists (with the same GUID as Alpha) AND
is active. This is an 'atomic' operation from user's point of view,
and at
no point does zpool list show more than one pool

5) zpool list shows only beta; scrubs fine, life is good..

V. Beta exists on disk and is active and healthy. Alpha is no more. (Except
as a ghost in a cache file)

6) Reboot into OS.

Kernel messages mention booting off of beta/<bootfs path>. Good, good...

VI. Beta exists at this point certainly, as boot progresses. The ghost of
Alpha is still lurking, however.

Before we drop to a prompt, the zpool.cache is parsed, and Alpha springs
back into existence beside Beta. NOTE: This is where I'm suggesting a cache
file shouldn't trump the on-disk metadata (name mismatch) and running state
(GUID collision) and lead to corruption.

VII. Alpha and Beta exist together violating the Pauli exclusion principle
as it applies to zpools.

7) Despair. Try to export one or the other. No good. Errors start to show
up in zpool status. Rebooting fails in various colorful ways. Trying to
import the pool again via Live CD is equally fruitless.

VIII. Your zpool is cooked. By a config file. Hope you weren't using ZFS
for integrity.

I would maintain that it is the combination of the import of an
already
> imported zpool which causes the double entry in the zpool.cache in
> combination with a recent rename of the zpool.
>
> Would (did) the import without the force fail ? The last host this zpool
> was imported on was _this_ host, so the ?is this zpool used by another
> host? check would not stop such an import.

Yes, it fails without the -f, and helpfully suggests using the -f to import
it. I don't think it's already imported in a way that most people
(including Oracle) describe it.

Or, is the zpool rename not updating the name in the zpool.cache file ?


As the zpool rename is occurring in the live cd system; the cache file
(living on the filesystems of the pool to be renamed) isn't accessible
during the import/rename step, and isn't known about by the active OS. I
suspect using a temporary and updating the cache file post-rename
pre-reboot would work, just like deleting the cache file before reboot
worked.

I still, however, maintain an errant cache file shouldn't trump both
on-disk state (my name... is Beta) and  in-memory state (but I already have
that GUID) and bring down a pool.

  - Eric

[1] http://docs.oracle.com/cd/E19253-01/819-5461/gazuf/index.html
>