zfs discuss - Sep 2006 - Comments on a ZFS multiple use of a pool, RFE.

I filed this RFE earlier, since there is no way for non sun personel
to see this RFE for a while I am posting it here, and asking for
feedback from the community.

[Fwd: CR 6470231 Created P5 opensolaris/triage-queue Add an inuse
check that is inforced even if import -f is used.]   Inbox
Assign a GTD Label to this Conversation: [Show]
Statuses:	 	Next Action, Action, Waiting On, SomeDay, Finished
Contexts:	 	Car, Desk, Email, Home, Office, Phone, Waiting
References:	 	ProjectHome, Reference
Misc.:
*Synopsis*: Add an inuse check that is inforced even if import -f is used.
http://bt2ws.central.sun.com/CrPrint?id=6470231


*Change Request ID*: 6470231

*Synopsis*: Add an inuse check that is inforced even if import -f is used.

 Product: solaris
 Category: opensolaris
 Subcategory: triage-queue
 Type: RFE
 Subtype:
 Status: 1-Dispatched
 Substatus:
 Priority: 5-Very Low
 Introduced In Release:
 Introduced In Build:
 Responsible Manager: karyn.ritter at sun.com
 Responsible Engineer:
 Initial Evaluator: opensolaris-bugs-triage at sun.com
 Keywords: opensolaris

=== *Description*
===========================================================Category
  kernel
Sub-Category
  zfs
Description
  Currently many people have been trying to import ZFS pools on
multiple systems at once. currently this is unsupported, and causes
massive data corruption to the pool.
If ZFS refuses to import any zfs pool that was used in the last 5
minutes, that was not cleanly exported. This prevents the filesystem
from being mounted on multiple systems at once.
Frequency
  Always
Regression
  No
Steps to Reproduce
  import the same storage pool on more than one machine or domain.

Expected Result
  #zpool import -f datapool1
Error: ZFS pool datapool1 is currently imported on another system, and
was accessed less than 5 minutes ago, ZFS currently does not currently
support concurrent access. If this filesystem is no longer in use on
the other system please export the filesystem from the other system or
try again in 5 minutes.

Actual Result
  #zpool import -f datapool1
#
a few minutes later the system crashes because of concurrent use.
Error Message(s)

There are several problems I can see:

- This is what the original ''-f'' flag is for.  I think a
better approach
  is to expand the default message of ''zpool import'' with more
  information, such as which was the last host to access the pool and
  when.  The point of ''-f'' is that you have recognized that
the pool
  is potentially in use, but as an administrator you''ve made a higher
  level determination that it is in fact safe to import.

- You are going to need a flag to override this behavior for clustering
  situations.  Forcing the user to always wait 5 minutes is
  unacceptable.

- By creating a new flag (lets say ''-F''), you are just going
to
  introduce more complexity, and customers will get equally used to
  issuing ''zpool import -fF'', and now you''re back to
the same problem
  all over again.

- A pool which is in use on another host but inactive for more than 5
  minutes will fail this check (since no transactions will have been
  pushed), but could potentially write data after the pool has been
  imported.

- This breaks existing behavior.  The CLI utilities are documented as
  commmitted (a.k.a stable), and breaking existing customer scripts
  isn''t acceptable.

This seems to take the wrong approach to the root problem.  Depending on
how you look at it, the real root problem is either:

a) ZFS is not a clustered filesystem, and actively using the same pool
   on multiple systems (even opening said pool) will corrupt data.

b) ''zpool import'' doesn''t present enough information
for an
   administrator to reliably determine if a pool is actually in use
   on multiple systems.

The former is obviously a ton of work and something we''re thinking
about
but won''t address any time soon.  The latter can be addressed by
presenting more useful information when ''zpool import'' is run
without
the ''-f'' flag.

- Eric

On Wed, Sep 13, 2006 at 12:14:06PM -0500, James Dickens
wrote:
> I filed this RFE earlier, since there is no way for non sun personel
> to see this RFE for a while I am posting it here, and asking for
> feedback from the community.
> 
> [Fwd: CR 6470231 Created P5 opensolaris/triage-queue Add an inuse
> check that is inforced even if import -f is used.]   Inbox
> Assign a GTD Label to this Conversation: [Show]
> Statuses:	 	Next Action, Action, Waiting On, SomeDay, Finished
> Contexts:	 	Car, Desk, Email, Home, Office, Phone, Waiting
> References:	 	ProjectHome, Reference
> Misc.:
> *Synopsis*: Add an inuse check that is inforced even if import -f is used.
> http://bt2ws.central.sun.com/CrPrint?id=6470231
> 
> 
> *Change Request ID*: 6470231
> 
> *Synopsis*: Add an inuse check that is inforced even if import -f is used.
> 
> Product: solaris
> Category: opensolaris
> Subcategory: triage-queue
> Type: RFE
> Subtype:
> Status: 1-Dispatched
> Substatus:
> Priority: 5-Very Low
> Introduced In Release:
> Introduced In Build:
> Responsible Manager: karyn.ritter at sun.com
> Responsible Engineer:
> Initial Evaluator: opensolaris-bugs-triage at sun.com
> Keywords: opensolaris
> 
> === *Description* 
> ===========================================================> Category
>  kernel
> Sub-Category
>  zfs
> Description
>  Currently many people have been trying to import ZFS pools on
> multiple systems at once. currently this is unsupported, and causes
> massive data corruption to the pool.
> If ZFS refuses to import any zfs pool that was used in the last 5
> minutes, that was not cleanly exported. This prevents the filesystem
> from being mounted on multiple systems at once.
> Frequency
>  Always
> Regression
>  No
> Steps to Reproduce
>  import the same storage pool on more than one machine or domain.
> 
> Expected Result
>  #zpool import -f datapool1
> Error: ZFS pool datapool1 is currently imported on another system, and
> was accessed less than 5 minutes ago, ZFS currently does not currently
> support concurrent access. If this filesystem is no longer in use on
> the other system please export the filesystem from the other system or
> try again in 5 minutes.
> 
> Actual Result
>  #zpool import -f datapool1
> #
> a few minutes later the system crashes because of concurrent use.
> Error Message(s)
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
--
Eric Schrock, Solaris Kernel Development       http://blogs.sun.com/eschrock

I think there are at least two separate issues here.

The first is that ZFS doesn''t support multiple hosts accessing the same
pool. That''s simply a matter of telling people. UFS doesn''t
support multiple hosts, but it doesn''t have any special features to
prevent administrators from *trying* it. They''ll "just"
corrupt their filesystem.

The second is that ZFS remembers pools and automatically imports them at boot
time. This is a bigger problem, because it means that if you create a pool on
host A, shut down host A, import the pool to host B, and then boot host A, your
pool is automatically destroyed.

The hostid solution that VxVM uses would catch this second problem, because when
A came up after its reboot, it would find that -- even though it had created the
pool -- it was not the last machine to access it, and could refuse to
automatically mount it. If the administrator really wanted it mounted, they
could force the issue. Relying on the administrator to know that they have to
remove a file (the ''zpool cache'') before they let the machine
come up out of single-user mode seems the wrong approach to me. ("By
default, we''ll shoot you in the foot, but we''ll give you a way
to unload the gun if you''re fast enough and if you remember.")

The hostid approach seems better to me than modifying the semantics of
"force." I honestly don''t think the problem is administrators
who don''t know what they''re doing; I think the problem is that
our defaults are wrong in the case of shared storage.
 
 
This message posted from opensolaris.org

On 9/13/06, Eric Schrock <eric.schrock at sun.com>
wrote:
> There are several problems I can see:
>
> - This is what the original ''-f'' flag is for.  I think a
better approach
>   is to expand the default message of ''zpool import'' with
more
>   information, such as which was the last host to access the pool and
>   when.  The point of ''-f'' is that you have recognized
that the pool
>   is potentially in use, but as an administrator you''ve made a
higher
>   level determination that it is in fact safe to import.
>
this would not be the first time that Solaris overrided an administive
command, because its just not safe or sane  to do so. For example.

rm -rf /
> - You are going to need a flag to override this behavior for clustering
>   situations.  Forcing the user to always wait 5 minutes is
>   unacceptable.
>
wouldn''t it be more likely to for a clustering solution to use libzfs
or we could add another  method to zfs for failing over in clustering
solutions, this method would then check to see if the OS and the pool
supported clustering at the time of import. Since Clustering support
is not yet released for ZFS we have clean slate on how ZFS deals with
it.
> - By creating a new flag (lets say ''-F''), you are just
going to
>   introduce more complexity, and customers will get equally used to
>   issuing ''zpool import -fF'', and now you''re
back to the same problem
>   all over again.
>
if 5 minutes is to long, perhaps it could reduced this to 2 minutes
and make ZFS update a value stored on the pool once a minute that it
is in use. we could update the pool in use  flag more often but it
seems excessive since its only a courner case anyway.

Another possible method to handle this case, but its more work but
would not impact existing fast paths, would be for zpool to watch the
pool if it appears to have been accessed in the last X minutes and not
exported, is to have ZFS watch the devices and see if any other disk
commands occur from the old host, if any occurs its obvious that the
administrator is putting the system in to a case where it will crash
the system, so isn''t it better to fail the import then crash? Any
extra delay this check imposes would not break existing specifications
because importing devices is not guaranteed fixed performance because
ZFS needs to find the devices  and verify that all components of pool
are in tact and not failed.


James
> - A pool which is in use on another host but inactive for more than 5
>   minutes will fail this check (since no transactions will have been
>   pushed), but could potentially write data after the pool has been
>   imported.
>
> - This breaks existing behavior.  The CLI utilities are documented as
>   commmitted (a.k.a stable), and breaking existing customer scripts
>   isn''t acceptable.
the existing behaviour is broken, ZFS should return EBUSY, if ZFS can
determine that the pool  is in active use without extreme measures.

This is the same behaviour that happens should a user try and mount a
filesystem twice on one system.

James

>
> This seems to take the wrong approach to the root problem.  Depending on
> how you look at it, the real root problem is either:
>
> a) ZFS is not a clustered filesystem, and actively using the same pool
>    on multiple systems (even opening said pool) will corrupt data.
>
> b) ''zpool import'' doesn''t present enough
information for an
>    administrator to reliably determine if a pool is actually in use
>    on multiple systems.
>
> The former is obviously a ton of work and something we''re thinking
about
> but won''t address any time soon.  The latter can be addressed by
> presenting more useful information when ''zpool import'' is
run without
> the ''-f'' flag.
>
> - Eric
>
> On Wed, Sep 13, 2006 at 12:14:06PM -0500, James Dickens wrote:
> > I filed this RFE earlier, since there is no way for non sun personel
> > to see this RFE for a while I am posting it here, and asking for
> > feedback from the community.
> >
> > [Fwd: CR 6470231 Created P5 opensolaris/triage-queue Add an inuse
> > check that is inforced even if import -f is used.]   Inbox
> > Assign a GTD Label to this Conversation: [Show]
> > Statuses:             Next Action, Action, Waiting On, SomeDay,
Finished
> > Contexts:             Car, Desk, Email, Home, Office, Phone, Waiting
> > References:           ProjectHome, Reference
> > Misc.:
> > *Synopsis*: Add an inuse check that is inforced even if import -f is
used.
> > http://bt2ws.central.sun.com/CrPrint?id=6470231
> >
> >
> > *Change Request ID*: 6470231
> >
> > *Synopsis*: Add an inuse check that is inforced even if import -f is
used.
> >
> > Product: solaris
> > Category: opensolaris
> > Subcategory: triage-queue
> > Type: RFE
> > Subtype:
> > Status: 1-Dispatched
> > Substatus:
> > Priority: 5-Very Low
> > Introduced In Release:
> > Introduced In Build:
> > Responsible Manager: karyn.ritter at sun.com
> > Responsible Engineer:
> > Initial Evaluator: opensolaris-bugs-triage at sun.com
> > Keywords: opensolaris
> >
> > === *Description*
> > ===========================================================> >
Category
> >  kernel
> > Sub-Category
> >  zfs
> > Description
> >  Currently many people have been trying to import ZFS pools on
> > multiple systems at once. currently this is unsupported, and causes
> > massive data corruption to the pool.
> > If ZFS refuses to import any zfs pool that was used in the last 5
> > minutes, that was not cleanly exported. This prevents the filesystem
> > from being mounted on multiple systems at once.
> > Frequency
> >  Always
> > Regression
> >  No
> > Steps to Reproduce
> >  import the same storage pool on more than one machine or domain.
> >
> > Expected Result
> >  #zpool import -f datapool1
> > Error: ZFS pool datapool1 is currently imported on another system, and
> > was accessed less than 5 minutes ago, ZFS currently does not currently
> > support concurrent access. If this filesystem is no longer in use on
> > the other system please export the filesystem from the other system or
> > try again in 5 minutes.
> >
> > Actual Result
> >  #zpool import -f datapool1
> > #
> > a few minutes later the system crashes because of concurrent use.
> > Error Message(s)
> > _______________________________________________
> > zfs-discuss mailing list
> > zfs-discuss at opensolaris.org
> > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
> --
> Eric Schrock, Solaris Kernel Development      
http://blogs.sun.com/eschrock
>

On Wed, Sep 13, 2006 at 02:29:55PM -0500, James Dickens
wrote:
>
> this would not be the first time that Solaris overrided an administive
> command, because its just not safe or sane  to do so. For example.
> 
> rm -rf /
As I''ve repeated before, and will continue to repeat, it''s not
actually
possible for ZFS to determine whether a pool is in active use (short of
making ZFS a cluster-aware filesystem).  Adding arbitrary delays
doesn''t
change this fact, and only less likely.  I''ve given you examples of
where this behavior is safe and sane and useful, so the above
simplification (upon which most of the other arguments are based) isn''t
really valid.

I''m curious why you didn''t comment on my other suggestion
(displaying
last accessed host and time as part of ''zpool import''), which
seems to
solve your problem by giving the administrator the data they need to
make an appropriate decision.

As Anton and other have mentioned in previous discussions there seems to
be several clear RFEs that everyone can agree with:

1. Store the hostid, hostname, and last time written as part of the
   label.

2. During auto-import (aka open), if the the hostid is different from
   our own, fault the pool and generate an appropriate FMA event.

3. During manual import, display the last hostname and time accessed if
   the hostid is not our own, and the pool is still marked ACTIVE.

This prevents administrators from shooting themselves in the foot, while
still allowing explicit cluster failover to operate with more
information than was available before.

- Eric

--
Eric Schrock, Solaris Kernel Development       http://blogs.sun.com/eschrock

On 9/13/06, Eric Schrock <eric.schrock at sun.com>
wrote:
> On Wed, Sep 13, 2006 at 02:29:55PM -0500, James Dickens wrote:
> >
> > this would not be the first time that Solaris overrided an administive
> > command, because its just not safe or sane  to do so. For example.
> >
> > rm -rf /
>
> As I''ve repeated before, and will continue to repeat,
it''s not actually
> possible for ZFS to determine whether a pool is in active use (short of
> making ZFS a cluster-aware filesystem).  Adding arbitrary delays
doesn''t
> change this fact, and only less likely.  I''ve given you examples
of
> where this behavior is safe and sane and useful, so the above
> simplification (upon which most of the other arguments are based)
isn''t
> really valid.
>
I disagree with this, isn''t there way to track when the last read was?
 Even computing the last write access time that you are already
recommending, then sleeping for 30 seconds and recompute the value by
accessing the disk again and compare,even a read on the pool will
possibly cause a write if ATIME tracking is enabled on the filesystem,
if someone is accessing the pool we are importing underneath us, it is
a dead give away that we are about to explode if we continue on this
path.
> I''m curious why you didn''t comment on my other suggestion
(displaying
> last accessed host and time as part of ''zpool import''),
which seems to
> solve your problem by giving the administrator the data they need to
> make an appropriate decision.
>
its a good suggestion, it just doesn''t go far enough in my opinion.
> As Anton and other have mentioned in previous discussions there seems to
> be several clear RFEs that everyone can agree with:
>
> 1. Store the hostid, hostname, and last time written as part of the
>    label.
>
> 2. During auto-import (aka open), if the the hostid is different from
>    our own, fault the pool and generate an appropriate FMA event.
>
> 3. During manual import, display the last hostname and time accessed if
>    the hostid is not our own, and the pool is still marked ACTIVE.
>
> This prevents administrators from shooting themselves in the foot, while
> still allowing explicit cluster failover to operate with more
> information than was available before.
>
if this is what the community decides I can live with this, I may even
provide a patch for OpenSolaris distros that does the more intensive
check, it seems to be an easy fix once you have complete #1.

James

> - Eric
>
> --
> Eric Schrock, Solaris Kernel Development      
http://blogs.sun.com/eschrock
>

Anton B. Rang schrieb:
> The hostid solution that VxVM uses would catch this second problem,
 > because when A came up after its reboot, it would find that -- even
 > though it had created the pool -- it was not the last machine to access
 > it, and could refuse to automatically mount it. If the administrator
 > really wanted it mounted, they could force the issue. Relying on the
 > administrator to know that they have to remove a file (the
 > ''zpool cache'') before they let the machine come up out
of single-user
 > mode seems the wrong approach to me. ("By default, we''ll
shoot you in
 > the foot, but we''ll give you a way to unload the gun if
you''re fast
 > enough and if you remember.")

I haven''t tried: Does ZFS try to -f (force) import of zpools in 
/etc/zfs/zpool.cache or does it just do a normal import and fails if the disks 
seem to be in use elsewhere, e.g. after a reboot of a proably failed and later
repaired machine?


Just to clear some things up. The OP who started the whole discussion would 
have had the same problems with VxVM as he has now with ZFS. If you force an 
import of a disk group on one host while it is still active on another host 
won''t make the DG magically disappear on the other one.

The corresponding flag to "zpool import -f" is "vxdg import
-C". If you issue
this command you could also end up with the same DG imported on more than one 
host. Because on VxVM there is usually another level of indirection (volumes 
ontop of the DG which may contain filesystems and also have to manually mount) 
just importing a DG is normally harmless.

So also with VxVM you can shoot yourself in the foot.

On host B
	B# vxdg -C import DG
	B# vxvol -g DG startall
	B# mount /dev/vx/dsk/DG/filesys /some/where
	B# do_someting on /some/where

while still on host A
	A# do_something on /some/where

Instead of a zpool.cache file VxVM uses the hostid (not to be confused with 
the host-id, normally just the ordinary hostname `uname -n` of the machine) to 
know which DGs it should mount automatically. Additionally each DG (or more 
precisely: each disk) has an autoimport flag which also has to be turned on to 
make the DG autoimported during bootup.

So to mimic VxVM in ZFS the solution would simply be: add an autoimport flag 
to the zpool.




Daniel

On September 14, 2006 1:25:01 AM +0200 Daniel Rock <solaris at
deadcafe.de> wrote:
> Just to clear some things up. The OP who started the whole discussion would
have had the same
> problems with VxVM as he has now with ZFS. If you force an import of a disk
group on one host
> while it is still active on another host won''t make the DG
magically disappear on the other one.
The OP was just showing a test case.  On a real system your HA software
would exchange a heartbeat and not do a double import.  The problem with
zfs is that after the original system fails and the second system imports
the pool, the original system also tries to import on [re]boot, and the OP
didn''t know how to disable this.

-frank

James Dickens wrote:
> On 9/13/06, Eric Schrock <eric.schrock at sun.com> wrote:
>> On Wed, Sep 13, 2006 at 02:29:55PM -0500, James Dickens wrote:
>> >
>> > this would not be the first time that Solaris overrided an
administive
>> > command, because its just not safe or sane  to do so. For example.
>> >
>> > rm -rf /
>>
>> As I''ve repeated before, and will continue to repeat,
it''s not actually
>> possible for ZFS to determine whether a pool is in active use (short of
>> making ZFS a cluster-aware filesystem).  Adding arbitrary delays
doesn''t
>> change this fact, and only less likely.  I''ve given you
examples of
>> where this behavior is safe and sane and useful, so the above
>> simplification (upon which most of the other arguments are based)
isn''t
>> really valid.
>>
> I disagree with this, isn''t there way to track when the last read
was?
Track ?  Doesn''t that imply a write. Writing is a bad idea because it
can cause otherwise spun-down disks to need to be spun-up - which is
bad for power consumption and possibly for the disk time to failure.

-- 
Darren J Moffat

On 9/14/06, Darren J Moffat <Darren.Moffat at sun.com>
wrote:
> James Dickens wrote:
> > On 9/13/06, Eric Schrock <eric.schrock at sun.com> wrote:
> >> On Wed, Sep 13, 2006 at 02:29:55PM -0500, James Dickens wrote:
> >> >
> >> > this would not be the first time that Solaris overrided an
administive
> >> > command, because its just not safe or sane  to do so. For
example.
> >> >
> >> > rm -rf /
> >>
> >> As I''ve repeated before, and will continue to repeat,
it''s not actually
> >> possible for ZFS to determine whether a pool is in active use
(short of
> >> making ZFS a cluster-aware filesystem).  Adding arbitrary delays
doesn''t
> >> change this fact, and only less likely.  I''ve given you
examples of
> >> where this behavior is safe and sane and useful, so the above
> >> simplification (upon which most of the other arguments are based)
isn''t
> >> really valid.
> >>
> > I disagree with this, isn''t there way to track when the last
read was?
>
> Track ?  Doesn''t that imply a write. Writing is a bad idea because
it
> can cause otherwise spun-down disks to need to be spun-up - which is
> bad for power consumption and possibly for the disk time to failure.
>
eric was allready talking about printing the last time a disk was
accessed when a disk was about to be imported, my idea would be run
that check twice, once initially and if it looks like it could be
still in use, like the pool wasn''t exported and last write occurred in
the last 30 seconds or a few minutes, it would sleep, then run the
check again, it the drive showed up changed ( even if it was just
because of an updated atime) the import would fail with EBUSY.  The
drives would all ready be spun up during all of this because were
importing them.

James Dickens
uadmin.blogspot.com

> --
> Darren J Moffat
>

> The OP was just showing a test case.  On a real system your HA software
> would exchange a heartbeat and not do a double import.  The problem with
> zfs is that after the original system fails and the second system imports
> the pool, the original system also tries to import on [re]boot, and the OP
> didn''t know how to disable this.
Ok, I see. I don''t think he showed that because he didn''t
mention a reboot.

If you *never* want to import a pool automatically on reboot you just have to
delete the /etc/zfs/zpool.cache file before the zfs module is being loaded. This
could be integrated into SMF.

ZFS should normally be loaded during svc:/system/filesystem/local. Just create a
service with filesystem/local as a dependent which removes /etc/zfs/zpool.cache.

Like:

/lib/svc/method/zpool-clear

#!/sbin/sh
. /lib/svc/share/smf_include.sh
/usr/bin/rm -f /etc/zfs/zpool.cache
exit $SMF_EXIT_OK


/var/svc/manifest/system/zpool-clear.xml

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM
"/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type=''manifest''
name=''none:zpool-clear''>
<service
        name=''system/zpool/clear''
        type=''service''
        version=''1''>
        <create_default_instance enabled=''true'' />
        <single_instance/>
        <dependency
                name=''usr''
                grouping=''require_all''
                restart_on=''none''
                type=''service''>
                <service_fmri
value=''svc:/system/filesystem/usr'' />
        </dependency>
        <dependent
                name=''fs-local''
                grouping=''require_all''
                restart_on=''none''>
                <service_fmri
value=''svc:/system/filesystem/local'' />
        </dependent>
        <exec_method
                type=''method''
                name=''start''
                exec=''/lib/svc/method/zpool-clear''
                timeout_seconds=''0'' />
        <exec_method
                type=''method''
                name=''stop''
                exec='':true''
                timeout_seconds=''0'' />
        <property_group name=''startd''
type=''framework''>
                <propval name=''duration''
type=''astring'' value=''transient'' />
        </property_group>
        <template>
                <common_name>
                        <loctext xml:lang=''C''>
                                Clears the zpool cache file
                        </loctext>
                </common_name>
        </template>
</service>
</service_bundle>

(sorry for the poor formating but this !$&%()# web interface is driving me
nuts)

Then do a "svccfg import /var/svc/manifest/system/zpool-clear.xml".
Now /etc/zfs/zpool.cache should disappear on each reboot before the zfs module
is loaded.
 
 
This message posted from opensolaris.org

> If you *never* want to import a pool automatically on reboot you just have
to delete the
> /etc/zfs/zpool.cache file before the zfs module is being loaded.
> This could be integrated into SMF.
Or you could always use import -R / create -R for your pool management.  Of
course, there''s no way to set a global default for these, so you have
to remember it each time, making the SMF solution more attractive....
 
 
This message posted from opensolaris.org

> > If you *never* want to import a pool automatically on reboot you just
have to delete the
> > /etc/zfs/zpool.cache file before the zfs module is being loaded.
> > This could be integrated into SMF.
> 
> Or you could always use import -R / create -R for your pool management.  Of
course, there''s no way to set a global default for these, so you have
to remember it each time, making the SMF solution more attractive....
Perfect.  (although I have to try it).  In a cluster framework, the
cluster can remember to do it each time, so that shouldn''t be an issue.


-- 
Darren Dunham                                           ddunham at taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

James Dickens wrote:
>
> eric was allready talking about printing the last time a disk was
> accessed when a disk was about to be imported, my idea would be run
> that check twice, once initially and if it looks like it could be
> still in use, like the pool wasn''t exported and last write
occurred in
> the last 30 seconds or a few minutes, it would sleep, then run the
> check again, it the drive showed up changed ( even if it was just
> because of an updated atime) the import would fail with EBUSY.  The
> drives would all ready be spun up during all of this because were
> importing them.

You''re assuming that the system that owns the pool is up and running at
the times you''re making your checks. Or that a certain span of wall 
clock time is a good indication of what is in use.

In any situation where two entities are unclear as to device ownership 
they go to a third entity. You can say thats a cluster node or that its 
a somewhat intelligent person. However, trying to work around the issue 
is going to lead to a disaster at some point. Seen it in action plenty 
of times. If people want to use ZFS commands to try and get around the 
issue then have at it but please don''t ask the folks working on ZFS to 
assemble the gun, purchase the ammo, load the weapon, and point it in 
the general direction of your lower locomotive extremities. :)

Hello Darren,

Thursday, September 14, 2006, 5:42:20 PM, you wrote:
>> > If you *never* want to import a pool automatically on reboot you
just have to delete the
>> > /etc/zfs/zpool.cache file before the zfs module is being loaded.
>> > This could be integrated into SMF.
>> 
>> Or you could always use import -R / create -R for your pool management.
Of course, there''s no way to set a global default for these, so you
have to remember it each time, making the SMF solution more attractive....
DD> Perfect.  (although I have to try it).  In a cluster framework, the
DD> cluster can remember to do it each time, so that shouldn''t be an
issue.



And that''s exactly what SC32 does.


-- 
Best regards,
 Robert                            mailto:rmilkowski at task.gda.pl
                                       http://milek.blogspot.com

Hello James,


I belive that storing hostid, etc. in a label and checking if it
matches on auto-import is the right solution.
Before it''s implemented you can use -R right now with home-clusters
and don''t worry about auto-import.

However maybe doing (optional) SCSI reservation on a pool would be a
good idea? Of course as an extra switch during/after import.
I know not all devices supports it but still. ZFS would allow either
reserve all disks in a pool or none.


-- 
Best regards,
 Robert                            mailto:rmilkowski at task.gda.pl
                                       http://milek.blogspot.com

Robert Milkowski wrote:
> Hello James,
> 
> I belive that storing hostid, etc. in a label and checking if it
> matches on auto-import is the right solution.
> Before it''s implemented you can use -R right now with
home-clusters
> and don''t worry about auto-import.
hostid isn''t sufficient (got a scar), so perhaps several identifying
characteristics would work.  There is merit to this.
> However maybe doing (optional) SCSI reservation on a pool would be a
> good idea? Of course as an extra switch during/after import.
> I know not all devices supports it but still. ZFS would allow either
> reserve all disks in a pool or none.
Hopefully we like ourselves too much to try to use SCSI reservations.
The road is littered with the dead or bleeding carcases of people who
expected SCSI reservations to actually work.  Heck, it took over 8 years
for even basic (by today''s standards) SCSI functionality to be widely
implemented.
  -- richard