I have a system with ZFS root that imports another zpool from a start
method. It uses a separate cache file for this zpool, like this:
if [ -f $CCACHE ]
then
echo "Importing $CPOOL with cache $CCACHE"
zpool import -o cachefile=$CCACHE -c $CCACHE $CPOOL
else
echo "Importing $CPOOL with device scan"
zpool import -o cachefile=$CCACHE $CPOOL
fi
It also exports that zpool from the stop method, which has the side
effect of deleting the cache. This all works nicely when the server
is rebooted.
What will happen when the server is halted without running the stop
method, so that that zpool is not exported? I know that there is a
flag in the zpool that indicates when it''s been exported cleanly. The
cache file will exist when the server reboots. Will the import fail
with the `The pool was last accessed by another system.'' error, or
will the import succeed? Does the cache change the import behavior?
Does it recognize that the server is the same system? I don''t want
to include the `-f'' flag in the commands above when it''s not
needed.
--
-Gary Mills- -Unix Group- -Computer and Network Services-
Q? are you intent to import this zpool to different host? Sent from my iPad Hung-Sheng Tsao ( LaoTsao) Ph.D On Aug 29, 2011, at 14:13, Gary Mills <mills at cc.umanitoba.ca> wrote:> I have a system with ZFS root that imports another zpool from a start > method. It uses a separate cache file for this zpool, like this: > > if [ -f $CCACHE ] > then > echo "Importing $CPOOL with cache $CCACHE" > zpool import -o cachefile=$CCACHE -c $CCACHE $CPOOL > else > echo "Importing $CPOOL with device scan" > zpool import -o cachefile=$CCACHE $CPOOL > fi > > It also exports that zpool from the stop method, which has the side > effect of deleting the cache. This all works nicely when the server > is rebooted. > > What will happen when the server is halted without running the stop > method, so that that zpool is not exported? I know that there is a > flag in the zpool that indicates when it''s been exported cleanly. The > cache file will exist when the server reboots. Will the import fail > with the `The pool was last accessed by another system.'' error, or > will the import succeed? Does the cache change the import behavior? > Does it recognize that the server is the same system? I don''t want > to include the `-f'' flag in the commands above when it''s not needed. > > -- > -Gary Mills- -Unix Group- -Computer and Network Services- > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Richard Elling
2011-Aug-30 00:24 UTC
[zfs-discuss] Does the zpool cache file affect import?
Hi Gary, We use this method to implement NexentaStor HA-Cluster and, IIRC, Solaris Cluster uses shared cachefiles, too. More below... On Aug 29, 2011, at 11:13 AM, Gary Mills wrote:> I have a system with ZFS root that imports another zpool from a start > method. It uses a separate cache file for this zpool, like this: > > if [ -f $CCACHE ] > then > echo "Importing $CPOOL with cache $CCACHE" > zpool import -o cachefile=$CCACHE -c $CCACHE $CPOOL > else > echo "Importing $CPOOL with device scan" > zpool import -o cachefile=$CCACHE $CPOOL > fi > > It also exports that zpool from the stop method, which has the side > effect of deleting the cache. This all works nicely when the server > is rebooted. > > What will happen when the server is halted without running the stop > method, so that that zpool is not exported? I know that there is a > flag in the zpool that indicates when it''s been exported cleanly. The > cache file will exist when the server reboots. Will the import fail > with the `The pool was last accessed by another system.'' error, or > will the import succeed?The pool metadata has the host''s ID stored. When you export the pool, the host''s ID is cleared. So as long as you import on the same host, it won''t pring the "last accessed by another system" message.> Does the cache change the import behavior?Only pools in /etc/zfs/zpool.cache are imported automatically at boot. To import your pool, add a start script.> Does it recognize that the server is the same system? I don''t want > to include the `-f'' flag in the commands above when it''s not needed.It should just work. Use "zdb -C /my/cache/file" for exploration and troubleshooting. -- richard
On Mon, Aug 29, 2011 at 07:56:16PM -0400, LaoTsao wrote:> Q? > are you intent to import this zpool to different host?Yes, it can be imported on another server. That part works when it has been exported cleanly first. I was concerned about a possible import failure when the original server lost power.> Sent from my iPadSent from my Sun type 6 keyboard. -- -Gary Mills- -Unix Group- -Computer and Network Services-
On Mon, Aug 29, 2011 at 05:24:18PM -0700, Richard Elling wrote:> We use this method to implement NexentaStor HA-Cluster and, IIRC, > Solaris Cluster uses shared cachefiles, too. More below...Mine''s a cluster too, with quite a simple design.> On Aug 29, 2011, at 11:13 AM, Gary Mills wrote: > > I have a system with ZFS root that imports another zpool from a start > > method. It uses a separate cache file for this zpool, like this: > > > > if [ -f $CCACHE ] > > then > > echo "Importing $CPOOL with cache $CCACHE" > > zpool import -o cachefile=$CCACHE -c $CCACHE $CPOOL > > else > > echo "Importing $CPOOL with device scan" > > zpool import -o cachefile=$CCACHE $CPOOL > > fi > > > > It also exports that zpool from the stop method, which has the side > > effect of deleting the cache. This all works nicely when the server > > is rebooted. > > > > What will happen when the server is halted without running the stop > > method, so that that zpool is not exported? I know that there is a > > flag in the zpool that indicates when it''s been exported cleanly. The > > cache file will exist when the server reboots. Will the import fail > > with the `The pool was last accessed by another system.'' error, or > > will the import succeed? > > The pool metadata has the host''s ID stored. When you export the pool, > the host''s ID is cleared. So as long as you import on the same host, > it won''t pring the "last accessed by another system" message.That''s what I''d hoped. So, if the active node reboots, that zpool will be imported again, perhaps more slowly because import is investigating possible damage. This is good.> > Does the cache change the import behavior? > > Only pools in /etc/zfs/zpool.cache are imported automatically at boot. > To import your pool, add a start script.Yes, that''s it above.> > Does it recognize that the server is the same system? I don''t want > > to include the `-f'' flag in the commands above when it''s not needed. > > It should just work. Use "zdb -C /my/cache/file" for exploration and > troubleshooting.Excellent! -- -Gary Mills- -Unix Group- -Computer and Network Services-
"Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D."
2011-Aug-30 11:59 UTC
[zfs-discuss] Does the zpool cache file affect import?
IMHO, you need to know how to recover the Zpool if the data/metadata get corrupt due to import by two hosts may be things are improved for zfs recently. you design cluster by certain rules, hope that all SA(not just you) will follow your rules, otherwise some SA may just import zpool that create cachefile and next time the system will want to import the zpool in cachefile that already imported by other hosts Sun cluster use its own cachefile to control the cache, but as any design it could not prevent SA to import zpool by hand:-( my 2c On 8/29/2011 9:03 PM, Gary Mills wrote:> On Mon, Aug 29, 2011 at 07:56:16PM -0400, LaoTsao wrote: >> Q? >> are you intent to import this zpool to different host? > Yes, it can be imported on another server. That part works when it > has been exported cleanly first. I was concerned about a possible > import failure when the original server lost power. > >> Sent from my iPad > Sent from my Sun type 6 keyboard. >-------------- next part -------------- A non-text attachment was scrubbed... Name: laotsao.vcf Type: text/x-vcard Size: 642 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20110830/473f3e79/attachment.vcf>
Garrett D''Amore
2011-Aug-30 14:29 UTC
[zfs-discuss] Does the zpool cache file affect import?
On 08/30/2011 04:59 AM, "Hung-Sheng Tsao (Lao Tsao ??) Ph.D." wrote:> > IMHO, you need to know how to recover the Zpool if the data/metadata > get corrupt due to import by two hosts > may be things are improved for zfs recently. > you design cluster by certain rules, hope that all SA(not just you) > will follow your rules, otherwise some SA may just import zpool that > create cachefile and next time the system will want to import the > zpool in cachefile that already imported by other hosts > Sun cluster use its own cachefile to control the cache, but as any > design it could not prevent SA to import zpool by hand:-(ZFS has some safeguards to protect against this error -- but those safeguards are bypassed when using "-f" to import a pool. Its absolutely vital that system administrators are taught to try without -f first, and the only use -f to import a pool when they are *absolutely certain* that no other system can be accessing the pool concurrently. Clusters require a greater degree of care and skill than ordinary systems. This is one reason that I try to steer folks towards application redundancy rather than cluster configurations, when redundancy at the application layer is available. - Garrett> my 2c > > On 8/29/2011 9:03 PM, Gary Mills wrote: >> On Mon, Aug 29, 2011 at 07:56:16PM -0400, LaoTsao wrote: >>> Q? >>> are you intent to import this zpool to different host? >> Yes, it can be imported on another server. That part works when it >> has been exported cleanly first. I was concerned about a possible >> import failure when the original server lost power. >> >>> Sent from my iPad >> Sent from my Sun type 6 keyboard. >> > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20110830/6fa32308/attachment.html>