James Dickens
2006-Sep-13 17:14 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
I filed this RFE earlier, since there is no way for non sun personel to see this RFE for a while I am posting it here, and asking for feedback from the community. [Fwd: CR 6470231 Created P5 opensolaris/triage-queue Add an inuse check that is inforced even if import -f is used.] Inbox Assign a GTD Label to this Conversation: [Show] Statuses: Next Action, Action, Waiting On, SomeDay, Finished Contexts: Car, Desk, Email, Home, Office, Phone, Waiting References: ProjectHome, Reference Misc.: *Synopsis*: Add an inuse check that is inforced even if import -f is used. http://bt2ws.central.sun.com/CrPrint?id=6470231 *Change Request ID*: 6470231 *Synopsis*: Add an inuse check that is inforced even if import -f is used. Product: solaris Category: opensolaris Subcategory: triage-queue Type: RFE Subtype: Status: 1-Dispatched Substatus: Priority: 5-Very Low Introduced In Release: Introduced In Build: Responsible Manager: karyn.ritter at sun.com Responsible Engineer: Initial Evaluator: opensolaris-bugs-triage at sun.com Keywords: opensolaris === *Description* ===========================================================Category kernel Sub-Category zfs Description Currently many people have been trying to import ZFS pools on multiple systems at once. currently this is unsupported, and causes massive data corruption to the pool. If ZFS refuses to import any zfs pool that was used in the last 5 minutes, that was not cleanly exported. This prevents the filesystem from being mounted on multiple systems at once. Frequency Always Regression No Steps to Reproduce import the same storage pool on more than one machine or domain. Expected Result #zpool import -f datapool1 Error: ZFS pool datapool1 is currently imported on another system, and was accessed less than 5 minutes ago, ZFS currently does not currently support concurrent access. If this filesystem is no longer in use on the other system please export the filesystem from the other system or try again in 5 minutes. Actual Result #zpool import -f datapool1 # a few minutes later the system crashes because of concurrent use. Error Message(s)
Eric Schrock
2006-Sep-13 17:28 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
There are several problems I can see: - This is what the original ''-f'' flag is for. I think a better approach is to expand the default message of ''zpool import'' with more information, such as which was the last host to access the pool and when. The point of ''-f'' is that you have recognized that the pool is potentially in use, but as an administrator you''ve made a higher level determination that it is in fact safe to import. - You are going to need a flag to override this behavior for clustering situations. Forcing the user to always wait 5 minutes is unacceptable. - By creating a new flag (lets say ''-F''), you are just going to introduce more complexity, and customers will get equally used to issuing ''zpool import -fF'', and now you''re back to the same problem all over again. - A pool which is in use on another host but inactive for more than 5 minutes will fail this check (since no transactions will have been pushed), but could potentially write data after the pool has been imported. - This breaks existing behavior. The CLI utilities are documented as commmitted (a.k.a stable), and breaking existing customer scripts isn''t acceptable. This seems to take the wrong approach to the root problem. Depending on how you look at it, the real root problem is either: a) ZFS is not a clustered filesystem, and actively using the same pool on multiple systems (even opening said pool) will corrupt data. b) ''zpool import'' doesn''t present enough information for an administrator to reliably determine if a pool is actually in use on multiple systems. The former is obviously a ton of work and something we''re thinking about but won''t address any time soon. The latter can be addressed by presenting more useful information when ''zpool import'' is run without the ''-f'' flag. - Eric On Wed, Sep 13, 2006 at 12:14:06PM -0500, James Dickens wrote:> I filed this RFE earlier, since there is no way for non sun personel > to see this RFE for a while I am posting it here, and asking for > feedback from the community. > > [Fwd: CR 6470231 Created P5 opensolaris/triage-queue Add an inuse > check that is inforced even if import -f is used.] Inbox > Assign a GTD Label to this Conversation: [Show] > Statuses: Next Action, Action, Waiting On, SomeDay, Finished > Contexts: Car, Desk, Email, Home, Office, Phone, Waiting > References: ProjectHome, Reference > Misc.: > *Synopsis*: Add an inuse check that is inforced even if import -f is used. > http://bt2ws.central.sun.com/CrPrint?id=6470231 > > > *Change Request ID*: 6470231 > > *Synopsis*: Add an inuse check that is inforced even if import -f is used. > > Product: solaris > Category: opensolaris > Subcategory: triage-queue > Type: RFE > Subtype: > Status: 1-Dispatched > Substatus: > Priority: 5-Very Low > Introduced In Release: > Introduced In Build: > Responsible Manager: karyn.ritter at sun.com > Responsible Engineer: > Initial Evaluator: opensolaris-bugs-triage at sun.com > Keywords: opensolaris > > === *Description* > ===========================================================> Category > kernel > Sub-Category > zfs > Description > Currently many people have been trying to import ZFS pools on > multiple systems at once. currently this is unsupported, and causes > massive data corruption to the pool. > If ZFS refuses to import any zfs pool that was used in the last 5 > minutes, that was not cleanly exported. This prevents the filesystem > from being mounted on multiple systems at once. > Frequency > Always > Regression > No > Steps to Reproduce > import the same storage pool on more than one machine or domain. > > Expected Result > #zpool import -f datapool1 > Error: ZFS pool datapool1 is currently imported on another system, and > was accessed less than 5 minutes ago, ZFS currently does not currently > support concurrent access. If this filesystem is no longer in use on > the other system please export the filesystem from the other system or > try again in 5 minutes. > > Actual Result > #zpool import -f datapool1 > # > a few minutes later the system crashes because of concurrent use. > Error Message(s) > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock
Anton B. Rang
2006-Sep-13 19:29 UTC
[zfs-discuss] Re: Comments on a ZFS multiple use of a pool, RFE.
I think there are at least two separate issues here. The first is that ZFS doesn''t support multiple hosts accessing the same pool. That''s simply a matter of telling people. UFS doesn''t support multiple hosts, but it doesn''t have any special features to prevent administrators from *trying* it. They''ll "just" corrupt their filesystem. The second is that ZFS remembers pools and automatically imports them at boot time. This is a bigger problem, because it means that if you create a pool on host A, shut down host A, import the pool to host B, and then boot host A, your pool is automatically destroyed. The hostid solution that VxVM uses would catch this second problem, because when A came up after its reboot, it would find that -- even though it had created the pool -- it was not the last machine to access it, and could refuse to automatically mount it. If the administrator really wanted it mounted, they could force the issue. Relying on the administrator to know that they have to remove a file (the ''zpool cache'') before they let the machine come up out of single-user mode seems the wrong approach to me. ("By default, we''ll shoot you in the foot, but we''ll give you a way to unload the gun if you''re fast enough and if you remember.") The hostid approach seems better to me than modifying the semantics of "force." I honestly don''t think the problem is administrators who don''t know what they''re doing; I think the problem is that our defaults are wrong in the case of shared storage. This message posted from opensolaris.org
James Dickens
2006-Sep-13 19:29 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
On 9/13/06, Eric Schrock <eric.schrock at sun.com> wrote:> There are several problems I can see: > > - This is what the original ''-f'' flag is for. I think a better approach > is to expand the default message of ''zpool import'' with more > information, such as which was the last host to access the pool and > when. The point of ''-f'' is that you have recognized that the pool > is potentially in use, but as an administrator you''ve made a higher > level determination that it is in fact safe to import. >this would not be the first time that Solaris overrided an administive command, because its just not safe or sane to do so. For example. rm -rf /> - You are going to need a flag to override this behavior for clustering > situations. Forcing the user to always wait 5 minutes is > unacceptable. >wouldn''t it be more likely to for a clustering solution to use libzfs or we could add another method to zfs for failing over in clustering solutions, this method would then check to see if the OS and the pool supported clustering at the time of import. Since Clustering support is not yet released for ZFS we have clean slate on how ZFS deals with it.> - By creating a new flag (lets say ''-F''), you are just going to > introduce more complexity, and customers will get equally used to > issuing ''zpool import -fF'', and now you''re back to the same problem > all over again. >if 5 minutes is to long, perhaps it could reduced this to 2 minutes and make ZFS update a value stored on the pool once a minute that it is in use. we could update the pool in use flag more often but it seems excessive since its only a courner case anyway. Another possible method to handle this case, but its more work but would not impact existing fast paths, would be for zpool to watch the pool if it appears to have been accessed in the last X minutes and not exported, is to have ZFS watch the devices and see if any other disk commands occur from the old host, if any occurs its obvious that the administrator is putting the system in to a case where it will crash the system, so isn''t it better to fail the import then crash? Any extra delay this check imposes would not break existing specifications because importing devices is not guaranteed fixed performance because ZFS needs to find the devices and verify that all components of pool are in tact and not failed. James> - A pool which is in use on another host but inactive for more than 5 > minutes will fail this check (since no transactions will have been > pushed), but could potentially write data after the pool has been > imported. > > - This breaks existing behavior. The CLI utilities are documented as > commmitted (a.k.a stable), and breaking existing customer scripts > isn''t acceptable.the existing behaviour is broken, ZFS should return EBUSY, if ZFS can determine that the pool is in active use without extreme measures. This is the same behaviour that happens should a user try and mount a filesystem twice on one system. James> > This seems to take the wrong approach to the root problem. Depending on > how you look at it, the real root problem is either: > > a) ZFS is not a clustered filesystem, and actively using the same pool > on multiple systems (even opening said pool) will corrupt data. > > b) ''zpool import'' doesn''t present enough information for an > administrator to reliably determine if a pool is actually in use > on multiple systems. > > The former is obviously a ton of work and something we''re thinking about > but won''t address any time soon. The latter can be addressed by > presenting more useful information when ''zpool import'' is run without > the ''-f'' flag. > > - Eric > > On Wed, Sep 13, 2006 at 12:14:06PM -0500, James Dickens wrote: > > I filed this RFE earlier, since there is no way for non sun personel > > to see this RFE for a while I am posting it here, and asking for > > feedback from the community. > > > > [Fwd: CR 6470231 Created P5 opensolaris/triage-queue Add an inuse > > check that is inforced even if import -f is used.] Inbox > > Assign a GTD Label to this Conversation: [Show] > > Statuses: Next Action, Action, Waiting On, SomeDay, Finished > > Contexts: Car, Desk, Email, Home, Office, Phone, Waiting > > References: ProjectHome, Reference > > Misc.: > > *Synopsis*: Add an inuse check that is inforced even if import -f is used. > > http://bt2ws.central.sun.com/CrPrint?id=6470231 > > > > > > *Change Request ID*: 6470231 > > > > *Synopsis*: Add an inuse check that is inforced even if import -f is used. > > > > Product: solaris > > Category: opensolaris > > Subcategory: triage-queue > > Type: RFE > > Subtype: > > Status: 1-Dispatched > > Substatus: > > Priority: 5-Very Low > > Introduced In Release: > > Introduced In Build: > > Responsible Manager: karyn.ritter at sun.com > > Responsible Engineer: > > Initial Evaluator: opensolaris-bugs-triage at sun.com > > Keywords: opensolaris > > > > === *Description* > > ===========================================================> > Category > > kernel > > Sub-Category > > zfs > > Description > > Currently many people have been trying to import ZFS pools on > > multiple systems at once. currently this is unsupported, and causes > > massive data corruption to the pool. > > If ZFS refuses to import any zfs pool that was used in the last 5 > > minutes, that was not cleanly exported. This prevents the filesystem > > from being mounted on multiple systems at once. > > Frequency > > Always > > Regression > > No > > Steps to Reproduce > > import the same storage pool on more than one machine or domain. > > > > Expected Result > > #zpool import -f datapool1 > > Error: ZFS pool datapool1 is currently imported on another system, and > > was accessed less than 5 minutes ago, ZFS currently does not currently > > support concurrent access. If this filesystem is no longer in use on > > the other system please export the filesystem from the other system or > > try again in 5 minutes. > > > > Actual Result > > #zpool import -f datapool1 > > # > > a few minutes later the system crashes because of concurrent use. > > Error Message(s) > > _______________________________________________ > > zfs-discuss mailing list > > zfs-discuss at opensolaris.org > > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > > -- > Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock >
Eric Schrock
2006-Sep-13 20:48 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
On Wed, Sep 13, 2006 at 02:29:55PM -0500, James Dickens wrote:> > this would not be the first time that Solaris overrided an administive > command, because its just not safe or sane to do so. For example. > > rm -rf /As I''ve repeated before, and will continue to repeat, it''s not actually possible for ZFS to determine whether a pool is in active use (short of making ZFS a cluster-aware filesystem). Adding arbitrary delays doesn''t change this fact, and only less likely. I''ve given you examples of where this behavior is safe and sane and useful, so the above simplification (upon which most of the other arguments are based) isn''t really valid. I''m curious why you didn''t comment on my other suggestion (displaying last accessed host and time as part of ''zpool import''), which seems to solve your problem by giving the administrator the data they need to make an appropriate decision. As Anton and other have mentioned in previous discussions there seems to be several clear RFEs that everyone can agree with: 1. Store the hostid, hostname, and last time written as part of the label. 2. During auto-import (aka open), if the the hostid is different from our own, fault the pool and generate an appropriate FMA event. 3. During manual import, display the last hostname and time accessed if the hostid is not our own, and the pool is still marked ACTIVE. This prevents administrators from shooting themselves in the foot, while still allowing explicit cluster failover to operate with more information than was available before. - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock
James Dickens
2006-Sep-13 21:08 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
On 9/13/06, Eric Schrock <eric.schrock at sun.com> wrote:> On Wed, Sep 13, 2006 at 02:29:55PM -0500, James Dickens wrote: > > > > this would not be the first time that Solaris overrided an administive > > command, because its just not safe or sane to do so. For example. > > > > rm -rf / > > As I''ve repeated before, and will continue to repeat, it''s not actually > possible for ZFS to determine whether a pool is in active use (short of > making ZFS a cluster-aware filesystem). Adding arbitrary delays doesn''t > change this fact, and only less likely. I''ve given you examples of > where this behavior is safe and sane and useful, so the above > simplification (upon which most of the other arguments are based) isn''t > really valid. >I disagree with this, isn''t there way to track when the last read was? Even computing the last write access time that you are already recommending, then sleeping for 30 seconds and recompute the value by accessing the disk again and compare,even a read on the pool will possibly cause a write if ATIME tracking is enabled on the filesystem, if someone is accessing the pool we are importing underneath us, it is a dead give away that we are about to explode if we continue on this path.> I''m curious why you didn''t comment on my other suggestion (displaying > last accessed host and time as part of ''zpool import''), which seems to > solve your problem by giving the administrator the data they need to > make an appropriate decision. >its a good suggestion, it just doesn''t go far enough in my opinion.> As Anton and other have mentioned in previous discussions there seems to > be several clear RFEs that everyone can agree with: > > 1. Store the hostid, hostname, and last time written as part of the > label. > > 2. During auto-import (aka open), if the the hostid is different from > our own, fault the pool and generate an appropriate FMA event. > > 3. During manual import, display the last hostname and time accessed if > the hostid is not our own, and the pool is still marked ACTIVE. > > This prevents administrators from shooting themselves in the foot, while > still allowing explicit cluster failover to operate with more > information than was available before. >if this is what the community decides I can live with this, I may even provide a patch for OpenSolaris distros that does the more intensive check, it seems to be an easy fix once you have complete #1. James> - Eric > > -- > Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock >
Daniel Rock
2006-Sep-13 23:25 UTC
[zfs-discuss] Re: Comments on a ZFS multiple use of a pool, RFE.
Anton B. Rang schrieb:> The hostid solution that VxVM uses would catch this second problem,> because when A came up after its reboot, it would find that -- even > though it had created the pool -- it was not the last machine to access > it, and could refuse to automatically mount it. If the administrator > really wanted it mounted, they could force the issue. Relying on the > administrator to know that they have to remove a file (the > ''zpool cache'') before they let the machine come up out of single-user > mode seems the wrong approach to me. ("By default, we''ll shoot you in > the foot, but we''ll give you a way to unload the gun if you''re fast > enough and if you remember.") I haven''t tried: Does ZFS try to -f (force) import of zpools in /etc/zfs/zpool.cache or does it just do a normal import and fails if the disks seem to be in use elsewhere, e.g. after a reboot of a proably failed and later repaired machine? Just to clear some things up. The OP who started the whole discussion would have had the same problems with VxVM as he has now with ZFS. If you force an import of a disk group on one host while it is still active on another host won''t make the DG magically disappear on the other one. The corresponding flag to "zpool import -f" is "vxdg import -C". If you issue this command you could also end up with the same DG imported on more than one host. Because on VxVM there is usually another level of indirection (volumes ontop of the DG which may contain filesystems and also have to manually mount) just importing a DG is normally harmless. So also with VxVM you can shoot yourself in the foot. On host B B# vxdg -C import DG B# vxvol -g DG startall B# mount /dev/vx/dsk/DG/filesys /some/where B# do_someting on /some/where while still on host A A# do_something on /some/where Instead of a zpool.cache file VxVM uses the hostid (not to be confused with the host-id, normally just the ordinary hostname `uname -n` of the machine) to know which DGs it should mount automatically. Additionally each DG (or more precisely: each disk) has an autoimport flag which also has to be turned on to make the DG autoimported during bootup. So to mimic VxVM in ZFS the solution would simply be: add an autoimport flag to the zpool. Daniel
Frank Cusack
2006-Sep-13 23:43 UTC
[zfs-discuss] Re: Comments on a ZFS multiple use of a pool, RFE.
On September 14, 2006 1:25:01 AM +0200 Daniel Rock <solaris at deadcafe.de> wrote:> Just to clear some things up. The OP who started the whole discussion would have had the same > problems with VxVM as he has now with ZFS. If you force an import of a disk group on one host > while it is still active on another host won''t make the DG magically disappear on the other one.The OP was just showing a test case. On a real system your HA software would exchange a heartbeat and not do a double import. The problem with zfs is that after the original system fails and the second system imports the pool, the original system also tries to import on [re]boot, and the OP didn''t know how to disable this. -frank
Darren J Moffat
2006-Sep-14 09:35 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
James Dickens wrote:> On 9/13/06, Eric Schrock <eric.schrock at sun.com> wrote: >> On Wed, Sep 13, 2006 at 02:29:55PM -0500, James Dickens wrote: >> > >> > this would not be the first time that Solaris overrided an administive >> > command, because its just not safe or sane to do so. For example. >> > >> > rm -rf / >> >> As I''ve repeated before, and will continue to repeat, it''s not actually >> possible for ZFS to determine whether a pool is in active use (short of >> making ZFS a cluster-aware filesystem). Adding arbitrary delays doesn''t >> change this fact, and only less likely. I''ve given you examples of >> where this behavior is safe and sane and useful, so the above >> simplification (upon which most of the other arguments are based) isn''t >> really valid. >> > I disagree with this, isn''t there way to track when the last read was?Track ? Doesn''t that imply a write. Writing is a bad idea because it can cause otherwise spun-down disks to need to be spun-up - which is bad for power consumption and possibly for the disk time to failure. -- Darren J Moffat
James Dickens
2006-Sep-14 10:02 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
On 9/14/06, Darren J Moffat <Darren.Moffat at sun.com> wrote:> James Dickens wrote: > > On 9/13/06, Eric Schrock <eric.schrock at sun.com> wrote: > >> On Wed, Sep 13, 2006 at 02:29:55PM -0500, James Dickens wrote: > >> > > >> > this would not be the first time that Solaris overrided an administive > >> > command, because its just not safe or sane to do so. For example. > >> > > >> > rm -rf / > >> > >> As I''ve repeated before, and will continue to repeat, it''s not actually > >> possible for ZFS to determine whether a pool is in active use (short of > >> making ZFS a cluster-aware filesystem). Adding arbitrary delays doesn''t > >> change this fact, and only less likely. I''ve given you examples of > >> where this behavior is safe and sane and useful, so the above > >> simplification (upon which most of the other arguments are based) isn''t > >> really valid. > >> > > I disagree with this, isn''t there way to track when the last read was? > > Track ? Doesn''t that imply a write. Writing is a bad idea because it > can cause otherwise spun-down disks to need to be spun-up - which is > bad for power consumption and possibly for the disk time to failure. >eric was allready talking about printing the last time a disk was accessed when a disk was about to be imported, my idea would be run that check twice, once initially and if it looks like it could be still in use, like the pool wasn''t exported and last write occurred in the last 30 seconds or a few minutes, it would sleep, then run the check again, it the drive showed up changed ( even if it was just because of an updated atime) the import would fail with EBUSY. The drives would all ready be spun up during all of this because were importing them. James Dickens uadmin.blogspot.com> -- > Darren J Moffat >
Daniel Rock
2006-Sep-14 14:28 UTC
[zfs-discuss] Re: Re: Comments on a ZFS multiple use of a pool, RFE.
> The OP was just showing a test case. On a real system your HA software > would exchange a heartbeat and not do a double import. The problem with > zfs is that after the original system fails and the second system imports > the pool, the original system also tries to import on [re]boot, and the OP > didn''t know how to disable this.Ok, I see. I don''t think he showed that because he didn''t mention a reboot. If you *never* want to import a pool automatically on reboot you just have to delete the /etc/zfs/zpool.cache file before the zfs module is being loaded. This could be integrated into SMF. ZFS should normally be loaded during svc:/system/filesystem/local. Just create a service with filesystem/local as a dependent which removes /etc/zfs/zpool.cache. Like: /lib/svc/method/zpool-clear #!/sbin/sh . /lib/svc/share/smf_include.sh /usr/bin/rm -f /etc/zfs/zpool.cache exit $SMF_EXIT_OK /var/svc/manifest/system/zpool-clear.xml <?xml version="1.0"?> <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type=''manifest'' name=''none:zpool-clear''> <service name=''system/zpool/clear'' type=''service'' version=''1''> <create_default_instance enabled=''true'' /> <single_instance/> <dependency name=''usr'' grouping=''require_all'' restart_on=''none'' type=''service''> <service_fmri value=''svc:/system/filesystem/usr'' /> </dependency> <dependent name=''fs-local'' grouping=''require_all'' restart_on=''none''> <service_fmri value=''svc:/system/filesystem/local'' /> </dependent> <exec_method type=''method'' name=''start'' exec=''/lib/svc/method/zpool-clear'' timeout_seconds=''0'' /> <exec_method type=''method'' name=''stop'' exec='':true'' timeout_seconds=''0'' /> <property_group name=''startd'' type=''framework''> <propval name=''duration'' type=''astring'' value=''transient'' /> </property_group> <template> <common_name> <loctext xml:lang=''C''> Clears the zpool cache file </loctext> </common_name> </template> </service> </service_bundle> (sorry for the poor formating but this !$&%()# web interface is driving me nuts) Then do a "svccfg import /var/svc/manifest/system/zpool-clear.xml". Now /etc/zfs/zpool.cache should disappear on each reboot before the zfs module is loaded. This message posted from opensolaris.org
Anton B. Rang
2006-Sep-14 14:57 UTC
[zfs-discuss] Re: Re: Comments on a ZFS multiple use of a pool, RFE.
> If you *never* want to import a pool automatically on reboot you just have to delete the > /etc/zfs/zpool.cache file before the zfs module is being loaded. > This could be integrated into SMF.Or you could always use import -R / create -R for your pool management. Of course, there''s no way to set a global default for these, so you have to remember it each time, making the SMF solution more attractive.... This message posted from opensolaris.org
Darren Dunham
2006-Sep-14 15:42 UTC
[zfs-discuss] Re: Re: Comments on a ZFS multiple use of a pool,
> > If you *never* want to import a pool automatically on reboot you just have to delete the > > /etc/zfs/zpool.cache file before the zfs module is being loaded. > > This could be integrated into SMF. > > Or you could always use import -R / create -R for your pool management. Of course, there''s no way to set a global default for these, so you have to remember it each time, making the SMF solution more attractive....Perfect. (although I have to try it). In a cluster framework, the cluster can remember to do it each time, so that shouldn''t be an issue. -- Darren Dunham ddunham at taos.com Senior Technical Consultant TAOS http://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. >
Torrey McMahon
2006-Sep-14 16:29 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
James Dickens wrote:> > eric was allready talking about printing the last time a disk was > accessed when a disk was about to be imported, my idea would be run > that check twice, once initially and if it looks like it could be > still in use, like the pool wasn''t exported and last write occurred in > the last 30 seconds or a few minutes, it would sleep, then run the > check again, it the drive showed up changed ( even if it was just > because of an updated atime) the import would fail with EBUSY. The > drives would all ready be spun up during all of this because were > importing them.You''re assuming that the system that owns the pool is up and running at the times you''re making your checks. Or that a certain span of wall clock time is a good indication of what is in use. In any situation where two entities are unclear as to device ownership they go to a third entity. You can say thats a cluster node or that its a somewhat intelligent person. However, trying to work around the issue is going to lead to a disaster at some point. Seen it in action plenty of times. If people want to use ZFS commands to try and get around the issue then have at it but please don''t ask the folks working on ZFS to assemble the gun, purchase the ammo, load the weapon, and point it in the general direction of your lower locomotive extremities. :)
Robert Milkowski
2006-Sep-17 09:29 UTC
[zfs-discuss] Re: Re: Comments on a ZFS multiple use of a pool,
Hello Darren, Thursday, September 14, 2006, 5:42:20 PM, you wrote:>> > If you *never* want to import a pool automatically on reboot you just have to delete the >> > /etc/zfs/zpool.cache file before the zfs module is being loaded. >> > This could be integrated into SMF. >> >> Or you could always use import -R / create -R for your pool management. Of course, there''s no way to set a global default for these, so you have to remember it each time, making the SMF solution more attractive....DD> Perfect. (although I have to try it). In a cluster framework, the DD> cluster can remember to do it each time, so that shouldn''t be an issue. And that''s exactly what SC32 does. -- Best regards, Robert mailto:rmilkowski at task.gda.pl http://milek.blogspot.com
Robert Milkowski
2006-Sep-17 09:42 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
Hello James, I belive that storing hostid, etc. in a label and checking if it matches on auto-import is the right solution. Before it''s implemented you can use -R right now with home-clusters and don''t worry about auto-import. However maybe doing (optional) SCSI reservation on a pool would be a good idea? Of course as an extra switch during/after import. I know not all devices supports it but still. ZFS would allow either reserve all disks in a pool or none. -- Best regards, Robert mailto:rmilkowski at task.gda.pl http://milek.blogspot.com
Richard Elling - PAE
2006-Sep-18 18:29 UTC
[zfs-discuss] Comments on a ZFS multiple use of a pool, RFE.
Robert Milkowski wrote:> Hello James, > > I belive that storing hostid, etc. in a label and checking if it > matches on auto-import is the right solution. > Before it''s implemented you can use -R right now with home-clusters > and don''t worry about auto-import.hostid isn''t sufficient (got a scar), so perhaps several identifying characteristics would work. There is merit to this.> However maybe doing (optional) SCSI reservation on a pool would be a > good idea? Of course as an extra switch during/after import. > I know not all devices supports it but still. ZFS would allow either > reserve all disks in a pool or none.Hopefully we like ourselves too much to try to use SCSI reservations. The road is littered with the dead or bleeding carcases of people who expected SCSI reservations to actually work. Heck, it took over 8 years for even basic (by today''s standards) SCSI functionality to be widely implemented. -- richard