Tim Cook
2006-Dec-22 15:28 UTC
[zfs-discuss] Re: Difference between ZFS and UFS with one LUN froma SAN
This may not be the answer you''re looking for, but I don''t know if it''s something you''ve thought of. If you''re pulling a LUN from an expensive array, with multiple HBA''s in the system, why not run mpxio? If you ARE running mpxio, there shouldn''t be an issue with a path dropping. I have the setup above in my test lab and pull cables all the time and have yet to see a zfs kernel panic. Is this something you''ve considered? I haven''t seen the bug in question, but I definitely have not run into it when running mpxio. --Tim -----Original Message----- From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Shawn Joy Sent: Friday, December 22, 2006 7:35 AM To: zfs-discuss at opensolaris.org Subject: [zfs-discuss] Re: Difference between ZFS and UFS with one LUN froma SAN OK, But lets get back to the original question. Does ZFS provide you with less features than UFS does on one LUN from a SAN (i.e is it less stable).>ZFS on the contrary checks every block it reads and is able to find the >mirror >or reconstruct the data in a raidz config. >Therefore ZFS uses only valid data and is able to repair the datablocks>automatically. >This is not possible in a traditional filesystem/volume manager >configuration.The above is fine. If I have two LUNs. But my original question was if I only have one LUN. What about kernel panics from ZFS if for instance access to one controller goes away for a few seconds or minutes. Normally UFS would just sit there and warn I have lost access to the controller. Then when the controller returns, after a short period, the warnings go away and the LUN continues to operate. The admin can then research further into why the controller went away. With ZFS, the above will panic the system and possibly cause other coruption on other LUNs due to this panic? I believe this was discussed in other threads? I also believe there is a bug filed against this? If so when should we expect this bug to be fixed? My understanding of ZFS is that it functions better in an environment where we have JBODs attached to the hosts. This way ZFS takes care of all of the redundancy? But what about SAN enviroments where customers have spend big money to invest in storage. I know of one instance where a customer has a growing need for more storage space. There environemt uses many inodes. Due to the UFS inode limitation, when creating LUNs over one TB, they would have to quadrulpe the about of storage usesd in there SAN in order to hold all of the files. A possible solution to this inode issue would be ZFS. However they have experienced kernel panics in there environment when a controller dropped of line. Any body have a solution to this? Shawn This message posted from opensolaris.org _______________________________________________ zfs-discuss mailing list zfs-discuss at opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Shawn Joy
2006-Dec-22 16:03 UTC
[zfs-discuss] Re: Difference between ZFS and UFS with one LUN froma SAN
No, I have not played with this. As I do not have access to my customer site. They have tested this themselves. It is unclear if they implemented this on a MPXIO/SSTM device. I will ask this question. Thanks, Shawn Tim Cook wrote:> This may not be the answer you''re looking for, but I don''t know if it''s > something you''ve thought of. If you''re pulling a LUN from an expensive > array, with multiple HBA''s in the system, why not run mpxio? If you ARE > running mpxio, there shouldn''t be an issue with a path dropping. I have > the setup above in my test lab and pull cables all the time and have yet > to see a zfs kernel panic. Is this something you''ve considered? I > haven''t seen the bug in question, but I definitely have not run into it > when running mpxio. > > --Tim > > -----Original Message----- > From: zfs-discuss-bounces at opensolaris.org > [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Shawn Joy > Sent: Friday, December 22, 2006 7:35 AM > To: zfs-discuss at opensolaris.org > Subject: [zfs-discuss] Re: Difference between ZFS and UFS with one LUN > froma SAN > > OK, > > But lets get back to the original question. > > Does ZFS provide you with less features than UFS does on one LUN from a > SAN (i.e is it less stable). > >> ZFS on the contrary checks every block it reads and is able to find the >> mirror >> or reconstruct the data in a raidz config. >> Therefore ZFS uses only valid data and is able to repair the data > blocks >> automatically. >> This is not possible in a traditional filesystem/volume manager >> configuration. > > The above is fine. If I have two LUNs. But my original question was if I > only have one LUN. > > What about kernel panics from ZFS if for instance access to one > controller goes away for a few seconds or minutes. Normally UFS would > just sit there and warn I have lost access to the controller. Then when > the controller returns, after a short period, the warnings go away and > the LUN continues to operate. The admin can then research further into > why the controller went away. With ZFS, the above will panic the system > and possibly cause other coruption on other LUNs due to this panic? I > believe this was discussed in other threads? I also believe there is a > bug filed against this? If so when should we expect this bug to be > fixed? > > > My understanding of ZFS is that it functions better in an environment > where we have JBODs attached to the hosts. This way ZFS takes care of > all of the redundancy? But what about SAN enviroments where customers > have spend big money to invest in storage. I know of one instance where > a customer has a growing need for more storage space. There environemt > uses many inodes. Due to the UFS inode limitation, when creating LUNs > over one TB, they would have to quadrulpe the about of storage usesd in > there SAN in order to hold all of the files. A possible solution to this > inode issue would be ZFS. However they have experienced kernel panics in > there environment when a controller dropped of line. > > Any body have a solution to this? > > Shawn > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-- Shawn Joy Systems Support Specialist Sun Microsystems, Inc. 1550 Bedford Highway, Suite 302 Bedford, Nova Scotia B4A 1E6 CA Phone 902-832-6213 Fax 902-835-6321 Email Shawn.Joy at Sun.COM
Jason J. W. Williams
2006-Dec-22 16:55 UTC
[zfs-discuss] Re: Difference between ZFS and UFS with one LUN froma SAN
Just for what its worth, when we rebooted a controller in our array (we pre-moved all the LUNs to the other controller), despite using MPXIO ZFS kernel panicked. Verified that all the LUNs were on the correct controller when this occurred. Its not clear why ZFS thought it lost a LUN but it did. We have done cable pulling using ZFS/MPXIO before and that works very well. It may well be array-related in our case, but I hate anyone to have a false sense of security. -J On 12/22/06, Tim Cook <tim.cook at qlogic.com> wrote:> This may not be the answer you''re looking for, but I don''t know if it''s > something you''ve thought of. If you''re pulling a LUN from an expensive > array, with multiple HBA''s in the system, why not run mpxio? If you ARE > running mpxio, there shouldn''t be an issue with a path dropping. I have > the setup above in my test lab and pull cables all the time and have yet > to see a zfs kernel panic. Is this something you''ve considered? I > haven''t seen the bug in question, but I definitely have not run into it > when running mpxio. > > --Tim > > -----Original Message----- > From: zfs-discuss-bounces at opensolaris.org > [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Shawn Joy > Sent: Friday, December 22, 2006 7:35 AM > To: zfs-discuss at opensolaris.org > Subject: [zfs-discuss] Re: Difference between ZFS and UFS with one LUN > froma SAN > > OK, > > But lets get back to the original question. > > Does ZFS provide you with less features than UFS does on one LUN from a > SAN (i.e is it less stable). > > >ZFS on the contrary checks every block it reads and is able to find the > >mirror > >or reconstruct the data in a raidz config. > >Therefore ZFS uses only valid data and is able to repair the data > blocks > >automatically. > >This is not possible in a traditional filesystem/volume manager > >configuration. > > The above is fine. If I have two LUNs. But my original question was if I > only have one LUN. > > What about kernel panics from ZFS if for instance access to one > controller goes away for a few seconds or minutes. Normally UFS would > just sit there and warn I have lost access to the controller. Then when > the controller returns, after a short period, the warnings go away and > the LUN continues to operate. The admin can then research further into > why the controller went away. With ZFS, the above will panic the system > and possibly cause other coruption on other LUNs due to this panic? I > believe this was discussed in other threads? I also believe there is a > bug filed against this? If so when should we expect this bug to be > fixed? > > > My understanding of ZFS is that it functions better in an environment > where we have JBODs attached to the hosts. This way ZFS takes care of > all of the redundancy? But what about SAN enviroments where customers > have spend big money to invest in storage. I know of one instance where > a customer has a growing need for more storage space. There environemt > uses many inodes. Due to the UFS inode limitation, when creating LUNs > over one TB, they would have to quadrulpe the about of storage usesd in > there SAN in order to hold all of the files. A possible solution to this > inode issue would be ZFS. However they have experienced kernel panics in > there environment when a controller dropped of line. > > Any body have a solution to this? > > Shawn > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Robert Milkowski
2006-Dec-22 21:18 UTC
[zfs-discuss] Re: Difference between ZFS and UFS with one LUN froma SAN
Hello Jason, Friday, December 22, 2006, 5:55:38 PM, you wrote: JJWW> Just for what its worth, when we rebooted a controller in our array JJWW> (we pre-moved all the LUNs to the other controller), despite using JJWW> MPXIO ZFS kernel panicked. Verified that all the LUNs were on the JJWW> correct controller when this occurred. Its not clear why ZFS thought JJWW> it lost a LUN but it did. We have done cable pulling using ZFS/MPXIO JJWW> before and that works very well. It may well be array-related in our JJWW> case, but I hate anyone to have a false sense of security. Did you first check (with format for example) if LUNs were really accessible? If MPxIO worked ok and at least one path is ok then ZFS won''t panic. -- Best regards, Robert mailto:rmilkowski at task.gda.pl http://milek.blogspot.com
Torrey McMahon
2007-Jan-10 16:26 UTC
[zfs-discuss] Re: Difference between ZFS and UFS with one LUN
Sounds like a bug. What was the panic message? Jason J. W. Williams wrote:> Just for what its worth, when we rebooted a controller in our array > (we pre-moved all the LUNs to the other controller), despite using > MPXIO ZFS kernel panicked. Verified that all the LUNs were on the > correct controller when this occurred. Its not clear why ZFS thought > it lost a LUN but it did. We have done cable pulling using ZFS/MPXIO > before and that works very well. It may well be array-related in our > case, but I hate anyone to have a false sense of security. > > -J > > On 12/22/06, Tim Cook <tim.cook@qlogic.com> wrote: >> This may not be the answer you''re looking for, but I don''t know if it''s >> something you''ve thought of. If you''re pulling a LUN from an expensive >> array, with multiple HBA''s in the system, why not run mpxio? If you ARE >> running mpxio, there shouldn''t be an issue with a path dropping. I have >> the setup above in my test lab and pull cables all the time and have yet >> to see a zfs kernel panic. Is this something you''ve considered? I >> haven''t seen the bug in question, but I definitely have not run into it >> when running mpxio. >> >> --Tim >> >> -----Original Message----- >> From: zfs-discuss-bounces@opensolaris.org >> [mailto:zfs-discuss-bounces@opensolaris.org] On Behalf Of Shawn Joy >> Sent: Friday, December 22, 2006 7:35 AM >> To: zfs-discuss@opensolaris.org >> Subject: [zfs-discuss] Re: Difference between ZFS and UFS with one LUN >> froma SAN >> >> OK, >> >> But lets get back to the original question. >> >> Does ZFS provide you with less features than UFS does on one LUN from a >> SAN (i.e is it less stable). >> >> >ZFS on the contrary checks every block it reads and is able to find the >> >mirror >> >or reconstruct the data in a raidz config. >> >Therefore ZFS uses only valid data and is able to repair the data >> blocks >> >automatically. >> >This is not possible in a traditional filesystem/volume manager >> >configuration. >> >> The above is fine. If I have two LUNs. But my original question was if I >> only have one LUN. >> >> What about kernel panics from ZFS if for instance access to one >> controller goes away for a few seconds or minutes. Normally UFS would >> just sit there and warn I have lost access to the controller. Then when >> the controller returns, after a short period, the warnings go away and >> the LUN continues to operate. The admin can then research further into >> why the controller went away. With ZFS, the above will panic the system >> and possibly cause other coruption on other LUNs due to this panic? I >> believe this was discussed in other threads? I also believe there is a >> bug filed against this? If so when should we expect this bug to be >> fixed? >> >> >> My understanding of ZFS is that it functions better in an environment >> where we have JBODs attached to the hosts. This way ZFS takes care of >> all of the redundancy? But what about SAN enviroments where customers >> have spend big money to invest in storage. I know of one instance where >> a customer has a growing need for more storage space. There environemt >> uses many inodes. Due to the UFS inode limitation, when creating LUNs >> over one TB, they would have to quadrulpe the about of storage usesd in >> there SAN in order to hold all of the files. A possible solution to this >> inode issue would be ZFS. However they have experienced kernel panics in >> there environment when a controller dropped of line. >> >> Any body have a solution to this? >> >> Shawn >> > > >_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Jason J. W. Williams
2007-Jan-10 16:26 UTC
[zfs-discuss] Re: Difference between ZFS and UFS with one
Hi Robert, MPxIO had correctly moved the paths. More than one path to controller A was OK, and one patch to controller A for each LUN was active when controller B was rebooted. I have a hunch that the array was at fault, because it also rebooted a Windows server with LUNs only on Controller A. In the case of the Windows server Engenios RDAC was handling multipathing. Overall, not a big deal, I just wouldn''t trust the array to do a hitless commanded controller failover or firmware upgrade. -J On 12/22/06, Robert Milkowski <rmilkowski@task.gda.pl> wrote:> Hello Jason, > > Friday, December 22, 2006, 5:55:38 PM, you wrote: > > JJWW> Just for what its worth, when we rebooted a controller in our array > JJWW> (we pre-moved all the LUNs to the other controller), despite using > JJWW> MPXIO ZFS kernel panicked. Verified that all the LUNs were on the > JJWW> correct controller when this occurred. Its not clear why ZFS thought > JJWW> it lost a LUN but it did. We have done cable pulling using ZFS/MPXIO > JJWW> before and that works very well. It may well be array-related in our > JJWW> case, but I hate anyone to have a false sense of security. > > Did you first check (with format for example) if LUNs were really > accessible? If MPxIO worked ok and at least one path is ok then ZFS > won''t panic. > > -- > Best regards, > Robert mailto:rmilkowski@task.gda.pl > http://milek.blogspot.com > >_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Torrey McMahon
2007-Jan-10 16:26 UTC
[zfs-discuss] Re: Difference between ZFS and UFS with one LUN
A LUN "going away" should not cause a panic. (The obvious exception being the boot LUN) If mpxio saw the LUN move and everything moved ... then it''s a bug. The panic backtrace will point to the guilty party in any case. Jason J. W. Williams wrote:> Hi Robert, > > MPxIO had correctly moved the paths. More than one path to controller > A was OK, and one patch to controller A for each LUN was active when > controller B was rebooted. I have a hunch that the array was at > fault, because it also rebooted a Windows server with LUNs only on > Controller A. In the case of the Windows server Engenios RDAC was > handling multipathing. Overall, not a big deal, I just wouldn''t trust > the array to do a hitless commanded controller failover or firmware > upgrade. > > -J > > On 12/22/06, Robert Milkowski <rmilkowski@task.gda.pl> wrote: >> Hello Jason, >> >> Friday, December 22, 2006, 5:55:38 PM, you wrote: >> >> JJWW> Just for what its worth, when we rebooted a controller in our >> array >> JJWW> (we pre-moved all the LUNs to the other controller), despite using >> JJWW> MPXIO ZFS kernel panicked. Verified that all the LUNs were on the >> JJWW> correct controller when this occurred. Its not clear why ZFS >> thought >> JJWW> it lost a LUN but it did. We have done cable pulling using >> ZFS/MPXIO >> JJWW> before and that works very well. It may well be array-related >> in our >> JJWW> case, but I hate anyone to have a false sense of security. >> >> Did you first check (with format for example) if LUNs were really >> accessible? If MPxIO worked ok and at least one path is ok then ZFS >> won''t panic. >> >> -- >> Best regards, >> Robert mailto:rmilkowski@task.gda.pl >> http://milek.blogspot.com_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss