Hi, Using the opensolaris installer I''ve created a raidz array from two 500GB hdds, but the installer keeps seening two hdds, not the array I''ve just made. How do I install opensolaris on raidz array? Thanks! This message posted from opensolaris.org
Alex wrote:> Hi, > > Using the opensolaris installer I''ve created a raidz array from two 500GB hdds, but the installer keeps seening two hdds, not the array I''ve just made. > How do I install opensolaris on raidz array? >You don''t. Today, only simple or mirrored vdevs are usable for ZFS boot devices. -- richard
On 02 June, 2008 - Richard Elling sent me these 0,5K bytes:> Alex wrote: > > Hi, > > > > Using the opensolaris installer I''ve created a raidz array from two > > 500GB hdds, but the installer keeps seening two hdds, not the array > > I''ve just made. How do I install opensolaris on raidz array? > > > > You don''t. Today, only simple or mirrored vdevs are > usable for ZFS boot devices.A two disk raidz has no advantages over a two disk mirror, but it does have disadvantages (slower and you can''t boot from it ;) /Tomas -- Tomas ?gren, stric at acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Ume? `- Sysadmin at {cs,acc}.umu.se
What do you mean about "mirrored vdevs" ? RAID1 hardware? Because I have only ICH9R and opensolaris doesn''t know about it. Would be network boot a good idea? This message posted from opensolaris.org
> What do you mean about "mirrored vdevs" ? RAID1 > hardware? Because I have only ICH9R and opensolaris > doesn''t know about it.No, he means a mirror created by zfs. This message posted from opensolaris.org
He means that you can have two types of pool as your root pool: 1. A single physical disk. 2. A ZFS mirror. Usually this means 2 disks. RAIDZ arrays are not supported as root pools (at the moment). Cheers Andrew. This message posted from opensolaris.org
Andrew sez: ...> RAIDZ arrays are not supported as root pools (at the > moment). > > Cheers > > Andrew.I appreciate that this is quite a substantial work. Just off the top of my head, each member of the RAIDZ has to have the same boot block information, then as you bring the RAIDZ up you have to make sure you have sufficient spindles in the group to boot etc etc. In short, not easy. But still, is there any estimate, WAG, SWAG or target for the ability to boot from a RAIDZ? Oh, they should also fix Thumper and Thumper2 to have 2 slots for mirrored OS away from the big honking storage. Cheers, This message posted from opensolaris.org
John Kotches wrote:> Oh, they should also fix Thumper and Thumper2 to have 2 slots for mirrored OS away from the big honking storage. >In general, I agree. However, the data does not necessarily support this as a solution and there is a point of diminishing return. Years ago, disk MTBFs were on the order of 250,000 hours. Today, enterprise class disks, like the ones Sun sells, are on the order of 1.2-1.6 Million hours. High quality CFs are on the order of 3-4 Million hours. What difference does that make? A relatively easy way to get the intuition is to use annual failure rate (AFR) instead of MTBF. I use AFR for 24x7x365 hours per year. Single device: 250,000 hours MTBF = 3.5% AFR 1,500,000 hours MTBF = 0.584% AFR 3,000,000 hours MTBF = 0.292% AFR 4,000,000 hours MTBF = 0.219% AFR OK, so we can immediately see that the newer systems is 12x-15x more reliable than the systems we got burned by a few years ago. But suppose we mirror them. For a redundant system we want to worry about the unscheduled mean time between system interruptions, because that is when we feel pain. We call that the annual system interruption rate (ASIR). This is not the AFR because we can suffer a disk failure, and fix it, without causing an interruption. Or, another way to look at it, for single disks, ASIR == AFR. For these calculations, I''ll use a 40 hour MTTR so that we won''t get interrupted during a hot date on Friday night :-) Mirrored device: 250,000 hours MTBF -> 0.0014% ASIR 1,500,000 hours MTBF -> 0.000039% ASIR 3,000,000 hours MTBF -> 0.0000097% ASIR [*] 4,000,000 hours MTBF -> 0.0000055% ASIR [*] at this point, you might have more success buying California SuperLOTTO lottery tickets once per week and winning the big prize once per year. So the math proves that you can still get much better ASIR for the mirrored disk case. But at some point you won''t worry about it any more. I like to refer to this mental phenomenon with the axiom "reliability trumps availability." Basically, once a disk becomes reliable enough, I won''t worry about mirroring it. If you look at the AFR numbers you''ll note that things have improved significantly already, and you know that there is some point between 3.5% AFR and 0.0014% ASIR that probably represents your comfort level. If that point is 0.3% AFR, then a single CF boot disk is a reasonable design decision. But there is more to the story. The typical MTBF model, like used above, assumes that the disk actually breaks such that it ceases to function. What we find in practice is that you will be much more likely to lose data than have the disk completely fail. This failure mode exists in CFs too, but is rarely characterized in a useful manner. A good solution with ZFS is to set the copies property, zfs set copies=2. For disks or CFs, this will decrease the annualized data loss percentage by a few orders of magnitude. It won''t quite reach the comfort level of mirroring across devices, but it will be much better than having only a single copy of the data. Though I may not always advocate redundant devices, I almost always advocate data redundancy :-) Besides, you can always mirror someplace else... any writable, bootable device will do. -- richard
Relling sez:> In general, I agree. However, the data does not > necessarily support > this as a solution and there is a point of > diminishing return.I sent a reply via e-mail to Richard as well; I basically said something along these lines... You missed my point though. It''s nothing at all to do with the MTBF, and everything to do with keeping the 48 drives for a series of symmetric arrays. Never mind that you really don''t need 1TB drives for the OS ;-) So a nice pair of 73GB or 146GB drives via SAS would be nice. You could do this yourself on a SAS add-on card but it defeats the purpose. I should point out that MTBFs are not reliable on small groupings of disks and a mirrored pair is definitely a small grouping :-) This message posted from opensolaris.org
John Kotches wrote:> Relling sez: > >> In general, I agree. However, the data does not >> necessarily support >> this as a solution and there is a point of >> diminishing return. >> > > I sent a reply via e-mail to Richard as well; I basically said something along these lines... > > You missed my point though. It''s nothing at all to do with the MTBF, and everything to do with keeping the 48 drives for a series of symmetric arrays. Never mind that you really don''t need 1TB drives for the OS ;-) >You missed my point. A CF for boot is a fairly good design, but it is questionable whether mirrored CFs is worth it. Set copies=2 and be happy. -- richard
Err... I think you guys are talking cross purposes here. Either I''m missing the point completely, or are you not aware that the Thumper2 includes a flash storage slot so the OS doesn''t use any of the 48 disks any more? No, it''s not mirrored, but Richard was trying to point out that it''s likely Sun felt the benefit of mirroring that CF drive would be minimal. This message posted from opensolaris.org
>>>>> "r" == Ross <myxiplx at hotmail.com> writes:r> the benefit of mirroring that CF drive would be minimal. rather short-sighted. What if you want to replace the CF with a bigger or faster one without shutting down? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080720/32189afa/attachment.bin>
On Sun, 20 Jul 2008, Miles Nordin wrote:>>>>>> "r" == Ross <myxiplx at hotmail.com> writes: > > r> the benefit of mirroring that CF drive would be minimal. > > rather short-sighted. What if you want to replace the CF with a > bigger or faster one without shutting down?Assuming that you are using zfs root, you just snapshot the filesystem and send it to some other system where you build the replacement CF card. Of course the bit of data which changes before the CF card is replaced will be lost unless you take special care. A shutdown is required in order to replace the card. Presuming that the card is easily reached, a tech should be able to swap it out in a few minutes. Regardless, I can''t imagine any reason why you would want to install a larger or faster card. Ideally the card should be just big enough to serve the purpose since larger cards will be less reliable. The boot/root filesystems should be fairly static. The only time you should notice card performance is when the system is booting. Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/