The host for this is up in the air. I''d hope I could use a Shuttle XPC. It''s an 8 drive USB enclosure. The total bandwidth to all 8 drives would be 480Mbps, which is fine for me. I was hoping to do a RAID-Z or RAID-Z2. I would have it export the drives as JBOD. http://fwdepot.com/thestore/product_info.php/products_id/1623 If I get my math right and understand ZFS... If I have 8x750 gig disks using RAID-Z I will have 7x750 space available. This will provide standard RAID-5 style redundancy. (6x750 for RAID-Z2) My confusion is that I see a lot of people suggesting cutting it up for mirrored RAID-Z setups. The main reason for this is performance right? I am fine with standard USB 2.0 performance and RAID-5 (or 6) functionality but with the added capability of data integrity checking/self-healing/etc/etc. I don''t need to build some crazy array. Just something for SOHO use, sharing files over samba to a couple Windows machines + a media player. Side note: Is this right? "ditto" blocks are extra parity blocks stored on the same disk (won''t prevent total disk failures, but could provide data recovery if enough parity is available) Thanks, mike
On Thu, 10 May 2007, mike wrote:> The host for this is up in the air. I''d hope I could use a Shuttle XPC. > > It''s an 8 drive USB enclosure. The total bandwidth to all 8 drives > would be 480Mbps, which is fine for me. I was hoping to do a RAID-Z or > RAID-Z2. I would have it export the drives as JBOD. > > http://fwdepot.com/thestore/product_info.php/products_id/1623My personal opinion is that USB is not robust enough under (Open)Solaris to provide the reliability that someone considering ZFS is looking for. I base this on experience with two 7 port powered USB hubs, each with 4 * 2Gb Kingston flash drives, connected via 2 ports to a Solaris (update 3) desktop box which runs ZFS on two internal 500Gb drives. I see about 24 to 28Mb/Sec (bytes) maximum of bandwidth over each USB bus. One time, after disconnecting one hub (to show someone the hub with 4*USB drives) it hung the OS and reset the box. A subsequent import of the ZFS volume that was disconnected, failed. (Yes it was exported, but failed to import). So my take on USB is ... it''s not sufficiently robust - and a USB related failure is likely to cause loss of the entire ZFS dataset; i.e., its likely to trash more that one drive in a raidz config. I''d be interested in hearing other opinions on USB connected drives under (Open)Solaris ....> If I get my math right and understand ZFS... If I have 8x750 gig disks > using RAID-Z I will have 7x750 space available. This will provide > standard RAID-5 style redundancy. (6x750 for RAID-Z2)Correct.> My confusion is that I see a lot of people suggesting cutting it up > for mirrored RAID-Z setups. The main reason for this is performance > right? I am fine with standard USB 2.0 performance and RAID-5 (or 6)Usually, because you want to take advantage of the operational characteristics of different storage topologies to meet different end uses. For example, on one server here, with 10 SATA disk drives, they are configured as: - a 5-way raidz - a 3-way zfs mirror - a 2-way zfs mirror The 5-way raidz is ideal when you want to re-assemble a DVD image after downloading 5 *.zip components - and other operations requiring large file, mainly sequencial access type, work patterns. The 3-way mirror is ideal for software development activity and where data reliability is of paramount importance. The 2-way mirror is a good general purpose all-arounder. But there is "overlap", in terms of usability/applicability, between all these configurations. But ZFS allows you to easily setup different configs and evaluate their operational characteristics based on *your* usage scenario. That is one of the beautiful characteristics of the well engineered and highly user friendly ZFS human interface.> functionality but with the added capability of data integrity > checking/self-healing/etc/etc. I don''t need to build some crazy array. > Just something for SOHO use, sharing files over samba to a couple > Windows machines + a media player.Suggestion - try two 4-way raidz pools.> Side note: Is this right? "ditto" blocks are extra parity blocks > stored on the same disk (won''t prevent total disk failures, but could > provide data recovery if enough parity is available)Yes. See Richard Ellings'' excellent blog titled "ZFS, copies, and data protection", where one picture is truely worth 1,000 words. Regards, Al Hopper Logical Approach Inc, Plano, TX. al at logical-approach.com Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007 http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/
> > Side note: Is this right? "ditto" blocks are extra parity blocks > > stored on the same disk (won''t prevent total disk failures, but could > > provide data recovery if enough parity is available) > > Yes. See Richard Ellings'' excellent blog titled "ZFS, copies, and data > protection", where one picture is truely worth 1,000 words.I wouldn''t refer to them as "extra parity blocks". They''re separate copies of the data. The pool that contains the filesystem may or may not be using parity. -- Darren Dunham ddunham at taos.com Senior Technical Consultant TAOS http://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. >
thanks for the reply. On 5/10/07, Al Hopper <al at logical-approach.com> wrote:> My personal opinion is that USB is not robust enough under (Open)Solaris > to provide the reliability that someone considering ZFS is looking for. > I base this on experience with two 7 port powered USB hubs, each with 4 * > 2Gb Kingston flash drives, connected via 2 ports to a Solaris (update 3) > desktop box which runs ZFS on two internal 500Gb drives. I see about 24 > to 28Mb/Sec (bytes) maximum of bandwidth over each USB bus. One time, > after disconnecting one hub (to show someone the hub with 4*USB drives) it > hung the OS and reset the box. A subsequent import of the ZFS volume that > was disconnected, failed. (Yes it was exported, but failed to import). > So my take on USB is ... it''s not sufficiently robust - and a USB related > failure is likely to cause loss of the entire ZFS dataset; i.e., its > likely to trash more that one drive in a raidz config.this is exactly the kind of feedback i was hoping for. i''m wondering if some people consider firewire to be better in opensolaris? my goal is not for hot-pluggable storage; i''m fine with taking the machine down to replace drives, etc. i don''t intend on disconnecting the USB cables or powering off the enclosure on purpose - but of course random unplanned events can happen, which is one of the reasons i want to use ZFS to begin with :)> I''d be interested in hearing other opinions on USB connected drives > under (Open)Solaris ....me too!> Suggestion - try two 4-way raidz pools.wouldn''t that bring usable space down to 2 pairs of 3x750? can those be combined into a single filesystem (for a total of 6x750 usable, but underlying would actually be the equivalent of 2 4x750 raid5 setups) why would you suggest this? would it improve some performance rather than having a raid-z2 on the 8 disks instead? thanks again!
mike wrote:> this is exactly the kind of feedback i was hoping for. > > i''m wondering if some people consider firewire to be better in opensolaris?I''ve written some about a 4-drive Firewire-attached box based on the Oxford 911 chipset, and I''ve had I/O grind to a halt in the face of media errors - see bugid 6539587. I haven''t played with USB drives enough to trust them more, but this was a hole I fell in with Firewire. I''ve had fabulous luck with a Firewire attached DVD burner, though. Rob T
mike wrote:> thanks for the reply. > > On 5/10/07, Al Hopper <al at logical-approach.com> wrote: > >> Suggestion - try two 4-way raidz pools. > > wouldn''t that bring usable space down to 2 pairs of 3x750? > > can those be combined into a single filesystem (for a total of 6x750 > usable, but underlying would actually be the equivalent of 2 4x750 > raid5 setups) > > why would you suggest this? would it improve some performance rather > than having a raid-z2 on the 8 disks instead? >You should run some tests with your workload, with ZFS it is very quick and easy to create and destroy pools, so you can evaluate many combinations in a short time. For my workload, four 2-way mirrors gave me the best redundancy/performance tradeoff. Ian
Robert Thurlow wrote:> I''ve written some about a 4-drive Firewire-attached box based on the > Oxford 911 chipset, and I''ve had I/O grind to a halt in the face of > media errors - see bugid 6539587. I haven''t played with USB drives > enough to trust them more, but this was a hole I fell in with Firewire. > I''ve had fabulous luck with a Firewire attached DVD burner, though.6539587 does not seem to be visible on the opensolaris bugs database. :-/ -Manoj