>>>> -------- Original Message -------- >>>>On 5/10/07, Al Hopper <al at logical-approach.com> wrote: >>>> >>>>> My personal opinion is that USB is not robust enough under (Open)Solaris >>>>> to provide the reliability that someone considering ZFS is looking for. >>>>> I base this on experience with two 7 port powered USB hubs, each with 4 * >>>>> 2Gb Kingston flash drives, connected via 2 ports to a Solaris (update 3) >>>>> desktop box which runs ZFS on two internal 500Gb drives. I see about 24 >>>>> to 28Mb/Sec (bytes) maximum of bandwidth over each USB bus. One time, >>>>> after disconnecting one hub (to show someone the hub with 4*USB drives) it >>>>> hung the OS and reset the box. A subsequent import of the ZFS volume that >>>>> was disconnected, failed. (Yes it was exported, but failed to import). >>>>> So my take on USB is ... it''s not sufficiently robust - and a USB related >>>>> failure is likely to cause loss of the entire ZFS dataset; i.e., its >>>>> likely to trash more that one drive in a raidz config.I am interested in your this comment on USB. But it seems too general and not helpful to solve problems. Several issues have been mixed together which may not necessarily be USB''s fault. If you believe there is a USB issue, a better practice is to file a bug. And please make sure the problem is reproducible and be detailed in problem description. :-) I play with USB devices a lot and seldom see hotplugging hangs a system. The hang looks very exceptional to me. Could you experiment more with the devices and combinations of filesystem configuration? e.g., if you put on the drives UFS instead of ZFS, would it hang? Is there a way that you can reproduce the hang much more reliably? Another question is if you are using ZFS on USB drives, the system hangs due to non-usb related reason and you reset the box, can data integrity on the USB drives be ensured? Yet another question is if you are using non-USB drives, the system hangs due to whatever reason and you reset the box, can data integrity on the non-USB drives be ensured? And how, by SW or HW? We need to think of the questions and make clear if such kind of data loss is particular to USB or not before coming to a conclusion too quickly.>>>> >>>>> I''d be interested in hearing other opinions on USB connected drives >>>>> under (Open)Solaris .... >>>>Any bus can have errors. USB is nothing particular, just the chance of encountering errors is bigger since USB device is cheap. But isn''t the file system expected to handle possible errors? Thanks, Sophia
On Fri, 11 May 2007, Sophia Li wrote:> >>>> -------- Original Message -------- > >>>>On 5/10/07, Al Hopper <al at logical-approach.com> wrote: > >>>> > >>>>> My personal opinion is that USB is not robust enough under (Open)Solaris > >>>>> to provide the reliability that someone considering ZFS is looking for. > >>>>> I base this on experience with two 7 port powered USB hubs, each with 4 * > >>>>> 2Gb Kingston flash drives, connected via 2 ports to a Solaris (update 3) > >>>>> desktop box which runs ZFS on two internal 500Gb drives. I see about 24 > >>>>> to 28Mb/Sec (bytes) maximum of bandwidth over each USB bus. One time, > >>>>> after disconnecting one hub (to show someone the hub with 4*USB drives) it > >>>>> hung the OS and reset the box. A subsequent import of the ZFS volume that > >>>>> was disconnected, failed. (Yes it was exported, but failed to import). > >>>>> So my take on USB is ... it''s not sufficiently robust - and a USB related > >>>>> failure is likely to cause loss of the entire ZFS dataset; i.e., its > >>>>> likely to trash more that one drive in a raidz config. > > I am interested in your this comment on USB. But it seems too general > and not helpful to solve problems. Several issues have been mixed > together which may not necessarily be USB''s fault. If you believe thereAgreed.> is a USB issue, a better practice is to file a bug. And please make sure > the problem is reproducible and be detailed in problem description. :-)Understood.> I play with USB devices a lot and seldom see hotplugging hangs a system. > The hang looks very exceptional to me. Could you experiment more with > the devices and combinations of filesystem configuration? e.g., if you > put on the drives UFS instead of ZFS, would it hang? Is there a way that > you can reproduce the hang much more reliably?I really can''t experiment with this particular machine, because its my main desktop that drives a 22" and 30" LCDs, has about 18 Gnome workspaces and 80+ windows active. If the Xserver dies it take 10 to 15 minutes just to get everything setup again so that I can get my productive development environment in place. And that is aside from the ZFS mirrored pool on the machine that has 45+ filesystems and 6 zones defined. So - experimenting with it is not possible for now. ... more below ...> Another question is if you are using ZFS on USB drives, the system hangs > due to non-usb related reason and you reset the box, can data integrity > on the USB drives be ensured? > > Yet another question is if you are using non-USB drives, the system > hangs due to whatever reason and you reset the box, can data integrity > on the non-USB drives be ensured? And how, by SW or HW? > > We need to think of the questions and make clear if such kind of data > loss is particular to USB or not before coming to a conclusion too quickly.The only conclusion I''ve reached is that attaching 6 or 8 750Gb disk drives via USB for use as ZFS pool is not a good idea - because USB is not robust enough to guarantee, that in the event of a USB failure or "event", that no more than one disk configured in a raidz configuration will be negatively impacted. If more than one disk drive in a raidz storage pool is negatively impacted, then you''ll loose the entire pool. USB is not an appropriate bus to support that type of usage scenario IMHO. USB is fine as a demonstrator of ZFS capabilities by connecting multiple USB flash drives (as I''ve done) and can also be used as a way to archive files reliably on a removable media (the flash drives). And using ZFS with flash drives solves the problem of corrupted or bad sectors on low-cost flash drives - which may or may not have been 100% tested before they were sold.> > >>>> > >>>>> I''d be interested in hearing other opinions on USB connected drives > >>>>> under (Open)Solaris .... > >>>> > > Any bus can have errors. USB is nothing particular, just the chance of > encountering errors is bigger since USB device is cheap. But isn''t theEvery bus topology has appropriate and inappropriate uses - and USB is no exception to that rule.> file system expected to handle possible errors?see above. Regards, Al Hopper Logical Approach Inc, Plano, TX. al at logical-approach.com Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007 http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/
Al Hopper wrote:> I really can''t experiment with this particular machine, because its my > main desktop that drives a 22" and 30" LCDs, has about 18 Gnome workspaces > and 80+ windows active. If the Xserver dies it take 10 to 15 minutes just > to get everything setup again so that I can get my productive development > environment in place. And that is aside from the ZFS mirrored pool on the > machine that has 45+ filesystems and 6 zones defined. So - experimenting > with it is not possible for now. ... more below ... >Ok, I can understand.>>Another question is if you are using ZFS on USB drives, the system hangs >>due to non-usb related reason and you reset the box, can data integrity >>on the USB drives be ensured? >> >>Yet another question is if you are using non-USB drives, the system >>hangs due to whatever reason and you reset the box, can data integrity >>on the non-USB drives be ensured? And how, by SW or HW? >> >>We need to think of the questions and make clear if such kind of data >>loss is particular to USB or not before coming to a conclusion too quickly. > > > The only conclusion I''ve reached is that attaching 6 or 8 750Gb disk > drives via USB for use as ZFS pool is not a good idea - because USB is not > robust enough to guarantee, that in the event of a USB failure or "event", > that no more than one disk configured in a raidz configuration will be > negatively impacted. If more than one disk drive in a raidz storage > pool is negatively impacted, then you''ll loose the entire pool. USB is > not an appropriate bus to support that type of usage scenario IMHO. > > USB is fine as a demonstrator of ZFS capabilities by connecting multiple > USB flash drives (as I''ve done) and can also be used as a way to archive > files reliably on a removable media (the flash drives). And using ZFS > with flash drives solves the problem of corrupted or bad sectors on > low-cost flash drives - which may or may not have been 100% tested before > they were sold.So do you think it is a limitation native to USB bus or it is caused by Solaris defects? I don''t play much with other types of disks. Why are other types of disks robust than USB? Is it ensured by different hardware technology? Thanks, Sophia