i have come across an interesting article at : http://www.anandtech.com/IT/showdoc.aspx?i=2859&p=5 it`s about sata vs. sas/scsi realiability , telling that typical desktop sata drives ".....on average experience an Unrecoverable Error every 12.5 terabytes written or read (EUR of 1 in 1014 bits)." since the 1TB drive is out very soon, this really makes me afraid of data integrity on my backup disks, so the question is: will zfs help detect/prevent such single-bit errors ? i`m somewhat sure, that it will help if i use raid1 setup with ZFS - it`s self healing will detect those single-bit-errors and correct this - but what about single disk setup ? can zfs protect my data from such single-bit-errors with a single drive ? regards roland This message posted from opensolaris.org
On 1/13/07, roland <devzero at web.de> wrote:> i have come across an interesting article at : > > http://www.anandtech.com/IT/showdoc.aspx?i=2859&p=5 > > it`s about sata vs. sas/scsi realiability , telling that typical desktop sata drives > ".....on average experience an Unrecoverable Error every 12.5 terabytes written or read (EUR of 1 in 1014 bits)." > > since the 1TB drive is out very soon, this really makes me afraid of data integrity on my backup disks, so the question is: > > will zfs help detect/prevent such single-bit errors ? >zfs will detect a single bit error, if you are using raid either raidz or mirroring it will fix the error.> i`m somewhat sure, that it will help if i use raid1 setup with ZFS - it`s self healing will detect those single-bit-errors and correct this - but what about single disk setup ? >if you aren''t using mirroring or raidz the error will be detected but won''t be repaired. with the possible exception of metablocks that hold information about the files and disk structures there are multiple copy of these, and they can be used should the error occur in one of those blocks.> can zfs protect my data from such single-bit-errors with a single drive ? >nope.. but it can tell you that it has occurred. James Dickens uadmin.blogspot.com> regards > roland > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
thanks for your infos!> > can zfs protect my data from such single-bit-errors with a single drive ? > > >nope.. but it can tell you that it has occurred.can it also tell (or can i use a tool to determine), which data/file is affected by this error (and needs repair/restore from backup) ? This message posted from opensolaris.org
On 1/13/07, roland <devzero at web.de> wrote:> thanks for your infos! > > > > can zfs protect my data from such single-bit-errors with a single drive ? > > > > >nope.. but it can tell you that it has occurred. > > can it also tell (or can i use a tool to determine), which data/file is affected by this error (and needs repair/restore from backup) ? > >with current versions it prints out the inode of the damaged file, you can use "find -inode xxx" to fnd the bad file there is allready request for enhancement to print the name and path to the bad file. James> This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
On 13-Jan-07, at 11:52 AM, roland wrote:> i have come across an interesting article at : > > http://www.anandtech.com/IT/showdoc.aspx?i=2859&p=5 > > it`s about sata vs. sas/scsi realiability , telling that typical > desktop sata drives > ".....on average experience an Unrecoverable Error every 12.5 > terabytes written or read (EUR of 1 in 1014 bits)." > > since the 1TB drive is out very soon, this really makes me afraid > of data integrity on my backup disks, so the question is: > > will zfs help detect/prevent such single-bit errors ? > > i`m somewhat sure, that it will help if i use raid1 setup with ZFS > - it`s self healing will detect those single-bit-errors and correct > this - but what about single disk setup ? > > can zfs protect my data from such single-bit-errors with a single > drive ?I suppose you could have redundancy between slices. But this is fairly pointless since it won''t protect against the certainty of eventual whole-drive problems. --T> > regards > roland > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
roland wrote:> i have come across an interesting article at : > > http://www.anandtech.com/IT/showdoc.aspx?i=2859&p=5 > > it`s about sata vs. sas/scsi realiability , telling that typical desktop sata drives > ".....on average experience an Unrecoverable Error every 12.5 terabytes written or read (EUR of 1 in 1014 bits)." > > since the 1TB drive is out very soon, this really makes me afraid of data integrity on my backup disks, so the question is:The Hitachi 7K1000 announced recently has a spec of 1 UER in 10^15 bits. This is the typical spec for an enterprise class drive, while consumer class drives tend to be speced at 1 UER in 10^14 bits. Our (Sun) field data shows better results than that, but you might want to stay conservative in your design.> will zfs help detect/prevent such single-bit errors ? > > i`m somewhat sure, that it will help if i use raid1 setup with ZFS - it`s self healing will detect those single-bit-errors and correct this - but what about single disk setup ? > > can zfs protect my data from such single-bit-errors with a single drive ?Yes, but you will need to mirror across slices. This isn''t as bad as it sounds because the majority of failures are not whole drive failures. Given a choice, it is better to have two drives. IIRC, the price range for the Hitachis will be in the $400, so for less than $1000 you can get a mirrored TByte. To put this in perspective, in 1998 that would have cost $1M. Next year, it might be as low as $300 (my observation is that the price of disks asymptotically approaches around $150 before they are EOLed and replaced with higher density models.) -- richard
Richard Elling wrote:> roland wrote: >> i have come across an interesting article at : >> http://www.anandtech.com/IT/showdoc.aspx?i=2859&p=5 >>Can anyone comment on the claims or conclusions of the article itself? It seems to me that they are not always clear about what they are talking about. Many times they say only ''SATA'' and other times ''enterprise SATA'' or ''desktop SATA'' Likwise, somtimes they use the term SAS/SCSI, other times just ''enterprise'' without specifying SAS/SCSI or SATA. I''m not clear on why the interconnect technology would have any affect on the reliability of the mechanics or electronics of the drive? I do beleive that the manufacturer''s could be targeting different customers with the different types of drives, but it''s not clear from that article how Enterprise SATA drives compare to Enterprise SAS/SCSI drives. All I can get from the article for sure is don''t use SATA desktop drives in a server. Is 1 bit out of 10^14 really equal to 1 bit in 12.5TB read? Does that really translate to an 8% chance of a read error while trying to reconstruct a 1TB disk in a 5 disk RAID5 array? Something tells me that someones statistics calculations are off... I thought these problems were much rarer? -Kyle
Kyle McDonald wrote:> Richard Elling wrote: >> roland wrote: >>> i have come across an interesting article at : >>> http://www.anandtech.com/IT/showdoc.aspx?i=2859&p=5 >>> > Can anyone comment on the claims or conclusions of the article itself? > > It seems to me that they are not always clear about what they are > talking about. > > Many times they say only ''SATA'' and other times ''enterprise SATA'' or > ''desktop SATA'' > Likwise, somtimes they use the term SAS/SCSI, other times just > ''enterprise'' without specifying SAS/SCSI or SATA. > > I''m not clear on why the interconnect technology would have any affect > on the reliability of the mechanics or electronics of the drive?The interconnect doesn''t have any affect on the mechanics. I think it is just a market segmentation description. A rather poor one, too.> I do beleive that the manufacturer''s could be targeting different > customers with the different types of drives, but it''s not clear from > that article how Enterprise SATA drives compare to Enterprise SAS/SCSI > drives. All I can get from the article for sure is don''t use SATA > desktop drives in a server. > > Is 1 bit out of 10^14 really equal to 1 bit in 12.5TB read?10^14 bits / 8 bits/byte = 12.5 TBytes.> Does that really translate to an 8% chance of a read error while trying > to reconstruct a 1TB disk in a 5 disk RAID5 array?Yes.> Something tells me that someones statistics calculations are off... I > thought these problems were much rarer?I believe these are rarer, for newer drives at least. Over an expected 5 year lifetime, this error rate may be closer to reality. -- richard
James Dickens wrote:> On 1/13/07, roland <devzero at web.de> wrote: > >> thanks for your infos! >> >> > > can zfs protect my data from such single-bit-errors with a single >> drive ? >> > > >> >nope.. but it can tell you that it has occurred. >> >> can it also tell (or can i use a tool to determine), which data/file >> is affected by this error (and needs repair/restore from backup) ? >> >> > with current versions it prints out the inode of the damaged file, you > can use "find -inode xxx" to fnd the bad file there is allready > request for enhancement to print the name and path to the bad file. >The fix for: 6410433 ''zpool status -v'' would be more useful with filenames http://bugs.opensolaris.org/view_bug.do?bug_id=6410433 went back last night, and will be in snv_57. No more requiring admins to issue a ''find -inode XXX'' silliness for damaged files. eric