Which consumer-priced 1.5TB drives do people currently recommend? I had zero read/write/checksum errors so far in 2 years with my trusty old Western Digital WD7500AAKS drives, but now I want to upgrade to a new set of drives that are big, reliable and cheap. As of Jan 2010 it seems the price sweet spot is the 1.5TB drives. As I had a lot of success with Western Digital drives I thought I would stick with WD. However, this time I might have to avoid Western Digital (see below), so I wondered which other recent drives people have found to be decent drives. WD15EADS: The model I was looking at was the WD15EADS. The older 4-platter WD15EADS-00R6B0 revision seems to work OK, from what I found, but I prefer fewer platters from noise, vibration, heat & reliability perspectives. The newer 3-platter WD15EADS-00P8B0 revision seems to have serious problems - see links below. WD15EARS: Also, very recently WD brought out a 3-platter WD15EARS-00Z5B1 revision, based on ''Advanced format'' where it uses 4KB sector sizes instead of the old traditional 512 byte sector sizes. Again, these drives seem to have serious issues - see links below. Does ZFS handle this new 4KB sector size automatically and transparently, or does something need to be done for it work? Reference: 1. On synology site, seems like older 4-platter 1.5TB EADS OK (WD15EADS-00R6B0), but newer 3 platter EADS have problems (WD15EADS-00P8B0): http://forum.synology.com/enu/viewtopic.php?f=151&t=19131&sid=c1c446863595a5addb8652a4af2d09ca 2. A mac user has problems with WD15EARS-00Z5B1: http://community.wdc.com/t5/Desktop/WD-1-5TB-Green-drives-Useful-as-door-stops/td-p/1217/page/2 (WD 1.5TB Green drives - Useful as door stops) http://community.wdc.com/t5/Desktop/WDC-WD15EARS-00Z5B1-awful-performance/m-p/5242 (WDC WD15EARS-00Z5B1 awful performance) Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
We''re in the process of upgrading our storage servers from Seagate RE.2 500 GB and WD 500 GB "black" drives to WD 1.5 TB "green" drives (ones with 512B sectors). So far, no problems to report. We''ve replaced 6 out of 8 drives in one raidz2 vdev so far (1 drive each weekend). re-silver times have dropped from over 80 hours for the first drive to just under 60 for the 6th (pool is 10TB with <150 GB free). No checksum errors of any kind reported so far, no drive timeouts reported by the controller, everything is working as per normal. We''re running ZFSv13 on FreeBSD 7.2-STABLE. -- This message posted from opensolaris.org
Which drive model/revision number are you using? I presume you are using the 4-platter version: WD15EADS-00R6B0, but perhaps I am wrong. Also did you run WDTLER.EXE on the drives first, to hasten error reporting times? -- This message posted from opensolaris.org
I''ve been having good luck with Samsung "green" 1.5TB drives. I have had 1 DOA, but I currently have 10 of them, so that''s not so bad. In that size purchase, I''ve had one bad from just about any manufacturer. I''ve avoided WD for RAID because of the error handling stuff kicking drives out of arrays. I don''t know if that''s currently an issue though. And with Seagate''s recent record, I didn''t feel confident in their larger drives. I was concerned about the 5400RPM speed being a problem, but I can read over 100MB/s from the array, and 95% on my use is over a gigabit LAN, so they are more than fast enough for my needs. I just set up a new array with them, 6 in raidz2. The replacement time is high enough that I decided the extra parity was worth the cost, even for a home server. I need 2 more drives, then I''ll migrate my other 4 from the older array over as well into another 6 drive raidz2 and add it to the pool. I have decided to treat HDDs as completely untrustworthy. So when I get new drives I test them by creating a temporary pool in a mirror config and filling the drives up by copying data from the primary array. Then do a scrub. When it''s done, if you get no errors, and no other errors in dmesg, then wait a week or so and do another scrub test. I found a bad SATA hotswap backplane and a bad drive this way. There are probably faster ways, but this works for me. -- This message posted from opensolaris.org
Good to hear about the Samsungs working for you. Which model/revision are you using? I think your cautious method of not trusting drives, and making an initial trial mirror is a good one, and I might well do what you''ve done for the next batch I buy. I also use RAID-Z2 vdevs and it feels a lot safer than having only single parity. -- This message posted from opensolaris.org
HD154UI/1AG01118 They have been great drives for a home server. Enterprise users probably need faster drives for most uses, but they work great for me. -- This message posted from opensolaris.org
Thanks. Newegg shows quite a good customer rating for that drive: 70% rated it with 5 stars, and 11% with four stars, with 240 ratings. Seems like some people have complained about them sleeping - presumable to save power, although others report they don''t, so I''ll need to look into that more. Did yours sleep? Also, someone reported some issues with smartctl and understanding some of the attributes. Does checking your drive temperatures using smartctl work? Like with this script: http://breden.org.uk/2008/05/16/home-fileserver-drive-temps/ -- This message posted from opensolaris.org
I see also that Samsung have very recently released the HD203WI 2TB 4-platter model. It seems to have good customer ratings so far at newegg.com, but currently there are only 13 reviews so it''s a bit early to tell if it''s reliable. Has anyone tried this model with ZFS? Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
Constantin Gonzalez
2010-Jan-20 13:42 UTC
[zfs-discuss] Best 1.5TB drives for consumer RAID?
Hi,
I''m using 2 x 1.5 TB drives from Samsung (EcoGreen, I believe) in my
current
home server. One reported 14 Read errors a few weeks ago, roughly 6 months after
install, which went away during the next scrub/resilver.
This remembered me to order a 3rd drive, a 2.0 TB WD20EADS from Western Digital
and I now have a 3-way mirror, which is effectively a 2-way mirror with its
hot-spare already synced in.
The idea behind notching up the capacity is threefold:
- No "sorry, this disk happens to have 1 block too few" problems on
attach.
- When the 1.5 TB disks _really_ break, I''ll just order another 2 TB
one and
use the opportunity to upgrade pool capacity. Since at least one of the 1.5TB
drives will still be attached, there won''t be any "slightly
smaller drive"
problems either when attaching the second 2TB drive.
- After building in 2 bigger drives, it becomes easy to figure out which of the
drives to phase out. Just go for the smaller drives. This solves the headache
of trying to figure out the right drive to build out when you replace drives
that aren''t hot spares and don''t have blinking lights.
Frankly, I don''t care whether the Samsung or the WD drives are better
or worse,
they''re both consumer drives and they''re both dirt cheap. Just
assume that
they''ll break soon (since you''re probably using them more
intensely than their
designed purpose) and make sure their replacements are already there.
It also helps mixing vendors, so one glitch that affect multiple disks in the
same batch won''t affect your setup too much. (And yes, I broke that
rule with
my initial 2 Samsung drives but I''m now glad I have both vendors :)).
Hope this helps,
Constantin
Simon Breden wrote:> I see also that Samsung have very recently released the HD203WI 2TB
4-platter model.
>
> It seems to have good customer ratings so far at newegg.com, but currently
there are only 13 reviews so it''s a bit early to tell if it''s
reliable.
>
> Has anyone tried this model with ZFS?
>
> Cheers,
> Simon
>
> http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/
--
Sent from OpenSolaris, http://www.opensolaris.org/
Constantin Gonzalez Sun Microsystems GmbH, Germany
Principal Field Technologist http://blogs.sun.com/constantin
Tel.: +49 89/4 60 08-25 91 http://google.com/search?q=constantin+gonzalez
Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel
Vorsitzender des Aufsichtsrates: Martin Haering
Hi Constantin, It''s good to hear your setup with the Samsung drives is working well. Which model/revision are they? My personal preference is to use drives of the same model & revision. However, in order to help ensure that the drives will perform reliably, I prefer to do a fair amount of research first, in order to find drives that are reported by many users to be working reliably in their systems. I did this for my current WD7500AAKS drives and have never seen even one read/write or checksum error in 2 years - they have worked flawlessly. As a crude method of checking reliability of any particular drive, I take a look at newegg.com and see the percentage of users rating the drives with 4 or 5 stars, and read the problems listed to see what kind of problems the drives may have. If you read the WDC links I list in the first post above, there does appear to be some problem that many users are experiencing with the most recent revisions of the WD Green ''EADS'' drives and also the new Green models in the ''EARS'' range. I don''t know the cause of the problem though. I did wonder if the problems people are experiencing might be caused by spindown/power-saving features of the drives, which might cause a long delay before data is accessible again after spin-up, but this is just a guess. For now, I am looking at the 1.5TB Samsung HD154UI (revision 1AG01118 ?), or possibly the 2TB Samsung HD203WI when more user ratings are available. Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
Well, I''ve purchased 5 Barracuda LP 1.5TB. They ran very queit, cool, 5 in a cage and the vibration are nearly zero. reliability ? Well every HDD is unreliable, every major brand at this time have problems, so go for the best bang for the bucks. In my country Seagate have the best RMA service, with tournaround in about 1 week or so, WD is 3-4 weeks. Samsung have no direct RMA service, Hitachi well have a foot out HDD business IMHO, no attractive product at moment. The enterprise SATA class HDD is a joke, same costructions like the consumers line only longer warranties but with a helfy money premium. If you need of a real enterprise class HDD you want a SAS not a SATA. -- This message posted from opensolaris.org
Interesting question. The answer I came to, perhaps through lack of information and experience, is that there isn''t a best 1.5tb drive. I decided that 1.5tb is too big, and that it''s better to use more and smaller devices so I could get to raidz3. The reasoning came after reading the case for triple-parity raid. The curves showing time to failure versus time to resilver a single lost drive. Time to failure will remain constant-ish, while time to resilver will increase as the number of bits inside a single drive increases, largely because the input/output bandwidth is going to increase only very slowly. The bigger the number of bits in a single drive compared to the time to write a new, full disk worth of bits, the bigger the window for a second-drive failure. Hence, the third parity version is desirable. In general, more drives of smaller capacity within reason for a vdev, the less exposure to a double fault. This led me to look at sub-terabyte drives, and that''s how I accidentally found those 0.75GB raid-rated drives, although the "raid rated" wasn''t what I was looking for. I was after the best cost/bit in a six-drive batch with a top cost limit. After reading through the "best practices" stuff, I clumsily decided that a six- or seven-drive raidz3 would be a good idea. And I have a natural leaning to stay !OFF! the leading edge of technology where keeping data reliable is involved. It''s a personal quirk I learned by getting several scars to remind me. How''s that for a mismash of misunderstanding? 8-) -- This message posted from opensolaris.org
<p>I just took a look at customer feedback on this drive here. 36% rate with one star, which I would consider alarming. Take a look here, ordered from lowest rating to highest rating. Note the recency of the comments and the descriptions:</p> <a href="http://www.newegg.com/Product/ProductReview.aspx?Item=22-148-412&SortField=3&SummaryType=0&Pagesize=10&SelectedRating=-1&PurchaseMark=&VideoOnlyMark=False&VendorMark=&Page=1&Keywords=%28keywords%29">Seagate Barracuda LP ST31500541AS 1.5TB 5900 RPM</a> <p>Is this the model you mean? If so, I might look at some other alternative possibilities.</p> <p>So, we have apparently problematic newest revision WD Green ''EADS'' and ''EARS'' models, and an apparently problematic Seagate model described here.</p> <p>That leaves Hitachi and Samsung.</p> <p>I had past ''experiences'' with post IBM ''deathstar'' Hitachi drives, so I think for now I shall be looking into the Samsungs, as from the customer reviews it seems these could be the most reliable consumer-priced high-capacity drives available right now.</p> <p>It does seem that it is proving to be a big challenge for the drive manufacturers to produce reliable high-capacity consumer-priced drives. Maybe this is Samsung''s opportunity to prove how good they are?</p> <a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16822152175&Tpk=HD154UI">Samsung 1.5TB HD154UI 3-platter drive</a> <a href="http://www.newegg.com/Product/Product.aspx?Item=N82E16822152202&Tpk=HD203WI">Samsung 2TB HD203WI 4-platter drive</a> -- This message posted from opensolaris.org
On Sat, 23 Jan 2010, R.G. Keen wrote:> The reasoning came after reading the case for triple-parity raid. > The curves showing time to failure versus time to resilver a single > lost drive. Time to failure will remain constant-ish, while time to > resilver will increase as the number of bits inside a single drive > increases, largely because the input/output bandwidth is going to > increase only very slowly. The bigger the number of bits in a single > drive compared to the time to write a new, full disk worth of bits, > the bigger the window for a second-drive failure. Hence, the third > parity version is desirable.Resilver time is definitely important criteria. Besides the number of raw bits to transfer from the drive, you will also find that today''s super-capacity SATA drives rotate more slowly, which increases access times. Since resilver is done in (roughly) the order that data was written, access time will be important to resilver times. A pool which has "aged" due to many snapshots, file updates, and file deletions, will require more seeks. The smaller drives are more responsive so their improved access time will help reduce resilver times. In other words, I think that you are making a wise choice. :-) Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
In general I agree completely with what you are saying. Making reliable large capacity drives does appear to have become very difficult for the drive manufacturers, judging by the many sad comments from drive buyers listed on popular, highly-trafficked sales outlets'' websites, like newegg. And I think your 750GB choice should be a good one. I''m currently using 750GB drives (WD7500AAKS) and they have worked flawlessly over the last 2 years. But knowing that drives don''t last forever, it''s time I looked for some new ones, assuming they can be reasonably assumed to be reliable from customer ratings and reports. If there''s one manufacturer that *may* possibly have proved the exception, it might be Samsung with their 1.5TB and 2TB drives -- see my post just a little further up. And using triple parity RAID-Z3 does seem a good idea now when using these higher capacity drives. Or perhaps RAID-Z2 with a hot spare? I don''t know which is better -- I guess RAID-Z3 is better, AND having a spare available ready to replace a failed drive when it happens. But I think I read that unused drive bearings seize up if unused so I don''t know. Any comments? For resilvering to be required, I presume this will occur mostly in the event of a mechanical failure. Soft failures like bad sectors will presumably not require resilvering of the whole drive to occur, as these types of error are probably easily fixable by re-writing the bad sector(s) elsewhere using available parity data in redundant arrays. So in this case larger capacities and resilvering time shouldn''t become an issue, right? And there''s one big item of huge importance here, which is often overlooked by people, and that is the fact that one should always have a reasonably current backup available. Home RAID users often pay out the money for a high-capacity NAS and then think they''re safe from failure, but a backup is still required to guard against loss. I do have a separate Solaris / ZFS machine dedicated to backups, but I do admit to not using it enough -- something I should improve. It contains a backup but an old one. Part of the reason for that is that to save money, I filled it with old drives of varying capacity in a *non-redundant* config to maximise available space from smaller drives mixed with larger drives. Being non-redundant, I shouldn''t depend on its integrity, as there is a high likelihood of it containing multiple latent errors (bit rot). What might be a good idea for a backup box, is to use a large case to house all your old drives using multiple matched drive-capacity redundant vdevs. This way, each time you upgrade, you can still make use of your old drives in your backup machine, without disturbing the backup pool - i.e. simply adding a new vdev each time... -- This message posted from opensolaris.org
On Jan 23, 2010, at 12:04, Simon Breden wrote:> And I think your 750GB choice should be a good one. I''m currently > using 750GB drives (WD7500AAKS) and they have worked flawlessly over > the last 2 years. But knowing that drives don''t last forever, it''s > time I looked for some new ones, assuming they can be reasonably > assumed to be reliable from customer ratings and reports. > > If there''s one manufacturer that *may* possibly have proved the > exception, it might be Samsung with their 1.5TB and 2TB drives -- > see my post just a little further up.Have your storage needs expanded such that you''ve outgrown your current capacity? It may seem counter-intuitive, but is it worth considering replacing your current 750 GB drives with newer 750 GB drives, instead of going to a larger size? Would simply buying new drives be sufficient to get a new warranty, and presumably a device that has less wear on it? More is not always better (though it is more :).> For resilvering to be required, I presume this will occur mostly in > the event of a mechanical failure. Soft failures like bad sectors > will presumably not require resilvering of the whole drive to occur, > as these types of error are probably easily fixable by re-writing > the bad sector(s) elsewhere using available parity data in redundant > arrays. So in this case larger capacities and resilvering time > shouldn''t become an issue, right?Correct. Though it''s recommended to run a ''scrub'' on a regular (weekly?) basis to make sure data corruption / bit flipping is caught early. This will take some time and eat I/O, but can be done during low traffic times (overnight?). Scrubbing (like resilvering) is only done over used blocks, and not over the entire drive(s).> And there''s one big item of huge importance here, which is often > overlooked by people, and that is the fact that one should always > have a reasonably current backup available. Home RAID users often > pay out the money for a high-capacity NAS and then think they''re > safe from failure, but a backup is still required to guard against > loss.Depends on what the NAS is used for. It may be backup volume for the desktops / laptops of the house. In which case it''s not /that/ essential for a backup of the backup to be done--though copying the data to an external drive regularly, and taking that offsite (work) would be useful in the case of fire or burglary. Of course if the NAS is the ''primary'' data store for any data, and you''re not replicating that data anywhere, you''re tempting fate. There are two types of computer users: those have experienced catastrophic data failure, and those that will. I use OS X at home and have a FireWire drive for Time Machine, but I also purchased a FW dock and two stand-alone hard drives in which I use SuperDuper! to clone my boot volume to every Sunday. Then on Monday I take the drive ("A") to work, and bring back the one I have there (Disk "B"). The syncing takes about 25 minutes each week with minimal effort (plug-in drive, launch SD!, press "Copy").
On January 23, 2010 8:04:50 AM -0800 "R.G. Keen" <keen at geofex.com> wrote:> The answer I came to, perhaps through lack of information and experience, > is that there isn''t a best 1.5tb drive. I decided that 1.5tb is too big, > and that it''s better to use more and smaller devices so I could get to > raidz3.Please explain. I don''t understand how smaller devices gets you to raidz3. With smaller devices, you probably have less need for raidz3 as you have more redundancy? It''s the larger drives that forces you to add more parity. -frank
Reading through your post brought back many memories of how I used to manage my data. I also found SuperDuper and Carbon Copy Cloner great for making a duplicate of my Mac''s boot drive, which also contained my data. After juggling around with cloning boot/data drives and using non-redundant Time Machine backups etc, plus some manual copies here and there, I said ''there must be a better way'' and so the long search ended up with the idea of having fairly ''dumb'' boot drives containing OS and apps for each desktop PC and moving the data itself onto a redundant RAID NAS using ZFS. I won''t bore you with the details any more -- see the link below if it''s interesting. BTW, I still use SuperDuper for cloning my boot drive and it IS terrific. Regardless of where the data is, one still needs to do backups, like you say. Indeed, I know all about scrub and do that regularly and that is a great tool to guard against silent failure aka bit rot. Once your data is centralised, making data backups becomes easier, although other problems like the human factor still come into play :) If I left my backup system switched on 24/7 it would in theory be fairly easy to (1) automate NAS snapshots and then (2) automate zfs sends of the incremental differences between snapshots, but I don''t want to spend the money on electricity for that. And when buying drives every few years, I always try to take advantage of Moore''s law. Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
On Sat, 23 Jan 2010, Simon Breden wrote:> <p>I just took a look at customer feedback on this drive here. 36% rate with one star, which I would consider alarming. Take a look here, ordered from lowest rating to highest rating. Note the recency of the comments and the descriptions:</p> > > <a href="http://www.newegg.com/Product/ProductReview.aspx?Item=22-148-412&SortField=3&SummaryType=0&Pagesize=10&SelectedRating=-1&PurchaseMark=&VideoOnlyMark=False&VendorMark=&Page=1&Keywords=%28keywords%29">Seagate Barracuda LP ST31500541AS 1.5TB 5900 RPM</a> > > <p>Is this the model you mean? If so, I might look at some other alternative possibilities.</p>This looks like a really good drive for use with zfs. Be sure to use a mirror configuration and keep in mind that zfs supports an arbitrary number of mirrors so that you can run six or ten of these drives in parallel so that there are enough working drives remaining to keep up with RMAed units. Be sure to mark any failed drive using a sledgehammer so that you don''t accidentally use it again by mistake. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Hi Bob, Why do you consider that model a good drive? Why do you like to use mirrors instead of something like RAID-Z2 / RAID-Z3? And how many drives do you (recommend to) use within each mirror vdev? Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
Just to jump in. Did you guys ever consider to shortstroke a larger sata disk? I''m not familiar with this, but read a lot about it; Since the drive cache gets larger on the bigger drives. Bringing back a disk to roughly 25% of its capicity would give better cache ratio and less seektime. So 2TB would become 500GB, but better then a normal 500GB SATA. ( Or in your case, swing it down to 750Gb ) Regards, Armand ----- Original Message ----- From: "Simon Breden" <sbreden at gmail.com> To: <zfs-discuss at opensolaris.org> Sent: Saturday, January 23, 2010 7:53 PM Subject: Re: [zfs-discuss] Best 1.5TB drives for consumer RAID?> Hi Bob, > > Why do you consider that model a good drive? > > Why do you like to use mirrors instead of something like RAID-Z2 / > RAID-Z3? > > And how many drives do you (recommend to) use within each mirror vdev? > > Cheers, > Simon > > http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
On Sat, 23 Jan 2010, Simon Breden wrote:> Why do you consider that model a good drive?This is a good model of drive to test zfs''s redundancy/resiliency support. It is surely recommended for anyone who does not have the resources to simulate drive failure.> Why do you like to use mirrors instead of something like RAID-Z2 / RAID-Z3?Because raidz3 only supports tripple redundancy but mirrors can support much more.> And how many drives do you (recommend to) use within each mirror vdev?Ten for this model of drive. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
On Sat, 23 Jan 2010, A. Krijgsman wrote:> Just to jump in. > > Did you guys ever consider to shortstroke a larger sata disk? > I''m not familiar with this, but read a lot about it; > > Since the drive cache gets larger on the bigger drives. > Bringing back a disk to roughly 25% of its capicity would give better cache > ratio and less seektime.Consider that a drive cache may be 16MB but the ZFS ARC cache can span up to 128GB of RAM in current servers, or much larger if SSDs are used to add a L2ARC. It seems to me that once the drive cache is large enough to contain a full drive track, that it is big enough. Perhaps a large drive cache may help with write performance. GB beats MB any day of the week.> So 2TB would become 500GB, but better then a normal 500GB SATA. > ( Or in your case, swing it down to 750Gb )Or you could buy a smaller enterprise drive which is short-stroked by design. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Ha ha -- regarding the drive comments, it looks like my humour detector was working just fine ;-) And regarding mirror vdevs etc, I can see the usefulness of being able to build a mirror vdev of multiple drives for cases where you have really critical data -- e.g. a single 4-drive mirror vdev. I suppose regular backups can help with critical data too. I use a 2-drive mirror vdev for ZFS boot, but prefer RAID-Z2 for the main data pool, although I may consider RAID-Z3 in future. For 2-drive mirror vdevs, 2 drives die and data=toast. A RAID-Z2 vdev would still be readable, although whether you''d have enough time/luck to survive the required two resilvers is debateable. With mirrors though, I suppose resilver time would be quicker. I expect you would have some insightful comments in this area. Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
On Jan 23, 2010, at 12:12 PM, Bob Friesenhahn wrote:> On Sat, 23 Jan 2010, A. Krijgsman wrote: > >> Just to jump in. >> >> Did you guys ever consider to shortstroke a larger sata disk? >> I''m not familiar with this, but read a lot about it; >> >> Since the drive cache gets larger on the bigger drives. >> Bringing back a disk to roughly 25% of its capicity would give better cache ratio and less seektime. > > Consider that a drive cache may be 16MB but the ZFS ARC cache can span up to 128GB of RAM in current servers, or much larger if SSDs are used to add a L2ARC.Wimpy servers! To rewrite for 2010, Consider that a drive cache may be 16MB but the ZFS ARC cache can span up to 4 TB of RAM in current servers, or much larger if SSDs are used to add a L2ARC. :-) -- richard
On Jan 23, 2010, at 8:04 AM, R.G. Keen wrote:> Interesting question. > > The answer I came to, perhaps through lack of information and experience, is that there isn''t a best 1.5tb drive. I decided that 1.5tb is too big, and that it''s better to use more and smaller devices so I could get to raidz3.My theory is that drives cost $100. When the price is > $100, the drive is manufactured. When the price is < $100, the drive is EOL and the manufacturer is flushing the inventory. Recently, 1.5 TB drives went below $100. So, if you consider avoiding the leading edge by buying EOL product, then it might not sound like such a good idea :-) -- richard
How does a previously highly rated drive that costed >$100 suddenly become substandard when it costs <$100 ? I can think of possible reasons, but they might not be printable here ;-) Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
On Sat, Jan 23, 2010 at 12:30:01PM -0800, Simon Breden wrote:> And regarding mirror vdevs etc, I can see the usefulness of being > able to build a mirror vdev of multiple drives for cases where you > have really critical data -- e.g. a single 4-drive mirror vdev. I > suppose regular backups can help with critical data too.Multi-way mirrors have lots of uses: - seek independence, for heavily read-biased loads (writes tend to kill this quickly by forcing all drives to seek together). - faster resilver times with less impact to production load (resilver reads are a particular case of the above) - capacity upgrades without losing redundancy in the process (note this is inherently n+1, proof by induction for arbitrary mirrors) - lots of variations of the "attach another mirror, sync and detach" workflow that "zpool clone" was created to support, whether for backup or reporting or remote replication or test systems or .. - "burning in" or qualifying new drives, to work out early failures before putting them in service (easy way to amplify a test workload by say 10x). Watch for slow units, as well as bad data/scrub fails. Just as good for amplifying test workload for controllers and other components. and.. um.. - testing dedup (make a n-way mirror out of n zvols on the same dedup''ed pool; comstar optional :) More seriously, though, it''s for some of these scenarios that the zfs limitation of not being able to layer pool types (easily) is most irritating (raidz of mirrors, mirror of raidz). Again, that''s in part because of practices developed previously; zfs may eventually offer even better solutions, but not yet. -- Dan. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100124/df63ec9a/attachment.bin>
On Sat, Jan 23, 2010 at 09:04:31AM -0800, Simon Breden wrote:> For resilvering to be required, I presume this will occur mostly in > the event of a mechanical failure. Soft failures like bad sectors > will presumably not require resilvering of the whole drive to occur, > as these types of error are probably easily fixable by re-writing > the bad sector(s) elsewhere using available parity data in redundant > arrays. So in this case larger capacities and resilvering time > shouldn''t become an issue, right?Correct. However, consider that it''s actually the *heads* that contribute most to errors accumulating; over time they lose the ability to read with the same sensitivity, for example. Of course this shows up first in some areas of the platter that already had slightly more marginal surface quality. This is why smart and similar systems consider both the absolute number of bad sectors, as well as the rate of discovery, as predictors of device failure.> What might be a good idea for a backup box, is to use a large case > to house all your old drives using multiple matched drive-capacity > redundant vdevs. This way, each time you upgrade, you can still make > use of your old drives in your backup machine, without disturbing > the backup pool - i.e. simply adding a new vdev each time...This is basically my scheme at home - current generation drives are in service, the previous generation go in the backup pool, and the set before that become "backup tapes". Every so often the same thing happens with the servers/chassis/controller/housing stuff, too. It''s coming up to time for exactly one of those changeovers now. I always have a bunch of "junk" data in the main pool that really isn''t worth backing up, which helps deal with the size difference. There''s no need to constantly add vdevs to the backup pool, just do replacement upgrades the same as you did with your primary pool. I, too, will admit to not being as diligent at putting the scheme into regular practice as theory would demand. I may also relocate the backup pool at a neigbours house soon (or, really, trade backup pool space with him). -- Dan. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100124/874b88e9/attachment.bin>
On Sat, Jan 23, 2010 at 11:41 AM, Frank Cusack <fcusack at fcusack.com> wrote:> On January 23, 2010 8:04:50 AM -0800 "R.G. Keen" <keen at geofex.com> wrote: > >> The answer I came to, perhaps through lack of information and experience, >> is that there isn''t a best 1.5tb drive. I decided that 1.5tb is too big, >> and that it''s better to use more and smaller devices so I could get to >> raidz3. >> > > Please explain. I don''t understand how smaller devices gets you to > raidz3. With smaller devices, you probably have less need for raidz3 > as you have more redundancy? It''s the larger drives that forces you > to add more parity. > > -frankSmaller devices get you to raid-z3 because they cost less money. Therefore, you can afford to buy more of them. -- --Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100123/a7b044cd/attachment.html>
On January 23, 2010 5:17:16 PM -0600 Tim Cook <tim at cook.ms> wrote:> Smaller devices get you to raid-z3 because they cost less money. > Therefore, you can afford to buy more of them.I sure hope you aren''t ever buying for my company! :) :) Smaller devices cost more $/GB; ie they are more expensive. -frank
On January 23, 2010 1:20:13 PM -0800 Richard Elling> My theory is that drives cost $100.Obviously you''re not talking about Sun drives. :) -frank
Hey Dan, Thanks for the reply. Yes, I''d forgotten that it''s often the heads that degrade -- something like lubricant buildup, IIRC. As well as SMART data, which I must admit to never looking at, presumably scrub errors are also a good indication of looming trouble due to head problems etc? As I''ve seen zero read/write/checksum errors after regular scrubs over 2 years, hopefully this is a reasonably good sign of r/w head health. Good to see you''re already using a backup solution I have envisaged using. It seems to make sense: making use of old kit for backups to help preserve ROI on drive purchases -- even, no especially, for home users. Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
On Sat, Jan 23, 2010 at 5:39 PM, Frank Cusack <fcusack at fcusack.com> wrote:> On January 23, 2010 5:17:16 PM -0600 Tim Cook <tim at cook.ms> wrote: > >> Smaller devices get you to raid-z3 because they cost less money. >> Therefore, you can afford to buy more of them. >> > > I sure hope you aren''t ever buying for my company! :) :) > > Smaller devices cost more $/GB; ie they are more expensive. > >First off, smaller devices don''t necessarily cost more $/GB, but that''s not really the point. For instance, the cheapest drive per GB is a 1.5TB drive today. The third cheapest is a 1TB drive. 2TB drives aren''t even in the top ten. Regardless that''s a great theory when you have an unlimited budget. When you''ve got a home system and X amount of dollars to spend, $/GB means absolutely nothing when you need a certain number of drives to have the redundancy you require. -- --Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100123/1ba2d165/attachment.html>
On Jan 23, 2010, at 3:47 PM, Frank Cusack wrote:> On January 23, 2010 1:20:13 PM -0800 Richard Elling >> My theory is that drives cost $100. > > Obviously you''re not talking about Sun drives. :)Don''t confuse cost with price :-) -- richard
On Sat, Jan 23, 2010 at 06:39:25PM -0500, Frank Cusack wrote:> On January 23, 2010 5:17:16 PM -0600 Tim Cook <tim at cook.ms> wrote: >> Smaller devices get you to raid-z3 because they cost less money. >> Therefore, you can afford to buy more of them. > > I sure hope you aren''t ever buying for my company! :) :) > > Smaller devices cost more $/GB; ie they are more expensive.Usually, other than the very largest (most recent) drives, that are still at a premium price. However, it all depends on your budget considerations. Budget applies not only to currency. You may be more constrained by available controller ports, motherboard slots, case drive bays, noise, power, heat or other factors. Even if it still comes back to currency units, adding more ports or drive bays can easily outweigh the cost of the drives to go on/in them, especially in the consumer market. There''s usually a big step where just one more drive means a totally different solution. If you''re targetting total available space, small drives really do cost more for the same space, when all these factors are counted. That''s what sells the bigger drives, despite the premium. The other constraint is redundancy - I need N drives (raidz3 in the OP''s case), the smaller size is "big enough" and maybe the only way to also be "cheap enough". -- Dan. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100124/a29feaa4/attachment.bin>
> <p>I just took a look at customer feedback on this > drive here. 36% rate with one star, which I would > consider alarming. Take a look here, ordered from > lowest rating to highest rating. Note the recency of > the comments and the descriptions:</p> >Every people vote in different way for the same things. A lot of 1 star are about DOA. Maybe Neweggs have a bad batch (happens sometimes) The Bad news propagate much faster that the good ones. A angry user is more probable to post a bad review that a happy user. 120 review are a tiny sample to make a decision. look in the other way 50% is 4-5 stars, 1/2 is very happy. I''ve said that every brand have problem at moment. At 1.5TB there''s few choice. For me the RMA service have a important role, because I expect to use it. I don''t expect to see the five HDDs running flawless for 3 years. Samsung have no direct RMA service in my country, so the tournaround is a shoot in the dark, few weeks, maybe a month. Pick a rock solid product is more a matter of luck. I''ve a Deskstar DTLA da 13GB, runnings strong after nealy 11 years or so ! maybe it''s the last ones alive. Reading all over internet it should be dead 12 years ago. If the failure rate was 36% Seagate was toast. The Barracuda LP can''t be the right driver for everyone, but if you look at 1.5TB cheap consumer driver, that run cool&quiet it deserve strong cosideration. It''s better on the paper that the WD green, (stardard sector, no start/stop cycle after 8 sec of inactivity), Hitachi have no 1.5TB at the moment. samsung have a new recent model, the 7200.11 is a old product on 4 platters. -- This message posted from opensolaris.org
>I just took a look at customer feedback on this > drive here. 36% rate with one star, which I would > consider alarming. Take a look here, ordered from lowest rating to highest rating. Note the recency of the comments and the descriptions:Every people vote in different way for the same things. A lot of 1 star are about DOA. Maybe Neweggs have a bad batch (happens sometimes) The Bad news propagate much faster that the good ones. A angry user is more probable to post a bad review that a happy user. 120 review are a tiny sample to make a decision. look in the other way 50% is 4-5 stars, 1/2 is very happy. I''ve said that every brand have problem at moment. At 1.5TB there''s few choice. For me the RMA service have a important role, because I expect to use it. I don''t expect to see the five HDDs running flawless for 3 years. Samsung have no direct RMA service in my country, so the tournaround is a shoot in the dark, few weeks, maybe a month. Pick a rock solid product is more a matter of luck. I''ve a 13GB Deskstar DTLA, runnings strong after nealy 11 years or so ! maybe it''s the last ones alive. Reading all over internet it should be dead 12 years ago. If the failure rate was 36% Seagate was toast. The Barracuda LP can''t be the right driver for everyone, but if you look at 1.5TB cheap consumer driver, that run cool&quiet it deserve strong cosideration. It''s better on the paper that the WD green, (stardard sector, no start/stop cycle after 8 sec of inactivity), Hitachi have no 1.5TB at the moment. samsung have a new recent model, the 7200.11 is a old product on 4 platters. -- This message posted from opensolaris.org
On January 23, 2010 6:09:49 PM -0600 Tim Cook <tim at cook.ms> wrote:> When you''ve got a home system and X amount of dollars > to spend, $/GB means absolutely nothing when you need a certain number of > drives to have the redundancy you require.Don''t you generally need a certain amount of GB? I know I plan my storage based on how much data I have, even my home systems. And THEN add in the overhead for redundancy. If we''re talking about such a small amount of storage ("home") that the $/GB is not a factor (ie, even with the most expensive $/GB drives we won''t exceed the budget and we don''t have better things to spend the money on anyway) then raidz3 seems unnecessary. I mean, just do a triple mirror of the 1.5TB drives rather than say (6) .5TB drives in a raidz3. -frank
On Sat, Jan 23, 2010 at 7:57 PM, Frank Cusack <fcusack at fcusack.com> wrote:> On January 23, 2010 6:09:49 PM -0600 Tim Cook <tim at cook.ms> wrote: > >> When you''ve got a home system and X amount of dollars >> to spend, $/GB means absolutely nothing when you need a certain number of >> drives to have the redundancy you require. >> > > Don''t you generally need a certain amount of GB? I know I plan my > storage based on how much data I have, even my home systems. And THEN > add in the overhead for redundancy. If we''re talking about such a > small amount of storage ("home") that the $/GB is not a factor (ie, > even with the most expensive $/GB drives we won''t exceed the budget and > we don''t have better things to spend the money on anyway) then raidz3 > seems unnecessary. I mean, just do a triple mirror of the 1.5TB drives > rather than say (6) .5TB drives in a raidz3. > > -frank >I bet you''ll get the same performance out of 3x1.5TB drives you get out of 6x500GB drives too. Are you really trying to argue people should never buy anything but the largest drives available? I hope YOU aren''t ever buying for MY company. -- --Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100123/76cf696b/attachment.html>
Mirko wrote:> Well, I''ve purchased 5 Barracuda LP 1.5TB. > They ran very queit, cool, 5 in a cage and the vibration are nearly zero. > > reliability ? Well every HDD is unreliable, every major brand at this time have problems, so go for the best bang for the bucks. > > In my country Seagate have the best RMA service, with tournaround in about 1 week or so, WD is 3-4 weeks. Samsung have no direct RMA service, Hitachi well have a foot out HDD business IMHO, no attractive product at moment.I really wonder why the Hitachi 2GB is the cheapest in Singapore Seagate and WD price the 1TB around S$125 and 2TB around S$305 However Hitachi 1TB is around S$125 and 2TB around S$248, quite a steal. Since this is anomaly, anybody know what technology difference did Hitachi make to hit that price? Or do I miss a prophecy of disaster related to their business or technology? All of them are on 3 years warranty. Thinking of replacing my (8 +1)x 500 GB Seagate Barracuda with the 2TB disks.
These days, I''ve switched to 2.5" SATA laptop drives for large-storage requirements. They''re going to cost more $/GB than 3.5" drives, but they''re still not horrible ($100 for a 500GB/7200rpm Seagate Momentus). They''re also easier to cram large numbers of them in smaller spaces, so it''s easier to get larger number of spindles in the same case. Not to mention being lower-power than equivalent 3.5" drives. My sole problem is finding well-constructed high-density 2.5" hot-swap bay/chassis setups. If anyone has a good recommendation for a 1U or 2U JBOD chassis for 2.5" drives, that would really be helpful. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
On Sun, Jan 24, 2010 at 08:36, Erik Trimble <Erik.Trimble at sun.com> wrote:> These days, I''ve switched to 2.5" SATA laptop drives for large-storage > requirements. > They''re going to cost more $/GB than 3.5" drives, but they''re still not > horrible ($100 for a 500GB/7200rpm Seagate Momentus). They''re also easier > to cram large numbers of them in smaller spaces, so it''s easier to get > larger number of spindles in the same case. Not to mention being lower-power > than equivalent 3.5" drives. > > My sole problem is finding well-constructed high-density 2.5" hot-swap > bay/chassis setups. > If anyone has a good recommendation for a 1U or 2U JBOD chassis for 2.5" > drives, that would really be helpful. > > Erik, try this one on for size;http://www.supermicro.com/products/accessories/mobilerack/CSE-M28E1.cfm Supermicro has a number of variations on this theme, but I deployed this one at a client site, and - so far - no complaints. I''m not sure I''d run one of these personally, because it seems that drives would tend to run hotter than if individually stacked in a conventional PC case***....but....write some of that off to me being excessively conservative when it comes to cooling. That said, it''s Supermicro after all, and they tend to sell well engineered gear. ***Unless you stuffed it with SSD''s - </wet messy hysterical laughter> -Me -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100124/a9736952/attachment.html>
I''m in the process of building semi-beefy file/general-purpose-server(Lynnfield Xeon, 4GB Ecc) and hard drive choice is the problem. I''ve been googling for a day and a half now and the main points seem to be: - ~all consumer class drives have the same problem with TLER/ERC/CCTL - ~all "for raid" drives are 30-50% more expensive AND use more power due to being tuned for performance(even the WD GB RE4 2TB drive uses more power than the equivalent plain WD GB 2TB) So for me there isn''t a perfect choice as I intented to use 6 1,5TB drives in RAIDZ2 and then later on add more similar 6drive vdevs when needed. 1,5TB WD/Samsung drives cost ~85? a pop, or 510? for 6. 1TB Samsung raid drives cost 100? a pop and to get to ~6TB usable with RAIDZ2 would mean 8 drives and 800?. 2TB drives would be even more expensive and there is some problems wiht the WD 2TB RE4 firmware so that isn''t nice option either. I think that I''m likely going to take the plunge with the 1,5TB WD drives, because at least the preceding 1TB models have worked well for me, being silent and low power(albeit they aren''t being used in raid). -- This message posted from opensolaris.org
> a 1U or 2U JBOD chassis for 2.5" drives,from http://supermicro.com/products/nfo/chassis_storage.cfm the E1 (single) or E2 (dual) options have a SAS expander so http://supermicro.com/products/chassis/2U/?chs=216 fits your build or build it your self with http://supermicro.com/products/accessories/mobilerack/CSE-M28E2.cfm
Kjetil Torgrim Homme
2010-Jan-24 16:26 UTC
[zfs-discuss] Best 1.5TB drives for consumer RAID?
Tim Cook <tim at cook.ms> writes:> On Sat, Jan 23, 2010 at 7:57 PM, Frank Cusack <fcusack at fcusack.com> wrote: > > I mean, just do a triple mirror of the 1.5TB drives rather than > say (6) .5TB drives in a raidz3. > > I bet you''ll get the same performance out of 3x1.5TB drives you get > out of 6x500GB drives too.no, it will be much better. you get 3 independent disks available for reading, so 3x the IOPS. in a 6x500 GB setup all disks will need to operate in tandem, for both reading and writing. even if the larger disks are slower than small disks, the difference is not even close to such a factor. perhaps 20% fewer IOPS?> Are you really trying to argue people should never buy anything but > the largest drives available?I don''t think that''s Frank''s point. the key here is the advantages of a (triple) mirroring over RAID-Z. it just so happens that it makes economic sense. (think about savings in power, too.) -- Kjetil T. Homme Redpill Linpro AS - Changing the game
Let me start this off with a personal philosophy statement. In technical matters, there is almost never a ?best?. There only the best compromise given the objective you?re trying to achieve. If you change the objectives even slightly, you may get wildly different ?best compromise? answers. I came to zfs looking for the answer to the following problem: 1. I need data backups for data chunks bigger than are practical or simple for DVDs. 2. While I need a lot of bits, I don?t necessarily need the most bits I can afford. 3. Elimination of silent bit-rot is a major priority. Yes, I?m looking at a RAID for backup. But I want it as a backup store that may not be spinning 24/7/365. I don?t plan to use the RAID for on-line work. That is on the individual hard drives in the various machines being used. Another backup may well be on DVD or other static medium for more critical data. I?ll just blather a bit. The most durable data backup medium humans have come up with was invented about 4000-6000 years ago. It?s fired cuniform tablets as used in the Middle East. Perhaps one could include stone carvings of Egyptian and/or Maya cultures in that. The second most durable medium we have is ink on paper/papyrus. There are records that are still readable on this medium from 3000-4000 years ago. That?s after being weathered and buried for kiloyears, no or not much human help with preservation. The modern computer era has nothing that even comes close. Our data storage media are largely temporary measures. We are very much like the performer I remember from the Ed Sullivan show (egad, I?m old!) who had an array of vertical rods upon which he spun ceramic plates, manipulating the rods to keep the plates spinning because if they ever spun down, the plate would fall off and break. How many of you can read a 3.5? floppy disk? A 5.25? floppy? An 8? floppy? A 1? data tape from the 1960s? Our modern data preservation relies on recopying the data onto new data media every so often before the mechanism to read the old medium becomes obsolete or irreparable and the medium itself decays. We are exactly an analog to the plate-spinner with our data. Worse yet, our media are not perfect. An otherwise perfectly-written record will accumulate errors and eventually become unusable. Cuniform does too, but the scale is very, different. Having evaluated my data needs, I have some data that I need to be readable to modestly past my death. This puts the archival time in decades, not centuries or millennia. This line of reasoning led me to zfs. It?s for the background scrub, item #3. I can buy new media as it becomes available to increase the storage modestly as affordable (item #2). And I can?t bet on a really archival data storage technology becoming available. It may not get there in my lifetime. Given that, here?s my cut on some of the questions here: ?Sweet spot for disks?: I don?t need the absolute most bits per dollar. While it would be nice to have that, it?s not crucial. Every disk size was once the sweet spot. Being a little back from the leading edge is probably a good thing for reliability (I have all these scars?) and when I have enough, cool. It?s better for my special case to have the most reliable data reading setup than it is to have the most bits. ?Fewer/bigger versus more/smaller drives?: Tim and Frank have worked this one over. I made the choice based on wanting to get a raidz3 setup, for which more disks are needed than raidz or raidz2. This idea comes out of the time-to-resilver versus time to failure line of reasoning. ?Disk drives cost $100?: yes, I fully agree, with minor exceptions. End of marketing, which is where the cost per drive drops significantly, is different from end of life ? I hope! In my case, I got 0.75TB drives for $58 each. The cost per bit is bigger than buying 1TB or 1.5TB drives, all right, but I can buy more of them, and that lets me put another drive on for the next level of error correction data. In my special, very limited set of criteria, a few extra disks are better than modestly more bits. This will obviously change when I have exhausted the number of bits available. Ideally, at that time, 50TB disks will be the sweet spot. Another issue I had not put out consciously until I wrote this is that this is a learning system for me. I?m new to Solaris and zfs. Lowest entry cost matters while I use up my ?classroom workbook? system. Frankly, I flirted around with buying batches of old 100gb drives for the learner system. This had the advantage of (1) running the cost even lower and (2) having a few failures be likely to ensure that I test the recovery parts of zfs, not simply think to myself ?Kewl, it all works fine!? right up until the first failure and find out that I can?t recover. Kind of like you really, really should have your fire extinguishers tested or replaced from time to time. But none of this is hard and fast math. It?s what seems right based on the specific set of criteria. I decided, wrong or right, that I needed to learn to use raidz3 for extra belts and suspenders; that suggested I needed more disk drives, and that made 1.5TB drives in quantity expensive. A deal on 750GB drives iced that one down. It was interesting, if modestly irrelevant that these were raid-rated drives. If your criteria is most bits per dollar, the reasoning flips around a lot. If it?s fewest dollars, not fewest dollars versus learning, the reasoning changes. If it?s speed, it changes. If it?s thermal issues, it changes. Where you?re going has a huge effect on the vehicle you choose. -- This message posted from opensolaris.org
On January 23, 2010 8:23:08 PM -0600 Tim Cook <tim at cook.ms> wrote:> I bet you''ll get the same performance out of 3x1.5TB drives you get out of > 6x500GB drives too.Yup. And if that''s the case, probably you want to go with the 3 drives because your operating costs (power consumption) will be less.> Are you really trying to argue people should never > buy anything but the largest drives available?No. Are you really so dense that you extrapolate my argument to an extremely broad catch-all? There are other reasons besides cost that people might want to buy smaller drives. And, e.g., if your data set isn''t that large, don''t spend money for space you don''t need. The post that I was responding to claimed smaller drives *allowed* him to get to raidz3. I challenged that as incorrect. It''s the larger drives that *require* raidz3 because resilver time is longer. So far I''ve seen no argument to the contrary. Just a side argument about cost which I happen to disagree with. And a followup side argument about planning for redundancy which I also disagree with. Let''s say you need 3TB of storage. That''s a lot for most home uses. The actual amount doesn''t matter as the costs will scale. So you buy 5 1.5TB drives. 4 (2+2) in a raidz2 plus a hot spare. For the sake of this argument, let''s say you''ve done the math and raidz2 meets your redundancy requirement, based on time to resilver. More likely, a home user has not done the math but that''s besides the point. Now let''s do it with .5GB drives. A quick survey shows me they come in at about a 10% discount to the 1.5TB drives. I''m being generous because I can''t even find .5GB drives, but I see that 320GB drives are about 10% less. If you want to get even "cheaper", 250GB drives are about 50% less cost than 1.5TB drives (which by my argument, which you refute, makes them 3x more expensive but whatever). So with .5GB drives you need 6+3 drives -- because the smaller drives "allows" you to get to raidz2, plus a hot spare. That''s twice as many drives, however you are only paying 10% less per drive. PLUS with this many drives you now need a pretty big chassis. Plus your power costs are now quite a bit higher. Please put together a scenario for me where smaller drives cost less.> I hope YOU aren''t ever buying for MY company.Rest assured, I won''t be. -frank
On January 24, 2010 11:45:57 AM +1100 Daniel Carosone <dan at geek.com.au> wrote:> On Sat, Jan 23, 2010 at 06:39:25PM -0500, Frank Cusack wrote: >> Smaller devices cost more $/GB; ie they are more expensive. > > Usually, other than the very largest (most recent) drives, that are > still at a premium price.Yes, I should have clarified that. -frank
Rob Logan wrote:>> a 1U or 2U JBOD chassis for 2.5" drives, >> > from http://supermicro.com/products/nfo/chassis_storage.cfm > the E1 (single) or E2 (dual) options have a SAS expander so > http://supermicro.com/products/chassis/2U/?chs=216 > fits your build or build it your self with > http://supermicro.com/products/accessories/mobilerack/CSE-M28E2.cfI''m aware of the Supermicro chassis, and, while they''re nice, I''m after an external JBOD chassis, not a server chassis. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
On January 24, 2010 8:41:00 AM -0800 Erik Trimble <Erik.Trimble at Sun.COM> wrote:> an external JBOD chassis, not a server chassis.<http://h18006.www1.hp.com/storage/disk_storage/msa_diskarrays/drive_enclosures/index.html>
On January 24, 2010 8:26:07 AM -0800 "R.G. Keen" <keen at geofex.com> wrote:> In my case, I got 0.75TB drives > for $58 each. The cost per bit is bigger than buying 1TB or 1.5TB drives, > all right, but I can buy more of them, and that lets me put another drive > on for the next level of error correction data.That''s the point I was arguing against. You did not respond to my argument, and you don''t have to now, but as long as you keep stating this without correcting me I will keep responding. The size of the drive has nothing to do with letting you put another drive on, for more redundancy. If you want more redundancy, you *have* to buy more drives, whether big or small. If you''re implying that because of the lower cost you can afford to buy the additional drive, that also clearly incorrect as the cost per bit is more, so in fact you spend more with the smaller drives PLUS the cost for the additional redundancy. Also, smaller drives require LESS redundancy for the same level of availability, not more. Of course, because drives are only available in discrete sizes you may end up with the same raidz level (1,2 or 3) anyway. -frank
On January 24, 2010 8:26:07 AM -0800 "R.G. Keen" <keen at geofex.com> wrote:> ?Fewer/bigger versus more/smaller drives?: Tim and Frank have worked > this one over. I made the choice based on wanting to get a raidz3 setup, > for which more disks are needed than raidz or raidz2. This idea comes out > of the time-to-resilver versus time to failure line of reasoning.Sorry I missed this part of your post before responding just a moment ago. If you want raidz3, you will spend more money on larger drives if your data still fits into N smaller drives. If you only have .75TB of data, then of course it is a waste to get 1.5TB drives because you still need 5 (1+3)+1 of them and you should definitely use the cheaper .75TB drives. But you''d do even better to use a triple mirror of the smaller drives. Once the size of your data exceeds the size of the smaller drive, and you have to buy 2 of them just for the data part (not incl. the parity), it''s now more expensive to use the smaller drives. In the above paragraph you haven''t mentioned cost at all, but since you did talk elsewhere about the cost of the smaller drives being cheaper, I wanted to make it clear you are spending more money by using the smaller drives. -frank
On Sun, Jan 24, 2010 at 10:38 AM, Frank Cusack <fcusack at fcusack.com> wrote:> On January 23, 2010 8:23:08 PM -0600 Tim Cook <tim at cook.ms> wrote: > >> I bet you''ll get the same performance out of 3x1.5TB drives you get out of >> 6x500GB drives too. >> > > Yup. And if that''s the case, probably you want to go with the 3 drives > because your operating costs (power consumption) will be less.Nope.> > > Are you really trying to argue people should never >> buy anything but the largest drives available? >> > > No. Are you really so dense that you extrapolate my argument to an > extremely broad catch-all? There are other reasons besides cost that > people might want to buy smaller drives. And, e.g., if your data set > isn''t that large, don''t spend money for space you don''t need. > > The post that I was responding to claimed smaller drives *allowed* him > to get to raidz3. I challenged that as incorrect. It''s the larger > drives that *require* raidz3 because resilver time is longer. So far > I''ve seen no argument to the contrary. Just a side argument about > cost which I happen to disagree with. And a followup side argument > about planning for redundancy which I also disagree with. >You''re calling me dense and you think the sole purpose of him using raid-z3 is resilver time? Hey guys, let''s throw self-healing out the window when using consumer drives because it''s cheaper per GB to buy a larger drive size. You''re trying to convince the OP not to use raid-z3 because you either haven''t read up, or find no benefit to some very real, and very useful features he will see with it.> Let''s say you need 3TB of storage. That''s a lot for most home uses. > The actual amount doesn''t matter as the costs will scale. So you > buy 5 1.5TB drives. 4 (2+2) in a raidz2 plus a hot spare. For the > sake of this argument, let''s say you''ve done the math and raidz2 > meets your redundancy requirement, based on time to resilver. More > likely, a home user has not done the math but that''s besides the point. > > Now let''s do it with .5GB drives. A quick survey shows me they come > in at about a 10% discount to the 1.5TB drives. I''m being generous > because I can''t even find .5GB drives, but I see that 320GB drives > are about 10% less. If you want to get even "cheaper", 250GB drives > are about 50% less cost than 1.5TB drives (which by my argument, which > you refute, makes them 3x more expensive but whatever). > > So with .5GB drives you need 6+3 drives -- because the smaller drives > "allows" you to get to raidz2, plus a hot spare. That''s twice as many > drives, however you are only paying 10% less per drive. PLUS with this > many drives you now need a pretty big chassis. Plus your power costs > are now quite a bit higher. > > Please put together a scenario for me where smaller drives cost less. >10%? I''m not sure where you shop, but no. The cheapest 500GB is $39.99. The cheapest 1.5TB is $97.99. http://www.zipzoomfly.com/jsp/ProductDetail.jsp?ProductCode=10010737 http://www.newegg.com//Product/Product.aspx?Item=N82E16822148516 $359.91 for the 500GB drives. $391.96 for the 1.5TB drives. -- --Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100124/345b4e74/attachment.html>
On Sun, Jan 24, 2010 at 11:41, Erik Trimble <Erik.Trimble at sun.com> wrote:> Rob Logan wrote: >>> >>> a 1U or 2U JBOD chassis for 2.5" drives, >>> >> >> from http://supermicro.com/products/nfo/chassis_storage.cfm the E1 >> (single) or E2 (dual) options have a SAS expander so >> http://supermicro.com/products/chassis/2U/?chs=216 >> fits your build or build it your self with >> http://supermicro.com/products/accessories/mobilerack/CSE-M28E2.cf > > I''m aware of the Supermicro chassis, and, while they''re nice, I''m after an > external JBOD chassis, not a server chassis.Depending on your needs, you may find adding a CSE-PTJBOD-CB1 card and a CBL-0166L assembly (which functionally turn the chassis into a JBOD) may suffice. The total cost for such a system is about $1150 shipped to me from Provantage.com, which is substantially less than any big-name JBOD with similar qualities I can find. Will
On Jan 24, 2010, at 8:26 AM, R.G. Keen wrote:> > ?Disk drives cost $100?: yes, I fully agree, with minor exceptions. End of marketing, which is where the cost per drive drops significantly, is different from end of life ? I hope!http://en.wikipedia.org/wiki/End-of-life_(product) Some vendors use EOSL to signify the end of service life, after which time the product will no longer be serviced or warranted. The actual EOSL can change since the product is no longer manufactured. For hardware, when the warranty reserve is exhausted, EOSL may be forced. For consumer disks, I think a model of 1 year of life + 3 years warranty is fairly consistent. For enterprise disks, the enterprise market demands longer warranty periods, like 5 years. Warranty is a non-zero cost for the manufacturer that is reflected in the price. -- richard
On 24-Jan-10, at 11:26 AM, R.G. Keen wrote:> ... > I?ll just blather a bit. The most durable data backup medium humans > have come up with was invented about 4000-6000 years ago. It?s > fired cuniform tablets as used in the Middle East. Perhaps one > could include stone carvings of Egyptian and/or Maya cultures in > that. ... > > The modern computer era has nothing that even comes close. ...> And I can?t bet on a really archival data storage technology > becoming available. It may not get there in my lifetime.A better digital archival medium may already exist: http://hardware.slashdot.org/story/09/11/13/019202/Synthetic-Stone- DVD-Claimed-To-Last-1000-Years --Toby
On Sun, Jan 24, 2010 at 19:34, Toby Thain <toby at telegraphics.com.au> wrote:> > On 24-Jan-10, at 11:26 AM, R.G. Keen wrote: > > ... >> >> I?ll just blather a bit. The most durable data backup medium humans have >> come up with was invented about 4000-6000 years ago. It?s fired cuniform >> tablets as used in the Middle East. Perhaps one could include stone carvings >> of Egyptian and/or Maya cultures in that. ... >> >> The modern computer era has nothing that even comes close. ... >> > > And I can?t bet on a really archival data storage technology becoming >> available. It may not get there in my lifetime. >> > > > A better digital archival medium may already exist: > http://hardware.slashdot.org/story/09/11/13/019202/Synthetic-Stone- > DVD-Claimed-To-Last-1000-YearsThat would be nice - but - I have to wonder how they would test it in order to justify the actual lifespan claim. Seems like the first real aging test would be down the road aways. Just as well it''s a start-up I guess. Mezzanine funding round...to come... -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100124/00f03136/attachment.html>
Colin Raven <colin at clearcutnetworks.com> wrote:> > A better digital archival medium may already exist: > > http://hardware.slashdot.org/story/09/11/13/019202/Synthetic-Stone- > > DVD-Claimed-To-Last-1000-Years > > > That would be nice - but - I have to wonder how they would test it in order > to justify the actual lifespan claim. Seems like the first real aging test > would be down the road aways. Just as well it''s a start-up I guess. > Mezzanine funding round...to come...100 years apply to the DVD media that was made in 1998 by Pioneer and TDK. The cost of a single media these days was ~ 70 Euro. I would guess that current media will not last as long as media from 1998. BluRay BD-R media is not based on organic dye but on metall-nitrides. I would guess that BD-R for this reason lasts longer and is not affected by UV rays. The article mentined above does not contain enough information to be able to judge on it. J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
On January 24, 2010 Frank Cusack wrote:>That''s the point I was arguing against.Yes, that''s correct.>You did not respond to my argument, and you don''t have to now,Thanks for the permission. I''ll need that someday.>but as long as you keep stating this without correcting me I will keep responding.As is your perfect right. You have my permission to say anything you like; I do like a guy who''s persistent. But if you''ll re-read what I wrote, I think you may have left out my implicit phrase "... for me..." in describing my problem and my solution. I would never presume to correct you. Your situation is almost certainly different from mine. I do however believe that situations like the one we are describing must be described in context, because when we start discussing failure rates and economics as well as different objectives, the scoring polynomials get complicated fast.>The size of the drive has nothing to do with letting you put another drive on, for >more redundancy.That is correct. I can obviously buy more drives no matter how big they are. As I used to tell girls in bars, I''m not really this tall, I''m just sitting on my wallet. It was remarkably effective. 8-) "Just buy more, bigger drives" is a good solution, if the sheer number of bits or lowest cost per bit is what you''re after. However, I may not have been clear about my point. More, smaller drives, within limits may be better in some circumstances, depending on how you look at the problem. I thought that I was clear about the "within limits/practicality/reason" approach, in my rambling. If not, let me say it. No generality is worth a d..., including this one. There is only fitness for context. Do what''s practical within your context. And have a good time at it. But contexts are rarely as simple as "get more bits" or "get the densest drives" when you have multiple objectives. As I obviously muffed explaining, more and smaller drives, for not too much cost penalty (where we all get to define how much smaller and how much more cost penalty in our own personal contexts) leads to a lower recover/resilver time per failed disk than grabbing the densest, most bit stuffed drives; this is because to a first order, the time to get bits onto a new disk is proportional to the number of bits. This changes in steps as the interfaces get faster, but is likely to not keep up with the sheer number of bits. I know this because I read it on the internet and that makes it true, right? 8-) I''ve got that reference here somewhere if it''s pertinent. Obviously, taken to extreme, a zillion one-bit drives is not a cost effective solution. We''re all here at least partly because a single zettabyte drive has some issues as well. I suspect that on this continuum, there is not a single answer of X drives of Y size is the best answer for all situations. Maybe there are two answers, perhaps even three. In my own, personal, dim-witted, benighted context of evaluation, for which I reserve the right to be wrong, I judged that for me, at this time, in the technological context I happen to be in right now, it''s better to get a few more drives to get more increments of physical failure possibility if that also includes more redundancy mechanisms (i.e. raidz3) so that I have a better chance of correcting silent bit errors before a second drive failure makes the data lost. In that limited, miniscule, probably useless context where the I/O rates onto and off the disk are the same for all the drives being considered, I judge that cutting the time to re-silver a replacement disk may be more desirable than getting the absolute most disks. It''s a different viewpoint, albeit probably limited and shortsighted as I''ve stated. But I thought it might be helpful to someone else, if only as a starter idea for thinking about disk drives on a basis other than bits/dollar. Perhaps the most bits per dollar is, in fact, the best strategy; however, it ought to be checked out if there are other ways to looking at it, depending on what you''re trying to do.> smaller drives require LESS redundancy for the same level of availabilityMaybe that''s the problem. I''m not after availability. Back in the bad old days when I sold my soul to a giant corporation every day, we distinguished between reliability, availability, and serviceability. I''m after serviceability. Reliability and availability are close, but not the same concepts. Anyway, thanks for hanging in there and helping me figure it out. I guess I just had this simple minded idea that all situations of evaluating how many drives of what density might not reduce to only the lowest cost per bit at any given time. Silly, right? 8-) R.G. -- This message posted from opensolaris.org
Registered: 7/7/05 Re: [zfs-discuss] Best 1.5TB drives for consumer RAID? Posted: Jan 24, 2010 11:20 AM in response to: r.g. Click to reply to this thread Reply On January 24, 2010 Frank wrote:>Sorry I missed this part of your post before responding just a moment >ago. If you want raidz3, you will spend more money on larger drives >if your data still fits into N smaller drives.Thanks. That''s a clearer explanation of what I was on about.> I wanted to make it clear you are spending more money by using the smaller > drives.S''True. I do know that. Even more importantly to someone doing a backup server or a NAS, it takes more electricity to run more drives, and that my cost more over time than the drives themselves. I do apologize for the snottier parts of my reply to your first note, which I am editing. I did not get a chance to read this note from you before responding. R.G. _______________________________________________ -- This message posted from opensolaris.org
Another issue with all this arithmetic: one needs to factor in the cost of additional spare disks (what were you going to resilver onto?). I look at it like this: you purchase the same number of total disks (active + hot spare + cold spare), and raidz2 vs raidz3 simply moves a disk from one of the spare columns to the active column. Raidz3 gives you longer to find (potentially even purchase, at future cheaper prices) the replacement disk, which might simply be a replacement cold spare. On Sun, Jan 24, 2010 at 12:20:37PM -0500, Frank Cusack wrote:> But you''d do even better to use a triple mirror of the > smaller drives.By R.G''s now well-stated and clear objectives and criteria for "better", no. He''d need at least a 4-way mirror, to survive 3 faults. Sorry to nit-pick your arithmetic again :) Assuming 4 active disks total (for capital cost equivalence), when it comes to other criteria, like random-IOPS, most of use would prefer the mirror, of those two configs. If those criteria are not as important to you, there may still be reasons to prefer raidz3, perhaps because different data patterns are written to each disk, protecting you against some potential data-sensitive deficiency in the on-disk error recovery. If you want any more than 3 faults, your only choice is mirrors (for now). By - or even well before - that stage, you should be looking at off-site replication. -- Dan. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 194 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100125/56755b44/attachment.bin>
What an entertaining discussion! Hope the following adds to the entertainment value :). Any comments on this Dec. 2005 study on disk failure and error rates? http://research.microsoft.com/apps/pubs/default.aspx?id=64599 Seagate says their 1.5TB consumer grade drives are good for 24*365 operation. http://labs.google.com/papers/disk_failures.pdf implies yes. This paper is quite interesting. Power cycles - bad. High temps - not so bad... The specs say an annualized failure rate of 0.34% and mean time between failures of 750,000 hours. 8760/750,000 = 1.17%. Hmmm. So around one disk in a hundred will fail each year? What does that mean to a system with a simple mirror if one disk in 20 will fail in 5 years? What is the MTTDL of a mirrored pair of consumer grade 1.5TB drives, or the probability of a single data loss (say) during a 5 year period, perhaps compared to the probability (say) of winning the lottery :-), or being hit by a 20 ton meteor, assuming at least one device failure? The OP originally asked "Best 1.5TB drives for consumer RAID?". Despite the entertainment value of the comments, it isn''t clear that this has been answered. I suspect the OP was expecting a discussion of WD vs. Seagate vs. Hitachi, etc., but the discussion didn''t go that way, perhaps because they are equally good (or bad) based on the TLER discussion? Has anyone actually experienced an extended timeout from one of these drives (from any manufacturer) causing a problem? On 01/24/10 11:26 AM, R.G. Keen wrote:> The most durable data backup medium humans have come up with was > invented about 4000-6000 years ago. It''s fired cuniform tablets as > used in the Middle East. Perhaps one could include stone carvings of > Egyptian and/or Maya cultures in that.Just trying to picture 1.5TB of data as cuniform tablets :-). On 01/24/10 12:44 PM, Tim Cook wrote:> 10%? I''m not sure where you shop, but no. The cheapest 500GB is > $39.99. The cheapest 1.5TB is $97.99. > > $359.91 for the 500GB drives. > $391.96 for the 1.5TB drives.You are comparing 9*500GB vs. 4*1.5TB? Wouldn''t you need 12*500GB for an apples-to-apples comparison? Surely a 3-way mirror is very unlikely to lose any bits under any reasonable scenario (see above pdfs). $97.99 * 3 = $293.97. Also, since newer drives tend to use less absolute power, 9*500GB drives are probably going to use significantly more than 3*1.5TB drives. perhaps 4 or 5 times as much; they''d be proportionately more difficult to cool too, which might reduce reliability. 16.5W (ST31500541AS) vs. 75W (WD5000AAKS) when idle - that''s about $50/year at $0.10 per KWH, not including cooling + extra controllers, fans, and a beefier power supply. Sounds like much more than a $200 advantage over 3 years, enough for 2 more 1.5TB drives for backups/spares. Does the 32GB vs 8GB of cache count for anything? On 01/23/10 04:20 PM, Richard Elling wrote:> My theory is that drives cost $100. When the price is> $100, the drive is > manufactured. When the price is< $100, the drive is EOL and the manufacturer > is flushing the inventory. Recently, 1.5 TB drives went below $100.Yes, for Seagate, a newish model, ST31500541AS (5900RPM 1.5GB drive). AFAIK the older ST31500341AS (7200RPM 1.5GB drive) is still over $100. The 2 month or so old ST31500541AS can hardly be EOLed at this point! However, it escapes me why anyone would not want to spend the extra $10 for the faster and more mature drive unless power consumption is important. On 01/23/10 01:15 PM, Bob Friesenhahn wrote:> This looks like a really good drive for use with zfs. Be sure to use aAren''t /all/ drives good for use with ZFS? Personally, AFAIK, I''ve never lost a single bit of data using ZFS, even using grotty hardware (well, had to use copies=2 on a mirror of very old IDE drives, but what other FS lets you do that?).> mirror configuration and keep in mind that zfs supports an arbitrary > number of mirrors so that you can run six or ten of these drives in > parallel so that there are enough working drives remaining to keep up > with RMAed units.You seem to have it in for Seagate :-). Newegg by default displays reviews worst to best. The review statistics as of 23 Jan 2010) were: ST31500341AS (older, 7200RPM 1.5GB drive) Excellent 911 - 49% Good 233 - 12% Average 113 - 6% Poor 123 - 6% Very Poor 514 - 27% ST31500541AS (more recent, 5900RPM 1.5GB drive). Excellent 64 - 35% Good 26 - 15% Average 11 - 6% Poor 14 - 8% Very Poor 64 - 36% The approval rate seems to get better as the product matures, surprise, surprise. Also, many of the "Very Poor" seemed to have problems that are not the fault of the drive ("stuck at 73% of capacity...").> Be sure to mark any failed drive using a sledgehammer so that you don''t > accidentally use it again by mistake.In my experience, failed drives rarely resuscitate themselves :-), so it''s pretty easy to tell them apart. However every Seagate 1.5TB drive ever sold is still under warranty, so I suggest rather than smashing them, RMA them. They don''t seem to be that bad, based on the professional reviews - actually some are almost ecstatic. Has anyone reading this who uses them had one fail? Cheers -- Frank
On Sun, 24 Jan 2010, Frank Middleton wrote:> > You seem to have it in for Seagate :-). Newegg by default displays reviews > worst to best. The review statistics as of 23 Jan 2010) were: > > ST31500341AS (older, 7200RPM 1.5GB drive) > > Excellent 911 - 49% > Good 233 - 12% > Average 113 - 6% > Poor 123 - 6% > Very Poor 514 - 27%It is always wise to consider that people with good experiences have little incentive to come back and give a rating whereas people with bad experiences have considerable incentive to do so. The text of the comments is much more useful than raw percentages. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
On Jan 24, 2010, at 8:26 PM, Frank Middleton wrote:> What an entertaining discussion! Hope the following adds to the > entertainment value :). > > Any comments on this Dec. 2005 study on disk failure and error rates? > http://research.microsoft.com/apps/pubs/default.aspx?id=64599 > > Seagate says their 1.5TB consumer grade drives are good for 24*365 > operation. http://labs.google.com/papers/disk_failures.pdf implies yes. > This paper is quite interesting. Power cycles - bad. High temps - not > so bad... > > The specs say an annualized failure rate of 0.34% and mean time between > failures of 750,000 hours. 8760/750,000 = 1.17%. Hmmm. So around one > disk in a hundred will fail each year? What does that mean to a system with > a simple mirror if one disk in 20 will fail in 5 years?Unfortunately, this is marketeering and you need to look at the footnotes to get the real story. http://blogs.sun.com/relling/entry/awesome_disk_afr_or_is> What is the MTTDL of a mirrored pair of consumer grade 1.5TB drives, > or the probability of a single data loss (say) during a 5 year period, > perhaps compared to the probability (say) of winning the lottery :-), > or being hit by a 20 ton meteor, assuming at least one device failure?MTTDL using model[2] for a Seagate ST31500341AS and 7x24x365 operation: MTBF = 700,000 hours UER = 1 error per 1e14 bits read, max 1.5 TB = 2,930,277,168 512-byte sectors Precon_fail = 2,930,277,168 * 512 bytes/sector * 8 bits/byte / 1e-14 = 0.12 MTTDL = 700,000 / (2 * 0.12) = 2,916,666 hours This is not that great, really. To bring it back to something a bit more understandable, it is an annualized data loss rate of 0.3 %. references for above: http://www.seagate.com/staticfiles/support/disc/manuals/desktop/Barracuda%207200.11/100507013e.pdf> The OP originally asked "Best 1.5TB drives for consumer RAID?". Despite > the entertainment value of the comments, it isn''t clear that this has been > answered. I suspect the OP was expecting a discussion of WD vs. Seagate > vs. Hitachi, etc., but the discussion didn''t go that way, perhaps because > they are equally good (or bad) based on the TLER discussion? Has anyone > actually experienced an extended timeout from one of these drives (from > any manufacturer) causing a problem?Extended timeouts lead to manual intervention, not a change in the probability of data loss. In other words, they affect the MTTR, not the reliability. For a 7x24x365 deployments, MTTR is a concern because it impacts availability. For home use, perhaps not so much. -- richard
On 24 janv. 2010, at 08:36, Erik Trimble wrote:> These days, I''ve switched to 2.5" SATA laptop drives for large-storage requirements. > They''re going to cost more $/GB than 3.5" drives, but they''re still not horrible ($100 for a 500GB/7200rpm Seagate Momentus). They''re also easier to cram large numbers of them in smaller spaces, so it''s easier to get larger number of spindles in the same case. Not to mention being lower-power than equivalent 3.5" drives.Ditto. After doing a quick check of the power consumption of various drives it''s clear that 2.5" drives are significantly less power-hungry, and with 500GB drives it''s entirely reasonable for many workloads as far as capacity requirements go. Even with 5400RPM mechanisms, it''s more than enough for most home server IOPS requirements, especially if you throw a few more axes at the server.> My sole problem is finding well-constructed high-density 2.5" hot-swap bay/chassis setups. > If anyone has a good recommendation for a 1U or 2U JBOD chassis for 2.5" drives, that would really be helpful.Not cheap, but I''ve used the HP MSA70 a while ago and was quite happy with the results (<http://h18006.www1.hp.com/storage/disk_storage/msa_diskarrays/drive_enclosures/msa70/specs.html>) And Dell has recently joined the crowd with the MD1120. I''ve used the MD1000 enclosures with 3.5" drives in many installations. (<http://www.dell.com/us/en/business/storage/storage_powervault_md1120/pd.aspx?refid=storage_powervault_md1120&s=bsd&cs=04>) Although both of those models talk about supporting Nearline SATA so I don''t know if they''ll take a regular off the shelf SATA laptop drive. Outside of that range, I''ve recently been looking at rebuilding my home storage server with a full-sized tower (something like: <http://www.xcase.co.uk/PC-A17A-Aluminium-Silver-Case-No-PSU-p/lili-a17a.htm>) and filling in front facing bays with multiple SuperMicro 8in2 chassis (<http://www.supermicro.com/products/accessories/mobilerack/CSE-M28E2.cfm>) which have include an 2x expander and are can be cascaded internally, so you should be able to add modules as capacity requirement grow. Cheers, Erik
> Extended timeouts lead to manual intervention, not a > change in the > probability of data loss. In other words, they > affect the MTTR, not > the reliability. For a 7x24x365 deployments, MTTR is > a concern because > it impacts availability. For home use, perhaps not so > much. > -- richardAs a home user, not running 24/7, replacing a problematic drive manually is not a problem. My main concern would be not to lose the whole array due to additional failing drives during the recovery process. Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
> Any comments on this Dec. 2005 study on disk failure > and error rates? > http://research.microsoft.com/apps/pubs/default.aspx?i > d=64599Will take a read...> The OP originally asked "Best 1.5TB drives for > consumer RAID?". Despite > the entertainment value of the comments, it isn''t > clear that this has been > answered. I suspect the OP was expecting a discussion > of WD vs. Seagate > vs. Hitachi, etc., but the discussion didn''t go that > way, perhaps because > they are equally good (or bad) based on the TLER > discussion? Has anyone > actually experienced an extended timeout from one of > these drives (from > any manufacturer) causing a problem?>From what I''ve managed to discover, rightly or wrongly, here is how I see it:* it appears as if the most recent revisions of some models of the WD Green ''EADS'' and newer Advanced Format ''EARS'' drives have some problem which puts me off using them for now (see links in first post + google). They also appear to have disabled user setting of TLER. * Some Seagate 1.5TB models, like the one discussed in this discussion, appear to have low user ratings, and many of the user comments mention clicking noises & failures. * Hitachi models I don''t know enough about yet, but I would rather avoid using 5-platter models like one of the 2TB models. * Samsung have a 3-platter 1.5TB model (HD154UI), which seems to have quite high user satisfaction levels and you can set the error reporting time, but it will not persist after power off. * Samsung also have a 4-platter 2TB model (HD203WI), which appears to have excellent user ratings, and no DOAs listed, but as there are only a small number of ratings left (<20), it is too early to make a judgement, but early data seems to be very promising. Based on the above, and with further reading required, at this stage, I will almost certainly be choosing the Samsung HD154UI. But let''s keep an eye on the HD203WI, because when the price drops a bit further and if more positive data appears, this might be a great model to consider for those people replacing / upgrading drives. And regarding your reply here to a comment from Bob on the Seagate model discussed:> You seem to have it in for Seagate :-). Newegg by > default displays reviews > worst to best.Bob was joking around about the Seagates :) And newegg don''t list reviews/ratings by default in worst to best order -- I posted that link using that order so that it was easy to see the kind of problems people were commonly listing. The things one wants to see before choosing.> > Be sure to mark any failed drive using a > sledgehammer so that you don''t > > accidentally use it again by mistake.Again, humour alert from Bob :) Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
With the absolutely deplorable reliability of drives >1TB why would one even waste their money? The 500GB RE2/3 and NS drives are very reliable and <$.12/gb. I get new drives off ebay all the time. NAS speed is all about spindles. 6 spindles will always outrun a setup with 3. Almost any mid-size ATX case can hold 6+ drives. Personally I use 4-in-3 units in a 9-bay case and a SAS expander. 16-20 drives in a $60 case. The 4-in-3 are like $25 each and a good 12v/60A power supply $100. It''s bigger than a Dell MD1000 or other specialized design but it works great and is cheap.
I''ve been using 10 Samsung eco greens in a raidz2 on freebsd for about 6 months. (Yeah I know it''s above 9, the performance is fine for my usage though) Haven''t had any problems. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100125/60135e84/attachment.html>
We have the WDC WD15EADS-00P8B0 1.5 TB Caviar Green drives. Unfortunately, these drives have the "fixed" firmware and the 8 second idle timeout cannot be changed. Since we starting replacing these drives in our pool about 6 weeks ago (replacing 1 drive per week), the drives has registered almost 40,000 Load Cycles (head parking cycles). At this rate, they won''t last more than a year. :( Neither the wdidle3 nor the wdtler utilities will work with these drives. The RE2/RE4-GP drives can be configured with a 5 minute idle timeout. -- This message posted from opensolaris.org
> We have the WDC WD15EADS-00P8B0 1.5 TB Caviar Green > drives. > > Unfortunately, these drives have the "fixed" firmware > and the 8 second idle timeout cannot be changed. > Since we starting replacing these drives in our pool > about 6 weeks ago (replacing 1 drive per week), the > drives has registered almost 40,000 Load Cycles > (head parking cycles). At this rate, they won''t > last more than a year. :( Neither the wdidle3 nor > the wdtler utilities will work with these drives.Thanks for posting your experiences here. This could be where the attempt to use less energy too aggressively ends up making the drives fail prematurely, and thus costing you more in the long run... I wonder if someone has done the math of the cost of failed drives versus money saved due to drives using less energy? How many load cycles are those drives quoted to be good for? As the revision ''00P8B0'' was the one quoted in the initial post''s WDC / Synology links, I would appreciate any further reliability / problem feedback you have regarding using these drives in a RAID array. Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
Good news. Are those the HD154UI models? Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
On 25-Jan-10, at 2:59 PM, Freddie Cash wrote:> We have the WDC WD15EADS-00P8B0 1.5 TB Caviar Green drives. > > Unfortunately, these drives have the "fixed" firmware and the 8 > second idle timeout cannot be changed.That sounds like a laptop spec, not a server spec! How silly. Maybe you can set up a tickle job to stop them idling during busy periods. :( --Toby> Since we starting replacing these drives in our pool about 6 weeks > ago (replacing 1 drive per week), the drives has registered almost > 40,000 Load Cycles (head parking cycles). At this rate, they won''t > last more than a year. :( Neither the wdidle3 nor the wdtler > utilities will work with these drives. > > The RE2/RE4-GP drives can be configured with a 5 minute idle timeout. > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
On January 24, 2010 12:20:55 PM -0800 "R.G. Keen" <keen at geofex.com> wrote:> I do apologize for the snottier parts of my reply to your first note, > which I am editing. I did not get a chance to read this note from you > before responding.Oh not at all. Snotty is as snotty does. um, what that is supposed to mean is -- I deserved it. :) I''m sure I''ve said this 3 times already in different ways, but I just thought you were generalizing that if you bought smaller drives, which are cheaper, since the drives are cheaper that allows you to buy more of them and thus have more parity. This violated a primary assumption of mine, that you always buys drives solely based on how much data you need to store, and then you factor in the level of redundancy you require. By that assumption, you would always want to buy the drives which are the least $/GB and which your data still fits into nicely. (If you have 2.1TB of data, you wouldn''t buy 2x2TB drives, and please ignore the loss due to base 10 vs base 2 and filesystem overhead in that statement.) I see that your goals are completely different. -frank
> > On 25-Jan-10, at 2:59 PM, Freddie Cash wrote: > > > We have the WDC WD15EADS-00P8B0 1.5 TB Caviar Green > drives. > > > > Unfortunately, these drives have the "fixed" > firmware and the 8 > > second idle timeout cannot be changed. > > That sounds like a laptop spec, not a server spec! > How silly. Maybe you can set up a tickle job to stop them idling > during busy periods. :(I''m running a 5-second loop using smartctl to twiddle the drives. So far, so good. Not sure how things will run long-term, though. :( -- This message posted from opensolaris.org
> With the absolutely deplorable reliability of drives > >1TB why would one even waste their money? The 500GB > RE2/3 and NS drives are very reliable and <$.12/gb. I > get new drives off ebay all the time.Perhaps because you only have X number of drive bays, and your dataset is larger than the current size of the pool? Whereby the only easy way to expand the pool is to increase the size of the drives. -- This message posted from opensolaris.org
On Sat, Jan 16, 2010 at 9:47 AM, Simon Breden <sbreden at gmail.com> wrote:> Which consumer-priced 1.5TB drives do people currently recommend?I happened to be looking at the Hitachi product information, and noticed that the Deskstar 7K2000 appears to be supported in RAID configurations. One of the applications listed is "Video editing arrays". http://www.hitachigst.com/portal/site/en/products/deskstar/7K2000/ -B -- Brandon High : bhigh at freaks.com If violence doesn''t solve your problem, you''re not using enough of it.
On Tue, Feb 2, 2010 at 1:38 PM, Brandon High <bhigh at freaks.com> wrote:> On Sat, Jan 16, 2010 at 9:47 AM, Simon Breden <sbreden at gmail.com> wrote: > > Which consumer-priced 1.5TB drives do people currently recommend? > > I happened to be looking at the Hitachi product information, and > noticed that the Deskstar 7K2000 appears to be supported in RAID > configurations. One of the applications listed is "Video editing > arrays". > > http://www.hitachigst.com/portal/site/en/products/deskstar/7K2000/ > >I''ve been having good success with the Western Digital Caviar Black drives...which are cousins of their Enterprise RE3 platform. AFAIK, you''re stuck at 1TB or 2TB capacities but I''ve managed to get some good deals on them... -marc -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100202/c52b6d9b/attachment.html>
The thing that puts me off the 7K2000 is that it is a 5 platter model. The latest 2TB drives use 4 x 500GB platters. A bit less noise, vibration and heat, in theory :) And the latest 1.5TB drives use only 3 x 500GB platters. -- This message posted from opensolaris.org
IIRC the Black range are meant to be the ''performance'' models and so are a bit noisy. What''s your opinion? And the 2TB models are not cheap either for a home user. The 1TB seem a good price. And from what little I read, it seems you can control the error reporting time with the WDTLER.EXE utility :) Cheers, Simon -- This message posted from opensolaris.org
I''m running the 500GB models myself, but I wouldn''t say they''re overly noisy....and I''ve been doing ZFS/iSCSI/IOMeter/Bonnie++ stress testing with them. They "whine" rather than "click" FYI. -marc On Tue, Feb 2, 2010 at 2:58 PM, Simon Breden <sbreden at gmail.com> wrote:> IIRC the Black range are meant to be the ''performance'' models and so are a > bit noisy. What''s your opinion? And the 2TB models are not cheap either for > a home user. The 1TB seem a good price. And from what little I read, it > seems you can control the error reporting time with the WDTLER.EXE utility > :) > > Cheers, > Simon > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100202/b6593037/attachment.html>
On February 2, 2010 11:58:12 AM -0800 Simon Breden <sbreden at gmail.com> wrote:> IIRC the Black range are meant to be the ''performance'' models and so are > a bit noisy. What''s your opinion? And the 2TB models are not cheap either > for a home user. The 1TB seem a good price. And from what little I read,It depends what you mean by cheap. As we''ve recently learned, cheaper is not necessarily cheaper. :) What I mean is, it depends how much data you have. If 2TB drives allow you to use only 1 chassis, you save on power consumption. Fewer spindles also will save on power consumption. However, w/ 2TB drives you may need to add more parity (raidz2 vs raidz1, e.g.) to meet your reliability requirements -- the time to resilver 2TB may not meet your MTTDR reqs. So you have to include your reliability needs when you figure cost. That said, I doubt 2TB drives represent good value for a home user. They WILL fail more frequently and as a home user you aren''t likely to be keeping multiple spares on hand to avoid warranty replacement time. -frank
On Tue, Feb 2, 2010 at 3:11 PM, Frank Cusack <frank+lists/zfs at linetwo.net>wrote:> > That said, I doubt 2TB drives represent good value for a home user. > They WILL fail more frequently and as a home user you aren''t likely > to be keeping multiple spares on hand to avoid warranty replacement > time.I''m having a hard time convincing myself to go beyond 500GB....both for performance (I''m trying to build something with reasonable IOPS) and reliability reasons. -marc -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100202/9f0a3d91/attachment.html>
Le 02/02/2010 20:26, Marc Nicholas a ?crit :> On Tue, Feb 2, 2010 at 1:38 PM, Brandon High <bhigh at freaks.com > <mailto:bhigh at freaks.com>> wrote: > > On Sat, Jan 16, 2010 at 9:47 AM, Simon Breden <sbreden at gmail.com > <mailto:sbreden at gmail.com>> wrote: > > Which consumer-priced 1.5TB drives do people currently recommend? > > I happened to be looking at the Hitachi product information, and > noticed that the Deskstar 7K2000 appears to be supported in RAID > configurations. One of the applications listed is "Video editing > arrays". > > http://www.hitachigst.com/portal/site/en/products/deskstar/7K2000/ > > > I''ve been having good success with the Western Digital Caviar Black > drives...which are cousins of their Enterprise RE3 platform. AFAIK, > you''re stuck at 1TB or 2TB capacities but I''ve managed to get some > good deals on them... > > -marc >We''re running HDS722020ALA330 (revisions A20N and A28A - 8 of each) for 5 weeks now without any problems. We also have 14 WD20EADS (at least 3 different versions starting with 00S, 00R, 32S - can''t tell right now, inquiry is truncated and the disks are in the servers). All of these disks were bought at the same time (about 5 weeks ago) and while we had no problems with the hitachi drives, we already had 4 failures with the WD, which apparently can not be linked to the server, since both servers are the same and the disks are mixed on the controllers (part of them WD, part of them hitachi). On the performance side, Hitachi drive seem more responsive (disks are configured in raidz3 and the limiting factor in IOs is the WD drives). Regarding the sequential write perf we made some tests with dd, writing at the beginning of the disk and at the end of the disk (512MB each time). Hitachi sustains 130MB/s at the beginning of the drive and 75MB/s at the end. For WD, the figures are 110MB/s and 55MB/s. These figures are from memory. We decided that we won''t buy any WD drives. When warranty''s over they''ll get replaced with hitachis. --arnaud -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100203/54eba60a/attachment.html>
smartmontools doesn''t work with my controllers. I can try it again when the 2 new drives I''ve ordered arrive. I''ll try connecting to the motherboard ports and see if that works with smartmontools. I haven''t noticed any sleeping with the drives. I don''t get any lag accessing the array or any error messages about them disappearing. -- This message posted from opensolaris.org
That''s a pity about smartmontools not working. Which controllers are you using? Good news about no sleeping though, although perhaps not so economical. I think I''d rather burn a bit more power and have drives that respond properly than weird timeout issues some people seem to be experiencing with some of the green low power drives. Cheers, Simon -- This message posted from opensolaris.org
Sounds good. I was taking a look at the 1TB Caviar Black drives which are WD1001FALS I think. They seem to have superb user ratings and good reliability comments from many people. I consider these "full fat" drives as opposed to the LITE (green) drives, as they spin at 7200 rpm instead of 5400 rpm, have higher performance and burn more juice than the Green models, but they have superb reviews from almost everyone regarding behaviour and reliability, and at the end of the day, we need good, reliable drives that work well in a RAID system. I can get them for around the same price as the cheapest 1.5TB green drives from Samsung. Somewhere I saw people saying that WDTLER.EXE works to allow reduction of the error reporting time like the enterprise RE versions (RAID Edition). However I then saw another user saying on the newer revisions WD have disabled this. I need to check a bit more to see what''s really the case. Cheers, Simon http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ -- This message posted from opensolaris.org
As I previously mentioned, I''m pretty happy with the 500GB Caviar Blacks that I have :) One word of caution: failure and rebuild times with 1TB+ drives can be a concern. How many spindles were you planning? -marc On 2/3/10, Simon Breden <sbreden at gmail.com> wrote:> Sounds good. > > I was taking a look at the 1TB Caviar Black drives which are WD1001FALS I > think. > They seem to have superb user ratings and good reliability comments from > many people. > > I consider these "full fat" drives as opposed to the LITE (green) drives, as > they spin at 7200 rpm instead of 5400 rpm, have higher performance and burn > more juice than the Green models, but they have superb reviews from almost > everyone regarding behaviour and reliability, and at the end of the day, we > need good, reliable drives that work well in a RAID system. > > I can get them for around the same price as the cheapest 1.5TB green drives > from Samsung. > Somewhere I saw people saying that WDTLER.EXE works to allow reduction of > the error reporting time like the enterprise RE versions (RAID Edition). > However I then saw another user saying on the newer revisions WD have > disabled this. I need to check a bit more to see what''s really the case. > > Cheers, > Simon > > http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/ > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- Sent from my mobile device
Probably 6 in a RAID-Z2 vdev. Cheers, Simon -- This message posted from opensolaris.org
I think you''ll do just fine then. And I think the extra platter will work to your advantage. -marc On 2/3/10, Simon Breden <sbreden at gmail.com> wrote:> Probably 6 in a RAID-Z2 vdev. > > Cheers, > Simon > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- Sent from my mobile device
Hi Arnaud, which type of controller is this? Regards, Tonmaus -- This message posted from opensolaris.org
Le 04/02/10 16:57, Tonmaus a ?crit :> Hi Arnaud, > > which type of controller is this? > > Regards, > > Tonmaus >I use two LSI SAS3081E-R in each server (16 hard disk trays, passive backplane AFAICT, no expander). Works very well. Arnaud
Hi again, thanks for the answer. Another thing that came to my mind is that you mentioned that you mixed the disks among the controllers. Does that mean you mixed them as well among pools? Unsurprisingly, the WD20EADS is slower than the Hitachi that is a fixed 7200 rpm drive. I wonder what impact that would have if you use them as vdevs of the same pool. Cheers, Tonmaus -- This message posted from opensolaris.org
Supermicro USAS-L8i controllers. I agree with you, I''d much rather have the drives respond properly and promptly than save a little power if that means I''m going to get strange errors from the array. And these are the "green" drives, they just don''t seem to cause me any problems. The issues people have noted with WD have made me stay away from them as just about every drive I own lives in some kind of RAID sometime in its life. I have a couple laptop drives that are single, all desktops have at least a mirror. I''m a little nuts and would probably install mirrors in the laptops if there were somewhere to put them. :) -- This message posted from opensolaris.org
Le 04/02/10 20:26, Tonmaus a écrit : Hi again, thanks for the answer. Another thing that came to my mind is that you mentioned that you mixed the disks among the controllers. Does that mean you mixed them as well among pools? Unsurprisingly, the WD20EADS is slower than the Hitachi that is a fixed 7200 rpm drive. I wonder what impact that would have if you use them as vdevs of the same pool. Cheers, Tonmaus Yes, we mixed them among controllers and pools. We''ve done something that''s not recommended : a 15 disk raidz3 pool. Disks are as follows : c3 (LSI SAS) has : - 1x 64 GB Intel X25E - 3 x 2TB WD20EADS - 4 x 2TB Hitachi c2 (LSI SAS) has : - 4 x 2TB WD20EADS - 4 x 2TB Hitachi c5 (motherboard ICH10 if I remember well) has : - 1x160GB 2,5'''' WD - DVD All the 2TB drivers are in the raidz3 zpool named tank (we''ve been very innovative here ;-). X25E is sliced in 20GB for the system, 1GB for ZIL for tank, the rest as cache for tank. The 2,5'''' 160GB WD was not initially part of the setup since we were planning to slice the 2TB drives in 32GB for the system (mirrored accross all drives) and the rest for the big zpool, while the X25E was just there for the ZIL and the cache, but two things we''ve read on lists and forums made us change our minds : - the disk write cache is disabled when you''re not using the whole drive - some reports on this list about X25E loosing up to 256 cache flushes in case of power failures. So we bought this 160GB disk (it was really the last thing that could fit in the chassis) and sliced it in the same way as the X25E. The system and the ZIL are mirrored between the X25E and the WD160. We do not use the WD160 for the cache : we thought it would be better to save IOPS on this disk for the ZIL mirror. I don''t know wether it''s a good idea to mirror the ZIL on such a disk but we prefer having slower setup and not loose that much cache flushes on power failure. Regarding the perfs obtained by using only Hitachi disks, I can''t tell, I haven''t tested it, and can''t do it right now as the system is in preproduction testing. Also, I should have mentionned in my previous post that some WD20EADS (the 32SB0) have shorter reponse times (as reported by iostat). They''re even "faster" than the Hitachi : I''ve seen them quite a few times in the range 0.3 to 1.5 ms, which seems far to short for this kind of drives. I suspect they''re sort of dropping flush requests. Add to it that 2 out of 3 failed WD20EADS were 32SB0 and you get the picture... Note they might also be hybrid drives with some flash memory which allows quick acknoledgment of writes, but I think we would have heard of such a feature on this list. Arnaud _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss