A little OT - but I've seen a few opinions voiced here by various admins and I'd like to benefit. Currently running a single combined server for multiple operations - fileserver, mailserver, webserver, virtual server, and whatever else pops up. Current incarnation of the machine, after the last rebuild, is an AMD Opteron 4180 with a Supermicro MB using ATI SB700 chipset - which means it supports SATA 3.0. Current storage is one o/s drive, and a 4-drive RAID10 array. The RAID10 is using the Linux softraid via mdadm. The drives are Seagates - ST3160811AS. So it's 320GB of storage, and SATA 1.5. I'm seeing some warnings & errors in my logs & dmesg - and Google tells me this can the result of several factors, including the hard drives. I haven't seen any SMART warnings as yet - but I am getting a little nervous and thinking about upgrading the storage. At this time, I'm just interested in recommendations for hard drives & partitioning strategy. At the time I created the RAID10 array - I was still in the early stages of learning these technologies (not that I know that much more now!) so I probably didn't take advantage of any of the advanced settings such as stripe size, cluster size, etc. I am using XFS. As part of the potential upgrade, I'm considering changing to RAID6 - seems a bit more efficient use of space. I see no reason for SSD - I think a set of reasonable 7200rpm drives should be just fine. What I don't know is, compared to my current 4 drive RAID10 with SATA 1.5, would even a single SATA 3 drive be comparable in terms of seek performance? Should I stick with the RAID-10? -- Daniel
SATA 1 vs SATA 3 won't change the seek performance of the drive, unless your stay fully within the drives cache. Only 2 things affect how many iops you can get from a drive. The physical diameter and the speed. You can affect the physical diameter of the drive by selecting how much of the drive you use, like only using 200gigs out of a 500gig drive. If you do opt for raid6, that would be a huge cpu overhead, and if you stick with 4 drives, not worth it, every write would hit atleast 3 drives then, instead of just two they do currently, with no gain of space. Quoting "Daniel L. Miller" <dmiller at amfes.com>:> A little OT - but I've seen a few opinions voiced here by various > admins and I'd like to benefit. > > Currently running a single combined server for multiple operations - > fileserver, mailserver, webserver, virtual server, and whatever else > pops up. Current incarnation of the machine, after the last > rebuild, is an AMD Opteron 4180 with a Supermicro MB using ATI SB700 > chipset - which means it supports SATA 3.0. > > Current storage is one o/s drive, and a 4-drive RAID10 array. The > RAID10 is using the Linux softraid via mdadm. The drives are > Seagates - ST3160811AS. So it's 320GB of storage, and SATA 1.5. > > I'm seeing some warnings & errors in my logs & dmesg - and Google > tells me this can the result of several factors, including the hard > drives. I haven't seen any SMART warnings as yet - but I am getting > a little nervous and thinking about upgrading the storage. > > At this time, I'm just interested in recommendations for hard drives > & partitioning strategy. At the time I created the RAID10 array - I > was still in the early stages of learning these technologies (not > that I know that much more now!) so I probably didn't take advantage > of any of the advanced settings such as stripe size, cluster size, > etc. I am using XFS. > > As part of the potential upgrade, I'm considering changing to RAID6 > - seems a bit more efficient use of space. I see no reason for SSD > - I think a set of reasonable 7200rpm drives should be just fine. > What I don't know is, compared to my current 4 drive RAID10 with > SATA 1.5, would even a single SATA 3 drive be comparable in terms of > seek performance? Should I stick with the RAID-10? > -- > Daniel
On 8/30/2011 5:43 PM, Daniel L. Miller wrote:> A little OT - but I've seen a few opinions voiced here by various admins > and I'd like to benefit.RAID-10 is fine (note that the default mdadm RAID10 isn't actually RAID10, but it works well enough). RAID-6 won't be faster (and will probably be worse) although RAID-6 does do a bit better in a double-drive failure over RAID-10. The only way to get more performance out of (4) drives is to switch to 10k or 15k SAS (or SSDs). For more information - see the Linux RAID mailing list: http://vger.kernel.org/majordomo-info.html One problematic issue with consumer-grade SATA drives (which may or may not bite you) is that they will not time out on errors fast enough to keep mdadm happy. The "enterprise" grade drives are better about this (such as the ES.2 series), but for smaller arrays (6 drives or less) it's not as big of a deal. For bigger arrays, it's a definite issue, especially if you try and do RAID-6 over 8+ drives. If you're getting SMART errors, then it's time to swap the drives out. If mdadm is reporting sync errors or dropping drives from the array, then get your backups squared away ASAP before fiddling. My knee-jerk reaction when I hear 4-drive RAID-10 is that it has no hot-spare. Which means that as soon as 1 drive fails you're in dangerous territory (make sure it pages you automatically) since the array can't automatically repair. Make sure you can properly identify the drive that fails (via the serial numbers) and don't try a hot-swap. (Take a look at /dev/disk/by-id, /dev/disk/by-uuid, etc. Export a copy of that information on a daily/weekly basis off of the machine. In a software RAID environment, it gives you better information about which drive serial # failed rather then relying on lights.) Our mail server is 3-way RAID1 (triple mirror) for the OS and mail queue with a 5-disk RAID-10 (4+spare) for mail storage.
On 8/31/11, Daniel L. Miller <dmiller at amfes.com> wrote:> I'm seeing some warnings & errors in my logs & dmesg - and Google tells > me this can the result of several factors, including the hard drives. I > haven't seen any SMART warnings as yet - but I am getting a little > nervous and thinking about upgrading the storage.Possible to post some of those error messages?> As part of the potential upgrade, I'm considering changing to RAID6 - > seems a bit more efficient use of space. I see no reason for SSD - IIt might be more efficient but you'll take a 3x IOPS penalty for writes vs RAID 1/10. Depending on your exact workload, it is most likely going to be a bad idea since you mention that you don't actually need more space.> think a set of reasonable 7200rpm drives should be just fine. What I > don't know is, compared to my current 4 drive RAID10 with SATA 1.5, > would even a single SATA 3 drive be comparable in terms of seek > performance? Should I stick with the RAID-10?Seek performance is largely a matter of I/O latency, SATA 3 only gives you more bandwidth to play with. A SATA 3 4200RPM HDD will seek slower than a SATA 1 7200RPM HDD. For random I/O, more spindles/drives are usually the way to deal with it, like others suggested, splitting your drives into RAID 1 nodes and balancing different workloads on each node is probably your best bet if you don't mind doing it manually.