> Actually I have 3 supermicro servers with 12 4TB SATA disks each and 2 > SSD (in each server) > Each server also has one dual port DDR Infiniband card. > > I would like to create a scale-out storage infrastructure (primary > used by web servers), totally HA and fault tollerant. > I was thinking about 1 brick for each SATA disks in Distributed > Dispersed mode. Replica set to 3 (so, actually, only 12*4TB=48TB would > be available) > > What do you suggest? Is Distributed Dispersed good for my environment > or should I go with Distributed Replicated ? > > In replicated mode, I can always access to raw files , in case of > disaster, this would not be possible with dispersed mode, right? > > Which are pro and cons between replicated and dispersed modes? > > We plan to add up to 10 servers (all with 12*4 SATA disks) in the near > future ending to 336TB of available and replicated space. > > Any suggestions?The key tradeoffs here are storage utilization vs. performance. In general, erasure codes (disperse) will give better storage utilization than replication for the same level of performance. However, this might not be the case for N=3. With replication, that will protect against two failures. However, from the admin guide section on disperse: "redundancy_ must be greater than 0, and the total number of bricks must be greater than 2 * _redundancy_" I interpret this to mean that for two-failure protection you would need at least five bricks. With three bricks disperse can only offer one-failure protection. In this case it's roughly equivalent to RAID-5, with only a 50% storage penalty vs. 100% for replica 2 offering the same protection. The other issue is performance. With disperse, all writes *and reads* must be done to all bricks, and at a stripe size equal to 512 times the number of bricks (minus those used for redundancy). This means more data transfer, especially for reads, and also more write contention than with replication. This being new code, some optimizations that already exist for replication do not yet exist for disperse even though they're applicable. Adding Xavier, who's the real expert on disperse, in case I got something wrong here.
Gandalf Corvotempesta
2014-Dec-23 10:08 UTC
[Gluster-users] New architecture: some advice needed
2014-12-22 13:56 GMT+01:00 Jeff Darcy <jdarcy at redhat.com>:> The key tradeoffs here are storage utilization vs. performance. In > general, erasure codes (disperse) will give better storage utilization > than replication for the same level of performance. However, this might > not be the case for N=3. With replication, that will protect against > two failures. However, from the admin guide section on disperse: > > "redundancy_ must be greater than 0, and the total number of bricks must > be greater than 2 * _redundancy_" > > I interpret this to mean that for two-failure protection you would need > at least five bricks. With three bricks disperse can only offer > one-failure protection. In this case it's roughly equivalent to RAID-5, > with only a 50% storage penalty vs. 100% for replica 2 offering the same > protection. > > The other issue is performance. With disperse, all writes *and reads* > must be done to all bricks, and at a stripe size equal to 512 times the > number of bricks (minus those used for redundancy). This means more > data transfer, especially for reads, and also more write contention than > with replication. This being new code, some optimizations that already > exist for replication do not yet exist for disperse even though they're > applicable.Thank you for the response. So, if I understood properly, disperse is space-optimized but has a performance penalty compared to a standard 'replication' What I would like is to get a RAID-6 equivalent, so, replica 2 should be used to get performance and redudancy, right ? More over, with replication, in case of issue, I can always read raw files from disks, this would not be possible in case of disperse, where each file is splitted in multiple chuncks, right? Any hardware suggestions for my environment ?