Gluster users - Feb 2017 - distribute replicated volume and tons of questions

On 02/22/17 11:25, Gandalf Corvotempesta wrote:
> 2017-02-22 19:27 GMT+01:00 Joe Julian <joe at julianfamily.org>:
>> I can't answer ZFS questions. I, personally, don't feel
it's worth all the
>> hype it's getting and I don't use it.
> the alternative would be XFS, but:
> 1) i'm using XFS on a backup server. I've *NEVER* seen so many
crashes
> in latest 2 month than in my 15 years by using ext[[2-4]
> 2) ZFS has native bit-rot protection and data scrubbing
So does gluster
> 3) ZFS has de-duplication and compression
dedup requires massive amounts of memory and is seldom worth it.
> 4) In case of hardcrash, try to run a XFS fsck on a server with 12x
> 8TB disks..... Probably you need a week or more. on our backup server
> (8TB raid-6) it took 18 hours.
Which is why I don't like building raid volumes that large.

Personally, I only use raid on the servers to allow the disk i/o to 
match the network i/o. If that means those 12 8TB disks need to be 3 
raid 0 volumes (bricks) where I do sharded replica 3 or disperse volumes 
to meet my redundancy requirements, then that's what I'll do.

This is where people panic about using raid 0. If you've got the 
redundancy, that shouldn't be that scary. Do the math and actually 
calculate your reliability. I can still get 6 nines with raid 0 bricks. 
Not to say you should use raid 0, just to keep an open mind about what 
possibilities exist and engineer to your SLA requirements rather than 
over engineering for things that may not matter in the long run.

2017-02-22 21:04 GMT+01:00 Joe Julian <joe at
julianfamily.org>:
> dedup requires massive amounts of memory and is seldom worth it.
Yes, but compression is usefull
> Which is why I don't like building raid volumes that large.
>
> Personally, I only use raid on the servers to allow the disk i/o to match
> the network i/o. If that means those 12 8TB disks need to be 3 raid 0
> volumes (bricks) where I do sharded replica 3 or disperse volumes to meet
my
> redundancy requirements, then that's what I'll do.
With gluster you could avoid raid, but you still need a filesystem on
each brick.
RAID or Non-RAID, an XFS fsck still need a week to fix (if able to
fix) a 12x 8TB chassis
in a non raid configuration. I don't think that fsck is run in
parallel, it will blow down the whole server.
> This is where people panic about using raid 0. If you've got the
redundancy,
> that shouldn't be that scary. Do the math and actually calculate your
> reliability. I can still get 6 nines with raid 0 bricks. Not to say you
> should use raid 0, just to keep an open mind about what possibilities exist
> and engineer to your SLA requirements rather than over engineering for
> things that may not matter in the long run.
I don't like RAID (that's why i'm migrating to gluster)
I prefere to use gluster on single bricks