Gluster users - Feb 2017 - distribute replicated volume and tons of questions

2017-02-22 21:04 GMT+01:00 Joe Julian <joe at
julianfamily.org>:
> dedup requires massive amounts of memory and is seldom worth it.
Yes, but compression is usefull
> Which is why I don't like building raid volumes that large.
>
> Personally, I only use raid on the servers to allow the disk i/o to match
> the network i/o. If that means those 12 8TB disks need to be 3 raid 0
> volumes (bricks) where I do sharded replica 3 or disperse volumes to meet
my
> redundancy requirements, then that's what I'll do.
With gluster you could avoid raid, but you still need a filesystem on
each brick.
RAID or Non-RAID, an XFS fsck still need a week to fix (if able to
fix) a 12x 8TB chassis
in a non raid configuration. I don't think that fsck is run in
parallel, it will blow down the whole server.
> This is where people panic about using raid 0. If you've got the
redundancy,
> that shouldn't be that scary. Do the math and actually calculate your
> reliability. I can still get 6 nines with raid 0 bricks. Not to say you
> should use raid 0, just to keep an open mind about what possibilities exist
> and engineer to your SLA requirements rather than over engineering for
> things that may not matter in the long run.
I don't like RAID (that's why i'm migrating to gluster)
I prefere to use gluster on single bricks

On 02/22/17 12:11, Gandalf Corvotempesta wrote:
> 2017-02-22 21:04 GMT+01:00 Joe Julian <joe at julianfamily.org>:
>> dedup requires massive amounts of memory and is seldom worth it.
> Yes, but compression is usefull
I've been using btrfs for that. In my own tests, btrfs has performed 
better for my use cases.
>
>> Which is why I don't like building raid volumes that large.
>>
>> Personally, I only use raid on the servers to allow the disk i/o to
match
>> the network i/o. If that means those 12 8TB disks need to be 3 raid 0
>> volumes (bricks) where I do sharded replica 3 or disperse volumes to
meet my
>> redundancy requirements, then that's what I'll do.
> With gluster you could avoid raid, but you still need a filesystem on
> each brick.
> RAID or Non-RAID, an XFS fsck still need a week to fix (if able to
> fix) a 12x 8TB chassis
> in a non raid configuration. I don't think that fsck is run in
> parallel, it will blow down the whole server.
fsck does run in parallel. A normal fsck of an xfs filesystem exits 
instantly because xfs_check is the command to check an xfs filesystem. 
It is normally run only when there is reason to believe that the 
filesystem has a consistency problem.
>
>> This is where people panic about using raid 0. If you've got the
redundancy,
>> that shouldn't be that scary. Do the math and actually calculate
your
>> reliability. I can still get 6 nines with raid 0 bricks. Not to say you
>> should use raid 0, just to keep an open mind about what possibilities
exist
>> and engineer to your SLA requirements rather than over engineering for
>> things that may not matter in the long run.
> I don't like RAID (that's why i'm migrating to gluster)
> I prefere to use gluster on single bricks