Gluster users - Oct 2020 - Low cost, extendable, failure tolerant home cloud storage question

Dear GlusterFS experts, I would like to have an advice (or more) from you.
Currently, I have a Qnap-419 (four disk) storage at home running in RAID5, this
allows me for one disk error (home storage project).
I would like to replace this storage with an another solution (also to overcome
storage bandwith limitation). Originally, I planned an ITX board with several
SATA ports and 4-6 HDD that I can extend later, so there should be plenty number
of SATA ports (also room for HDDs in the case). Also the possibility have to be
supported by the underlying filesystem.I found, that ZFS could do the trick.
Also stripe sets are available. Since a year or two, the stripe sets can also be
extended. So in my case, when there is a demand for more storage I can simply
add new disk(s) (1 or two) to the set and rebuild the stripe set (possibly
offline), then I will have some more space for my files without adding
additional volume or establish a new server. A few days ago, I figured out, that
a different approach would do it better and also can be more failure tolerant: 
like the cloud storage.
So my plan is changed to have small servers with storage (1 disk per server).
Than I started to search systems that fulfil my criteria.
Unfortunately, I found only GlusterFS that might be ok for this purpose, since I
do not want to have separete metadata server. Main conditions:
 - working like stripe sets, so unit failures will not cause data loss (1 or two
server failures are allowed)
 - scalable, and therefore
 - can be extended with one (or maybe two) server at a time (due to low budget)
 - cheap nodes (with 1-2GB of RAM) able to handle the task (like Rpi, Odroid XU4
or even HP T610)
 
As I read a lot about GlusterFS, I recetly concluded that this might also not be
possible with this FS.
At the beginning I thought, that I create 2+2 disperse volume (for optimal
sector size) that I can later extend to 4+2, even later 6+2 and continue.... Now
the question: Can I achieve this with GlusterFS? How? What configuration I must
choose? I am looking forward for advices or solutions with this tool.Regards,
Alex 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20201003/18ef1357/attachment.html>

>Main conditions:
> - working like stripe sets, so unit failures will not cause data loss (1 or
two server failures are allowed)
Gluster replica or dispersed volumes can tolerate that
> - scalable, and therefore
> - can be extended with one (or maybe two) server at a time (due to low
budget)
In order to be safe you need 'replica 3' or a disperse volumes.
In both cases extending by 1 brick (brick is not equal to node) is not possible
in most cases. For example in 'replica 3' you need to add 3 more bricks
(brick is a combination of 'server + directory' and it is recommended to
be on separate systems or it's a potential single point of failure).
Dispersed volumes also need to be extended in numbers of <disperse count>,
so if you have 4+2 (4 bricks , 2 are the maximum you can loose without dataloss
) - you need to add another 6 bricks to extend.
> - cheap nodes (with 1-2GB of RAM) able to handle the task (like Rpi,
>Odroid XU4 or even HP T610)
You need a little bit more ram for daily usage and most probably more cores as
healing of data in replica is demanding (dispersed volumes are like raid's
parity and require some cpu).
>As I read a lot about GlusterFS, I recetly concluded that this might also
>not be possible with this FS.
>At the beginning I thoug! ht, that I create 2+2 disperse volume (for
>optimal sector size) that I can later extend to 4+2, even later 6+2 and
>continue....
Actually a 2+2 disperse volume will be extended with another 4 disks and will
become a 2 * (2+2) , so you will have 2 subvolumes consisting of 2+2 and the
algorithm? will distribute the data to any of the 2 subvolumes.
?
>Now the question: Can I achieve this with GlusterFS? How? What
>configuration I must choose?
The idea with the ITX boards is not so bad. You can get 2 small systems and
create your erasure coding.

So you will have something like this:
ITX1:/gluster_bricks/HDD1/brick
ITX1:/gluster_bricks/HDD2/brick
ITX1:/gluster_bricks/HDD3/brick


ITX2:/gluster_bricks/HDD1/brick
ITX2:/gluster_bricks/HDD2/brick
ITX2:/gluster_bricks/HDD3/brick


Then when you create your dispersed volume you should take 3 bricks from ITX1
and 2 from ITX2 (4+2) .Once the volume is created you can expand it with another
set of disks .

In theory, failure of a system should be tolerable.

Yet, I would prefer the 'replica 3 arbiter 1' approach as it doesn't
take?so much space and extending will require only 2 data disks .
Have you thought about 'replica 3 arbiter 1' types of volumes ?
Of course you can build software/hardware raid and manage the disks underneath
without problems, but every time you change the raid shape you will need to wipe
the fs and 'reset-brick'.


Best Regards,
Strahil Nikolov