Serkan Çoban
2017-Mar-16 04:39 UTC
[Gluster-users] advice needed on configuring large gluster cluster
> Thanks Serkan, I appreciate the feedback. Do you know if there are any > associated performance issues with odd values of parity disks? e.g any > reason to not go with 16+3?, also are there any things to watch for in terms > of the number of distributed disperse sets?AFAIK, there is no performance concern for n for m+n configuration, n affects how many bricks/nodes will be tolerated in case of failure. Number of disperse sets are related with your workload and capacity. EC is not recommended for small files and random I/O so as I said before do your test with your prod workload.> > -Alastair > > On 15 March 2017 at 13:32, Serkan ?oban <cobanserkan at gmail.com> wrote: >> >> Please find my comments inline. >> >> > Hi >> > >> > we have a new gluster cluster we are planning on deploying. We will >> > have 24 >> > nodes each with JBOD, 39 8TB drives and 6, 900GB SSDs, and FDR IB >> > >> > We will not be using all of this as one volume , but I thought initially >> > of >> > using a distributed disperse volume. >> > >> > Never having attempted anything on this scale I have a couple of >> > questions >> > regarding EC and distibuted disperse volumes. >> > >> > Does a distributed dispersed volume have to start life as distributed >> > dispersed, or can I take a disperse volume and make it distributed by >> > adding bricks? >> Yes you can start with one subvolume and later you can increase the >> subvolumes. >> But be careful about planning, if you start with m+n EC configuration, >> you can add >> another m+n subvolume to it. >> > >> > Does an EC scheme of 24+4 seem reasonable? One requirement we will have >> > is >> > the need to tolerate two nodes down at once, as the nodes share a >> > chassis. >> > I assume that distributed disperse volumes can be expanded in a similar >> > fashion to distributed replicate volumes by adding additional disperse >> > brick >> > sets? >> It is recommended in m+n configuration m should be power of two. >> You can do 16+4 or 8+2. Higher m will cause slower healing but >> parallel self heal >> of EC volumes in 3.9+ will help. 8+2 configuration with one brick from >> every node will >> tolerate loss of two nodes. >> >> > >> > I would also like to consider adding a hot-tier using the SSDs, I >> > confess I >> > have not done much reading on tiering, but am hoping I can use a >> > different >> > volume form for the hot tier. Can I use create a disperse, or a >> > distributed >> > replicated? If I am smoking rainbows then I can consider setting up a >> > SSD >> > only distributed disperse volume. >> EC performance is quite good for our workload,I did not try any tier >> in front of it >> Test your workload without tier, if it works then KISS >> > >> > I'd also appreciate any feedback on likely performance issues and tuning >> > tips? >> You can find kernel performance tuning here: >> >> https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Linux%20Kernel%20Tuning/ >> You may also change client.event-threads, server.event-threads and >> heal related parameters >> but do not forget to test your workload after and before changing those >> values. >> > >> > Many Thanks >> > >> > -Alastair >> > >> > >> > >> > _______________________________________________ >> > Gluster-users mailing list >> > Gluster-users at gluster.org >> > http://lists.gluster.org/mailman/listinfo/gluster-users > >