On Mon, Apr 24, 2017 at 1:24 PM, Gandalf Corvotempesta < gandalf.corvotempesta at gmail.com> wrote:> Il 24 apr 2017 9:40 AM, "Ashish Pandey" <aspandey at redhat.com> ha scritto: > > > There is difference between server and bricks which we should understand. > When we say m+n = 6+2, then we are talking about the bricks. > Total number of bricks are m+n = 8. > > Now, these bricks could be anywhere on any server. The only thing is that > the server should be a part of cluster. > You can have all the 8 bricks on one server or on 8 different servers. > So, there is no *restriction* on number of servers when you add bricks. > However, the number of bricks which you want to add should be in multiple > of the > configuration you have. > > > This is clear but it doesn't change the result > As no one is using gluster to replicate data by loosing redundancy (it's > nonsense), adding bricks means adding servers > If our server are already full with no more available slots for adding > disks, the only solution is to add 8 servers more (at least 1 brick per > server) > > > > In you case it should be 8, 16, 24.... > > "can I add a single node moving from 6:2 to 7:2 and so on ?" > You can not make 6+2 config volume to 7+2 volume. You can not change the > *configuration* of an existing volume. > You can just add bricks in multiple to increase the storage capacity. > > > Yes and this is the worst thing in gluster: the almost zero flexibility >> Bigger the cluster, higher the cost to maintain it or expand it. >At least in case of EC it is with good reason. If you want to change volume's configuration from 6+2->7+2 you have to compute the encoding again and place different data on the resulting 9 bricks. Which has to be done for all files. It is better to just create a new volume with 7+2 and just copy the files on to this volume and remove the original files on volume with 6+2.> If you start with a 6:2 by using commodity hardware, you are screwed, your > next upgrade will be 8 servers with 1 disk/brick each. >Not true.> > Yes, gluster doesn't make use of any metadata server, but I really prefer > to add 2 metadata server and 1 storage server at once when needed than > avoid metadata servers but being forced to add a bounch of servers every > time > > More servers means more power cost, more hardware that could fails and so > on. > > Let's assume a replica 3 cluster. > If I need to add 2tb more, I have to add 3 servers with 2tb on each server. > Ceph, Lizard, Moose and others allow adding a single server/disk and then > they rebalance data aroud by freeing up the used space adding the new disk. >Didn't understand this math. If you want to add 2TB capacity to a volume that is 3-way replicated, you essentially need to add 6TB in whatever solution you have. At least 6TB with a single server. Which you can do even with Gluster.> > I thought that this lack of flexibility was addressed is some way in > latest version... >I think we had this discussion last July[1] with you that we can simulate the same things other storage solutions with metadata do by doing replace-bricks and rebalance. If you have a new server with 8 bricks then we can add a single server and make sure things are rebalanced with 6+2. Please note it is better to use data-bricks that is power of 2 like 4+2/8+2/16+4 etc than 6+2. Are you suggesting this process to be easier through commands, rather than for administrators to figure out how to place the data? [1] http://lists.gluster.org/pipermail/gluster-users/2016-July/027431.html> _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >-- Pranith -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170424/780b04a8/attachment.html>
2017-04-24 10:21 GMT+02:00 Pranith Kumar Karampuri <pkarampu at redhat.com>:> At least in case of EC it is with good reason. If you want to change > volume's configuration from 6+2->7+2 you have to compute the encoding again > and place different data on the resulting 9 bricks. Which has to be done for > all files. It is better to just create a new volume with 7+2 and just copy > the files on to this volume and remove the original files on volume with > 6+2.Ok, for EC this makes sense.> Didn't understand this math. If you want to add 2TB capacity to a volume > that is 3-way replicated, you essentially need to add 6TB in whatever > solution you have. At least 6TB with a single server. Which you can do even > with Gluster.Obviously, if you add a single 2TB disk in a replica 3, you won't get 2TB usable space but only 1/3, about 600GB> I think we had this discussion last July[1] with you that we can simulate > the same things other storage solutions with metadata do by doing > replace-bricks and rebalance. If you have a new server with 8 bricks then we > can add a single server and make sure things are rebalanced with 6+2. Please > note it is better to use data-bricks that is power of 2 like 4+2/8+2/16+4 > etc than 6+2.This is an hugly workaround and very prone to errors. Usually I prefere to not mess with my data, making multiple steps where any other SDS has this natively. Please take a look at LizardFS or MooseFS (or even Ceph). You can add a single disk and it will be automatically added and rebalanced without loosing redudancy in any single phase. If you add a 2TB disk on a replica 3, you'll end up automatically with +600GB You can also choose which file must be replicated where and how, if you need one replica on SSD and other replica (for the same file) on HDD, this is possible, or even one replica on local SSD, one replica on hdd on the same datacenter and the third replica on HDD on the dark side of the moon. I don' think this would be possible with gluster with fixed "files map" (I don't know the exact terms) because the lack of metadata server doesn't allow you to know where a file is, without assuring that is located in a fixed position across the whole cluster in gluster to archieve the same you have to run multiple commands, respect the proper order of command line arguments and so on. This is very, very, very risky and prone to errors. This is not a battle between two SDS and I don't want to be pedant (i'm just giving some suggestions), but it's a fact that these SDS are way more flexible than gluster (and on daily usage, far more cheaper) I'm hoping that newer version of gluster brings some more flexibility in brick placement/management.> Are you suggesting this process to be easier through commands, rather than > for administrators to figure out how to place the data?Yes, this for sure. SDS must ensure data resiliency always, so, whatever operation you'll do, data must be always replicated properly. If you need to make some dangerous operation, a "--force" must be used.
2017-04-24 10:21 GMT+02:00 Pranith Kumar Karampuri <pkarampu at redhat.com>:> Are you suggesting this process to be easier through commands, rather than > for administrators to figure out how to place the data? > > [1] http://lists.gluster.org/pipermail/gluster-users/2016-July/027431.htmlAdmin should always have the ability to choose where to place data, but something easier should be added, like in any other SDS. Something like: gluster volume add-brick gv0 new_brick if gv0 is a replicated volume, the add-brick should automatically add the new brick and rebalance data automatically, still keeping the required redundancy level In case admin would like to set a custom placement for data, it should specify a "force" argument or something similiar. tl;dr: as default, gluster should preserve data redundancy allowing users to add single bricks without having to think how to place data. This will make gluster way easier to manage and much less error prone, thus increasing the resiliency of the whole gluster. after all , if you have a replicated volume, is obvious that you want your data to be replicated and gluster should manage this on it's own. Is this something are you planning or considering for further implementation? I know that lack of metadata server (this is a HUGE advantage for gluster) means less flexibility, but as there is a manual workaround for adding single bricks, gluster should be able to handle this automatically.