Gluster users - May 2017 - Add single server

2017-05-01 20:30 GMT+02:00 Shyam <srangana at
redhat.com>:
> Yes, as a matter of fact, you can do this today using the CLI and creating
> nx2 instead of 1x2. 'n' is best decided by you, depending on the
growth
> potential of your cluster, as at some point 'n' wont be enough if
you grow
> by some nodes.
>
> But, when a brick is replaced we will fail to address "(a) ability to
retain
> replication/availability levels" as we support only homogeneous
replication
> counts across all DHT subvols. (I could be corrected on this when using
> replace-brick though)

Yes, but this is error prone.

I'm still thinking that saving (I don't know where, I don't know
how)
a mapping between
files and bricks would solve many issues and add much more flexibility.

On Tue, May 2, 2017 at 12:04 AM, Gandalf Corvotempesta <
gandalf.corvotempesta at gmail.com> wrote:
> 2017-05-01 20:30 GMT+02:00 Shyam <srangana at redhat.com>:
> > Yes, as a matter of fact, you can do this today using the CLI and
> creating
> > nx2 instead of 1x2. 'n' is best decided by you, depending on
the growth
> > potential of your cluster, as at some point 'n' wont be enough
if you
> grow
> > by some nodes.
> >
> > But, when a brick is replaced we will fail to address "(a)
ability to
> retain
> > replication/availability levels" as we support only homogeneous
> replication
> > counts across all DHT subvols. (I could be corrected on this when
using
> > replace-brick though)
>
>
> Yes, but this is error prone.
>
Why?

>
> I'm still thinking that saving (I don't know where, I don't
know how)
> a mapping between
> files and bricks would solve many issues and add much more flexibility.
>


-- 
Pranith
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170502/90e10d4f/attachment.html>

On Mon, May 1, 2017, at 02:34 PM, Gandalf Corvotempesta
wrote:
> I'm still thinking that saving (I don't know where, I don't
know how)
> a mapping between
> files and bricks would solve many issues and add much more flexibility.
Every system we've discussed has a map.  The differences are only in the
granularity, and how the map is stored.  Per-file maps inevitably become
a scaling problem, so a deterministic function is used to map individual
files into a much smaller number of buckets, placement groups, hash
ranges, or whatever.  Then information about those buckets and their
locations is stored somehow:

* Centrally - Lustre, HDFS, Moose/Lizard

* Distributed among a few servers - Ceph, possibly Gluster with DHT2

* Distributed among all servers - Gluster today

No matter which approach you use, you can manipulate the maps.  Without
changing the fundamental structure of Gluster, you could take a brick's
hash range and split it in two to create two bricks.  Then you could
quietly migrate the files in one brick to anywhere else in the
background.  That doesn't quite work today because the two bricks would
be trying to operate on the same directories, seeing each others' files,
etc.  Making it more transparent won't be easy, but the changes would be
pretty well localized to DHT.  Brick multiplexing can help too, because
it allows a volume to be created with many more bricks initially so
they'd already be in separate directories and ready to move.  Multiple
bricks living in one process also makes coordination during such
transitions much easier.  This has been part of my plan for years, not
only to support adding a single server but also to support more
sophisticated forms of tiering, quality of service, etc.

The big question as I see it is what we can do *in the near term* to
make N+1 addition easier on *existing* clusters.  That probably deserves
a separate answer, so I'll leave it for another time.