> It is very hard to compare them because they are structurally very
different. For example, GlusterFS performance will depend *a lot* on the
underlying file system performance. Ceph eliminated that factor by using
Bluestore.
> Ceph is very well performing for VM storage, since it's block based and
as such optimized for that. I haven't tested CephFS a lot (I used it but
only for very small storage) so I cannot speak for its performance, but I am
guessing it's not ideal. For large amount of files thus GlusterFS is still a
good choice.
Was your experience above based on using a sharded volume or a normal
one? When we worked with virtual machine images, we followed the volume
sharding advice. I don't have a comparison for Ceph handy. I was just
curious. It worked so well for us (but maybe our storage is "too
good")
that we found it hard to imagine it could be improved much. This was a
simple case though of a single VM, 3 gluster servers, a sharded volume,
and a raw virtual machine image. Probably a simpler case than yours.
Thank you for writing this and take care,
Erik
>
> One *MAJOR* advantage of Ceph over GlusterFS is tooling. Ceph's
self-analytics, status reporting and problem fixing toolset is just so far
beyond GlusterFS that it's really hard for me to recommend GlusterFS for any
but the most experienced sysadmins. It does come with the type of implementation
Ceph has chosen that they have to have such good tooling (because honestly,
poking around in binary data structures really wouldn't be practical for
most users), but whenever I had a problem with Ceph the solution was just a
couple of command line commands (even if it meant to remove a storage device,
wipe it and add it back), where with GlusterFS it means poking around in the
.glusterfs directory, looking up inode numbers, extended attributes etc. which
is a real pain if you have a multi-million-file filesystem to work on. And
that's not even with sharding or distributed volumes.
>
> Also, Ceph has been a lot more stable that GlusterFS for us. The amount of
hand-holding GlusterFS needs is crazy. With Ceph, there is this one bug (I think
in certain Linux kernel versions) where it sometimes reads only zeroes from disk
and complains about that and then you have to restart that OSD to not run into
problems, but that's one "swatch" process on each machine that
will do that automatically for us. I have run some Ceph clusters for several
years now and only once or twice I had to deal with problems. The several
GlusterFS clusters we operate constantly run into troubles. We now shut down all
GlusterFS clients before we reboot any GlusterFS node because it was near
impossible to reboot a single node without running into unrecoverable troubles
(heal entries that will not heal etc.). With Ceph we can achieve 100% uptime, we
regularly reboot our hosts one by one and some minutes later the Ceph cluster is
clean again.
>
> If others have more insights I'd be very happy to hear them.
>
> Stefan
>
>
> ----- Original Message -----
> > Date: Tue, 16 Jun 2020 20:30:34 -0700
> > From: Artem Russakovskii <archon810 at gmail.com>
> > To: Strahil Nikolov <hunter86_bg at yahoo.com>
> > Cc: gluster-users <gluster-users at gluster.org>
> > Subject: Re: [Gluster-users] State of Gluster project
> > Message-ID:
> > <CAD+dzQdf_TiPBSDj57hY=t8AQ=mACrxinPX7iU4hmuxNMo+omg at
mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Has anyone tried to pit Ceph against gluster? I'm curious what the
ups and
> > downs are.
> >
> > On Tue, Jun 16, 2020, 4:32 PM Strahil Nikolov <hunter86_bg at
yahoo.com> wrote:
> >
> >> Hey Mahdi,
> >>
> >> For me it looks like Red Hat are focusing more on CEPH than on
Gluster.
> >> I hope the project remains active, cause it's very difficult
to find a
> >> Software-defined Storage as easy and as scalable as Gluster.
> >>
> >> Best Regards,
> >> Strahil Nikolov
> >>
> >> ?? 17 ??? 2020 ?. 0:06:33 GMT+03:00, Mahdi Adnan <mahdi at
sysmin.io> ??????:
> >> >Hello,
> >> >
> >> > I'm wondering what's the current and future plan for
Gluster project
> >> >overall, I see that the project is not as busy as it was
before "at
> >> >least
> >> >this is what I'm seeing" Like there are fewer blogs
about what the
> >> >roadmap
> >> >or future plans of the project, the deprecation of Glusterd2,
even Red
> >> >Hat
> >> >Openshift storage switched to Ceph.
> >> >As the community of this project, do you feel the same? Is the
> >> >deprecation
> >> >of Glusterd2 concerning? Do you feel that the project is
slowing down
> >> >somehow? Do you think Red Hat is abandoning the project or
giving fewer
> >> >resources to Gluster?
> >> ________
> >>
> >>
> >>
> >> Community Meeting Calendar:
> >>
> >> Schedule -
> >> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> >> Bridge: https://bluejeans.com/441850968
> >>
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> https://lists.gluster.org/mailman/listinfo/gluster-users
> >>
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> >
<http://lists.gluster.org/pipermail/gluster-users/attachments/20200616/a1d0f142/attachment-0001.html
>
> ________
>
>
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://bluejeans.com/441850968
>
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users