Gluster users - Apr 2016 - Number of shards in .shard directory

Looking at migrating all our VM's to sharded gluster storage, 2.5TB of 
data in total. Underlying storage is zfs (on linux).

Small scale testing has been going well but one thing I am unsure of is 
whether a *large* number of small files in one directory is going to be 
a problem. If I migrate everything to a 4MB shard size there is going to 
be 655360 files. Will that be a problem for gluster? or for zfs?

-- 
Lindsay Mathieson

Hmm I don't have an answer to that question right now because I'm no
on-disk fs expert.
As far as gluster is concerned, I don't actually see any issue except for
the following:
as of today, gluster's entry self-heal algorithm crawls the whole parent
directory and compares the
source and sink to figure out what files need to be created/deleted on the
sink brick.

If the shards were out into multiple directories as opposed to just
/.shard, entry self-heal would take
lesser time and resources to complete in certain cases since the crawling
could possibly be confined to only those few directories and they would all
within them contain fewer shards.
But with granular entry self-heal feature which is going to be introduced
in 3.8, this will also be a non-issue since
there AFR will record the exact shards that were created/deleted while a
brick is down and only sync them to the sink brick once it is back up.

https://github.com/gluster/glusterfs-specs/blob/master/accepted/granular-entry-self-healing.md

-Krutika

-Krutika

On Sat, Apr 16, 2016 at 5:38 AM, Lindsay Mathieson <
lindsay.mathieson at gmail.com> wrote:
> Looking at migrating all our VM's to sharded gluster storage, 2.5TB of
> data in total. Underlying storage is zfs (on linux).
>
> Small scale testing has been going well but one thing I am unsure of is
> whether a *large* number of small files in one directory is going to be a
> problem. If I migrate everything to a 4MB shard size there is going to be
> 655360 files. Will that be a problem for gluster? or for zfs?
>
> --
> Lindsay Mathieson
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160417/59852868/attachment.html>