Gambit15
2017-Feb-15 16:08 UTC
[Gluster-users] Optimal shard size & self-heal algorithm for VM hosting?
Hey guys, I keep seeing different recommendations for the best shard sizes for VM images, from 64MB to 512MB. What's the benefit of smaller v larger shards? I'm guessing smaller shards are quicker to heal, but larger shards will provide better sequential I/O for single clients? Anything else? I also usually see "cluster.data-self-heal-algorithm: full" is generally recommended in these cases. Why not "diff"? Is it simply to reduce CPU load when there's plenty of excess network capacity? Thanks in advance, Doug -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170215/3603cee3/attachment.html>
Krutika Dhananjay
2017-Feb-16 10:22 UTC
[Gluster-users] Optimal shard size & self-heal algorithm for VM hosting?
On Wed, Feb 15, 2017 at 9:38 PM, Gambit15 <dougti+gluster at gmail.com> wrote:> Hey guys, > I keep seeing different recommendations for the best shard sizes for VM > images, from 64MB to 512MB. > > What's the benefit of smaller v larger shards? > I'm guessing smaller shards are quicker to heal, but larger shards will > provide better sequential I/O for single clients? Anything else? >That's the main difference. And also smaller shards provide better brick utilization and distribution of IO in distributed-replicated volumes as opposed to larger shards.> > I also usually see "cluster.data-self-heal-algorithm: full" is generally > recommended in these cases. Why not "diff"? Is it simply to reduce CPU load > when there's plenty of excess network capacity? >That's correct. diff heal requires rolling checksum to be computed for every 128KB chunk of the file on both source and sink bricks, which is CPU intensive, potentially affecting IO traffic. -Krutika> > Thanks in advance, > Doug > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170216/ce434dcf/attachment.html>