I have some machines over WAN, in a Replicate cluster, with around 50ms ~ 60ms between them. However, I have read-subvolume specified so that it will always use the local brick instead of the WAN. I just need this in order to replicate files as easily and as quickly as possible between various systems... My question is - Why is it still painfully slow when all I do is read operations? Even just listing a directory takes ages. I can understand about Write operations, but read? and from the local sub-volume?
Dear Community Users, We have recently made some code changes in an effort to improve small file and 'ls' performance. The patches are - selective readdirp - http://patches.gluster.com/patch/3203/ dht lookup revalidation optimization - http://patches.gluster.com/patch/3204/ updated write-behind default values - http://patches.gluster.com/patch/3223/ DISCLAIMER : These patches have not made it to any supported release yet and have not been tested yet. Don't use them in production. I am providing this information only as some advance notice for those in the community who might be interested in trying out these changes and provide feedback. Once these are fully tested they will make to an officially supported release. Regards, Tejas Bhise. ----- Original Message ----- From: "Count Zero" <countz at gmail.com> To: gluster-users at gluster.org Sent: Wednesday, May 5, 2010 9:31:22 AM Subject: [Gluster-users] Replicate over WAN? I have some machines over WAN, in a Replicate cluster, with around 50ms ~ 60ms between them. However, I have read-subvolume specified so that it will always use the local brick instead of the WAN. I just need this in order to replicate files as easily and as quickly as possible between various systems... My question is - Why is it still painfully slow when all I do is read operations? Even just listing a directory takes ages. I can understand about Write operations, but read? and from the local sub-volume? _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
On May 4, 2010, at 9:01 PM, Count Zero wrote:> I have some machines over WAN, in a Replicate cluster, with around 50ms ~ 60ms between them. > However, I have read-subvolume specified so that it will always use the local brick instead of the WAN. > I just need this in order to replicate files as easily and as quickly as possible between various systems... > > My question is - Why is it still painfully slow when all I do is read operations? > Even just listing a directory takes ages. > I can understand about Write operations, but read? and from the local sub-volume?Replicate is not really designed for a WAN environment. A couple of things that are probably affecting you are: 1) "lookup" (first access to any file or directory) needs to be sent to all subvolumes to gather information to determine if self-heal is needed. There is no way to fix this without losing the ability to self-heal. 2) "readdir" (ls) is always sent to the first subvolume. This is necessary to ensure consistent inode numbers. Perhaps you could ensure that the first subvolume is local? (Make sure the order of subvolumes is the same on all your clients.) ------------------------------ Vikas Gorur Engineer - Gluster, Inc. ------------------------------