Hi Yaniv, It may be my lack of knowledge, but I can't see how the fastest response time could differ from file to file. If that's true then it would be enough to only periodically test which node is fastest for this client, not having to do it for every single read. Thanks for the tip about read-hash-mode. I see the help is as below. Value 4 may help, but not if the latency is tested for every file read. Value 0 may help, but it depends how the children are ordered. Does anyone know more about how these work? Option: cluster.read-hash-mode Default Value: 1 Description: inode-read fops happen only on one of the bricks in replicate. AFR will prefer the one computed using the method specified using this option. 0 = first readable child of AFR, starting from 1st child. 1 = hash by GFID of file (all clients use same subvolume). 2 = hash by GFID of file and client PID. 3 = brick having the least outstanding read requests. 4 = brick having the least network ping latency. Thanks again. On Tue, 27 Jul 2021 at 19:16, Yaniv Kaul <ykaul at redhat.com> wrote:> > > On Tue, Jul 27, 2021 at 9:50 AM David Cunningham < > dcunningham at voisonics.com> wrote: > >> Hello, >> >> We have a replicated GlusterFS cluster, and my understanding is that the >> GlusterFS FUSE client will check the file with all nodes before doing a >> read. >> >> For our application it is not actually critical to be certain of having >> the latest version of a file, and it would be preferable to speed up the >> read by simply reading the file from the fastest node. This would be >> especially beneficial if some of the other nodes have higher latency from >> the client. >> > > How do you define, in real time, per file, which is the fastest node? > Maybe you are looking for read-hash-mode volume option? > Y. > >> >> Is it possible to do this? Thanks in advance for any assistance. >> >> -- >> David Cunningham, Voisonics Limited >> http://voisonics.com/ >> USA: +1 213 221 1092 >> New Zealand: +64 (0)28 2558 3782 >> ________ >> >> >> >> Community Meeting Calendar: >> >> Schedule - >> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC >> Bridge: https://meet.google.com/cpu-eiue-hvk >> Gluster-users mailing list >> Gluster-users at gluster.org >> https://lists.gluster.org/mailman/listinfo/gluster-users >> >-- David Cunningham, Voisonics Limited http://voisonics.com/ USA: +1 213 221 1092 New Zealand: +64 (0)28 2558 3782 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20210728/0b0a59ed/attachment.html>
First readable child... Isn't this the first brick in the subvolume ? David, you just want to skip the lookups and you need gluster to just serve the file, right ? I think that you can play a little bit with md-cache. There is a nice article (requires RH dev or real subscription): https://access.redhat.com/solutions/3019721 Just a short quotation from that article: Operations/Workload that are benefited:- Create, delete, ls (recursive), chmod, rename, du, and find will show marked improvement. - Directory enumerations with SMB have shown up to 8x improvement and similar tests with FUSE have also shown up to 2x improvement.- Small file creates show up to 2x improvement. This important caching enhancement is likely to improve access performance of tiered volumes as well.Testing has shown up to 3x improvements in small-file read performance. Best Regards,Strahil Nikolov On Wed, Jul 28, 2021 at 5:50, David Cunningham<dcunningham at voisonics.com> wrote: ________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users at gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20210728/ac400c42/attachment.html>
On Wed, Jul 28, 2021 at 5:50 AM David Cunningham <dcunningham at voisonics.com> wrote:> Hi Yaniv, > > It may be my lack of knowledge, but I can't see how the fastest response > time could differ from file to file. If that's true then it would be enough > to only periodically test which node is fastest for this client, not having > to do it for every single read. >In real life, the 'best' node is the one with the highest overall free resources, across CPU, network and disk IO. So it could change and it might change all the time. Network, disk saturation might be common, disk performing garbage collection, CPU being hogged by something, noisy neighbor, etc... Our latency check is indeed not per file, AFAIK. Y.> Thanks for the tip about read-hash-mode. I see the help is as below. Value > 4 may help, but not if the latency is tested for every file read. Value 0 > may help, but it depends how the children are ordered. Does anyone know > more about how these work? > > Option: cluster.read-hash-mode > Default Value: 1 > Description: inode-read fops happen only on one of the bricks in > replicate. AFR will prefer the one computed using the method specified > using this option. > 0 = first readable child of AFR, starting from 1st child. > 1 = hash by GFID of file (all clients use same subvolume). > 2 = hash by GFID of file and client PID. > 3 = brick having the least outstanding read requests. > 4 = brick having the least network ping latency. > > Thanks again. > > > On Tue, 27 Jul 2021 at 19:16, Yaniv Kaul <ykaul at redhat.com> wrote: > >> >> >> On Tue, Jul 27, 2021 at 9:50 AM David Cunningham < >> dcunningham at voisonics.com> wrote: >> >>> Hello, >>> >>> We have a replicated GlusterFS cluster, and my understanding is that the >>> GlusterFS FUSE client will check the file with all nodes before doing a >>> read. >>> >>> For our application it is not actually critical to be certain of having >>> the latest version of a file, and it would be preferable to speed up the >>> read by simply reading the file from the fastest node. This would be >>> especially beneficial if some of the other nodes have higher latency from >>> the client. >>> >> >> How do you define, in real time, per file, which is the fastest node? >> Maybe you are looking for read-hash-mode volume option? >> Y. >> >>> >>> Is it possible to do this? Thanks in advance for any assistance. >>> >>> -- >>> David Cunningham, Voisonics Limited >>> http://voisonics.com/ >>> USA: +1 213 221 1092 >>> New Zealand: +64 (0)28 2558 3782 >>> ________ >>> >>> >>> >>> Community Meeting Calendar: >>> >>> Schedule - >>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC >>> Bridge: https://meet.google.com/cpu-eiue-hvk >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> https://lists.gluster.org/mailman/listinfo/gluster-users >>> >> > > -- > David Cunningham, Voisonics Limited > http://voisonics.com/ > USA: +1 213 221 1092 > New Zealand: +64 (0)28 2558 3782 >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20210728/5261ca1b/attachment.html>