On Wed, Jul 28, 2021 at 5:50 AM David Cunningham <dcunningham at voisonics.com> wrote:> Hi Yaniv, > > It may be my lack of knowledge, but I can't see how the fastest response > time could differ from file to file. If that's true then it would be enough > to only periodically test which node is fastest for this client, not having > to do it for every single read. >In real life, the 'best' node is the one with the highest overall free resources, across CPU, network and disk IO. So it could change and it might change all the time. Network, disk saturation might be common, disk performing garbage collection, CPU being hogged by something, noisy neighbor, etc... Our latency check is indeed not per file, AFAIK. Y.> Thanks for the tip about read-hash-mode. I see the help is as below. Value > 4 may help, but not if the latency is tested for every file read. Value 0 > may help, but it depends how the children are ordered. Does anyone know > more about how these work? > > Option: cluster.read-hash-mode > Default Value: 1 > Description: inode-read fops happen only on one of the bricks in > replicate. AFR will prefer the one computed using the method specified > using this option. > 0 = first readable child of AFR, starting from 1st child. > 1 = hash by GFID of file (all clients use same subvolume). > 2 = hash by GFID of file and client PID. > 3 = brick having the least outstanding read requests. > 4 = brick having the least network ping latency. > > Thanks again. > > > On Tue, 27 Jul 2021 at 19:16, Yaniv Kaul <ykaul at redhat.com> wrote: > >> >> >> On Tue, Jul 27, 2021 at 9:50 AM David Cunningham < >> dcunningham at voisonics.com> wrote: >> >>> Hello, >>> >>> We have a replicated GlusterFS cluster, and my understanding is that the >>> GlusterFS FUSE client will check the file with all nodes before doing a >>> read. >>> >>> For our application it is not actually critical to be certain of having >>> the latest version of a file, and it would be preferable to speed up the >>> read by simply reading the file from the fastest node. This would be >>> especially beneficial if some of the other nodes have higher latency from >>> the client. >>> >> >> How do you define, in real time, per file, which is the fastest node? >> Maybe you are looking for read-hash-mode volume option? >> Y. >> >>> >>> Is it possible to do this? Thanks in advance for any assistance. >>> >>> -- >>> David Cunningham, Voisonics Limited >>> http://voisonics.com/ >>> USA: +1 213 221 1092 >>> New Zealand: +64 (0)28 2558 3782 >>> ________ >>> >>> >>> >>> Community Meeting Calendar: >>> >>> Schedule - >>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC >>> Bridge: https://meet.google.com/cpu-eiue-hvk >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> https://lists.gluster.org/mailman/listinfo/gluster-users >>> >> > > -- > David Cunningham, Voisonics Limited > http://voisonics.com/ > USA: +1 213 221 1092 > New Zealand: +64 (0)28 2558 3782 >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20210728/5261ca1b/attachment.html>
Il 2021-07-28 09:20 Yaniv Kaul ha scritto:> In real life, the 'best' node is the one with the highest overall free > resources, across CPU, network and disk IO. So it could change and it > might change all the time. > Network, disk saturation might be common, disk performing garbage > collection, CPU being hogged by something, noisy neighbor, etc... > > Our latency check is indeed not per file, AFAIK. > Y.Hi, while true, in small cluster often the "best" node is the local one. Once upon a time Gluster was able to prefer reads from the local node, but I seem to remember that the option was changed/removed in later version. Nowaday, how one can instruct Gluster to read from the local node rather then from remote ones? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8
I think you mean cluster.choose-local which is enabled by default. Yet, Gluster will check if the local copy is healthy. Best Regards, Strahil Nikolov ? ?????, 28 ??? 2021 ?., 10:20:55 ?. ???????+3, Yaniv Kaul <ykaul at redhat.com> ??????: On Wed, Jul 28, 2021 at 5:50 AM David Cunningham <dcunningham at voisonics.com> wrote:> Hi Yaniv, > > It may be my lack of knowledge, but I can't see how the fastest response time could differ from file to file. If that's true then it would be enough to only periodically test which node is fastest for this client, not having to do it for every single read.In real life, the 'best' node is the one with the highest overall free resources, across CPU, network and disk IO. So it could change and it might change all the time. Network, disk saturation might be common, disk performing garbage collection, CPU being hogged by something, noisy neighbor, etc... Our latency check is indeed not per file, AFAIK. Y.> > Thanks for the tip about read-hash-mode. I see the help is as below. Value 4 may help, but not if the latency is tested for every file read. Value 0 may help, but it depends how the children are ordered. Does anyone know more about how these work? > > Option: cluster.read-hash-mode > Default Value: 1 > Description: inode-read fops happen only on one of the bricks in replicate. AFR will prefer the one computed using the method specified using this option. > 0 = first readable child of AFR, starting from 1st child. > 1 = hash by GFID of file (all clients use same subvolume). > 2 = hash by GFID of file and client PID. > 3 = brick having the least outstanding read requests. > 4 = brick having the least network ping latency. > > Thanks again. > > > On Tue, 27 Jul 2021 at 19:16, Yaniv Kaul <ykaul at redhat.com> wrote: >> >> >> On Tue, Jul 27, 2021 at 9:50 AM David Cunningham <dcunningham at voisonics.com> wrote: >>> Hello, >>> >>> We have a replicated GlusterFS cluster, and my understanding is that the GlusterFS FUSE client will check the file with all nodes before doing a read. >>> >>> For our application it is not actually critical to be certain of having the latest version of a file, and it would be preferable to speed up the read by simply reading the file from the fastest node. This would be especially beneficial if some of the other nodes have higher latency from the client. >> >> How do you define, in real time, per file, which is the fastest?node? >> Maybe you are looking for read-hash-mode volume option? >> Y. >> >>> >>> Is it possible to do this? Thanks in advance for any assistance. >>> >>> -- >>> David Cunningham, Voisonics Limited >>> http://voisonics.com/ >>> USA: +1 213 221 1092 >>> New Zealand: +64 (0)28 2558 3782 >>> ________ >>> >>> >>> >>> Community Meeting Calendar: >>> >>> Schedule - >>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC >>> Bridge: https://meet.google.com/cpu-eiue-hvk >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> https://lists.gluster.org/mailman/listinfo/gluster-users >>> >> > > > -- > David Cunningham, Voisonics Limited > http://voisonics.com/ > USA: +1 213 221 1092 > New Zealand: +64 (0)28 2558 3782 >________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users at gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users