Raghavendra Gowdappa
2019-Feb-13 05:21 UTC
[Gluster-users] Disabling read-ahead and io-cache for native fuse mounts
On Tue, Feb 12, 2019 at 5:38 PM Raghavendra Gowdappa <rgowdapp at redhat.com> wrote:> All, > > We've found perf xlators io-cache and read-ahead not adding any > performance improvement. At best read-ahead is redundant due to kernel > read-ahead >One thing we are still figuring out is whether kernel read-ahead is tunable. From what we've explored, it _looks_ like (may not be entirely correct), ra is capped at 128KB. If that's the case, I am interested in few things: * Are there any realworld applications/usecases, which would benefit from larger read-ahead (Manoj says block devices can do ra of 4MB)? * Is the limit on kernel ra tunable a hard one? IOW, what does it take to make it to do higher ra? If its difficult, can glusterfs read-ahead provide the expected performance improvement for these applications that would benefit from aggressive ra (as glusterfs can support larger ra sizes)? I am still inclined to prefer kernel ra as I think its more intelligent and can identify more sequential patterns than Glusterfs read-ahead [1][2]. [1] https://www.kernel.org/doc/ols/2007/ols2007v2-pages-273-284.pdf [2] https://lwn.net/Articles/155510/ and at worst io-cache is degrading the performance for workloads that> doesn't involve re-read. Given that VFS already have both these > functionalities, I am proposing to have these two translators turned off by > default for native fuse mounts. > > For non-native fuse mounts like gfapi (NFS-ganesha/samba) we can have > these xlators on by having custom profiles. Comments? > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1665029 > > regards, > Raghavendra >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190213/29c0940b/attachment.html>
Manoj Pillai
2019-Feb-13 05:45 UTC
[Gluster-users] Disabling read-ahead and io-cache for native fuse mounts
On Wed, Feb 13, 2019 at 10:51 AM Raghavendra Gowdappa <rgowdapp at redhat.com> wrote:> > > On Tue, Feb 12, 2019 at 5:38 PM Raghavendra Gowdappa <rgowdapp at redhat.com> > wrote: > >> All, >> >> We've found perf xlators io-cache and read-ahead not adding any >> performance improvement. At best read-ahead is redundant due to kernel >> read-ahead >> > > One thing we are still figuring out is whether kernel read-ahead is > tunable. From what we've explored, it _looks_ like (may not be entirely > correct), ra is capped at 128KB. If that's the case, I am interested in few > things: > * Are there any realworld applications/usecases, which would benefit from > larger read-ahead (Manoj says block devices can do ra of 4MB)? >kernel read-ahead is adaptive but influenced by the read-ahead setting on the block device (/sys/block/<dev>/queue/read_ahead_kb), which can be tuned. For RHEL specifically, the default is 128KB (last I checked) but the default RHEL tuned-profile, throughput-performance, bumps that up to 4MB. It should be fairly easy to rig up a test where 4MB read-ahead on the block device gives better performance than 128KB read-ahead. -- Manoj * Is the limit on kernel ra tunable a hard one? IOW, what does it take to> make it to do higher ra? If its difficult, can glusterfs read-ahead provide > the expected performance improvement for these applications that would > benefit from aggressive ra (as glusterfs can support larger ra sizes)? > > I am still inclined to prefer kernel ra as I think its more intelligent > and can identify more sequential patterns than Glusterfs read-ahead [1][2]. > [1] https://www.kernel.org/doc/ols/2007/ols2007v2-pages-273-284.pdf > [2] https://lwn.net/Articles/155510/ > > and at worst io-cache is degrading the performance for workloads that >> doesn't involve re-read. Given that VFS already have both these >> functionalities, I am proposing to have these two translators turned off by >> default for native fuse mounts. >> >> For non-native fuse mounts like gfapi (NFS-ganesha/samba) we can have >> these xlators on by having custom profiles. Comments? >> >> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1665029 >> >> regards, >> Raghavendra >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20190213/121849a2/attachment.html>