Gluster users - Mar 2017 - Turning off readdirp in the entire stack on fuse mount

Hi all,

One of the grey areas (in terms of documentation) is how to turn off readdirplus
(aka readdirp) in the entire stack of glusterfs. The confusion is because, there
are couple of xlators which by default convert a readdir into readdirp. All of
the following steps need to be done to turn off readdirp entirely.

1. mount glusterfs with option --use-readdirp=no (disables readdirp between fuse
kernel module and glusterfs)

[root at unused glusterfs]# mount -t glusterfs booradley:/testvol -o
use-readdirp=no /mnt/glusterfs

[root at unused glusterfs]# ps ax | grep -i mnt
26096 ?        Ssl    0:00 /usr/local/sbin/glusterfs --use-readdirp=no
--volfile-server=booradley --volfile-id=/testvol /mnt/glusterfs

2. set performance.force-readdirp to false (prevents md-cache from converting
readdir calls to readdirp)
[root at unused glusterfs]# gluster volume set testvol
performance.force-readdirp off
volume set: success

3. set dht.force-readdirp to false (prevents dht from converting readdir into
readdirp
[root at unused glusterfs]# gluster volume set testvol dht.force-readdirp off
volume set: success

[root at unused glusterfs]# gluster volume info testvol
 
Volume Name: testvol
Type: Distribute
Volume ID: 007edfec-0e54-4d80-bef4-9aec5bcc1108
Status: Started
Snapshot Count: 0
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: booradley:/home/export/testvol
Options Reconfigured:
dht.force-readdirp: off
performance.force-readdirp: off
transport.address-family: inet
nfs.disable: on

Note that if readdir-ahead is turned on, by default it does prefetch directory
entries to fill its cache using readdirp. However, it doesn't implement
readdirp fop. Hence those prefetched entries are never consumed in the above
configuration. If option performance.readdir-ahead is set to off, we
wouldn't not witness a readdirp fop in the entire glusterfs stack (from fuse
till bricks).

regards,
Raghavendra

----- Original Message -----
> From: "Raghavendra Gowdappa" <rgowdapp at redhat.com>
> To: "Gluster Devel" <gluster-devel at gluster.org>,
"gluster-users" <gluster-users at gluster.org>
> Sent: Thursday, March 2, 2017 9:48:08 AM
> Subject: Turning off readdirp in the entire stack on fuse mount
> 
> Hi all,
> 
> One of the grey areas (in terms of documentation) is how to turn off
> readdirplus (aka readdirp) in the entire stack of glusterfs. The confusion
> is because, there are couple of xlators which by default convert a readdir
> into readdirp. All of the following steps need to be done to turn off
> readdirp entirely.
> 
> 1. mount glusterfs with option --use-readdirp=no (disables readdirp between
> fuse kernel module and glusterfs)
> 
> [root at unused glusterfs]# mount -t glusterfs booradley:/testvol -o
> use-readdirp=no /mnt/glusterfs
> 
> [root at unused glusterfs]# ps ax | grep -i mnt
> 26096 ?        Ssl    0:00 /usr/local/sbin/glusterfs --use-readdirp=no
> --volfile-server=booradley --volfile-id=/testvol /mnt/glusterfs
> 
> 2. set performance.force-readdirp to false (prevents md-cache from
converting
> readdir calls to readdirp)
> [root at unused glusterfs]# gluster volume set testvol
> performance.force-readdirp off
> volume set: success
> 
> 3. set dht.force-readdirp to false (prevents dht from converting readdir
into
> readdirp
> [root at unused glusterfs]# gluster volume set testvol dht.force-readdirp
off
> volume set: success
> 
> [root at unused glusterfs]# gluster volume info testvol
>  
> Volume Name: testvol
> Type: Distribute
> Volume ID: 007edfec-0e54-4d80-bef4-9aec5bcc1108
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 1
> Transport-type: tcp
> Bricks:
> Brick1: booradley:/home/export/testvol
> Options Reconfigured:
> dht.force-readdirp: off
> performance.force-readdirp: off
> transport.address-family: inet
> nfs.disable: on
> 
> Note that if readdir-ahead is turned on, by default it does prefetch
> directory entries to fill its cache using readdirp. However, it doesn't
> implement readdirp fop. 
s/readdirp fop/readdir fop/
> Hence those prefetched entries are never consumed in
> the above configuration. If option performance.readdir-ahead is set to off,
> we wouldn't not witness a readdirp fop in the entire glusterfs stack
(from
> fuse till bricks).
> 
> regards,
> Raghavendra