Thibault Godouet
2015-Aug-05 15:11 UTC
[Gluster-users] NFS file locking with gluster 3.7.3
Looking around I get the impression that file locking (NLM) may simply not be supported in glusterfs's built-in NFS server. I get the impression that Ganesha is aimed at supporting NFS better, and presumably supports locking well, so I should give it a try (If I understand well the performance is also likely to be higher, which is a nice bonus!) If someone could confirm this that would be useful to make sure I'm going in the right direction. Thanks, Thibault. On 4 Aug 2015 1:23 pm, "Thibault Godouet" <tibo92 at godouet.net> wrote:> Hi, > > > > I have a cluster of 2 servers running 3.7.3 with replication, and standard > NFS (no ganesha). This in on CentOS 6. > > I use CTDB with 2 virtual IPs (one for each server in a normal > situation) to share the volume over NFS and CIFS (samba). > > > > fnctl() file locking doesn't seem to work when the volume is mounted over > NFS. > > This is apparent with a 'svn info' (svn 1.8 if it made any difference) in > a local working copy: > > > > $ svn info > svn: E200033: Another process is blocking the working copy database, or > the underlying filesystem does not support file locking; if the working > copy is on a network filesystem, make sure file locking has been enabled on > the file server > svn: E200033: sqlite[S5]: database is locked, executing statement 'PRAGMA > synchronous=OFF;PRAGMA recursive_triggers=ON;PRAGMA foreign_keys=OFF;PRAGMA > locking_mode = NORMAL;' > > > > a strace shows: > > > > $ svn info > svn: E200033: Another process is blocking the working copy database, or > the underlying filesystem does not support file locking; if the working > copy is on a network filesystem, make sure file locking has been enabled on > the file server > svn: E200033: sqlite[S5]: database is locked, executing statement 'PRAGMA > synchronous=OFF;PRAGMA recursive_triggers=ON;PRAGMA foreign_keys=OFF;PRAGMA > locking_mode = NORMAL;' > > > > Everything seems to work fine on native Gluster (FUSE) mounts: the same > 'svn info' works nicely. > > I can't really use native mounts due to the performance hit (many small > files) and the fact I would need to install the gluster client software on > every server. > > > > Is fnctl() file locking supported in Gluster NFS mounts? If so, any idea > why it doesn't work for me? > > > > Thanks, > > Thibault. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150805/785aa722/attachment.html>
On Wed, Aug 05, 2015 at 04:11:47PM +0100, Thibault Godouet wrote:> Looking around I get the impression that file locking (NLM) may simply not > be supported in glusterfs's built-in NFS server.This is actually supported. But note that you can not run a userspace NLM implementation provided by a NFS-server (Gluster/NFS or NFS-Ganesha) on a system that acts as an NFS-client. The Linux kernel NFS-client uses the lockd kernel module, and there can be only one NLM implementation be registered at rpcbind. Whichever service (nfs-client or nfs-server) starts first, will be able to register itself, the 2nd one will (mostly silently) fail.> I get the impression that Ganesha is aimed at supporting NFS better, and > presumably supports locking well, so I should give it a try (If I > understand well the performance is also likely to be higher, which is a > nice bonus!)NFS-Ganesha offers more features than Gluster/NFS. The performance is highly dependent on the workload, Gluster/NFS can be faster for many of them. Cheers, Niels> > If someone could confirm this that would be useful to make sure I'm going > in the right direction. > > Thanks, > Thibault. > On 4 Aug 2015 1:23 pm, "Thibault Godouet" <tibo92 at godouet.net> wrote: > > > Hi, > > > > > > > > I have a cluster of 2 servers running 3.7.3 with replication, and standard > > NFS (no ganesha). This in on CentOS 6. > > > > I use CTDB with 2 virtual IPs (one for each server in a normal > > situation) to share the volume over NFS and CIFS (samba). > > > > > > > > fnctl() file locking doesn't seem to work when the volume is mounted over > > NFS. > > > > This is apparent with a 'svn info' (svn 1.8 if it made any difference) in > > a local working copy: > > > > > > > > $ svn info > > svn: E200033: Another process is blocking the working copy database, or > > the underlying filesystem does not support file locking; if the working > > copy is on a network filesystem, make sure file locking has been enabled on > > the file server > > svn: E200033: sqlite[S5]: database is locked, executing statement 'PRAGMA > > synchronous=OFF;PRAGMA recursive_triggers=ON;PRAGMA foreign_keys=OFF;PRAGMA > > locking_mode = NORMAL;' > > > > > > > > a strace shows: > > > > > > > > $ svn info > > svn: E200033: Another process is blocking the working copy database, or > > the underlying filesystem does not support file locking; if the working > > copy is on a network filesystem, make sure file locking has been enabled on > > the file server > > svn: E200033: sqlite[S5]: database is locked, executing statement 'PRAGMA > > synchronous=OFF;PRAGMA recursive_triggers=ON;PRAGMA foreign_keys=OFF;PRAGMA > > locking_mode = NORMAL;' > > > > > > > > Everything seems to work fine on native Gluster (FUSE) mounts: the same > > 'svn info' works nicely. > > > > I can't really use native mounts due to the performance hit (many small > > files) and the fact I would need to install the gluster client software on > > every server. > > > > > > > > Is fnctl() file locking supported in Gluster NFS mounts? If so, any idea > > why it doesn't work for me? > > > > > > > > Thanks, > > > > Thibault. > >> _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users