Lustre discuss - Mar 2009 - Group Lock in Lustre: write call is blocking

Hi,

I have seen the group lock Design at:
http://lists.lustre.org/pipermail/lustre-devel/2006-May/000071.html
I checked the group lock capabilities with the programme
parallel_grouplock.c:
http://lustre.sourcearchive.com/documentation/1.6.5/parallel__grouplock_8c-source.html

This program runs well, but ... it only tests the read function call.

I have a Lustre version 1.6.6, and on this release, the write is blocking
(I have to delete my job). I would like to use the write call.

Here is a little program to reproduce this.

Could anyone tell me if I made a mistake, if Lustre does not support the
group lock or if it is a bug in Lustre ?

Thanks very much for your help.

Best Regards,
P. Dev?ze

==============================================#include <sys/stat.h>
#include <sys/types.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>
#include "lustre_user.h"

char *strerror(int errnum);
#define FILEMODE S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH
#define FILENAME??????? "/mnt/fs_mpi/GL"

// Process takes a GROUP_LOCK and writes
// The write call will block ===> WHY ?

int main(int argc, char *argv[]) {
??????? int nb, rc, fd, gid = 1;

?????? if ((fd = open(FILENAME, O_RDWR | O_CREAT, FILEMODE)) == -1) {
??????????????? printf("Error opening file: %s\n", strerror(errno));
??????????????? exit(-1);
??????? }

??????? if ((rc = ioctl(fd, LL_IOC_GROUP_LOCK, gid)) == -1) {
??????????????? printf("Error ioctl GROUP_LOCK on file %d: %s\n", rc,
strerror(errno));
??????????????? exit(-1);
??????? }
??????? printf("GROUP_LOCK OK, we will write on file\n");

// This write call will block

??????? nb=write(fd, "Hello\n", 6);
??????? if (nb != 6) printf("write: %s\n", strerror(errno));
??????? printf("write done\n");

??????? printf("We will free the lock\n");
??????? if ((rc = ioctl(fd, LL_IOC_GROUP_UNLOCK, gid)) == -1) {
???????????????? printf("Error ioctl GROUP_UNLOCK on file %d: %s", rc,
strerror(errno));
??????? }
??????? printf("group lock is free\n");
??????? close(fd);
}
=============================================================

Hello!

On Mar 16, 2009, at 5:41 AM, pascal.deveze at bull.net
wrote:
> Could anyone tell me if I made a mistake, if Lustre does not support  
> the
> group lock or if it is a bug in Lustre ?
Thank you for bringing this to our attention.
Please file a bug. This is a bug in lustre introduced by "lockless io"
code.

Bye,
     Oleg

Hello,

On 16 March 2009 12:41:20 pascal.deveze at bull.net
wrote:
> Hi,
>
> I have seen the group lock Design at:
> http://lists.lustre.org/pipermail/lustre-devel/2006-May/000071.html
> I checked the group lock capabilities with the programme
> parallel_grouplock.c:
> http://lustre.sourcearchive.com/documentation/1.6.5/parallel__grouplo
>ck_8c-source.html
>
> This program runs well, but ... it only tests the read function call.
>
> I have a Lustre version 1.6.6, and on this release, the write is
> blocking (I have to delete my job). I would like to use the write
> call.
>
> Here is a little program to reproduce this.
>
> Could anyone tell me if I made a mistake, if Lustre does not support
> the group lock or if it is a bug in Lustre ?
Thanks for catching it!

Here is the fix:

Index: lustre/llite/file.c
==================================================================---
lustre/llite/file.c.orig    2009-03-18 23:22:17.000000000 +0300
+++ lustre/llite/file.c 2009-03-18 23:22:31.000000000 +0300
@@ -1144,7 +1144,7 @@ static int ll_is_file_contended(struct f
                 RETURN(0);
         }
         if (fd && (fd->fd_flags & LL_FILE_IGNORE_LOCK))
-                RETURN(1);
+                RETURN(0);
         if (test_bit(LLI_F_CONTENDED, &lli->lli_flags)) {
                 cfs_time_t cur_time = cfs_time_current();
                 cfs_time_t retry_time;

=================================================================# ./l
GROUP_LOCK OK, we will write on file
write done
We will free the lock
group lock is free
> Thanks very much for your help.
>
> Best Regards,
> P. Dev?ze
Thanks,
-- 
Alexander "Zam" Zarochentsev
Staff Engineer
Lustre Group, Sun Microsystems

Hi,

On 18 March 2009 23:03:12 Oleg Drokin wrote:
> Hello!
>
> On Mar 16, 2009, at 5:41 AM, pascal.deveze at bull.net wrote:
> > Could anyone tell me if I made a mistake, if Lustre does not
> > support the
> > group lock or if it is a bug in Lustre ?
>
> Thank you for bringing this to our attention.
> Please file a bug. This is a bug in lustre introduced by "lockless
> io" code.
I''ve created Bug 18859 for this problem.
>
> Bye,
>      Oleg
-- 
Alexander "Zam" Zarochentsev
Staff Engineer
Lustre Group, Sun Microsystems