Gluster users - May 2011 - Files are duplicated after renaming (with glusterfs+zfs-fuse)

Hi all,

I installed glusterfs (version 3.1.3) with zfs-fuse (0.6.9) as the
underlying filesystem.

After renaming a file, I found the file duplicated.

Following is my test scenario.

root at ubuntu:/# zpool create tank /dev/sdb
root at ubuntu:/# gluster volume create test-volume ubuntu:/tank/exp1
ubuntu:/exp2
root at ubuntu:/# gluster volume start test-volume
root at ubuntu:/# mount -t glusterfs ubuntu:/test-volume /mnt
root at ubuntu:/# cd /mnt
root at ubuntu:/mnt# touch test.cc
root at ubuntu:/mnt# mv test.cc test1.cc
root at ubuntu:/mnt# ll
total 7
drwxr-xr-x   2 root root       3 2011-05-31 11:38 ./
drwxr-xr-x 29 root root 4096 2011-05-27 13:40 ../
-rw-r--r--     1 root root      0  2011-05-31 11:38 test1.cc
-rw-r--r--     1 root root      0  2011-05-31 11:38 test1.cc

Is any way to fix this problem?

BTW, I found the ACL list of test1.cc in /tank/exp2 looks different:
----------  1 root root 0 2011-05-31 11:38 test1.cc
If I use ext4 as the underlying filesystem, the ACL list will look like:
---------T 1 root root 0 2011-05-31 11:38 test1.cc

I'm wondering what does the character 'T' mean. Is it related to the
problem?

Thank you!

Ken
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110531/e68bfc12/attachment.html>

Hi all,

I installed glusterfs (version 3.1.3) with zfs-fuse (0.6.9) as the
underlying filesystem.

After renaming a file, I found the file is duplicated.

Following is my test scenario.

root at ubuntu:/# zpool create tank /dev/sdb
root at ubuntu:/# gluster volume create test-volume ubuntu:/tank/exp1
ubuntu:/exp2
root at ubuntu:/# gluster volume start test-volume
root at ubuntu:/# mount -t glusterfs ubuntu:/test-volume /mnt
root at ubuntu:/# cd /mnt
root at ubuntu:/mnt# touch test.cc
root at ubuntu:/mnt# mv test.cc test1.cc
root at ubuntu:/mnt# ll
total 7
drwxr-xr-x   2 root root       3 2011-05-31 11:38 ./
drwxr-xr-x 29 root root 4096 2011-05-27 13:40 ../
-rw-r--r--     1 root root      0  2011-05-31 11:38 test1.cc
-rw-r--r--     1 root root      0  2011-05-31 11:38 test1.cc

Is any way to fix this problem?

BTW, I found the ACL list of test1.cc in /tank/exp2 looks different:
----------  1 root root 0 2011-05-31 11:38 test1.cc
If I use ext4 as the underlying filesystem, the ACL list will look like:
---------T 1 root root 0 2011-05-31 11:38 test1.cc

I'm wondering what does the character 'T' in the ACL list mean.
Is it related to the problem?

Thank you!

Ken
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110531/d9eeec9b/attachment.html>

On the backend to create the link file gluster does mknod (2)

try below operation on zfs-fuse.

ret = mkdir (path, S_IFREG | S_ISVTX, 0);

see whats the behavior.

Regards,
Amar

On Tue, May 31, 2011 at 3:20 PM, ??? <f92019 at csie.ntu.edu.tw> wrote:
>
> I tried the chmod command and fchmod(fd, S_ISVTX) system call.
> In both way I can successfully set the sticky bit in zfs-fuse.
>
> What may be the reason glusterFS failed to set the sticky bit?
>
> Thank you.
>
>
>  Ken
>
>
> 2011/5/31 Amar Tumballi <amar at gluster.com>
>
>> No problem.. thanks for trying out different backend... please get that
>> 'chmod 01000 file' issue fixed in zfs-fuse.. it may work as
backend for
>> glusterfs.
>>
>> -Amar
>>
>>
>> On Tue, May 31, 2011 at 11:27 AM, ??? <f92019 at csie.ntu.edu.tw>
wrote:
>>
>>> Thank you for the detailed answer!
>>>
>>> 2011/5/31 Amar Tumballi <amar at gluster.com>
>>>
>>>> Hi Ken,
>>>>
>>>> Find the answers inline.
>>>>
>>>>
>>>>
>>>>> I installed glusterfs (version 3.1.3) with zfs-fuse (0.6.9)
as the
>>>>> underlying filesystem.
>>>>>
>>>>> After renaming a file, I found the file is duplicated.
>>>>>
>>>>> As per the details given below, that is possible.
>>>>
>>>>>
>>>>> Is any way to fix this problem?
>>>>>
>>>>
>>>> Use a posix complaint filesystem as the backend.
>>>>
>>>>
>>>>>
>>>>> BTW, I found the ACL list of test1.cc in /tank/exp2 looks
different:
>>>>> ----------  1 root root 0 2011-05-31 11:38 test1.cc
>>>>> If I use ext4 as the underlying filesystem, the ACL list
will look
>>>>> like:
>>>>> ---------T 1 root root 0 2011-05-31 11:38 test1.cc
>>>>>
>>>>> I'm wondering what does the character 'T' in
the ACL list mean.
>>>>>  Is it related to the problem?
>>>>>
>>>>>
>>>> Yes. That is the reason of the problem. The character
'T' in the ACL
>>>> list means its having a stickybit set. (perm 01000). We treat
files with
>>>> permission 01000 (ie, ---------T type) as link files in
'cluster/distribute'
>>>> translator and filter it out in 'readdir()' call. If
the permission of that
>>>> link file is different, then it treats it as a regular file,
and hence
>>>> readdir() sends duplicate entries for the same file.
>>>>
>>>> Regards,
>>>> Amar
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110531/97184d5e/attachment.html>