Hello, Were having issues with macs writing to our gluster system. Gluster vol info at end. On a mac, if I make a file in the shell I get the following message: smoke:hunter david$ echo hello > test -bash: test: Operation not permitted And the file is made but is zero size. smoke:hunter david$ ls -l test -rw-r--r-- 1 david realise 0 Mar 3 08:44 test glusterfs/nfslog logs thus: [2011-03-03 08:44:10.379188] I [io-stats.c:333:io_stats_dump_fd] glustervol1: --- fd stats --- [2011-03-03 08:44:10.379222] I [io-stats.c:338:io_stats_dump_fd] glustervol1: Filename : /production/hunter/test Then try to open the file: smoke:hunter david$ cat test and get the following messages in the log: [2011-03-03 08:51:13.957319] I [afr-common.c:716:afr_lookup_done] glustervol1-replicate-0: background meta-data self-heal triggered. path: /production/hunter/test [2011-03-03 08:51:13.959466] I [afr-self-heal-common.c:1526:afr_self_heal_completion_cbk] glustervol1-replicate-0: background meta-data self-heal completed on /production/hunter/test If I do the same test on a linux machine (nfs) it's fine. We get the same issue on all the macs. They are 10.6.6. Gluster volume is mounted: /n/auto/gv1 -rw,hard,tcp,rsize=32768,wsize=32768,intr gus:/glustervol1 Other nfs mounts on mac (from linux servers) are OK We're using LDAP to authenticate on the macs, the gluster servers aren't bound into the LDAP domain. Any ideas? Thanks David g3:/var/log/glusterfs # gluster volume info Volume Name: glustervol1 Type: Distributed-Replicate Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: g1:/mnt/glus1 Brick2: g2:/mnt/glus1 Brick3: g3:/mnt/glus1 Brick4: g4:/mnt/glus1 Brick5: g1:/mnt/glus2 Brick6: g2:/mnt/glus2 Brick7: g3:/mnt/glus2 Brick8: g4:/mnt/glus2 Options Reconfigured: performance.stat-prefetch: 1 performance.cache-size: 1gb performance.write-behind-window-size: 1mb network.ping-timeout: 20 diagnostics.latency-measurement: off diagnostics.dump-fd-stats: on -- David Lloyd V Consultants www.v-consultants.co.uk
it's gluster 3.1.2 On 3 March 2011 09:01, David Lloyd <david.lloyd at v-consultants.co.uk> wrote:> Hello, > > Were having issues with macs writing to our gluster system. > Gluster vol info at end. > > On a mac, if I make a file in the shell I get the following message: > > smoke:hunter david$ echo hello > test > -bash: test: Operation not permitted > > > And the file is made but is zero size. > > smoke:hunter david$ ls -l test > -rw-r--r-- 1 david realise 0 Mar 3 08:44 test > > > glusterfs/nfslog logs thus: > > [2011-03-03 08:44:10.379188] I [io-stats.c:333:io_stats_dump_fd] > glustervol1: --- fd stats --- > > [2011-03-03 08:44:10.379222] I [io-stats.c:338:io_stats_dump_fd] > glustervol1: Filename : /production/hunter/test > > Then try to open the file: > > smoke:hunter david$ cat test > > and get the following messages in the log: > > [2011-03-03 08:51:13.957319] I [afr-common.c:716:afr_lookup_done] > glustervol1-replicate-0: background meta-data self-heal triggered. path: > /production/hunter/test > [2011-03-03 08:51:13.959466] I > [afr-self-heal-common.c:1526:afr_self_heal_completion_cbk] > glustervol1-replicate-0: background meta-data self-heal completed on > /production/hunter/test > > If I do the same test on a linux machine (nfs) it's fine. > > We get the same issue on all the macs. They are 10.6.6. > > Gluster volume is mounted: > /n/auto/gv1 -rw,hard,tcp,rsize=32768,wsize=32768,intr > gus:/glustervol1 > Other nfs mounts on mac (from linux servers) are OK > > We're using LDAP to authenticate on the macs, the gluster servers aren't > bound into the LDAP domain. > > Any ideas? > > Thanks > David > > > g3:/var/log/glusterfs # gluster volume info > Volume Name: glustervol1 > Type: Distributed-Replicate > Status: Started > Number of Bricks: 4 x 2 = 8 > Transport-type: tcp > Bricks: > Brick1: g1:/mnt/glus1 > Brick2: g2:/mnt/glus1 > Brick3: g3:/mnt/glus1 > Brick4: g4:/mnt/glus1 > Brick5: g1:/mnt/glus2 > Brick6: g2:/mnt/glus2 > Brick7: g3:/mnt/glus2 > Brick8: g4:/mnt/glus2 > Options Reconfigured: > performance.stat-prefetch: 1 > performance.cache-size: 1gb > performance.write-behind-window-size: 1mb > network.ping-timeout: 20 > diagnostics.latency-measurement: off > diagnostics.dump-fd-stats: on > > > > > > -- > David Lloyd > V Consultants > www.v-consultants.co.uk > > > >-- David Lloyd V Consultants www.v-consultants.co.uk tel: +44 7983 816501 skype: davidlloyd1243
Hello, I tried a small gluster setup a little while ago (gluster 3.1.2), and I experienced the exact same issue as you describe when mounting the volume on my mac. Mounting it from a centOS distro worked fine though... Unfortunatelly I do not have any clue on how this could be fixed. Would be interested to know what's going wrong here. Cheers, Etienne --- Etienne Lyard ISDC, Observatoire de Geneve Chemin d'Ecogia 16, CH-1290 VERSOIX +41 22 379 21 14 (direct) +41 22 379 21 00 (switchboard) +41 22 379 21 33 (fax) http://www.isdc.unige.ch/ Date: Thu, 10 Mar 2011 09:14:42 +0000 From: David Lloyd <david.lloyd at v-consultants.co.uk> Subject: Re: [Gluster-users] Mac / NFS problems To: "gluster-users at gluster.org" <gluster-users at gluster.org> Message-ID: <AANLkTi=L1v20PjmY_EHiU+S1fLQ=2GOr2wo-GigLW5eb at mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Hello, Thought I'd bring this up again to see if anyone else is having similar problems with Mac NFS and gluster 3.1.2. Cheers David
David Lloyd wrote:> Hello, > > Were having issues with macs writing to our gluster system. > Gluster vol info at end. > > On a mac, if I make a file in the shell I get the following message: > > smoke:hunter david$ echo hello > test > -bash: test: Operation not permitted >I can help if you can send the nfs.log file from the /etc/glusterd directory on the nfs server. Before your mount command, set the log-level to trace for nfs server and then run the echo command above. Unmount as soon as you see the error above and email me the nfs.log. -Shehjar> > And the file is made but is zero size. > > smoke:hunter david$ ls -l test > -rw-r--r-- 1 david realise 0 Mar 3 08:44 test > > > glusterfs/nfslog logs thus: > > [2011-03-03 08:44:10.379188] I [io-stats.c:333:io_stats_dump_fd] > glustervol1: --- fd stats --- > > [2011-03-03 08:44:10.379222] I [io-stats.c:338:io_stats_dump_fd] > glustervol1: Filename : /production/hunter/test > > Then try to open the file: > > smoke:hunter david$ cat test > > and get the following messages in the log: > > [2011-03-03 08:51:13.957319] I [afr-common.c:716:afr_lookup_done] > glustervol1-replicate-0: background meta-data self-heal triggered. path: > /production/hunter/test > [2011-03-03 08:51:13.959466] I > [afr-self-heal-common.c:1526:afr_self_heal_completion_cbk] > glustervol1-replicate-0: background meta-data self-heal completed on > /production/hunter/test > > If I do the same test on a linux machine (nfs) it's fine. > > We get the same issue on all the macs. They are 10.6.6. > > Gluster volume is mounted: > /n/auto/gv1 -rw,hard,tcp,rsize=32768,wsize=32768,intr > gus:/glustervol1 > Other nfs mounts on mac (from linux servers) are OK > > We're using LDAP to authenticate on the macs, the gluster servers aren't > bound into the LDAP domain. > > Any ideas? > > Thanks > David > > > g3:/var/log/glusterfs # gluster volume info > Volume Name: glustervol1 > Type: Distributed-Replicate > Status: Started > Number of Bricks: 4 x 2 = 8 > Transport-type: tcp > Bricks: > Brick1: g1:/mnt/glus1 > Brick2: g2:/mnt/glus1 > Brick3: g3:/mnt/glus1 > Brick4: g4:/mnt/glus1 > Brick5: g1:/mnt/glus2 > Brick6: g2:/mnt/glus2 > Brick7: g3:/mnt/glus2 > Brick8: g4:/mnt/glus2 > Options Reconfigured: > performance.stat-prefetch: 1 > performance.cache-size: 1gb > performance.write-behind-window-size: 1mb > network.ping-timeout: 20 > diagnostics.latency-measurement: off > diagnostics.dump-fd-stats: on > > > > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users