similar to: Slow write times to gluster disk

Are you sure using conv=sync is what you want? I normally use conv=fdatasync, I'll look up the difference between the two and see if it affects your test. -b ----- Original Message ----- > From: "Pat Haley" <phaley at mit.edu> > To: "Pranith Kumar Karampuri" <pkarampu at redhat.com> > Cc: "Ravishankar N" <ravishankar at redhat.com>,

On Fri, Jun 23, 2017 at 2:23 AM, Pat Haley <phaley at mit.edu> wrote: > > Hi, > > Today we experimented with some of the FUSE options that we found in the > list. > > Changing these options had no effect: > > gluster volume set test-volume performance.cache-max-file-size 2MB > gluster volume set test-volume performance.cache-refresh-timeout 4 > gluster

Hi All, Decided to try another tests of gluster mounted via FUSE vs gluster mounted via NFS, this time using the software we run in production (i.e. our ocean model writing a netCDF file). gluster mounted via NFS the run took 2.3 hr gluster mounted via FUSE: the run took 44.2 hr The only problem with using gluster mounted via NFS is that it does not respect the group write permissions which

Hi Guys, I was wondering what our next steps should be to solve the slow write times. Recently I was debugging a large code and writing a lot of output at every time step. When I tried writing to our gluster disks, it was taking over a day to do a single time step whereas if I had the same program (same hardware, network) write to our nfs disk the time per time-step was about 45 minutes.

On Fri, Jun 23, 2017 at 9:10 AM, Pranith Kumar Karampuri < pkarampu at redhat.com> wrote: > > > On Fri, Jun 23, 2017 at 2:23 AM, Pat Haley <phaley at mit.edu> wrote: > >> >> Hi, >> >> Today we experimented with some of the FUSE options that we found in the >> list. >> >> Changing these options had no effect: >> >>

Hi, Today we experimented with some of the FUSE options that we found in the list. Changing these options had no effect: gluster volume set test-volume performance.cache-max-file-size 2MB gluster volume set test-volume performance.cache-refresh-timeout 4 gluster volume set test-volume performance.cache-size 256MB gluster volume set test-volume performance.write-behind-window-size 4MB gluster

On Mon, Jun 26, 2017 at 7:40 PM, Pat Haley <phaley at mit.edu> wrote: > > Hi All, > > Decided to try another tests of gluster mounted via FUSE vs gluster > mounted via NFS, this time using the software we run in production (i.e. > our ocean model writing a netCDF file). > > gluster mounted via NFS the run took 2.3 hr > > gluster mounted via FUSE: the run took

On 06/27/2017 10:17 AM, Pranith Kumar Karampuri wrote: > The only problem with using gluster mounted via NFS is that it does not > respect the group write permissions which we need. > > We have an exercise coming up in the a couple of weeks. It seems to me > that in order to improve our write times before then, it would be good > to solve the group write permissions for gluster

Hi Soumya, One example, we have a common working directory dri_fleat in the gluster volume drwxrwsr-x 22 root dri_fleat 4.0K May 1 15:14 dri_fleat my user (phaley) does not own that directory but is a member of the group dri_fleat and should have write permissions. When I go to the nfs-mounted version and try to use the touch command I get the following ibfdr-compute-0-4(dri_fleat)%

Hi, I was wondering if there were any additional test we could perform to help debug the group write-permissions issue? Thanks Pat On 06/27/2017 12:29 PM, Pat Haley wrote: > > Hi Soumya, > > One example, we have a common working directory dri_fleat in the > gluster volume > > drwxrwsr-x 22 root dri_fleat 4.0K May 1 15:14 dri_fleat > > my user (phaley) does

Hi Soumya, When I originally did the tests I ran tcpdump on the client. I have rerun the tests, doing tcpdump on the server tcpdump -i any -nnSs 0 host 172.16.1.121 -w /root/capture_nfsfail.pcap The results are in the same place http://mseas.mit.edu/download/phaley/GlusterUsers/TestNFSmount/ capture_nfsfail.pcap has the results from the failed touch experiment capture_nfssucceed.pcap has

Hi Soumya, (1) In http://mseas.mit.edu/download/phaley/GlusterUsers/TestNFSmount/ I've placed the following 2 log files etc-glusterfs-glusterd.vol.log gdata.log The first has repeated messages about nfs disconnects. The second had the <fuse_mnt_direcotry>.log name (but not much information). (2) About the gluster-NFS native server: do you know where we can find documentation on

On 06/30/2017 07:56 PM, Pat Haley wrote: > > Hi, > > I was wondering if there were any additional test we could perform to > help debug the group write-permissions issue? Sorry for the delay. Please find response inline -- > > Thanks > > Pat > > > On 06/27/2017 12:29 PM, Pat Haley wrote: >> >> Hi Soumya, >> >> One example, we have a

Hi, On 07/07/2017 06:16 AM, Pat Haley wrote: > > Hi All, > > A follow-up question. I've been looking at various pages on nfs-ganesha > & gluster. Is there a version of nfs-ganesha that is recommended for > use with > > glusterfs 3.7.11 built on Apr 27 2016 14:09:22 > CentOS release 6.8 (Final) For glusterfs 3.7, nfs-ganesha-2.3-* version can be used. I see

Hi, We recently installed glusterfs 3.3.1. We have a 3 brick gluster system running that was being successfully mounted earlier. Yesterday we experienced a power outage and now after rebooting our systems, we are unable to mount this gluster file system. On the gluster client, a df -h command shows 41TB out of 55TB, while an ls command shows broken links for directories and missing files.

On 07/03/2017 09:01 PM, Pat Haley wrote: > > Hi Soumya, > > When I originally did the tests I ran tcpdump on the client. > > I have rerun the tests, doing tcpdump on the server > > tcpdump -i any -nnSs 0 host 172.16.1.121 -w /root/capture_nfsfail.pcap > > The results are in the same place > > http://mseas.mit.edu/download/phaley/GlusterUsers/TestNFSmount/ >

Hi, Just before the Thanksgiving break, we enabled quotas on the /home areas on the mseas-data server (running CentOS 5.8), using the following line in the updated /etc/fstab /dev/mapper/the_raid-lv_home /home ext3 defaults,usrquota,grpquota 1 0 Following the Thanksgiving reboot of mseas-data we have been experiencing problems with svn on mseas (our front-end machine, running

Hi Soumya, I just noticed some of the notes at the bottom. In particular * Till glusterfs-3.7, gluster-NFS (gNFS) gets enabled by default. The only requirement is that kernel-NFS has to be disabled for gluster-NFS to come up. Please disable kernel-NFS server and restart glusterd to start gNFS. In case of any issues with starting gNFS server, please look at

So far, those look the same client: [root at mseas FixOwn]# getfacl /gdata/bibliography/Work/GroupBib/trunk/ getfacl: Removing leading '/' from absolute path names # file: gdata/bibliography/Work/GroupBib/trunk/ # owner: phaley # group: mseasweb # flags: -s- user::rwx group::rwx other::r-x server: [root at mseas-data2 ~]# getfacl /gdata/bibliography/Work/GroupBib/trunk/ getfacl:

Hi We have just migrated our data to a new file server (more space, old server was showing its age). We have a volume for collaborative use, based on group membership. In our new server, the group write permissions are not being respected (e.g. the owner of a directory can still write to that directory but any other member of the associated group cannot, even though the directory clearly