similar to: Question on gluster interaction with Rocks

Hi Guys, I was wondering what our next steps should be to solve the slow write times. Recently I was debugging a large code and writing a lot of output at every time step. When I tried writing to our gluster disks, it was taking over a day to do a single time step whereas if I had the same program (same hardware, network) write to our nfs disk the time per time-step was about 45 minutes.

Hi, Today we experimented with some of the FUSE options that we found in the list. Changing these options had no effect: gluster volume set test-volume performance.cache-max-file-size 2MB gluster volume set test-volume performance.cache-refresh-timeout 4 gluster volume set test-volume performance.cache-size 256MB gluster volume set test-volume performance.write-behind-window-size 4MB gluster

On Mon, Jun 26, 2017 at 7:40 PM, Pat Haley <phaley at mit.edu> wrote: > > Hi All, > > Decided to try another tests of gluster mounted via FUSE vs gluster > mounted via NFS, this time using the software we run in production (i.e. > our ocean model writing a netCDF file). > > gluster mounted via NFS the run took 2.3 hr > > gluster mounted via FUSE: the run took

Are you sure using conv=sync is what you want? I normally use conv=fdatasync, I'll look up the difference between the two and see if it affects your test. -b ----- Original Message ----- > From: "Pat Haley" <phaley at mit.edu> > To: "Pranith Kumar Karampuri" <pkarampu at redhat.com> > Cc: "Ravishankar N" <ravishankar at redhat.com>,

On Fri, Jun 23, 2017 at 9:10 AM, Pranith Kumar Karampuri < pkarampu at redhat.com> wrote: > > > On Fri, Jun 23, 2017 at 2:23 AM, Pat Haley <phaley at mit.edu> wrote: > >> >> Hi, >> >> Today we experimented with some of the FUSE options that we found in the >> list. >> >> Changing these options had no effect: >> >>

Hi All, Decided to try another tests of gluster mounted via FUSE vs gluster mounted via NFS, this time using the software we run in production (i.e. our ocean model writing a netCDF file). gluster mounted via NFS the run took 2.3 hr gluster mounted via FUSE: the run took 44.2 hr The only problem with using gluster mounted via NFS is that it does not respect the group write permissions which

Hi Ben, Sorry this took so long, but we had a real-time forecasting exercise last week and I could only get to this now. Backend Hardware/OS: * Much of the information on our back end system is included at the top of http://lists.gluster.org/pipermail/gluster-users/2017-April/030529.html * The specific model of the hard disks is SeaGate ENTERPRISE CAPACITY V.4 6TB

On Fri, Jun 23, 2017 at 2:23 AM, Pat Haley <phaley at mit.edu> wrote: > > Hi, > > Today we experimented with some of the FUSE options that we found in the > list. > > Changing these options had no effect: > > gluster volume set test-volume performance.cache-max-file-size 2MB > gluster volume set test-volume performance.cache-refresh-timeout 4 > gluster

Hi We have been trying back-up a glusterfs (v3.1.4) area using the Tivoli TSM software to an off-site area. The back-up keeps failing with the following typical error messages 06/14/2011 22:22:58 ANS1587W I/O error reading file attributes for: /gdata/projects/philex/OAG/2011/May16/mdor3km10/coast_den2.in. errno = 22, Invalid argument 06/14/2011 22:22:59 ANS4007E Error processing

Hi, We have recently upgraded our gluster to 3.2.5 and have encountered the following error. Gluster seems somehow confused about one of the files it should be serving up, specifically /projects/philex/PE/2010/Oct18/arch07/BalbacFull_250_200_03Mar_3.png If I go to that directory and simply do an ls *.png I get ls: BalbacFull_250_200_03Mar_3.png: Structure needs cleaning (along with a listing

Trying the gluster client seems to fix the problem. On 09/02/2016 12:57 AM, Chris Murphy wrote: > On Thu, Sep 1, 2016, 8:11 AM Pat Haley <phaley at mit.edu> wrote: > >> For the enforcing=0, is that referring to SELinux? If so, we are not >> running SELinux. >> > > OK so neither that nor chcon nor context mount option apply. It's something > else. >

For the enforcing=0, is that referring to SELinux? If so, we are not running SELinux. On 08/31/2016 11:38 PM, Chris Murphy wrote: ---------------------------------------------------------------------------------------- > Try booting with enforcing=0 and if that fixes it, you need to find out > what security label is needed for gluster.

Hi We have just migrated our data to a new file server (more space, old server was showing its age). We have a volume for collaborative use, based on group membership. In our new server, the group write permissions are not being respected (e.g. the owner of a directory can still write to that directory but any other member of the associated group cannot, even though the directory clearly

So far, those look the same client: [root at mseas FixOwn]# getfacl /gdata/bibliography/Work/GroupBib/trunk/ getfacl: Removing leading '/' from absolute path names # file: gdata/bibliography/Work/GroupBib/trunk/ # owner: phaley # group: mseasweb # flags: -s- user::rwx group::rwx other::r-x server: [root at mseas-data2 ~]# getfacl /gdata/bibliography/Work/GroupBib/trunk/ getfacl:

Hi Soumya, One example, we have a common working directory dri_fleat in the gluster volume drwxrwsr-x 22 root dri_fleat 4.0K May 1 15:14 dri_fleat my user (phaley) does not own that directory but is a member of the group dri_fleat and should have write permissions. When I go to the nfs-mounted version and try to use the touch command I get the following ibfdr-compute-0-4(dri_fleat)%

Hi, We are looking for some guidance in installing an upgraded Python on our cluster. Our cluster was installed with Rocks 6.0, is running CentOs 6.2, and has python-2.6.6, gcc-4.4.6. We would like to install an upgraded version of Python along with the following modules NumPy Scipy (which will require a compatible version of the Atlas libraries) Mayavi (which will require a compatible

Hi, Just before the Thanksgiving break, we enabled quotas on the /home areas on the mseas-data server (running CentOS 5.8), using the following line in the updated /etc/fstab /dev/mapper/the_raid-lv_home /home ext3 defaults,usrquota,grpquota 1 0 Following the Thanksgiving reboot of mseas-data we have been experiencing problems with svn on mseas (our front-end machine, running

Hi, I was wondering if there were any additional test we could perform to help debug the group write-permissions issue? Thanks Pat On 06/27/2017 12:29 PM, Pat Haley wrote: > > Hi Soumya, > > One example, we have a common working directory dri_fleat in the > gluster volume > > drwxrwsr-x 22 root dri_fleat 4.0K May 1 15:14 dri_fleat > > my user (phaley) does

Hi, We have a cluster with 130 compute nodes with an NAS-type central storage under gluster (3 bricks, ~50TB). When we run large number of ocean models we can run into bottlenecks with many jobs trying to write to our central storage. It was suggested to us that we could also used gluster to unite the disks on the compute nodes into a single "disk" in which files would be written

For example the directory /gdata/bibliography/Work/GroupBib/trunk/ can be written in by user phaley but not by other users who are member of the group mseasweb. The directory has permissions [root at mseas ~]# ls -lh /gdata/bibliography/Work/GroupBib total 12K drwxrwsr-x 4 phaley mseasweb 4.0K Aug 30 12:31 trunk The parent directory (/gdata/bibliography/Work/GroupBib) has permissions [root