Gluster users - Jan 2015 - [Gluster-user] [Gluster-devel] Sybase backup server failed to write to Gluster NFS

Hi,

We have been having 5 sybase servers doing dump/export to Gluster NFS for couple
months and yesterday it started to give us these error on not able to write
files

The gluster NFS export is not full and we can still move and write files as
sybase unix user from the sybase servers.

There are no error logs on gluster nfs nor the bricks and etc-glusterfs logs and
no nfs client error on the sybase servers neither.

The NFS export was a replica 2 volume (3x2)

I created another NFS export from same gluster but a distributed only volume and
still giving out the same error.

Any Clue?

Thanks
Peter

Jan 20 20:04:17 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on device
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.e' cannot be
opened for write access. Mount another volume.
Jan 20 20:04:17 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged
        @session_id = 87,
        @devname =
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.e',
        @action = { 'PROCEED' | 'RETRY' | 'ABORT' }
Jan 20 20:04:26 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on device
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.a' cannot be
opened for write access. Mount another volume.
Jan 20 20:04:26 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged
        @session_id = 87,
        @devname =
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.a',
        @action = { 'PROCEED' | 'RETRY' | 'ABORT' }
Jan 20 20:05:41 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on device
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.d' cannot be
opened for write access. Mount another volume.
Jan 20 20:05:41 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged
        @session_id = 87,
        @devname =
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.d',
        @action = { 'PROCEED' | 'RETRY' | 'ABORT' }

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150122/ad58fc9c/attachment.html>

Hi Peter,

Can you please try manually mounting those volumes using any/other nfs 
client and check if you are able to perform write operations. Also 
please collect the gluster nfs log while doing so.

Thanks,
Soumya

On 01/22/2015 08:18 AM, Peter Auyeung wrote:
> Hi,
>
> We have been having 5 sybase servers doing dump/export to Gluster NFS
> for couple months and yesterday it started to give us these error on not
> able to write files
>
> The gluster NFS export is not full and we can still move and write files
> as sybase unix user from the sybase servers.
>
> There are no error logs on gluster nfs nor the bricks and etc-glusterfs
> logs and no nfs client error on the sybase servers neither.
>
> The NFS export was a replica 2 volume (3x2)
>
> I created another NFS export from same gluster but a distributed only
> volume and still giving out the same error.
>
> Any Clue?
>
> Thanks
> Peter
>
> Jan 20 20:04:17 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on
> device '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.e'
> cannot be opened for write access. Mount another volume.
> Jan 20 20:04:17 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged
>          @session_id = 87,
>          @devname >
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.e',
>          @action = { 'PROCEED' | 'RETRY' | 'ABORT'
}
> Jan 20 20:04:26 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on
> device '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.a'
> cannot be opened for write access. Mount another volume.
> Jan 20 20:04:26 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged
>          @session_id = 87,
>          @devname >
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.a',
>          @action = { 'PROCEED' | 'RETRY' | 'ABORT'
}
> Jan 20 20:05:41 2015: Backup Server: 6.53.1.1: OPERATOR: Volume on
> device '/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.d'
> cannot be opened for write access. Mount another volume.
> Jan 20 20:05:41 2015: Backup Server: 6.78.1.1: EXECUTE sp_volchanged
>          @session_id = 87,
>          @devname >
'/dbbackup01/db/full/pr_rssd_id_repsrv_rssd.F01-20-20-04.d',
>          @action = { 'PROCEED' | 'RETRY' | 'ABORT'
}
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>