On 05/04/2012 01:05 PM, Brian Candler wrote:> This should be a pretty easy issue to reproduce, at least it seems to
happen
> to me very often. (gluster-3.2.5)
>
> After storage backend(s) have been rebooted, the client mounts are often
> broken until you unmount and remount. Example from this morning: I had
> rebooted storage servers to upgrade them to ubuntu 12.04. Now at the
client
> side:
>
> $ ls /gluster/scratch
> ls: cannot access /gluster/scratch: Transport endpoint is not connected
> $ ls /gluster/scratch3
> dbbuild DBS
> $ sudo umount /gluster/scratch
> $ sudo mount /gluster/scratch
> $ ls /gluster/scratch
> dbbuild
> $
>
> Note that /gluster/scratch is a distributed volume (spread across servers
> 'storage2' and 'storage3'), whereas /gluster/scratch3 is a
single brick
> (server 'storage3' only).
>
> So *some* of the mounts do seem to automatically reconnect - not all are
> affected.
>
> But in future, I think it would be good if the FUSE client could
> automatically attempt to reconnect under whatever circumstance causes
> 'Transport endpoint is not connected'; clearly it *can* reconnect
if forced.
>
Are you sure the clients are not automatically remounted within 10
seconds of servers coming up? This was working fine from the time we had
networking code written.
Internally, there is a timer thread which makes sure we automatically
reconnect after 10seconds.
Please see if you can repeat the operations 2-3 times before doing a
umount/mount, it should have gotten reconnected.
If not, please file a bug report with the glusterfs logs (of the client
process).
Regards,
Amar