Pedro Damián Cruz Santiago
2010-Jan-25 09:37 UTC
[Gluster-users] ib-verbs on centos 5 hangs
The scenario: distro CentOS release 5.2 (Final) glusterfs: glusterfs-client-3.0.0-1.x86_64.rpm glusterfs-common-3.0.0-1.x86_64.rpm glusterfs-server-3.0.0-1.x86_64.rpm configuration (server & client) from: glusterfs-volgen -n storage --transport ib-verbs storage01-ib:/storage infiniband InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] mount command on client return: ... glusterfs#storage-ib-verbs.vol on /storage type fuse (rw,allow_other,default_permissions,max_read=131072) ... error: 1. Copy from local disk to directory /storage (glusterfs) pv < CentOS-5.2-x86_64-bin-DVD.iso > /storage/file 4.29GB 0:00:10 [ 429MB/s] [=================================>] 100% 2. after that when a run any command to gluster storage, this fails whit the next error: ls /storage ls: /storage: Transport endpoint is not connected df Filesystem Size Used Avail Use% Mounted on /dev/sda1 48G 11G 35G 23% / /dev/sda3 171G 188M 162G 1% /data tmpfs 7.9G 0 7.9G 0% /dev/shm df: `/storage': Transport endpoint is not connected before a couple of minutes these commands works whell !!! what is wrong with my configuration? The storage node (glusterfs) is a RAID 6 system of 18 HD, dual quad core whit 32GB RAM. The cliente node is too dual quad core system with 16GB RAM. Thanks in advanced.
Can you clarify, does your disk mount and work for a period of time, and then fail? Or does your disk mount, but not become active for a period of time? I run ib_verbs, and find that sometimes it takes a while for a connection to be setup, but that it does start working fine in 2-5 minutes. I think the problem is my fabric manager, but i haven't tried switching it yet. Thomas On Jan 25, 2010, at 4:37 AM, Pedro Dami?n Cruz Santiago wrote:> The scenario: > > distro > CentOS release 5.2 (Final) > > > glusterfs: > glusterfs-client-3.0.0-1.x86_64.rpm > glusterfs-common-3.0.0-1.x86_64.rpm > glusterfs-server-3.0.0-1.x86_64.rpm > > > configuration (server & client) from: > glusterfs-volgen -n storage --transport ib-verbs storage01-ib:/storage > > infiniband > InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] > > > mount command on client return: > ... > glusterfs#storage-ib-verbs.vol on /storage type fuse > (rw,allow_other,default_permissions,max_read=131072) > ... > > error: > > 1. Copy from local disk to directory /storage (glusterfs) > > pv < CentOS-5.2-x86_64-bin-DVD.iso > /storage/file > 4.29GB 0:00:10 [ 429MB/s] [=================================>] 100% > > 2. after that when a run any command to gluster storage, this fails whit the > next error: > ls /storage > ls: /storage: Transport endpoint is not connected > > df > Filesystem Size Used Avail Use% Mounted on > /dev/sda1 48G 11G 35G 23% / > /dev/sda3 171G 188M 162G 1% /data > tmpfs 7.9G 0 7.9G 0% /dev/shm > df: `/storage': Transport endpoint is not connected > > > before a couple of minutes these commands works whell !!! > > what is wrong with my configuration? > > The storage node (glusterfs) is a RAID 6 system of 18 HD, dual quad core > whit 32GB RAM. > The cliente node is too dual quad core system with 16GB RAM. > > Thanks in advanced. > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Hi, Can you send client and server log files? regards, 2010/1/25 Pedro Dami?n Cruz Santiago <xeuler at gmail.com>> The scenario: > > distro > CentOS release 5.2 (Final) > > > glusterfs: > glusterfs-client-3.0.0-1.x86_64.rpm > glusterfs-common-3.0.0-1.x86_64.rpm > glusterfs-server-3.0.0-1.x86_64.rpm > > > configuration (server & client) from: > glusterfs-volgen -n storage --transport ib-verbs storage01-ib:/storage > > infiniband > InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] > > > mount command on client return: > ... > glusterfs#storage-ib-verbs.vol on /storage type fuse > (rw,allow_other,default_permissions,max_read=131072) > ... > > error: > > 1. Copy from local disk to directory /storage (glusterfs) > > pv < CentOS-5.2-x86_64-bin-DVD.iso > /storage/file > 4.29GB 0:00:10 [ 429MB/s] [=================================>] 100% > > 2. after that when a run any command to gluster storage, this fails whit > the > next error: > ls /storage > ls: /storage: Transport endpoint is not connected > > df > Filesystem Size Used Avail Use% Mounted on > /dev/sda1 48G 11G 35G 23% / > /dev/sda3 171G 188M 162G 1% /data > tmpfs 7.9G 0 7.9G 0% /dev/shm > df: `/storage': Transport endpoint is not connected > > > before a couple of minutes these commands works whell !!! > > what is wrong with my configuration? > > The storage node (glusterfs) is a RAID 6 system of 18 HD, dual quad core > whit 32GB RAM. > The cliente node is too dual quad core system with 16GB RAM. > > Thanks in advanced. > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > >-- Raghavendra G