Sahina Bose
2015-May-27 05:54 UTC
[Gluster-users] [ovirt-users] Replica2 stripe2 hang on write to VM disk
[+ gluster list] On 05/25/2015 09:32 PM, paf1 at email.cz wrote:> Hello, > can anybody help me with hanging replica2 stripe2 datastore on 4 > nodes cluster ?? > > oVirt - ovirt-engine-lib-3.5.2.1-1.el7.centos.noarch > gluster - glusterfs-server-3.7.0-2.el7.x86_64 > VM - Centos 7.1 > > If I use any bigger write to VM disk ( eg 2-5GB ) storage hosted > virtual disk will hang = I/O error > created by : > gluster volume create 12KVM12SC4 replica 2 stripe 2 > 16.0.0.161:/STORAGES/SlowClass/p4/GFS1 > 16.0.0.162:/STORAGES/SlowClass/p4/GFS1 > 16.0.0.163:/STORAGES/SlowClass/p4/GFS1 > 16.0.0.164:/STORAGES/SlowClass/p4/GFS1 > > rhev-data-center-mnt-glusterSD-localhost:_12KVM12SC4.log > ----------------------------------------------------------------------------------- > > [2015-05-25 14:47:24.205609] I [rpc-clnt.c:1807:rpc_clnt_reconfig] > 0-12KVM12SC4-client-3: changing port to 49158 (from 0) > [2015-05-25 14:47:24.210824] I > [client-handshake.c:1405:select_server_supported_programs] > 0-12KVM12SC4-client-3: Using Program GlusterFS 3.3, Num (1298437), > Version (330) > [2015-05-25 14:47:24.211204] I > [client-handshake.c:1193:client_setvolume_cbk] 0-12KVM12SC4-client-3: > Connected to 12KVM12SC4-client-3, attached to remote volume > '/STORAGES/SlowClass/p4/GFS1'. > [2015-05-25 14:47:24.211225] I > [client-handshake.c:1203:client_setvolume_cbk] 0-12KVM12SC4-client-3: > Server and Client lk-version numbers are not same, reopening the fds > [2015-05-25 14:47:24.211275] I [MSGID: 108005] > [afr-common.c:3880:afr_notify] 0-12KVM12SC4-replicate-1: Subvolume > '12KVM12SC4-client-3' came back up; going online. > [2015-05-25 14:47:24.216465] I [fuse-bridge.c:5077:fuse_graph_setup] > 0-fuse: switched to graph 0 > [2015-05-25 14:47:24.216556] I > [client-handshake.c:187:client_set_lk_version_cbk] > 0-12KVM12SC4-client-3: Server lk version = 1 > [2015-05-25 14:47:24.216643] I [fuse-bridge.c:4007:fuse_init] > 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 > kernel 7.22 > [2015-05-25 14:47:24.217998] I > [afr-common.c:1673:afr_local_discovery_cbk] 0-12KVM12SC4-replicate-0: > selecting local read_child 12KVM12SC4-client-0 > [2015-05-25 14:47:29.737732] W [fuse-bridge.c:1080:fuse_setattr_cbk] > 0-glusterfs-fuse: 40: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only > file system) > [2015-05-25 14:49:18.266212] E > [client-handshake.c:1488:client_query_portmap_cbk] > 0-12KVM12SC4-client-2: failed to get the port number for remote > subvolume. Please run 'gluster volume status' on server to see if > brick process is running. > [2015-05-25 14:49:18.266274] I [client.c:2086:client_rpc_notify] > 0-12KVM12SC4-client-2: disconnected from 12KVM12SC4-client-2. Client > process will keep trying to connect to glusterd until brick's port is > available > [2015-05-25 14:49:19.346555] I [rpc-clnt.c:1807:rpc_clnt_reconfig] > 0-12KVM12SC4-client-2: changing port to 49158 (from 0) > [2015-05-25 14:49:19.351812] I > [client-handshake.c:1405:select_server_supported_programs] > 0-12KVM12SC4-client-2: Using Program GlusterFS 3.3, Num (1298437), > Version (330) > [2015-05-25 14:49:19.352169] I > [client-handshake.c:1193:client_setvolume_cbk] 0-12KVM12SC4-client-2: > Connected to 12KVM12SC4-client-2, attached to remote volume > '/STORAGES/SlowClass/p4/GFS1'. > [2015-05-25 14:49:19.352191] I > [client-handshake.c:1203:client_setvolume_cbk] 0-12KVM12SC4-client-2: > Server and Client lk-version numbers are not same, reopening the fds > [2015-05-25 14:49:19.352242] I [MSGID: 108002] > [afr-common.c:3959:afr_notify] 0-12KVM12SC4-replicate-1: Client-quorum > is met > [2015-05-25 14:49:19.352353] I > [client-handshake.c:187:client_set_lk_version_cbk] > 0-12KVM12SC4-client-2: Server lk version = 1 > [2015-05-25 14:49:27.843616] W [fuse-bridge.c:1263:fuse_err_cbk] > 0-glusterfs-fuse: 151: REMOVEXATTR() /__DIRECT_IO_TEST__ => -1 (No > data available) > [2015-05-25 14:49:58.356900] W [fuse-bridge.c:1263:fuse_err_cbk] > 0-glusterfs-fuse: 327: REMOVEXATTR() /__DIRECT_IO_TEST__ => -1 (No > data available) > > > # gluster volume status > Status of volume: 12KVM12SC4 > Gluster process TCP Port RDMA Port > Online Pid > ------------------------------------------------------------------------------ > > Brick 16.0.0.161:/STORAGES/SlowClass/p4/GFS > 1 49173 0 Y 17678 > Brick 16.0.0.162:/STORAGES/SlowClass/p4/GFS > 1 49158 0 Y 19184 > Brick 16.0.0.163:/STORAGES/SlowClass/p4/GFS > 1 49158 0 Y 9784 > Brick 16.0.0.164:/STORAGES/SlowClass/p4/GFS > 1 49158 0 Y 9327 > NFS Server on localhost 2049 0 Y 17697 > Self-heal Daemon on localhost N/A N/A Y 17708 > NFS Server on 16.0.0.162 2049 0 Y 19205 > Self-heal Daemon on 16.0.0.162 N/A N/A Y 19215 > NFS Server on 16.0.0.163 2049 0 Y 9806 > Self-heal Daemon on 16.0.0.163 N/A N/A Y 9813 > NFS Server on 16.0.0.164 2049 0 Y 9347 > Self-heal Daemon on 16.0.0.164 N/A N/A Y 9359 > > Task Status of Volume 12KVM12SC4 > ------------------------------------------------------------------------------ > > There are no active volume tasks > > > > any idea ?? > regs. > Pavel > _______________________________________________ > Users mailing list > Users at ovirt.org > http://lists.ovirt.org/mailman/listinfo/users