search for: rpcsvc_check_and_reply_error

...000-0 000-0000-0000-000000000000 [Invalid argument] ... gl-master-04 : etc-glusterfs-glusterd.vol.log [2017-06-23 16:35:18.872346] W [rpcsvc.c:270:rpcsvc_program_actor] 0-rpc-service: RPC program not available (req 1298437 330) for 10.0.1.203:65533 [2017-06-23 16:35:18.872421] E [rpcsvc.c:565:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully gl-master-04 : glustershd.log [2017-06-23 16:35:42.536840] E [MSGID: 108006] [afr-common.c:4323:afr_notify] 0-mvol1-replicate-1: All subvolumes are down. Going offline until atleast one of them comes back up. [2017-06-23 16:35:51.702413] E [...

...] > ... > > > gl-master-04 : etc-glusterfs-glusterd.vol.log > > [2017-06-23 16:35:18.872346] W [rpcsvc.c:270:rpcsvc_program_actor]? > 0-rpc-service: RPC program not available (req 1298437 330) for? > 10.0.1.203:65533 > [2017-06-23 16:35:18.872421] E? > [rpcsvc.c:565:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed? > to complete successfully > > gl-master-04 : glustershd.log > > [2017-06-23 16:35:42.536840] E [MSGID: 108006]? > [afr-common.c:4323:afr_notify] 0-mvol1-replicate-1: All subvolumes are? > down. Going offline until atleast one of them comes back u...

...gt;> gl-master-04 : etc-glusterfs-glusterd.vol.log >> >> [2017-06-23 16:35:18.872346] W [rpcsvc.c:270:rpcsvc_program_actor] >> 0-rpc-service: RPC program not available (req 1298437 330) for >> 10.0.1.203:65533 >> [2017-06-23 16:35:18.872421] E >> [rpcsvc.c:565:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed >> to complete successfully >> >> gl-master-04 : glustershd.log >> >> [2017-06-23 16:35:42.536840] E [MSGID: 108006] >> [afr-common.c:4323:afr_notify] 0-mvol1-replicate-1: All subvolumes are >> down. Going offline until atleast o...

Hi all I use Distributed-Replicate(12 x 2 = 24) hot tier plus Distributed-Replicate(36 x (6 + 2) = 288) cold tier with gluster3.8.4 for performance test. When i set client/server.event-threads as small values etc 2, it works ok. But if set client/server.event-threads as big values etc 32, the netconnects will always become un-available during the test, with following error messages in stree

...1. I am not sure how to install 3.11 without compiling it. Then when tried to start the gluster on the node the bricks were reported down (the other 2 nodes have still 3.8). No sure why. The logs were showing the below (even after rebooting the server): [2017-09-06 10:56:09.023777] E [rpcsvc.c:557:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully [2017-09-06 10:56:09.024122] E [server-helpers.c:395:server_alloc_frame] (-->/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x325) [0x7f2d0ec20905] -->/usr/lib64/glusterfs/3.10.5/xlator/protocol/server.so(+0x3006b) [0x7f2cfa4bf06b] -->/usr/...

...l 3.11 without > compiling it. > Then when tried to start the gluster on the node the bricks were reported > down (the other 2 nodes have still 3.8). No sure why. The logs were showing > the below (even after rebooting the server): > > [2017-09-06 10:56:09.023777] E [rpcsvc.c:557:rpcsvc_check_and_reply_error] > 0-rpcsvc: rpc actor failed to complete successfully > [2017-09-06 10:56:09.024122] E [server-helpers.c:395:server_alloc_frame] > (-->/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x325) [0x7f2d0ec20905] > -->/usr/lib64/glusterfs/3.10.5/xlator/protocol/server.so(+0x3006b) > [0x...

...1. I am not sure how to install 3.11 without compiling it. Then when tried to start the gluster on the node the bricks were reported down (the other 2 nodes have still 3.8). No sure why. The logs were showing the below (even after rebooting the server): [2017-09-06 10:56:09.023777] E [rpcsvc.c:557:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully [2017-09-06 10:56:09.024122] E [server-helpers.c:395:server_alloc_frame] (-->/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x325) [0x7f2d0ec20905] -->/usr/lib64/glusterfs/3.10.5/xlator/protocol/server.so(+0x3006b) [0x7f2cfa4bf06b] -->/usr/...

...ll 3.11 without > compiling it. > Then when tried to start the gluster on the node the bricks were reported > down (the other 2 nodes have still 3.8). No sure why. The logs were showing > the below (even after rebooting the server): > > [2017-09-06 10:56:09.023777] E [rpcsvc.c:557:rpcsvc_check_and_reply_error] > 0-rpcsvc: rpc actor failed to complete successfully > [2017-09-06 10:56:09.024122] E [server-helpers.c:395:server_alloc_frame] > (-->/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x325) [0x7f2d0ec20905] > -->/usr/lib64/glusterfs/3.10.5/xlator/protocol/server.so(+0x3006b) > [0x...

Do you see any improvement with 3.11.1 as that has a patch that improves perf for this kind of a workload Also, could you disable eager-lock and check if that helps? I see that max time is being spent in acquiring locks. -Krutika On Wed, Sep 6, 2017 at 1:38 PM, Abi Askushi <rightkicktech at gmail.com> wrote: > Hi Krutika, > > Is it anything in the profile indicating what is

Hi Krutika, Is it anything in the profile indicating what is causing this bottleneck? In case i can collect any other info let me know. Thanx On Sep 5, 2017 13:27, "Abi Askushi" <rightkicktech at gmail.com> wrote: Hi Krutika, Attached the profile stats. I enabled profiling then ran some dd tests. Also 3 Windows VMs are running on top this volume but did not do any stress

...sync-failed-syncs-after-fsync: on server.outstanding-rpc-limit: 512 -------------- next part -------------- [2018-02-22 18:07:45.241970] W [rpcsvc.c:265:rpcsvc_program_actor] 0-rpc-service: RPC program not available (req 1298437 330) for 172.17.1.61:1022 [2018-02-22 18:07:45.242018] E [rpcsvc.c:560:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully [2018-02-22 18:07:45.242681] W [rpcsvc.c:265:rpcsvc_program_actor] 0-rpc-service: RPC program not available (req 1298437 330) for 172.17.1.61:1022 [2018-02-22 18:07:45.242704] E [rpcsvc.c:560:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor fail...

...9] I [MSGID: 101190] [event-epoll.c:602:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2017-08-20 20:31:01.891406] W [rpcsvc.c:265:rpcsvc_program_actor] 0-rpc-service: RPC program not available (req 1298437 330) for 192.168.1.63:49148 [2017-08-20 20:31:01.891429] E [rpcsvc.c:557:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully [2017-08-20 20:31:02.746317] I [MSGID: 106493] [glusterd-rpc-ops.c:485:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 8f66df4a-e286-4c63-9b0b-257c1ccd08b0, host: web3.dasilva.network, port: 0 [2017-08-20 20:31:02.890721] I [rpc-cl...

...it. > > Then when tried to start the gluster on the node the bricks were reported > > down (the other 2 nodes have still 3.8). No sure why. The logs were > showing > > the below (even after rebooting the server): > > > > [2017-09-06 10:56:09.023777] E [rpcsvc.c:557:rpcsvc_check_and_reply_error] > > 0-rpcsvc: rpc actor failed to complete successfully > > [2017-09-06 10:56:09.024122] E [server-helpers.c:395:server_alloc_frame] > > (-->/lib64/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x325) [0x7f2d0ec20905] > > -->/usr/lib64/glusterfs/3.10.5/xlator/protocol/server.s...

On Mon, Aug 21, 2017 at 2:49 AM, Cesar da Silva <thunderlight1 at gmail.com> wrote: > Hi! > I am having same issue but I am running Ubuntu v16.04. > It does not mount during boot, but works if I mount it manually. I am > running the Gluster-server on the same machines (3 machines) > Here is the /tc/fstab file > > /dev/sdb1 /data/gluster ext4 defaults 0 0 > >

Hi! I am having same issue but I am running Ubuntu v16.04. It does not mount during boot, but works if I mount it manually. I am running the Gluster-server on the same machines (3 machines) Here is the /tc/fstab file /dev/sdb1 /data/gluster ext4 defaults 0 0 web1.dasilva.network:/www /mnt/glusterfs/www glusterfs defaults,_netdev,log-level=debug,log-file=/var/log/gluster.log 0 0