Hello, I am looking at the changes done for this bug 767229. I downloaded and installed 3.3.0beta3-1 which seems to have included the changes. But when I make myself a member of more than 16 groups, I can't access any directory nor files under my mounted glusterfs file system. I got a "Transport endpoint is not connected" error. I have no problem accessing the directories as root user, or as a different user who belongs to less than 16 groups. This is what I get in the log on the client machine: [2012-05-02 18:48:11.121567] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] 0-rpc: failed to encode call msg [2012-05-02 18:48:11.121600] E [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-0: Failed to build record header [2012-05-02 18:48:11.121615] W [rpc-clnt.c:1328:rpc_clnt_record] 0-backup-client-0: cannot build rpc-record [2012-05-02 18:48:11.121628] W [rpc-clnt.c:1467:rpc_clnt_submit] 0-backup-client-0: cannot build rpc-record [2012-05-02 18:48:11.121641] W [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-0: remote operation failed: Transport endpoint is not connected. Path: / [2012-05-02 18:48:11.121665] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] 0-rpc: failed to encode call msg [2012-05-02 18:48:11.121679] E [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-1: Failed to build record header [2012-05-02 18:48:11.121690] W [rpc-clnt.c:1328:rpc_clnt_record] 0-backup-client-1: cannot build rpc-record [2012-05-02 18:48:11.121701] W [rpc-clnt.c:1467:rpc_clnt_submit] 0-backup-client-1: cannot build rpc-record [2012-05-02 18:48:11.121727] W [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-1: remote operation failed: Transport endpoint is not connected. Path: / [2012-05-02 18:48:11.121749] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] 0-rpc: failed to encode call msg [2012-05-02 18:48:11.121763] E [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-2: Failed to build record header [2012-05-02 18:48:11.121775] W [rpc-clnt.c:1328:rpc_clnt_record] 0-backup-client-2: cannot build rpc-record [2012-05-02 18:48:11.121786] W [rpc-clnt.c:1467:rpc_clnt_submit] 0-backup-client-2: cannot build rpc-record [2012-05-02 18:48:11.121798] W [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-2: remote operation failed: Transport endpoint is not connected. Path: / [2012-05-02 18:48:11.121818] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] 0-rpc: failed to encode call msg [2012-05-02 18:48:11.121831] E [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-3: Failed to build record header [2012-05-02 18:48:11.121842] W [rpc-clnt.c:1328:rpc_clnt_record] 0-backup-client-3: cannot build rpc-record [2012-05-02 18:48:11.121853] W [rpc-clnt.c:1467:rpc_clnt_submit] 0-backup-client-3: cannot build rpc-record [2012-05-02 18:48:11.121865] W [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-3: remote operation failed: Transport endpoint is not connected. Path: / [2012-05-02 18:48:11.121885] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] 0-rpc: failed to encode call msg [2012-05-02 18:48:11.121898] E [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-4: Failed to build record header [2012-05-02 18:48:11.121909] W [rpc-clnt.c:1328:rpc_clnt_record] 0-backup-client-4: cannot build rpc-record [2012-05-02 18:48:11.121920] W [rpc-clnt.c:1467:rpc_clnt_submit] 0-backup-client-4: cannot build rpc-record [2012-05-02 18:48:11.121932] W [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-4: remote operation failed: Transport endpoint is not connected. Path: / [2012-05-02 18:48:11.121946] W [fuse-bridge.c:712:fuse_fd_cbk] 0-glusterfs-fuse: 12: OPENDIR() / => -1 (Transport endpoint is not connected) Is there anything I am missing? Thanks, ... ling
Hi, Can you please give it another try with a clean install? Also please make sure that both server and clients are of the same version. - Anush On 05/03/2012 07:20 AM, Ling Ho wrote:> Hello, > > I am looking at the changes done for this bug 767229. > > I downloaded and installed 3.3.0beta3-1 which seems to have included > the changes. But when I make myself a member of more than 16 groups, I > can't access any directory nor files under my mounted glusterfs file > system. I got a "Transport endpoint is not connected" error. I have > no problem accessing the directories as root user, or as a different > user who belongs to less than 16 groups. > > This is what I get in the log on the client machine: > > [2012-05-02 18:48:11.121567] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] > 0-rpc: failed to encode call msg > [2012-05-02 18:48:11.121600] E > [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-0: > Failed to build record header > [2012-05-02 18:48:11.121615] W [rpc-clnt.c:1328:rpc_clnt_record] > 0-backup-client-0: cannot build rpc-record> [2012-05-02 18:48:11.121628] W [rpc-clnt.c:1467:rpc_clnt_submit] > 0-backup-client-0: cannot build rpc-record > [2012-05-02 18:48:11.121641] W > [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-0: > remote operation failed: Transport endpoint is not connected. Path: / > [2012-05-02 18:48:11.121665] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] > 0-rpc: failed to encode call msg > [2012-05-02 18:48:11.121679] E > [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-1: > Failed to build record header > [2012-05-02 18:48:11.121690] W [rpc-clnt.c:1328:rpc_clnt_record] > 0-backup-client-1: cannot build rpc-record > [2012-05-02 18:48:11.121701] W [rpc-clnt.c:1467:rpc_clnt_submit] > 0-backup-client-1: cannot build rpc-record > [2012-05-02 18:48:11.121727] W > [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-1: > remote operation failed: Transport endpoint is not connected. Path: / > [2012-05-02 18:48:11.121749] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] > 0-rpc: failed to encode call msg > [2012-05-02 18:48:11.121763] E > [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-2: > Failed to build record header > [2012-05-02 18:48:11.121775] W [rpc-clnt.c:1328:rpc_clnt_record] > 0-backup-client-2: cannot build rpc-record > [2012-05-02 18:48:11.121786] W [rpc-clnt.c:1467:rpc_clnt_submit] > 0-backup-client-2: cannot build rpc-record > [2012-05-02 18:48:11.121798] W > [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-2: > remote operation failed: Transport endpoint is not connected. Path: / > [2012-05-02 18:48:11.121818] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] > 0-rpc: failed to encode call msg > [2012-05-02 18:48:11.121831] E > [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-3: > Failed to build record header > [2012-05-02 18:48:11.121842] W [rpc-clnt.c:1328:rpc_clnt_record] > 0-backup-client-3: cannot build rpc-record > [2012-05-02 18:48:11.121853] W [rpc-clnt.c:1467:rpc_clnt_submit] > 0-backup-client-3: cannot build rpc-record > [2012-05-02 18:48:11.121865] W > [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-3: > remote operation failed: Transport endpoint is not connected. Path: / > [2012-05-02 18:48:11.121885] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] > 0-rpc: failed to encode call msg > [2012-05-02 18:48:11.121898] E > [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-4: > Failed to build record header > [2012-05-02 18:48:11.121909] W [rpc-clnt.c:1328:rpc_clnt_record] > 0-backup-client-4: cannot build rpc-record > [2012-05-02 18:48:11.121920] W [rpc-clnt.c:1467:rpc_clnt_submit] > 0-backup-client-4: cannot build rpc-record > [2012-05-02 18:48:11.121932] W > [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-4: > remote operation failed: Transport endpoint is not connected. Path: / > [2012-05-02 18:48:11.121946] W [fuse-bridge.c:712:fuse_fd_cbk] > 0-glusterfs-fuse: 12: OPENDIR() / => -1 (Transport endpoint is not > connected) > > Is there anything I am missing? > > Thanks, > ... > ling > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Hi Ling, Can you please give me the steps used in testing the issue so that I could reproduce it locally. - Anush On 05/05/2012 06:24 AM, Ling Ho wrote:> Hi, > > I tried again, with clean install both on the server and client, and > recreated the volume. Both have these four rpms installed. > > glusterfs-3.3.0beta3-1 > glusterfs-server-3.3.0beta3-1 > glusterfs-rdma-3.3.0beta3-1 > glusterfs-fuse-3.3.0beta3-1 > > I tried both rdma and tcp protocol, and both gave the same results. > > Thanks, > ... > ling > > > On 05/02/2012 10:34 PM, Anush Shetty wrote: >> Hi, >> >> Can you please give it another try with a clean install? Also please >> make sure that both server and clients are of the same version. >> >> - >> Anush >> >> On 05/03/2012 07:20 AM, Ling Ho wrote: >>> Hello, >>> >>> I am looking at the changes done for this bug 767229. >>> >>> I downloaded and installed 3.3.0beta3-1 which seems to have included >>> the changes. But when I make myself a member of more than 16 groups, I >>> can't access any directory nor files under my mounted glusterfs file >>> system. I got a "Transport endpoint is not connected" error. I have >>> no problem accessing the directories as root user, or as a different >>> user who belongs to less than 16 groups. >>> >>> This is what I get in the log on the client machine: >>> >>> [2012-05-02 18:48:11.121567] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>> 0-rpc: failed to encode call msg >>> [2012-05-02 18:48:11.121600] E >>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-0: >>> Failed to build record header >>> [2012-05-02 18:48:11.121615] W [rpc-clnt.c:1328:rpc_clnt_record] >>> 0-backup-client-0: cannot build rpc-record >>> [2012-05-02 18:48:11.121628] W [rpc-clnt.c:1467:rpc_clnt_submit] >>> 0-backup-client-0: cannot build rpc-record >>> [2012-05-02 18:48:11.121641] W >>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-0: >>> remote operation failed: Transport endpoint is not connected. Path: / >>> [2012-05-02 18:48:11.121665] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>> 0-rpc: failed to encode call msg >>> [2012-05-02 18:48:11.121679] E >>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-1: >>> Failed to build record header >>> [2012-05-02 18:48:11.121690] W [rpc-clnt.c:1328:rpc_clnt_record] >>> 0-backup-client-1: cannot build rpc-record >>> [2012-05-02 18:48:11.121701] W [rpc-clnt.c:1467:rpc_clnt_submit] >>> 0-backup-client-1: cannot build rpc-record >>> [2012-05-02 18:48:11.121727] W >>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-1: >>> remote operation failed: Transport endpoint is not connected. Path: / >>> [2012-05-02 18:48:11.121749] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>> 0-rpc: failed to encode call msg >>> [2012-05-02 18:48:11.121763] E >>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-2: >>> Failed to build record header >>> [2012-05-02 18:48:11.121775] W [rpc-clnt.c:1328:rpc_clnt_record] >>> 0-backup-client-2: cannot build rpc-record >>> [2012-05-02 18:48:11.121786] W [rpc-clnt.c:1467:rpc_clnt_submit] >>> 0-backup-client-2: cannot build rpc-record >>> [2012-05-02 18:48:11.121798] W >>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-2: >>> remote operation failed: Transport endpoint is not connected. Path: / >>> [2012-05-02 18:48:11.121818] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>> 0-rpc: failed to encode call msg >>> [2012-05-02 18:48:11.121831] E >>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-3: >>> Failed to build record header >>> [2012-05-02 18:48:11.121842] W [rpc-clnt.c:1328:rpc_clnt_record] >>> 0-backup-client-3: cannot build rpc-record >>> [2012-05-02 18:48:11.121853] W [rpc-clnt.c:1467:rpc_clnt_submit] >>> 0-backup-client-3: cannot build rpc-record >>> [2012-05-02 18:48:11.121865] W >>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-3: >>> remote operation failed: Transport endpoint is not connected. Path: / >>> [2012-05-02 18:48:11.121885] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>> 0-rpc: failed to encode call msg >>> [2012-05-02 18:48:11.121898] E >>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-4: >>> Failed to build record header >>> [2012-05-02 18:48:11.121909] W [rpc-clnt.c:1328:rpc_clnt_record] >>> 0-backup-client-4: cannot build rpc-record >>> [2012-05-02 18:48:11.121920] W [rpc-clnt.c:1467:rpc_clnt_submit] >>> 0-backup-client-4: cannot build rpc-record >>> [2012-05-02 18:48:11.121932] W >>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-4: >>> remote operation failed: Transport endpoint is not connected. Path: / >>> [2012-05-02 18:48:11.121946] W [fuse-bridge.c:712:fuse_fd_cbk] >>> 0-glusterfs-fuse: 12: OPENDIR() / => -1 (Transport endpoint is not >>> connected) >>> >>> Is there anything I am missing? >>> >>> Thanks, >>> ... >>> ling >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >
Hi Anush, Sorry about not replaying earlier. I ran into hardware problems a couple of weeks ago and have not been able to do any testing. But a couple of days ago I managed to put together another test environment, and installed the latest beta and it is working as it should now. I can confirm that group permission works fine when a user belongs to more than 16 groups. I will report any problem if I see any on the official 3.3 release. Thanks, ... ling On 05/18/2012 07:10 AM, Anush Shetty wrote:> Hi Ling, > > Was not able to reproduce it locally. Do you have a script which can > reproduce this issue. > > Let me know. > > Thanks, > Anush > > On 05/17/2012 07:38 AM, Ling Ho wrote: >> Hi Anush, >> >> Did you have a chance to try to reproduce the error? Should I expect >> this feature/fix be supported in 3.3? >> >> Let me know if I am doing the right thing to test it, or if I should >> download a different tarball to try it. >> >> Thanks, >> ... >> ling >> >> On 05/07/2012 11:56 AM, Ling Ho wrote: >>> Hi Anush, >>> >>> This is how I see the error. I added myself into more than 16 posix >>> groups and I start seeing the "Transport endpoint is not connected" >>> error, and could no longer access the gluster file system anymore. I >>> have no problem when using another user (tstopr) that belongs to less >>> than 16 posix groups. Both server and client machines are running >>> RHEL5.8, kernel version 2.6.18-308.1.1, and glusterfs 3.3.0beta3-1 . >>> >>> Under 3.2.6-1, I could at least see the file system content as both >>> users. Of couse I have problem with group permissions because of the rpc >>> 16 gid limit. >>> >>> [root at pssrv105 x86_64]# mount -t glusterfs psanabackup02:backup >>> /tmp/mnt >>> [root at pssrv105 x86_64]# ls /tmp/mnt >>> backup_ana01_scratch.01-17-2012 backup_ana12_scratch.03-29-2012 >>> backup_ana02_scratch.01-17-2012 backup_ana12_scratch.03-30-2012 >>> backup_ana11_scratch.03-22-2012 backup_ana12_scratch.03-31-2012 >>> backup_ana11_scratch.03-29-2012 backup_ana12_scratch.04-02-2012 >>> backup_ana11_scratch.03-30-2012 backup_ana12_scratch.04-03-2012 >>> backup_ana11_scratch.03-31-2012 backup_ana12_scratch.04-05-2012 >>> backup_ana11_scratch.04-10-2012 backup_ana12_scratch.04-06-2012 >>> backup_ana11_scratch.04-11-2012 backup_ana12_scratch.04-12-2012 >>> backup_ana11_scratch.05-04-2012 backup_ana12_scratch.04-16-2012 >>> backup_ana11_scratch.current backup_ana12_scratch.current >>> backup_ana12_scratch.03-19-2012 test >>> backup_ana12_scratch.03-26-2012 >>> [root at pssrv105 x86_64]# su - ling >>> -bash-3.2$ cd /tmp/mnt >>> -bash-3.2$ ls -al >>> ls: .: Transport endpoint is not connected >>> -bash-3.2$ >>> -bash-3.2$ id >>> uid=10730(ling) gid=1107(re) >>> groups=1107(re),2293(amo00409),2294(amo02809),2295(amo01109),2306(amo14410),2307(amo10610),2308(amo10810),2309(amo15010),2310(amo10510),2341(ps-ioc),2352(amo19810),2353(amo22210),2392(ps-cxi),2406(amom0111),2475(cxi49812),10000(ps-users),10001(ps-pcds),10002(ps-icsmec) >>> >>> >>> -bash-3.2$ logout >>> [root at pssrv105 x86_64]# su - tstopr >>> pssrv105:~> cd /tmp/mnt >>> pssrv105:mnt> ls >>> backup_ana01_scratch.01-17-2012 backup_ana12_scratch.03-29-2012 >>> backup_ana02_scratch.01-17-2012 backup_ana12_scratch.03-30-2012 >>> backup_ana11_scratch.03-22-2012 backup_ana12_scratch.03-31-2012 >>> backup_ana11_scratch.03-29-2012 backup_ana12_scratch.04-02-2012 >>> backup_ana11_scratch.03-30-2012 backup_ana12_scratch.04-03-2012 >>> backup_ana11_scratch.03-31-2012 backup_ana12_scratch.04-05-2012 >>> backup_ana11_scratch.04-10-2012 backup_ana12_scratch.04-06-2012 >>> backup_ana11_scratch.04-11-2012 backup_ana12_scratch.04-12-2012 >>> backup_ana11_scratch.05-04-2012 backup_ana12_scratch.04-16-2012 >>> backup_ana11_scratch.current backup_ana12_scratch.current >>> backup_ana12_scratch.03-19-2012 test >>> backup_ana12_scratch.03-26-2012 >>> pssrv105:mnt> id >>> uid=10858(tstopr) gid=1109(xs) groups=1109(xs) >>> >>> >>> Thanks, >>> >>> ... >>> ling >>> >>> On 05/06/2012 11:10 PM, Anush Shetty wrote: >>>> Hi Ling, >>>> >>>> Can you please give me the steps used in testing the issue so that I >>>> could reproduce it locally. >>>> >>>> - >>>> Anush >>>> >>>> >>>> On 05/05/2012 06:24 AM, Ling Ho wrote: >>>>> Hi, >>>>> >>>>> I tried again, with clean install both on the server and client, and >>>>> recreated the volume. Both have these four rpms installed. >>>>> >>>>> glusterfs-3.3.0beta3-1 >>>>> glusterfs-server-3.3.0beta3-1 >>>>> glusterfs-rdma-3.3.0beta3-1 >>>>> glusterfs-fuse-3.3.0beta3-1 >>>>> >>>>> I tried both rdma and tcp protocol, and both gave the same results. >>>>> >>>>> Thanks, >>>>> ... >>>>> ling >>>>> >>>>> >>>>> On 05/02/2012 10:34 PM, Anush Shetty wrote: >>>>>> Hi, >>>>>> >>>>>> Can you please give it another try with a clean install? Also please >>>>>> make sure that both server and clients are of the same version. >>>>>> >>>>>> - >>>>>> Anush >>>>>> >>>>>> On 05/03/2012 07:20 AM, Ling Ho wrote: >>>>>>> Hello, >>>>>>> >>>>>>> I am looking at the changes done for this bug 767229. >>>>>>> >>>>>>> I downloaded and installed 3.3.0beta3-1 which seems to have included >>>>>>> the changes. But when I make myself a member of more than 16 >>>>>>> groups, I >>>>>>> can't access any directory nor files under my mounted glusterfs file >>>>>>> system. I got a "Transport endpoint is not connected" error. I have >>>>>>> no problem accessing the directories as root user, or as a different >>>>>>> user who belongs to less than 16 groups. >>>>>>> >>>>>>> This is what I get in the log on the client machine: >>>>>>> >>>>>>> [2012-05-02 18:48:11.121567] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>>>>>> 0-rpc: failed to encode call msg >>>>>>> [2012-05-02 18:48:11.121600] E >>>>>>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-0: >>>>>>> Failed to build record header >>>>>>> [2012-05-02 18:48:11.121615] W [rpc-clnt.c:1328:rpc_clnt_record] >>>>>>> 0-backup-client-0: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121628] W [rpc-clnt.c:1467:rpc_clnt_submit] >>>>>>> 0-backup-client-0: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121641] W >>>>>>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-0: >>>>>>> remote operation failed: Transport endpoint is not connected. >>>>>>> Path: / >>>>>>> [2012-05-02 18:48:11.121665] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>>>>>> 0-rpc: failed to encode call msg >>>>>>> [2012-05-02 18:48:11.121679] E >>>>>>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-1: >>>>>>> Failed to build record header >>>>>>> [2012-05-02 18:48:11.121690] W [rpc-clnt.c:1328:rpc_clnt_record] >>>>>>> 0-backup-client-1: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121701] W [rpc-clnt.c:1467:rpc_clnt_submit] >>>>>>> 0-backup-client-1: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121727] W >>>>>>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-1: >>>>>>> remote operation failed: Transport endpoint is not connected. >>>>>>> Path: / >>>>>>> [2012-05-02 18:48:11.121749] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>>>>>> 0-rpc: failed to encode call msg >>>>>>> [2012-05-02 18:48:11.121763] E >>>>>>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-2: >>>>>>> Failed to build record header >>>>>>> [2012-05-02 18:48:11.121775] W [rpc-clnt.c:1328:rpc_clnt_record] >>>>>>> 0-backup-client-2: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121786] W [rpc-clnt.c:1467:rpc_clnt_submit] >>>>>>> 0-backup-client-2: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121798] W >>>>>>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-2: >>>>>>> remote operation failed: Transport endpoint is not connected. >>>>>>> Path: / >>>>>>> [2012-05-02 18:48:11.121818] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>>>>>> 0-rpc: failed to encode call msg >>>>>>> [2012-05-02 18:48:11.121831] E >>>>>>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-3: >>>>>>> Failed to build record header >>>>>>> [2012-05-02 18:48:11.121842] W [rpc-clnt.c:1328:rpc_clnt_record] >>>>>>> 0-backup-client-3: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121853] W [rpc-clnt.c:1467:rpc_clnt_submit] >>>>>>> 0-backup-client-3: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121865] W >>>>>>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-3: >>>>>>> remote operation failed: Transport endpoint is not connected. >>>>>>> Path: / >>>>>>> [2012-05-02 18:48:11.121885] W [xdr-rpcclnt.c:88:rpc_request_to_xdr] >>>>>>> 0-rpc: failed to encode call msg >>>>>>> [2012-05-02 18:48:11.121898] E >>>>>>> [rpc-clnt.c:1268:rpc_clnt_record_build_record] 0-backup-client-4: >>>>>>> Failed to build record header >>>>>>> [2012-05-02 18:48:11.121909] W [rpc-clnt.c:1328:rpc_clnt_record] >>>>>>> 0-backup-client-4: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121920] W [rpc-clnt.c:1467:rpc_clnt_submit] >>>>>>> 0-backup-client-4: cannot build rpc-record >>>>>>> [2012-05-02 18:48:11.121932] W >>>>>>> [client3_1-fops.c:2528:client3_1_opendir_cbk] 0-backup-client-4: >>>>>>> remote operation failed: Transport endpoint is not connected. >>>>>>> Path: / >>>>>>> [2012-05-02 18:48:11.121946] W [fuse-bridge.c:712:fuse_fd_cbk] >>>>>>> 0-glusterfs-fuse: 12: OPENDIR() / => -1 (Transport endpoint is >>>>>>> not >>>>>>> connected) >>>>>>> >>>>>>> Is there anything I am missing? >>>>>>> >>>>>>> Thanks, >>>>>>> ... >>>>>>> ling >>>>>>> _______________________________________________ >>>>>>> Gluster-users mailing list >>>>>>> Gluster-users at gluster.org >>>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>>>>> _______________________________________________ >>>>>> Gluster-users mailing list >>>>>> Gluster-users at gluster.org >>>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users