# gluster volume info No volumes present # gluster peer status Number of Peers: 1 Hostname: 205.208.111.14 Uuid: 00000000-0000-0000-0000-000000000000 State: Connected to Peer (Connected) Forward and Reverse DNS lookups between all 3 servers work fine. Ping works fine. IB connectivity works fine (using tcp,rdma transport type). /var/log/glusterfs/etc-glusterfs-glusterd.vol.log on vothgluster3 [2012-09-14 15:16:27.528532] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.3.0 [2012-09-14 15:16:27.530778] I [glusterd.c:807:init] 0-management: Using /var/lib/glusterd as working directory [2012-09-14 15:16:28.088422] I [glusterd.c:95:glusterd_uuid_init] 0-glusterd: retrieved UUID: 924322ef-5e55-427b-a393-90903ccb7d16 [2012-09-14 15:16:28.097455] I [glusterd.c:265:glusterd_check_gsync_present] 0-glusterd: geo-replication module not installed in the system Given volfile: +------------------------------------------------------------------------------+ 1: volume management 2: type mgmt/glusterd 3: option working-directory /var/lib/glusterd 4: option transport-type socket,rdma 5: option transport.socket.keepalive-time 10 6: option transport.socket.keepalive-interval 2 7: option transport.socket.read-fail-log off 8: end-volume +------------------------------------------------------------------------------+ [2012-09-14 15:16:32.760625] I [glusterd-handler.c:1753:glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: 70ee8eed-224e-4031-8cfe-52004c5ccf47 [2012-09-14 15:16:32.760707] I [glusterd-handler.c:411:glusterd_friend_find] 0-glusterd: Unable to find peer by uuid [2012-09-14 15:16:32.760929] I [glusterd-handler.c:423:glusterd_friend_find] 0-glusterd: Unable to find hostname: 205.208.111.14 [2012-09-14 15:16:32.760950] I [glusterd-handler.c:1767:glusterd_handle_probe_query] 0-glusterd: Unable to find peerinfo for host: 205.208.111.14 (24007) [2012-09-14 15:16:32.766007] I [glusterd-handler.c:2204:glusterd_friend_add] 0-management: connect returned 0 [2012-09-14 15:16:32.766067] I [glusterd-handler.c:1793:glusterd_handle_probe_query] 0-glusterd: Responded to 205.208.111.14, op_ret: 0, op_errno: 0, ret: 0 [2012-09-14 15:16:32.766572] I [glusterd-handshake.c:397:glusterd_set_clnt_mgmt_program] 0-: Using Program glusterd mgmt, Num (1238433), Version (2) [2012-09-14 15:16:32.766620] I [glusterd-handshake.c:403:glusterd_set_clnt_mgmt_program] 0-: Using Program Peer mgmt, Num (1238437), Version (2) [2012-09-14 15:18:44.916221] I [glusterd-handler.c:813:glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2012-09-14 15:19:41.020394] I [glusterd-handler.c:860:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-09-14 15:22:40.807457] I [glusterd-handler.c:860:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-09-14 15:22:48.655202] I [glusterd-handler.c:813:glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req ----- Please let me know any other information to provide. Thanks in advance, Tom -- Tom Indelli tindelli at uchicago.edu<mailto:tindelli at uchicago.edu> Systems Administrator Physical Sciences Division The University of Chicago --_000_A4C9147B95CDDC4097015BC5551AEB6E0473AE80xmmbx01prodaduc_ Content-Type: text/html; charset="us-ascii" Content-ID: <69685171149C684D84D030E73DBB3660 at uchicago.edu> Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"> </head> <body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "> Hello, <div><br> </div> <div>I don''t have a great deal of experience yet with Gluster, and I''m having some tremendous difficulties adding an additional (3rd) server to the pool I have set up. This is running Gluster 3.3.0 on RHEL 6.3, using RPMs:</div> <div> <div># rpm -qa | grep gluster</div> <div>glusterfs-rdma-3.3.0-1.el6.x86_64</div> <div>glusterfs-server-3.3.0-1.el6.x86_64</div> <div>glusterfs-3.3.0-1.el6.x86_64</div> <div>glusterfs-fuse-3.3.0-1.el6.x86_64</div> </div> <div><br> </div> <div>When I first set it up, I was able to add the second server without an issue using <b>gluster peer probe vothgluster2</b>, however when I do the same here, it just hangs for a couple minutes, then returns to the prompt without successfully probing the server.</div> <div><br> </div> <div>From the first server:</div> <div> <div># gluster volume info</div> <div> </div> <div>Volume Name: home</div> <div>Type: Distribute</div> <div>Volume ID: a6f62691-a876-431a-a941-d3b76e197eeb</div> <div>Status: Started</div> <div>Number of Bricks: 4</div> <div>Transport-type: tcp,rdma</div> <div>Bricks:</div> <div>Brick1: vothgluster1:/export/brick1</div> <div>Brick2: vothgluster1:/export/brick2</div> <div>Brick3: vothgluster2:/export/brick1</div> <div>Brick4: vothgluster2:/export/brick2</div> <div>Options Reconfigured:</div> <div>nfs.rpc-auth-allow: 128.135.10.*,205.208.111.*,128.135.233.*,128.135.186.*,10.135.10.*,205.208.76.203</div> <div>nfs.disable: on</div> <div>nfs.register-with-portmap: on</div> <div>performance.io-cache: on</div> <div>nfs.port: 2049</div> <div>features.quota: off</div> <div>performance.cache-size: 64MB</div> <div>performance.read-ahead: off</div> <div>performance.write-behind-window-size: 1MB</div> <div>nfs.ports-insecure: on</div> </div> <div><br> </div> <div># gluster peer status</div> <div>Number of Peers: 2</div> <div><br> </div> <div>Hostname: vothgluster2</div> <div>Uuid: 44033fcb-f6da-4491-b860-d05bb3ce23f7</div> <div>State: Peer in Cluster (Connected)</div> <div><br> </div> <div>Hostname: vothgluster3</div> <div>Uuid: 924322ef-5e55-427b-a393-90903ccb7d16</div> <div>State: Establishing Connection (Connected)</div> <div><br> </div> <div>From the second server, gluster volume info reports the same.</div> <div> <div># gluster peer status</div> <div>Number of Peers: 1</div> <div><br> </div> <div>Hostname: 205.208.111.14</div> <div>Uuid: 70ee8eed-224e-4031-8cfe-52004c5ccf47</div> <div>State: Peer in Cluster (Connected)</div> </div> <div><br> </div> <div>From the new (problematic) server:</div> <div> <div># gluster volume info</div> <div>No volumes present</div> </div> <div><br> </div> <div> <div># gluster peer status</div> <div>Number of Peers: 1</div> <div><br> </div> <div>Hostname: 205.208.111.14</div> <div>Uuid: 00000000-0000-0000-0000-000000000000</div> <div>State: Connected to Peer (Connected)</div> </div> <div><br> </div> <div>Forward and Reverse DNS lookups between all 3 servers work fine. Ping works fine. IB connectivity works fine (using tcp,rdma transport type).</div> <div><br> </div> <div>/var/log/glusterfs/etc-glusterfs-glusterd.vol.log on vothgluster3</div> <div> <div>[2012-09-14 15:16:27.528532] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.3.0</div> <div>[2012-09-14 15:16:27.530778] I [glusterd.c:807:init] 0-management: Using /var/lib/glusterd as working directory</div> <div>[2012-09-14 15:16:28.088422] I [glusterd.c:95:glusterd_uuid_init] 0-glusterd: retrieved UUID: 924322ef-5e55-427b-a393-90903ccb7d16</div> <div>[2012-09-14 15:16:28.097455] I [glusterd.c:265:glusterd_check_gsync_present] 0-glusterd: geo-replication module not installed in the system</div> <div>Given volfile:</div> <div>+------------------------------------------------------------------------------+</div> <div> 1: volume management</div> <div> 2: type mgmt/glusterd</div> <div> 3: option working-directory /var/lib/glusterd</div> <div> 4: option transport-type socket,rdma</div> <div> 5: option transport.socket.keepalive-time 10</div> <div> 6: option transport.socket.keepalive-interval 2</div> <div> 7: option transport.socket.read-fail-log off</div> <div> 8: end-volume</div> <div><br> </div> <div>+------------------------------------------------------------------------------+</div> <div>[2012-09-14 15:16:32.760625] I [glusterd-handler.c:1753:glusterd_handle_probe_query] 0-glusterd: Received probe from uuid: 70ee8eed-224e-4031-8cfe-52004c5ccf47</div> <div>[2012-09-14 15:16:32.760707] I [glusterd-handler.c:411:glusterd_friend_find] 0-glusterd: Unable to find peer by uuid</div> <div>[2012-09-14 15:16:32.760929] I [glusterd-handler.c:423:glusterd_friend_find] 0-glusterd: Unable to find hostname: 205.208.111.14</div> <div>[2012-09-14 15:16:32.760950] I [glusterd-handler.c:1767:glusterd_handle_probe_query] 0-glusterd: Unable to find peerinfo for host: 205.208.111.14 (24007)</div> <div>[2012-09-14 15:16:32.766007] I [glusterd-handler.c:2204:glusterd_friend_add] 0-management: connect returned 0</div> <div>[2012-09-14 15:16:32.766067] I [glusterd-handler.c:1793:glusterd_handle_probe_query] 0-glusterd: Responded to 205.208.111.14, op_ret: 0, op_errno: 0, ret: 0</div> <div>[2012-09-14 15:16:32.766572] I [glusterd-handshake.c:397:glusterd_set_clnt_mgmt_program] 0-: Using Program glusterd mgmt, Num (1238433), Version (2)</div> <div>[2012-09-14 15:16:32.766620] I [glusterd-handshake.c:403:glusterd_set_clnt_mgmt_program] 0-: Using Program Peer mgmt, Num (1238437), Version (2)</div> <div>[2012-09-14 15:18:44.916221] I [glusterd-handler.c:813:glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req</div> <div>[2012-09-14 15:19:41.020394] I [glusterd-handler.c:860:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req</div> <div>[2012-09-14 15:22:40.807457] I [glusterd-handler.c:860:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req</div> <div>[2012-09-14 15:22:48.655202] I [glusterd-handler.c:813:glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req</div> </div> <div><br> </div> <div>-----</div> <div><br> </div> <div>Please let me know any other information to provide.</div> <div><br> </div> <div>Thanks in advance,</div> <div>Tom</div> <div><br> </div> <br> <br> <div> <div style=3D"color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "> <span class=3D"Apple-style-span" style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "> <div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "> <span class=3D"Apple-style-span" style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "> <div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "> <span class=3D"Apple-style-span" style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "> <div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "> <span class=3D"Apple-style-span" style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; border-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "> <div style=3D"font-size: medium; "><span style=3D"font-size: 12px; ">--</span></div> <div apple-content-edited=3D"true" style=3D"font-size: medium; "><span class=3D"Apple-style-span" style=3D"border-collapse: separate; border-spacing: 0px; "> <div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "> <span class=3D"Apple-style-span" style=3D"border-collapse: separate; font-size: 12px; border-spacing: 0px; "> <div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "> <span class=3D"Apple-style-span" style=3D"border-collapse: separate; border-spacing: 0px; "> <div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "> <span class=3D"Apple-style-span" style=3D"border-collapse: separate; border-spacing: 0px; "> <div>Tom Indelli</div> <div><a href=3D"mailto:tindelli at uchicago.edu">tindelli at uchicago.edu</a></div> <div>Systems Administrator</div> <div>Physical Sciences Division</div> <div>The University of Chicago</div> </span></div> </span></div> </span></div> </span></div> </span></div> </span></div> </span></div> </span></div> </div> </body> </html> --_000_A4C9147B95CDDC4097015BC5551AEB6E0473AE80xmmbx01prodaduc_--