Pranith Kumar Karampuri
2015-Jan-21 19:16 UTC
[Gluster-users] Fwd: to mailing list gluster.org
+ quota devs Pranith On 01/07/2015 07:37 PM, Bekk, Klaus (IKP) wrote:> >> Hi all, >> >> we are running 4 server with Ubuntu 14.04 as glusterfs server with >> 3.5.2-ubuntu1~trusty1. >> > >> When starting quotas on any of the volumes the brick will be not >> available after some time. So we have to disable quota on this volume >> and stop and start this volume. >> >> We have the following configuration >> >> root at ikpsrv01:/var/log/glusterfs# gluster volume info >> >> Volume Name: gltestn >> >> Type: Distribute >> >> Volume ID: e50bd30d-dbc8-4279-ac9d-2e557328a643 >> >> Status: Started >> >> Number of Bricks: 1 >> >> Transport-type: tcp >> >> Bricks: >> >> Brick1: ikpsrv01:/gltest2/gl >> >> Volume Name: glwww1 >> >> Type: Replicate >> >> Volume ID: 811e4c2a-4520-4e61-bb84-955afbdb9dff >> >> Status: Started >> >> Number of Bricks: 1 x 2 = 2 >> >> Transport-type: tcp >> >> Bricks: >> >> Brick1: ikpsrv03:/glwwwn/gl >> >> Brick2: ikpsrv01:/glwwwn2/gl >> >> Options Reconfigured: >> >> features.quota: on >> >> Volume Name: corsika >> >> Type: Distribute >> >> Volume ID: d6e07f3e-4b35-4736-9b71-afc3224f0f29 >> >> Status: Started >> >> Number of Bricks: 4 >> >> Transport-type: tcp >> >> Bricks: >> >> Brick1: ikpsrv03:/glcors3-1/gl >> >> Brick2: ikpsrv03:/glcors3-2/gl >> >> Brick3: ikpsrv02:/glcors2-1/gl >> >> Brick4: ikpsrv02:/glcors2-2/gl >> >> Volume Name: gldata >> >> Type: Distribute >> >> Volume ID: 82485d03-9ef4-4151-a2cc-881072e871a1 >> >> Status: Started >> >> Number of Bricks: 2 >> >> Transport-type: tcp >> >> Bricks: >> >> Brick1: ikpsrv02:/gldata2-1/gl >> >> Brick2: ikpsrv01:/gldata1-2/gl >> >> Options Reconfigured: >> >> features.quota: on >> >> Volume Name: gltest >> >> Type: Distribute >> >> Volume ID: 9e69afcc-d421-41c9-824b-7d9a31240072 >> >> Status: Started >> >> Number of Bricks: 2 >> >> Transport-type: tcp >> >> Bricks: >> >> Brick1: ikpsrv01:/gltest3/gl >> >> Brick2: ikpsrv01:/gltest4/gl >> >> Options Reconfigured: >> >> features.quota: off >> >> Volume Name: glusersold >> >> Type: Distribute >> >> Volume ID: 91a79e1a-4ff3-4191-b0ae-73357105774c >> >> Status: Started >> >> Number of Bricks: 1 >> >> Transport-type: tcp >> >> Bricks: >> >> Brick1: ikpsrv02:/glusers2-2/gl >> >> Volume Name: glusers >> >> Type: Distribute >> >> Volume ID: f48b1941-9c19-4311-bd9e-497660fbcc80 >> >> Status: Started >> >> Number of Bricks: 1 >> >> Transport-type: tcp >> >> Bricks: >> >> Brick1: ikpsrv02:/glusers2-1/gl >> >> Options Reconfigured: >> >> diagnostics.client-log-level: CRITICAL >> >> diagnostics.brick-log-level: CRITICAL >> >> features.quota: off >> >> When starting quotas on any of the volumes the brick will be not >> available after some time. So we have to disable quota on this volume >> and stop and start this volume. >> >> The command >> >> Gluster volume quota gltestn enable >> >> gives the following output in quotad.log >> >> [2015-01-07 12:01:55.522229] W [glusterfsd.c:1095:cleanup_and_exit] >> (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb455bd9fbd] >> (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb455ead182] >> (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xd5) [0x7fb4569a7265]))) >> 0-: received signum (15), shutting down >> >> [2015-01-07 12:01:56.534931] I [glusterfsd.c:1959:main] >> 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version >> 3.5.2 (/usr/sbin/glusterfs -s localhost --volfile-id gluster/quotad >> -p /var/lib/glusterd/quotad/run/quotad.pid -l >> /var/log/glusterfs/quotad.log -S >> /var/run/fa869d5552bf340ec7506747e21b2841.socket --xlator-option >> *replicate*.data-self-heal=off --xlator-option >> *replicate*.metadata-self-heal=off --xlator-option >> *replicate*.entry-self-heal=off) >> >> [2015-01-07 12:01:56.537061] I [socket.c:3561:socket_init] >> 0-socket.glusterfsd: SSL support is NOT enabled >> >> [2015-01-07 12:01:56.537134] I [socket.c:3576:socket_init] >> 0-socket.glusterfsd: using system polling thread >> >> [2015-01-07 12:01:56.537355] I [socket.c:3561:socket_init] >> 0-glusterfs: SSL support is NOT enabled >> >> [2015-01-07 12:01:56.537383] I [socket.c:3576:socket_init] >> 0-glusterfs: using system polling thread >> >> [2015-01-07 12:01:56.541480] I [graph.c:254:gf_add_cmdline_options] >> 0-glwww1-replicate-0: adding option 'entry-self-heal' for volume >> 'glwww1-replicate-0' with value 'off' >> >> [2015-01-07 12:01:56.541516] I [graph.c:254:gf_add_cmdline_options] >> 0-glwww1-replicate-0: adding option 'metadata-self-heal' for volume >> 'glwww1-replicate-0' with value 'off' >> >> [2015-01-07 12:01:56.541536] I [graph.c:254:gf_add_cmdline_options] >> 0-glwww1-replicate-0: adding option 'data-self-heal' for volume >> 'glwww1-replicate-0' with value 'off' >> >> [2015-01-07 12:01:56.543122] I [socket.c:3561:socket_init] >> 0-socket.quotad: SSL support is NOT enabled >> >> [2015-01-07 12:01:56.543157] I [socket.c:3576:socket_init] >> 0-socket.quotad: using system polling thread >> >> [2015-01-07 12:01:56.543404] I [dht-shared.c:311:dht_init_regex] >> 0-glwww1: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ >> >> [2015-01-07 12:01:56.547124] I [socket.c:3561:socket_init] >> 0-glwww1-client-1: SSL support is NOT enabled >> >> [2015-01-07 12:01:56.547158] I [socket.c:3576:socket_init] >> 0-glwww1-client-1: using system polling thread >> >> [2015-01-07 12:01:56.547728] I [socket.c:3561:socket_init] >> 0-glwww1-client-0: SSL support is NOT enabled >> >> [2015-01-07 12:01:56.547754] I [socket.c:3576:socket_init] >> 0-glwww1-client-0: using system polling thread >> >> [2015-01-07 12:01:56.547820] I [dht-shared.c:311:dht_init_regex] >> 0-gltestn: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ >> >> [2015-01-07 12:01:56.549471] I [socket.c:3561:socket_init] >> 0-gltestn-client-0: SSL support is NOT enabled >> >> [2015-01-07 12:01:56.549501] I [socket.c:3576:socket_init] >> 0-gltestn-client-0: using system polling thread >> >> [2015-01-07 12:01:56.549565] I [dht-shared.c:311:dht_init_regex] >> 0-gldata: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ >> >> [2015-01-07 12:01:56.551155] I [socket.c:3561:socket_init] >> 0-gldata-client-1: SSL support is NOT enabled >> >> [2015-01-07 12:01:56.551184] I [socket.c:3576:socket_init] >> 0-gldata-client-1: using system polling thread >> >> [2015-01-07 12:01:56.551744] I [socket.c:3561:socket_init] >> 0-gldata-client-0: SSL support is NOT enabled >> >> [2015-01-07 12:01:56.551770] I [socket.c:3576:socket_init] >> 0-gldata-client-0: using system polling thread >> >> [2015-01-07 12:01:56.551790] W [graph.c:329:_log_if_unknown_option] >> 0-quotad: option 'rpc-auth.auth-glusterfs' is not recognized >> >> [2015-01-07 12:01:56.551819] W [graph.c:329:_log_if_unknown_option] >> 0-quotad: option 'rpc-auth.auth-unix' is not recognized >> >> [2015-01-07 12:01:56.551838] W [graph.c:329:_log_if_unknown_option] >> 0-quotad: option 'rpc-auth.auth-null' is not recognized >> >> [2015-01-07 12:01:56.551854] W [graph.c:329:_log_if_unknown_option] >> 0-quotad: option 'glwww1.volume-id' is not recognized >> >> [2015-01-07 12:01:56.551870] W [graph.c:329:_log_if_unknown_option] >> 0-quotad: option 'gltestn.volume-id' is not recognized >> >> [2015-01-07 12:01:56.551885] W [graph.c:329:_log_if_unknown_option] >> 0-quotad: option 'gldata.volume-id' is not recognized >> >> [2015-01-07 12:01:56.551922] I [client.c:2294:notify] >> 0-glwww1-client-0: parent translators are ready, attempting connect >> on transport >> >> [2015-01-07 12:01:56.552579] I [client.c:2294:notify] >> 0-glwww1-client-1: parent translators are ready, attempting connect >> on transport >> >> [2015-01-07 12:01:56.553179] I [client.c:2294:notify] >> 0-gltestn-client-0: parent translators are ready, attempting connect >> on transport >> >> [2015-01-07 12:01:56.553757] I [client.c:2294:notify] >> 0-gldata-client-0: parent translators are ready, attempting connect >> on transport >> >> [2015-01-07 12:01:56.554372] I [client.c:2294:notify] >> 0-gldata-client-1: parent translators are ready, attempting connect >> on transport >> >> Final graph: >> >> +------------------------------------------------------------------------------+ >> >> 1: volume glwww1-client-0 >> >> 2: type protocol/client >> >> 3: option remote-host ikpsrv03 >> >> 4: option remote-subvolume /glwwwn/gl >> >> 5: option transport-type socket >> >> 6: option username 7c5931b4-d41e-40f7-978e-4b5a40f49481 >> >> 7: option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f >> >> 8: end-volume >> >> 9: >> >> 10: volume glwww1-client-1 >> >> 11: type protocol/client >> >> 12: option remote-host ikpsrv01 >> >> 13: option remote-subvolume /glwwwn2/gl >> >> 14: option transport-type socket >> >> 15: option username 7c5931b4-d41e-40f7-978e-4b5a40f49481 >> >> 16: option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f >> >> 17: end-volume >> >> 18: >> >> 19: volume glwww1-replicate-0 >> >> 20: type cluster/replicate >> >> 21: option data-self-heal off >> >> 22: option metadata-self-heal off >> >> 23: option entry-self-heal off >> >> 24: subvolumes glwww1-client-0 glwww1-client-1 >> >> 25: end-volume >> >> 26: >> >> 27: volume glwww1 >> >> 28: type cluster/distribute >> >> 29: subvolumes glwww1-replicate-0 >> >> 30: end-volume >> >> 31: >> >> 32: volume gltestn-client-0 >> >> 33: type protocol/client >> >> 34: option remote-host ikpsrv01 >> >> 35: option remote-subvolume /gltest2/gl >> >> 36: option transport-type socket >> >> 37: option username 76c8bf85-a5c5-445d-8e04-e596b6eb173b >> >> 38: option password ceec81fd-5afd-4d6b-a413-00b2688f4c62 >> >> 39: end-volume >> >> 40: >> >> 41: volume gltestn >> >> 42: type cluster/distribute >> >> 43: subvolumes gltestn-client-0 >> >> 44: end-volume >> >> 45: >> >> 46: volume gldata-client-0 >> >> 47: type protocol/client >> >> 48: option remote-host ikpsrv02 >> >> 49: option remote-subvolume /gldata2-1/gl >> >> 50: option transport-type socket >> >> 51: option username e9dcdbec-1110-4ab2-b667-63625126b246 >> >> 52: option password 05a9e37d-67a0-476b-88df-93d612a4e08e >> >> 53: end-volume >> >> 54: >> >> 55: volume gldata-client-1 >> >> 56: type protocol/client >> >> 57: option remote-host ikpsrv01 >> >> 58: option remote-subvolume /gldata1-2/gl >> >> 59: option transport-type socket >> >> 60: option username e9dcdbec-1110-4ab2-b667-63625126b246 >> >> 61: option password 05a9e37d-67a0-476b-88df-93d612a4e08e >> >> 62: end-volume >> >> 63: >> >> 64: volume gldata >> >> 65: type cluster/distribute >> >> 66: subvolumes gldata-client-0 gldata-client-1 >> >> 67: end-volume >> >> 68: >> >> 69: volume quotad >> >> 70: type features/quotad >> >> 71: option rpc-auth.auth-glusterfs on >> >> 72: option rpc-auth.auth-unix on >> >> 73: option rpc-auth.auth-null on >> >> 74: option transport.socket.listen-path /tmp/quotad.socket >> >> 75: option transport-type socket >> >> 76: option transport.address-family unix >> >> 77: option glwww1.volume-id glwww1 >> >> 78: option gltestn.volume-id gltestn >> >> 79: option gldata.volume-id gldata >> >> 80: subvolumes glwww1 gltestn gldata >> >> 81: end-volume >> >> 82: >> >> +------------------------------------------------------------------------------+ >> >> [2015-01-07 12:01:56.555618] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >> 0-gldata-client-0: changing port to 49160 (from 0) >> >> [2015-01-07 12:01:56.556529] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >> 0-glwww1-client-1: changing port to 49165 (from 0) >> >> [2015-01-07 12:01:56.556597] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >> 0-gltestn-client-0: changing port to 49166 (from 0) >> >> [2015-01-07 12:01:56.556643] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >> 0-gldata-client-1: changing port to 49178 (from 0) >> >> [2015-01-07 12:01:56.558382] W >> [client-callback.c:31:client_cbk_fetchspec] 0-glwww1-client-0: this >> function should not be called >> >> [2015-01-07 12:01:56.558484] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] >> 0-mgmt: Volume file changed >> >> [2015-01-07 12:01:56.558601] I >> [client-handshake.c:1677:select_server_supported_programs] >> 0-gldata-client-0: Using Program GlusterFS 3.3, Num (1298437), >> Version (330) >> >> [2015-01-07 12:01:56.559594] I >> [client-handshake.c:1677:select_server_supported_programs] >> 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), >> Version (330) >> >> [2015-01-07 12:01:56.559686] I >> [client-handshake.c:1677:select_server_supported_programs] >> 0-glwww1-client-1: Using Program GlusterFS 3.3, Num (1298437), >> Version (330) >> >> [2015-01-07 12:01:56.559768] I >> [client-handshake.c:1677:select_server_supported_programs] >> 0-gldata-client-1: Using Program GlusterFS 3.3, Num (1298437), >> Version (330) >> >> [2015-01-07 12:01:56.559852] I >> [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-0: >> Connected to 141.52.67.86:49160, attached to remote volume >> '/gldata2-1/gl'. >> >> [2015-01-07 12:01:56.559873] I >> [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-0: >> Server and Client lk-version numbers are not same, reopening the fds >> >> [2015-01-07 12:01:56.559996] I >> [client-handshake.c:1462:client_setvolume_cbk] 0-gltestn-client-0: >> Connected to 141.52.67.85:49166, attached to remote volume '/gltest2/gl'. >> >> [2015-01-07 12:01:56.560017] I >> [client-handshake.c:1474:client_setvolume_cbk] 0-gltestn-client-0: >> Server and Client lk-version numbers are not same, reopening the fds >> >> [2015-01-07 12:01:56.560089] I >> [client-handshake.c:450:client_set_lk_version_cbk] 0-gldata-client-0: >> Server lk version = 1 >> >> [2015-01-07 12:01:56.560139] I >> [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-1: >> Connected to 141.52.67.85:49165, attached to remote volume '/glwwwn2/gl'. >> >> [2015-01-07 12:01:56.560158] I >> [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-1: >> Server and Client lk-version numbers are not same, reopening the fds >> >> [2015-01-07 12:01:56.560204] I [afr-common.c:4131:afr_notify] >> 0-glwww1-replicate-0: Subvolume 'glwww1-client-1' came back up; going >> online. >> >> [2015-01-07 12:01:56.560262] I >> [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-1: >> Connected to 141.52.67.85:49178, attached to remote volume >> '/gldata1-2/gl'. >> >> [2015-01-07 12:01:56.560301] I >> [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-1: >> Server and Client lk-version numbers are not same, reopening the fds >> >> [2015-01-07 12:01:56.560385] I >> [client-handshake.c:450:client_set_lk_version_cbk] >> 0-gltestn-client-0: Server lk version = 1 >> >> [2015-01-07 12:01:56.560467] I >> [client-handshake.c:450:client_set_lk_version_cbk] 0-glwww1-client-1: >> Server lk version = 1 >> >> [2015-01-07 12:01:56.560546] I >> [client-handshake.c:450:client_set_lk_version_cbk] 0-gldata-client-1: >> Server lk version = 1 >> >> [2015-01-07 12:01:57.255911] I >> [glusterfsd-mgmt.c:1307:mgmt_getspec_cbk] 0-glusterfs: No change in >> volfile, continuing >> >> [2015-01-07 12:01:57.557002] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >> 0-glwww1-client-0: changing port to 49153 (from 0) >> >> [2015-01-07 12:01:57.557924] I >> [client-handshake.c:1677:select_server_supported_programs] >> 0-glwww1-client-0: Using Program GlusterFS 3.3, Num (1298437), >> Version (330) >> >> [2015-01-07 12:01:57.558360] I >> [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-0: >> Connected to 141.52.67.96:49153, attached to remote volume '/glwwwn/gl'. >> >> [2015-01-07 12:01:57.558405] I >> [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-0: >> Server and Client lk-version numbers are not same, reopening the fds >> >> [2015-01-07 12:01:57.558706] I >> [client-handshake.c:450:client_set_lk_version_cbk] 0-glwww1-client-0: >> Server lk version = 1 >> >> what means >> >> 2015-01-07 12:01:56.559594] I >> [client-handshake.c:1677:select_server_supported_programs] >> 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), >> Version (330) >> >> In the log output. >> >> Our glusterfs clients all have 3.5.2 glusterfs version. >> >> Can someone help? >> >> kind regards >> >> Klaus Bekk >> >> ---------------------------------------------------------- >> >> Karlsruher Institut of Technologie (KIT) >> >> Institut f?r Kernphysik (IKP) >> >> Dr. Klaus Bekk >> >> Hermann-von-Helmholtz-Platz 1 >> >> Campus Nord Geb?ude 425 >> >> 76344 Eggenstein-Leopoldshafen >> >> Telefon: +49 721 608-23382 >> >> Fax: +49 721 608-23321 >> >> E-Mail: Klaus.Bekk at kit.edu <mailto:Klaus.Bekk at kit.edu> >> >> http://www.kit.edu/ >> >> KIT -- University of the State of Baden-Wuerttemberg and >> >> National Research Center of the Helmholtz Association >> >> ---------------------------------------------------------- >> > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150122/b247d983/attachment.html>
Dear Sir, sorry, but I do not understand what you mean? What information you need? kind regards Klaus Bekk ---------------------------------------------------------- Karlsruher Institut of Technologie (KIT) Institut f?r Kernphysik (IKP) Dr. Klaus Bekk Hermann-von-Helmholtz-Platz 1 Campus Nord Geb?ude 425 76344 Eggenstein-Leopoldshafen Telefon: +49 721 608-23382 Fax: +49 721 608-23321 E-Mail: Klaus.Bekk at kit.edu http://www.kit.edu/ KIT - University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association ---------------------------------------------------------- Von: Pranith Kumar Karampuri [mailto:pkarampu at redhat.com] Gesendet: Mittwoch, 21. Januar 2015 20:17 An: Bekk, Klaus (IKP); gluster-users at gluster.org; Vijaikumar Mallikarjuna; Sachin Pandit Cc: Wochele, Doris (IKP) Betreff: Re: [Gluster-users] Fwd: to mailing list gluster.org + quota devs Pranith On 01/07/2015 07:37 PM, Bekk, Klaus (IKP) wrote: Hi all, we are running 4 server with Ubuntu 14.04 as glusterfs server with 3.5.2-ubuntu1~trusty1. When starting quotas on any of the volumes the brick will be not available after some time. So we have to disable quota on this volume and stop and start this volume. We have the following configuration root at ikpsrv01:/var/log/glusterfs# gluster volume info Volume Name: gltestn Type: Distribute Volume ID: e50bd30d-dbc8-4279-ac9d-2e557328a643 Status: Started Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: ikpsrv01:/gltest2/gl Volume Name: glwww1 Type: Replicate Volume ID: 811e4c2a-4520-4e61-bb84-955afbdb9dff Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: ikpsrv03:/glwwwn/gl Brick2: ikpsrv01:/glwwwn2/gl Options Reconfigured: features.quota: on Volume Name: corsika Type: Distribute Volume ID: d6e07f3e-4b35-4736-9b71-afc3224f0f29 Status: Started Number of Bricks: 4 Transport-type: tcp Bricks: Brick1: ikpsrv03:/glcors3-1/gl Brick2: ikpsrv03:/glcors3-2/gl Brick3: ikpsrv02:/glcors2-1/gl Brick4: ikpsrv02:/glcors2-2/gl Volume Name: gldata Type: Distribute Volume ID: 82485d03-9ef4-4151-a2cc-881072e871a1 Status: Started Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: ikpsrv02:/gldata2-1/gl Brick2: ikpsrv01:/gldata1-2/gl Options Reconfigured: features.quota: on Volume Name: gltest Type: Distribute Volume ID: 9e69afcc-d421-41c9-824b-7d9a31240072 Status: Started Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: ikpsrv01:/gltest3/gl Brick2: ikpsrv01:/gltest4/gl Options Reconfigured: features.quota: off Volume Name: glusersold Type: Distribute Volume ID: 91a79e1a-4ff3-4191-b0ae-73357105774c Status: Started Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: ikpsrv02:/glusers2-2/gl Volume Name: glusers Type: Distribute Volume ID: f48b1941-9c19-4311-bd9e-497660fbcc80 Status: Started Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: ikpsrv02:/glusers2-1/gl Options Reconfigured: diagnostics.client-log-level: CRITICAL diagnostics.brick-log-level: CRITICAL features.quota: off When starting quotas on any of the volumes the brick will be not available after some time. So we have to disable quota on this volume and stop and start this volume. The command Gluster volume quota gltestn enable gives the following output in quotad.log [2015-01-07 12:01:55.522229] W [glusterfsd.c:1095:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb455bd9fbd] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb455ead182] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xd5) [0x7fb4569a7265]))) 0-: received signum (15), shutting down [2015-01-07 12:01:56.534931] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.5.2 (/usr/sbin/glusterfs -s localhost --volfile-id gluster/quotad -p /var/lib/glusterd/quotad/run/quotad.pid -l /var/log/glusterfs/quotad.log -S /var/run/fa869d5552bf340ec7506747e21b2841.socket --xlator-option *replicate*.data-self-heal=off --xlator-option *replicate*.metadata-self-heal=off --xlator-option *replicate*.entry-self-heal=off) [2015-01-07 12:01:56.537061] I [socket.c:3561:socket_init] 0-socket.glusterfsd: SSL support is NOT enabled [2015-01-07 12:01:56.537134] I [socket.c:3576:socket_init] 0-socket.glusterfsd: using system polling thread [2015-01-07 12:01:56.537355] I [socket.c:3561:socket_init] 0-glusterfs: SSL support is NOT enabled [2015-01-07 12:01:56.537383] I [socket.c:3576:socket_init] 0-glusterfs: using system polling thread [2015-01-07 12:01:56.541480] I [graph.c:254:gf_add_cmdline_options] 0-glwww1-replicate-0: adding option 'entry-self-heal' for volume 'glwww1-replicate-0' with value 'off' [2015-01-07 12:01:56.541516] I [graph.c:254:gf_add_cmdline_options] 0-glwww1-replicate-0: adding option 'metadata-self-heal' for volume 'glwww1-replicate-0' with value 'off' [2015-01-07 12:01:56.541536] I [graph.c:254:gf_add_cmdline_options] 0-glwww1-replicate-0: adding option 'data-self-heal' for volume 'glwww1-replicate-0' with value 'off' [2015-01-07 12:01:56.543122] I [socket.c:3561:socket_init] 0-socket.quotad: SSL support is NOT enabled [2015-01-07 12:01:56.543157] I [socket.c:3576:socket_init] 0-socket.quotad: using system polling thread [2015-01-07 12:01:56.543404] I [dht-shared.c:311:dht_init_regex] 0-glwww1: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ [2015-01-07 12:01:56.547124] I [socket.c:3561:socket_init] 0-glwww1-client-1: SSL support is NOT enabled [2015-01-07 12:01:56.547158] I [socket.c:3576:socket_init] 0-glwww1-client-1: using system polling thread [2015-01-07 12:01:56.547728] I [socket.c:3561:socket_init] 0-glwww1-client-0: SSL support is NOT enabled [2015-01-07 12:01:56.547754] I [socket.c:3576:socket_init] 0-glwww1-client-0: using system polling thread [2015-01-07 12:01:56.547820] I [dht-shared.c:311:dht_init_regex] 0-gltestn: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ [2015-01-07 12:01:56.549471] I [socket.c:3561:socket_init] 0-gltestn-client-0: SSL support is NOT enabled [2015-01-07 12:01:56.549501] I [socket.c:3576:socket_init] 0-gltestn-client-0: using system polling thread [2015-01-07 12:01:56.549565] I [dht-shared.c:311:dht_init_regex] 0-gldata: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ [2015-01-07 12:01:56.551155] I [socket.c:3561:socket_init] 0-gldata-client-1: SSL support is NOT enabled [2015-01-07 12:01:56.551184] I [socket.c:3576:socket_init] 0-gldata-client-1: using system polling thread [2015-01-07 12:01:56.551744] I [socket.c:3561:socket_init] 0-gldata-client-0: SSL support is NOT enabled [2015-01-07 12:01:56.551770] I [socket.c:3576:socket_init] 0-gldata-client-0: using system polling thread [2015-01-07 12:01:56.551790] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'rpc-auth.auth-glusterfs' is not recognized [2015-01-07 12:01:56.551819] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'rpc-auth.auth-unix' is not recognized [2015-01-07 12:01:56.551838] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'rpc-auth.auth-null' is not recognized [2015-01-07 12:01:56.551854] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'glwww1.volume-id' is not recognized [2015-01-07 12:01:56.551870] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'gltestn.volume-id' is not recognized [2015-01-07 12:01:56.551885] W [graph.c:329:_log_if_unknown_option] 0-quotad: option 'gldata.volume-id' is not recognized [2015-01-07 12:01:56.551922] I [client.c:2294:notify] 0-glwww1-client-0: parent translators are ready, attempting connect on transport [2015-01-07 12:01:56.552579] I [client.c:2294:notify] 0-glwww1-client-1: parent translators are ready, attempting connect on transport [2015-01-07 12:01:56.553179] I [client.c:2294:notify] 0-gltestn-client-0: parent translators are ready, attempting connect on transport [2015-01-07 12:01:56.553757] I [client.c:2294:notify] 0-gldata-client-0: parent translators are ready, attempting connect on transport [2015-01-07 12:01:56.554372] I [client.c:2294:notify] 0-gldata-client-1: parent translators are ready, attempting connect on transport Final graph: +------------------------------------------------------------------------------+ 1: volume glwww1-client-0 2: type protocol/client 3: option remote-host ikpsrv03 4: option remote-subvolume /glwwwn/gl 5: option transport-type socket 6: option username 7c5931b4-d41e-40f7-978e-4b5a40f49481 7: option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f 8: end-volume 9: 10: volume glwww1-client-1 11: type protocol/client 12: option remote-host ikpsrv01 13: option remote-subvolume /glwwwn2/gl 14: option transport-type socket 15: option username 7c5931b4-d41e-40f7-978e-4b5a40f49481 16: option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f 17: end-volume 18: 19: volume glwww1-replicate-0 20: type cluster/replicate 21: option data-self-heal off 22: option metadata-self-heal off 23: option entry-self-heal off 24: subvolumes glwww1-client-0 glwww1-client-1 25: end-volume 26: 27: volume glwww1 28: type cluster/distribute 29: subvolumes glwww1-replicate-0 30: end-volume 31: 32: volume gltestn-client-0 33: type protocol/client 34: option remote-host ikpsrv01 35: option remote-subvolume /gltest2/gl 36: option transport-type socket 37: option username 76c8bf85-a5c5-445d-8e04-e596b6eb173b 38: option password ceec81fd-5afd-4d6b-a413-00b2688f4c62 39: end-volume 40: 41: volume gltestn 42: type cluster/distribute 43: subvolumes gltestn-client-0 44: end-volume 45: 46: volume gldata-client-0 47: type protocol/client 48: option remote-host ikpsrv02 49: option remote-subvolume /gldata2-1/gl 50: option transport-type socket 51: option username e9dcdbec-1110-4ab2-b667-63625126b246 52: option password 05a9e37d-67a0-476b-88df-93d612a4e08e 53: end-volume 54: 55: volume gldata-client-1 56: type protocol/client 57: option remote-host ikpsrv01 58: option remote-subvolume /gldata1-2/gl 59: option transport-type socket 60: option username e9dcdbec-1110-4ab2-b667-63625126b246 61: option password 05a9e37d-67a0-476b-88df-93d612a4e08e 62: end-volume 63: 64: volume gldata 65: type cluster/distribute 66: subvolumes gldata-client-0 gldata-client-1 67: end-volume 68: 69: volume quotad 70: type features/quotad 71: option rpc-auth.auth-glusterfs on 72: option rpc-auth.auth-unix on 73: option rpc-auth.auth-null on 74: option transport.socket.listen-path /tmp/quotad.socket 75: option transport-type socket 76: option transport.address-family unix 77: option glwww1.volume-id glwww1 78: option gltestn.volume-id gltestn 79: option gldata.volume-id gldata 80: subvolumes glwww1 gltestn gldata 81: end-volume 82: +------------------------------------------------------------------------------+ [2015-01-07 12:01:56.555618] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-gldata-client-0: changing port to 49160 (from 0) [2015-01-07 12:01:56.556529] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-glwww1-client-1: changing port to 49165 (from 0) [2015-01-07 12:01:56.556597] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-gltestn-client-0: changing port to 49166 (from 0) [2015-01-07 12:01:56.556643] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-gldata-client-1: changing port to 49178 (from 0) [2015-01-07 12:01:56.558382] W [client-callback.c:31:client_cbk_fetchspec] 0-glwww1-client-0: this function should not be called [2015-01-07 12:01:56.558484] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed [2015-01-07 12:01:56.558601] I [client-handshake.c:1677:select_server_supported_programs] 0-gldata-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-01-07 12:01:56.559594] I [client-handshake.c:1677:select_server_supported_programs] 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-01-07 12:01:56.559686] I [client-handshake.c:1677:select_server_supported_programs] 0-glwww1-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-01-07 12:01:56.559768] I [client-handshake.c:1677:select_server_supported_programs] 0-gldata-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-01-07 12:01:56.559852] I [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-0: Connected to 141.52.67.86:49160, attached to remote volume '/gldata2-1/gl'. [2015-01-07 12:01:56.559873] I [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-0: Server and Client lk-version numbers are not same, reopening the fds [2015-01-07 12:01:56.559996] I [client-handshake.c:1462:client_setvolume_cbk] 0-gltestn-client-0: Connected to 141.52.67.85:49166, attached to remote volume '/gltest2/gl'. [2015-01-07 12:01:56.560017] I [client-handshake.c:1474:client_setvolume_cbk] 0-gltestn-client-0: Server and Client lk-version numbers are not same, reopening the fds [2015-01-07 12:01:56.560089] I [client-handshake.c:450:client_set_lk_version_cbk] 0-gldata-client-0: Server lk version = 1 [2015-01-07 12:01:56.560139] I [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-1: Connected to 141.52.67.85:49165, attached to remote volume '/glwwwn2/gl'. [2015-01-07 12:01:56.560158] I [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-1: Server and Client lk-version numbers are not same, reopening the fds [2015-01-07 12:01:56.560204] I [afr-common.c:4131:afr_notify] 0-glwww1-replicate-0: Subvolume 'glwww1-client-1' came back up; going online. [2015-01-07 12:01:56.560262] I [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-1: Connected to 141.52.67.85:49178, attached to remote volume '/gldata1-2/gl'. [2015-01-07 12:01:56.560301] I [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-1: Server and Client lk-version numbers are not same, reopening the fds [2015-01-07 12:01:56.560385] I [client-handshake.c:450:client_set_lk_version_cbk] 0-gltestn-client-0: Server lk version = 1 [2015-01-07 12:01:56.560467] I [client-handshake.c:450:client_set_lk_version_cbk] 0-glwww1-client-1: Server lk version = 1 [2015-01-07 12:01:56.560546] I [client-handshake.c:450:client_set_lk_version_cbk] 0-gldata-client-1: Server lk version = 1 [2015-01-07 12:01:57.255911] I [glusterfsd-mgmt.c:1307:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2015-01-07 12:01:57.557002] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-glwww1-client-0: changing port to 49153 (from 0) [2015-01-07 12:01:57.557924] I [client-handshake.c:1677:select_server_supported_programs] 0-glwww1-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-01-07 12:01:57.558360] I [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-0: Connected to 141.52.67.96:49153, attached to remote volume '/glwwwn/gl'. [2015-01-07 12:01:57.558405] I [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-0: Server and Client lk-version numbers are not same, reopening the fds [2015-01-07 12:01:57.558706] I [client-handshake.c:450:client_set_lk_version_cbk] 0-glwww1-client-0: Server lk version = 1 what means 2015-01-07 12:01:56.559594] I [client-handshake.c:1677:select_server_supported_programs] 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) In the log output. Our glusterfs clients all have 3.5.2 glusterfs version. Can someone help? kind regards Klaus Bekk ---------------------------------------------------------- Karlsruher Institut of Technologie (KIT) Institut f?r Kernphysik (IKP) Dr. Klaus Bekk Hermann-von-Helmholtz-Platz 1 Campus Nord Geb?ude 425 76344 Eggenstein-Leopoldshafen Telefon: +49 721 608-23382 Fax: +49 721 608-23321 E-Mail: Klaus.Bekk at kit.edu<mailto:Klaus.Bekk at kit.edu> http://www.kit.edu/ KIT - University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association ---------------------------------------------------------- _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org<mailto:Gluster-users at gluster.org> http://www.gluster.org/mailman/listinfo/gluster-users -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150126/faede04c/attachment.html>
Pranith Kumar Karampuri
2015-Jan-28 06:43 UTC
[Gluster-users] Fwd: to mailing list gluster.org
Vijaikumar, Sachin will work on this quota issue. Pranith On 01/22/2015 12:46 AM, Pranith Kumar Karampuri wrote:> + quota devs > > Pranith > On 01/07/2015 07:37 PM, Bekk, Klaus (IKP) wrote: >> >>> Hi all, >>> >>> we are running 4 server with Ubuntu 14.04 as glusterfs server with >>> 3.5.2-ubuntu1~trusty1. >>> >> >>> When starting quotas on any of the volumes the brick will be not >>> available after some time. So we have to disable quota on this >>> volume and stop and start this volume. >>> >>> We have the following configuration >>> >>> root at ikpsrv01:/var/log/glusterfs# gluster volume info >>> >>> Volume Name: gltestn >>> >>> Type: Distribute >>> >>> Volume ID: e50bd30d-dbc8-4279-ac9d-2e557328a643 >>> >>> Status: Started >>> >>> Number of Bricks: 1 >>> >>> Transport-type: tcp >>> >>> Bricks: >>> >>> Brick1: ikpsrv01:/gltest2/gl >>> >>> Volume Name: glwww1 >>> >>> Type: Replicate >>> >>> Volume ID: 811e4c2a-4520-4e61-bb84-955afbdb9dff >>> >>> Status: Started >>> >>> Number of Bricks: 1 x 2 = 2 >>> >>> Transport-type: tcp >>> >>> Bricks: >>> >>> Brick1: ikpsrv03:/glwwwn/gl >>> >>> Brick2: ikpsrv01:/glwwwn2/gl >>> >>> Options Reconfigured: >>> >>> features.quota: on >>> >>> Volume Name: corsika >>> >>> Type: Distribute >>> >>> Volume ID: d6e07f3e-4b35-4736-9b71-afc3224f0f29 >>> >>> Status: Started >>> >>> Number of Bricks: 4 >>> >>> Transport-type: tcp >>> >>> Bricks: >>> >>> Brick1: ikpsrv03:/glcors3-1/gl >>> >>> Brick2: ikpsrv03:/glcors3-2/gl >>> >>> Brick3: ikpsrv02:/glcors2-1/gl >>> >>> Brick4: ikpsrv02:/glcors2-2/gl >>> >>> Volume Name: gldata >>> >>> Type: Distribute >>> >>> Volume ID: 82485d03-9ef4-4151-a2cc-881072e871a1 >>> >>> Status: Started >>> >>> Number of Bricks: 2 >>> >>> Transport-type: tcp >>> >>> Bricks: >>> >>> Brick1: ikpsrv02:/gldata2-1/gl >>> >>> Brick2: ikpsrv01:/gldata1-2/gl >>> >>> Options Reconfigured: >>> >>> features.quota: on >>> >>> Volume Name: gltest >>> >>> Type: Distribute >>> >>> Volume ID: 9e69afcc-d421-41c9-824b-7d9a31240072 >>> >>> Status: Started >>> >>> Number of Bricks: 2 >>> >>> Transport-type: tcp >>> >>> Bricks: >>> >>> Brick1: ikpsrv01:/gltest3/gl >>> >>> Brick2: ikpsrv01:/gltest4/gl >>> >>> Options Reconfigured: >>> >>> features.quota: off >>> >>> Volume Name: glusersold >>> >>> Type: Distribute >>> >>> Volume ID: 91a79e1a-4ff3-4191-b0ae-73357105774c >>> >>> Status: Started >>> >>> Number of Bricks: 1 >>> >>> Transport-type: tcp >>> >>> Bricks: >>> >>> Brick1: ikpsrv02:/glusers2-2/gl >>> >>> Volume Name: glusers >>> >>> Type: Distribute >>> >>> Volume ID: f48b1941-9c19-4311-bd9e-497660fbcc80 >>> >>> Status: Started >>> >>> Number of Bricks: 1 >>> >>> Transport-type: tcp >>> >>> Bricks: >>> >>> Brick1: ikpsrv02:/glusers2-1/gl >>> >>> Options Reconfigured: >>> >>> diagnostics.client-log-level: CRITICAL >>> >>> diagnostics.brick-log-level: CRITICAL >>> >>> features.quota: off >>> >>> When starting quotas on any of the volumes the brick will be not >>> available after some time. So we have to disable quota on this >>> volume and stop and start this volume. >>> >>> The command >>> >>> Gluster volume quota gltestn enable >>> >>> gives the following output in quotad.log >>> >>> [2015-01-07 12:01:55.522229] W [glusterfsd.c:1095:cleanup_and_exit] >>> (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fb455bd9fbd] >>> (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7fb455ead182] >>> (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xd5) >>> [0x7fb4569a7265]))) 0-: received signum (15), shutting down >>> >>> [2015-01-07 12:01:56.534931] I [glusterfsd.c:1959:main] >>> 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version >>> 3.5.2 (/usr/sbin/glusterfs -s localhost --volfile-id gluster/quotad >>> -p /var/lib/glusterd/quotad/run/quotad.pid -l >>> /var/log/glusterfs/quotad.log -S >>> /var/run/fa869d5552bf340ec7506747e21b2841.socket --xlator-option >>> *replicate*.data-self-heal=off --xlator-option >>> *replicate*.metadata-self-heal=off --xlator-option >>> *replicate*.entry-self-heal=off) >>> >>> [2015-01-07 12:01:56.537061] I [socket.c:3561:socket_init] >>> 0-socket.glusterfsd: SSL support is NOT enabled >>> >>> [2015-01-07 12:01:56.537134] I [socket.c:3576:socket_init] >>> 0-socket.glusterfsd: using system polling thread >>> >>> [2015-01-07 12:01:56.537355] I [socket.c:3561:socket_init] >>> 0-glusterfs: SSL support is NOT enabled >>> >>> [2015-01-07 12:01:56.537383] I [socket.c:3576:socket_init] >>> 0-glusterfs: using system polling thread >>> >>> [2015-01-07 12:01:56.541480] I [graph.c:254:gf_add_cmdline_options] >>> 0-glwww1-replicate-0: adding option 'entry-self-heal' for volume >>> 'glwww1-replicate-0' with value 'off' >>> >>> [2015-01-07 12:01:56.541516] I [graph.c:254:gf_add_cmdline_options] >>> 0-glwww1-replicate-0: adding option 'metadata-self-heal' for volume >>> 'glwww1-replicate-0' with value 'off' >>> >>> [2015-01-07 12:01:56.541536] I [graph.c:254:gf_add_cmdline_options] >>> 0-glwww1-replicate-0: adding option 'data-self-heal' for volume >>> 'glwww1-replicate-0' with value 'off' >>> >>> [2015-01-07 12:01:56.543122] I [socket.c:3561:socket_init] >>> 0-socket.quotad: SSL support is NOT enabled >>> >>> [2015-01-07 12:01:56.543157] I [socket.c:3576:socket_init] >>> 0-socket.quotad: using system polling thread >>> >>> [2015-01-07 12:01:56.543404] I [dht-shared.c:311:dht_init_regex] >>> 0-glwww1: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ >>> >>> [2015-01-07 12:01:56.547124] I [socket.c:3561:socket_init] >>> 0-glwww1-client-1: SSL support is NOT enabled >>> >>> [2015-01-07 12:01:56.547158] I [socket.c:3576:socket_init] >>> 0-glwww1-client-1: using system polling thread >>> >>> [2015-01-07 12:01:56.547728] I [socket.c:3561:socket_init] >>> 0-glwww1-client-0: SSL support is NOT enabled >>> >>> [2015-01-07 12:01:56.547754] I [socket.c:3576:socket_init] >>> 0-glwww1-client-0: using system polling thread >>> >>> [2015-01-07 12:01:56.547820] I [dht-shared.c:311:dht_init_regex] >>> 0-gltestn: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ >>> >>> [2015-01-07 12:01:56.549471] I [socket.c:3561:socket_init] >>> 0-gltestn-client-0: SSL support is NOT enabled >>> >>> [2015-01-07 12:01:56.549501] I [socket.c:3576:socket_init] >>> 0-gltestn-client-0: using system polling thread >>> >>> [2015-01-07 12:01:56.549565] I [dht-shared.c:311:dht_init_regex] >>> 0-gldata: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ >>> >>> [2015-01-07 12:01:56.551155] I [socket.c:3561:socket_init] >>> 0-gldata-client-1: SSL support is NOT enabled >>> >>> [2015-01-07 12:01:56.551184] I [socket.c:3576:socket_init] >>> 0-gldata-client-1: using system polling thread >>> >>> [2015-01-07 12:01:56.551744] I [socket.c:3561:socket_init] >>> 0-gldata-client-0: SSL support is NOT enabled >>> >>> [2015-01-07 12:01:56.551770] I [socket.c:3576:socket_init] >>> 0-gldata-client-0: using system polling thread >>> >>> [2015-01-07 12:01:56.551790] W [graph.c:329:_log_if_unknown_option] >>> 0-quotad: option 'rpc-auth.auth-glusterfs' is not recognized >>> >>> [2015-01-07 12:01:56.551819] W [graph.c:329:_log_if_unknown_option] >>> 0-quotad: option 'rpc-auth.auth-unix' is not recognized >>> >>> [2015-01-07 12:01:56.551838] W [graph.c:329:_log_if_unknown_option] >>> 0-quotad: option 'rpc-auth.auth-null' is not recognized >>> >>> [2015-01-07 12:01:56.551854] W [graph.c:329:_log_if_unknown_option] >>> 0-quotad: option 'glwww1.volume-id' is not recognized >>> >>> [2015-01-07 12:01:56.551870] W [graph.c:329:_log_if_unknown_option] >>> 0-quotad: option 'gltestn.volume-id' is not recognized >>> >>> [2015-01-07 12:01:56.551885] W [graph.c:329:_log_if_unknown_option] >>> 0-quotad: option 'gldata.volume-id' is not recognized >>> >>> [2015-01-07 12:01:56.551922] I [client.c:2294:notify] >>> 0-glwww1-client-0: parent translators are ready, attempting connect >>> on transport >>> >>> [2015-01-07 12:01:56.552579] I [client.c:2294:notify] >>> 0-glwww1-client-1: parent translators are ready, attempting connect >>> on transport >>> >>> [2015-01-07 12:01:56.553179] I [client.c:2294:notify] >>> 0-gltestn-client-0: parent translators are ready, attempting connect >>> on transport >>> >>> [2015-01-07 12:01:56.553757] I [client.c:2294:notify] >>> 0-gldata-client-0: parent translators are ready, attempting connect >>> on transport >>> >>> [2015-01-07 12:01:56.554372] I [client.c:2294:notify] >>> 0-gldata-client-1: parent translators are ready, attempting connect >>> on transport >>> >>> Final graph: >>> >>> +------------------------------------------------------------------------------+ >>> >>> 1: volume glwww1-client-0 >>> >>> 2: type protocol/client >>> >>> 3: option remote-host ikpsrv03 >>> >>> 4: option remote-subvolume /glwwwn/gl >>> >>> 5: option transport-type socket >>> >>> 6: option username 7c5931b4-d41e-40f7-978e-4b5a40f49481 >>> >>> 7: option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f >>> >>> 8: end-volume >>> >>> 9: >>> >>> 10: volume glwww1-client-1 >>> >>> 11: type protocol/client >>> >>> 12: option remote-host ikpsrv01 >>> >>> 13: option remote-subvolume /glwwwn2/gl >>> >>> 14: option transport-type socket >>> >>> 15: option username 7c5931b4-d41e-40f7-978e-4b5a40f49481 >>> >>> 16: option password 6ab1661b-4752-42a4-80cc-ca0bd0167a9f >>> >>> 17: end-volume >>> >>> 18: >>> >>> 19: volume glwww1-replicate-0 >>> >>> 20: type cluster/replicate >>> >>> 21: option data-self-heal off >>> >>> 22: option metadata-self-heal off >>> >>> 23: option entry-self-heal off >>> >>> 24: subvolumes glwww1-client-0 glwww1-client-1 >>> >>> 25: end-volume >>> >>> 26: >>> >>> 27: volume glwww1 >>> >>> 28: type cluster/distribute >>> >>> 29: subvolumes glwww1-replicate-0 >>> >>> 30: end-volume >>> >>> 31: >>> >>> 32: volume gltestn-client-0 >>> >>> 33: type protocol/client >>> >>> 34: option remote-host ikpsrv01 >>> >>> 35: option remote-subvolume /gltest2/gl >>> >>> 36: option transport-type socket >>> >>> 37: option username 76c8bf85-a5c5-445d-8e04-e596b6eb173b >>> >>> 38: option password ceec81fd-5afd-4d6b-a413-00b2688f4c62 >>> >>> 39: end-volume >>> >>> 40: >>> >>> 41: volume gltestn >>> >>> 42: type cluster/distribute >>> >>> 43: subvolumes gltestn-client-0 >>> >>> 44: end-volume >>> >>> 45: >>> >>> 46: volume gldata-client-0 >>> >>> 47: type protocol/client >>> >>> 48: option remote-host ikpsrv02 >>> >>> 49: option remote-subvolume /gldata2-1/gl >>> >>> 50: option transport-type socket >>> >>> 51: option username e9dcdbec-1110-4ab2-b667-63625126b246 >>> >>> 52: option password 05a9e37d-67a0-476b-88df-93d612a4e08e >>> >>> 53: end-volume >>> >>> 54: >>> >>> 55: volume gldata-client-1 >>> >>> 56: type protocol/client >>> >>> 57: option remote-host ikpsrv01 >>> >>> 58: option remote-subvolume /gldata1-2/gl >>> >>> 59: option transport-type socket >>> >>> 60: option username e9dcdbec-1110-4ab2-b667-63625126b246 >>> >>> 61: option password 05a9e37d-67a0-476b-88df-93d612a4e08e >>> >>> 62: end-volume >>> >>> 63: >>> >>> 64: volume gldata >>> >>> 65: type cluster/distribute >>> >>> 66: subvolumes gldata-client-0 gldata-client-1 >>> >>> 67: end-volume >>> >>> 68: >>> >>> 69: volume quotad >>> >>> 70: type features/quotad >>> >>> 71: option rpc-auth.auth-glusterfs on >>> >>> 72: option rpc-auth.auth-unix on >>> >>> 73: option rpc-auth.auth-null on >>> >>> 74: option transport.socket.listen-path /tmp/quotad.socket >>> >>> 75: option transport-type socket >>> >>> 76: option transport.address-family unix >>> >>> 77: option glwww1.volume-id glwww1 >>> >>> 78: option gltestn.volume-id gltestn >>> >>> 79: option gldata.volume-id gldata >>> >>> 80: subvolumes glwww1 gltestn gldata >>> >>> 81: end-volume >>> >>> 82: >>> >>> +------------------------------------------------------------------------------+ >>> >>> [2015-01-07 12:01:56.555618] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >>> 0-gldata-client-0: changing port to 49160 (from 0) >>> >>> [2015-01-07 12:01:56.556529] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >>> 0-glwww1-client-1: changing port to 49165 (from 0) >>> >>> [2015-01-07 12:01:56.556597] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >>> 0-gltestn-client-0: changing port to 49166 (from 0) >>> >>> [2015-01-07 12:01:56.556643] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >>> 0-gldata-client-1: changing port to 49178 (from 0) >>> >>> [2015-01-07 12:01:56.558382] W >>> [client-callback.c:31:client_cbk_fetchspec] 0-glwww1-client-0: this >>> function should not be called >>> >>> [2015-01-07 12:01:56.558484] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] >>> 0-mgmt: Volume file changed >>> >>> [2015-01-07 12:01:56.558601] I >>> [client-handshake.c:1677:select_server_supported_programs] >>> 0-gldata-client-0: Using Program GlusterFS 3.3, Num (1298437), >>> Version (330) >>> >>> [2015-01-07 12:01:56.559594] I >>> [client-handshake.c:1677:select_server_supported_programs] >>> 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), >>> Version (330) >>> >>> [2015-01-07 12:01:56.559686] I >>> [client-handshake.c:1677:select_server_supported_programs] >>> 0-glwww1-client-1: Using Program GlusterFS 3.3, Num (1298437), >>> Version (330) >>> >>> [2015-01-07 12:01:56.559768] I >>> [client-handshake.c:1677:select_server_supported_programs] >>> 0-gldata-client-1: Using Program GlusterFS 3.3, Num (1298437), >>> Version (330) >>> >>> [2015-01-07 12:01:56.559852] I >>> [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-0: >>> Connected to 141.52.67.86:49160, attached to remote volume >>> '/gldata2-1/gl'. >>> >>> [2015-01-07 12:01:56.559873] I >>> [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-0: >>> Server and Client lk-version numbers are not same, reopening the fds >>> >>> [2015-01-07 12:01:56.559996] I >>> [client-handshake.c:1462:client_setvolume_cbk] 0-gltestn-client-0: >>> Connected to 141.52.67.85:49166, attached to remote volume >>> '/gltest2/gl'. >>> >>> [2015-01-07 12:01:56.560017] I >>> [client-handshake.c:1474:client_setvolume_cbk] 0-gltestn-client-0: >>> Server and Client lk-version numbers are not same, reopening the fds >>> >>> [2015-01-07 12:01:56.560089] I >>> [client-handshake.c:450:client_set_lk_version_cbk] >>> 0-gldata-client-0: Server lk version = 1 >>> >>> [2015-01-07 12:01:56.560139] I >>> [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-1: >>> Connected to 141.52.67.85:49165, attached to remote volume >>> '/glwwwn2/gl'. >>> >>> [2015-01-07 12:01:56.560158] I >>> [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-1: >>> Server and Client lk-version numbers are not same, reopening the fds >>> >>> [2015-01-07 12:01:56.560204] I [afr-common.c:4131:afr_notify] >>> 0-glwww1-replicate-0: Subvolume 'glwww1-client-1' came back up; >>> going online. >>> >>> [2015-01-07 12:01:56.560262] I >>> [client-handshake.c:1462:client_setvolume_cbk] 0-gldata-client-1: >>> Connected to 141.52.67.85:49178, attached to remote volume >>> '/gldata1-2/gl'. >>> >>> [2015-01-07 12:01:56.560301] I >>> [client-handshake.c:1474:client_setvolume_cbk] 0-gldata-client-1: >>> Server and Client lk-version numbers are not same, reopening the fds >>> >>> [2015-01-07 12:01:56.560385] I >>> [client-handshake.c:450:client_set_lk_version_cbk] >>> 0-gltestn-client-0: Server lk version = 1 >>> >>> [2015-01-07 12:01:56.560467] I >>> [client-handshake.c:450:client_set_lk_version_cbk] >>> 0-glwww1-client-1: Server lk version = 1 >>> >>> [2015-01-07 12:01:56.560546] I >>> [client-handshake.c:450:client_set_lk_version_cbk] >>> 0-gldata-client-1: Server lk version = 1 >>> >>> [2015-01-07 12:01:57.255911] I >>> [glusterfsd-mgmt.c:1307:mgmt_getspec_cbk] 0-glusterfs: No change in >>> volfile, continuing >>> >>> [2015-01-07 12:01:57.557002] I [rpc-clnt.c:1729:rpc_clnt_reconfig] >>> 0-glwww1-client-0: changing port to 49153 (from 0) >>> >>> [2015-01-07 12:01:57.557924] I >>> [client-handshake.c:1677:select_server_supported_programs] >>> 0-glwww1-client-0: Using Program GlusterFS 3.3, Num (1298437), >>> Version (330) >>> >>> [2015-01-07 12:01:57.558360] I >>> [client-handshake.c:1462:client_setvolume_cbk] 0-glwww1-client-0: >>> Connected to 141.52.67.96:49153, attached to remote volume '/glwwwn/gl'. >>> >>> [2015-01-07 12:01:57.558405] I >>> [client-handshake.c:1474:client_setvolume_cbk] 0-glwww1-client-0: >>> Server and Client lk-version numbers are not same, reopening the fds >>> >>> [2015-01-07 12:01:57.558706] I >>> [client-handshake.c:450:client_set_lk_version_cbk] >>> 0-glwww1-client-0: Server lk version = 1 >>> >>> what means >>> >>> 2015-01-07 12:01:56.559594] I >>> [client-handshake.c:1677:select_server_supported_programs] >>> 0-gltestn-client-0: Using Program GlusterFS 3.3, Num (1298437), >>> Version (330) >>> >>> In the log output. >>> >>> Our glusterfs clients all have 3.5.2 glusterfs version. >>> >>> Can someone help? >>> >>> kind regards >>> >>> Klaus Bekk >>> >>> ---------------------------------------------------------- >>> >>> Karlsruher Institut of Technologie (KIT) >>> >>> Institut f?r Kernphysik (IKP) >>> >>> Dr. Klaus Bekk >>> >>> Hermann-von-Helmholtz-Platz 1 >>> >>> Campus Nord Geb?ude 425 >>> >>> 76344 Eggenstein-Leopoldshafen >>> >>> Telefon: +49 721 608-23382 >>> >>> Fax: +49 721 608-23321 >>> >>> E-Mail: Klaus.Bekk at kit.edu <mailto:Klaus.Bekk at kit.edu> >>> >>> http://www.kit.edu/ >>> >>> KIT -- University of the State of Baden-Wuerttemberg and >>> >>> National Research Center of the Helmholtz Association >>> >>> ---------------------------------------------------------- >>> >> >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://www.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150128/0978be1a/attachment.html>