Alexandre Besnard
2016-Jul-24 20:54 UTC
[Gluster-users] Geo-replication configuration issue
Anybody, any clue ?> On 20 Jul 2016, at 01:59, Alexandre Besnard <besnard.alexandre at gmail.com> wrote: > > Hello > > I deleted the content of /root/.ssh/authorized_keys on the slave (ks4) > > Then I configured passwordless authentication from the host (ks16): > > ssh-copy-id root at ks4 > > Also did from the host: > > gluster system:: execute gsec_create > > which created a file in /var/lib/glusterd/geo-replication/common_secret.pem.pub > > Then created the geo-replicated volume successfully: > > gluster volume geo-replication backupvol ks4::backupvol create push-pem force > > but still getting the same errors in the log after starting the volume: > > [2016-07-19 23:52:14.900583] I [monitor(monitor):266:monitor] Monitor: ------------------------------------------------------------ > [2016-07-19 23:52:14.900752] I [monitor(monitor):267:monitor] Monitor: starting gsyncd worker > [2016-07-19 23:52:14.958281] I [gsyncd(/gluster/backupvol):710:main_i] <top>: syncing: gluster://localhost:backupvol <gluster://localhost:backupvol> -> ssh://root at ks4:gluster://localhost:backupvol <ssh://root at ks4:gluster://localhost:backupvol> > [2016-07-19 23:52:14.958520] I [changelogagent(agent):73:__init__] ChangelogAgent: Agent listining... > [2016-07-19 23:52:15.81407] E [syncdutils(/gluster/backupvol):252:log_raise_exception] <top>: connection to peer is broken > [2016-07-19 23:52:15.81645] E [resource(/gluster/backupvol):226:errlog] Popen: command "ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-Aux8JK/b63292d563144e7818235d683516731d.sock root at ks4 /nonexistent/gsyncd --session-owner 3281242a-ab45-4a0d-99e5-2965b4ac5840 -N --listen --timeout 120 gluster://localhost:backupvol <gluster://localhost:backupvol>" returned with 255, saying: > [2016-07-19 23:52:15.81733] E [resource(/gluster/backupvol):230:logerr] Popen: ssh> key_load_public: invalid format > [2016-07-19 23:52:15.81804] E [resource(/gluster/backupvol):230:logerr] Popen: ssh> Permission denied (publickey,password). > [2016-07-19 23:52:15.81947] I [syncdutils(/gluster/backupvol):220:finalize] <top>: exiting. > [2016-07-19 23:52:15.82798] I [repce(agent):92:service_loop] RepceServer: terminating on reaching EOF. > [2016-07-19 23:52:15.82946] I [syncdutils(agent):220:finalize] <top>: exiting. > [2016-07-19 23:52:15.82858] I [monitor(monitor):333:monitor] Monitor: worker(/gluster/backupvol) died before establishing connection > > > Any thoughts ? > > >> On 19 Jul 2016, at 06:33, Aravinda <avishwan at redhat.com <mailto:avishwan at redhat.com>> wrote: >> >> Hi, >> >> Looks like Master Pem keys are not copied to Slave nodes properly, Please cleanup /root/.ssh/authorized_keys in Slave nodes and run Geo-rep create force again. >> >> gluster volume geo-replication <MASTERVOL> <SLAVEHOST>::<SLAVEVOL> create push-pem force >> >> Do you observe any errors related to hook scripts in glusterd log file? >> >> regards >> Aravinda >> >> On 07/18/2016 10:11 PM, Alexandre Besnard wrote: >>> Hello >>> >>> On a fresh Gluster 3.8 install, I am not able to configure a geo-replicated volume. Everything works fine up to starting of the volume however Gluster reports a faulty status. >>> >>> When looking at the logs (gluster_error): >>> >>> [2016-07-18 16:30:04.371686] I [cli.c:730:main] 0-cli: Started running gluster with version 3.8.0 >>> [2016-07-18 16:30:04.435854] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 >>> [2016-07-18 16:30:04.435921] I [socket.c:2468:socket_event_handler] 0-transport: disconnecting now >>> [2016-07-18 16:30:04.997986] I [input.c:31:cli_batch] 0-: Exiting with: 0 >>> >>> >>> >>> From the geo-replicated logs, it seems I have a SSH configuration issue: >>> >>> 2016-07-18 16:35:28.293524] I [monitor(monitor):266:monitor] Monitor: ------------------------------------------------------------ >>> [2016-07-18 16:35:28.293740] I [monitor(monitor):267:monitor] Monitor: starting gsyncd worker >>> [2016-07-18 16:35:28.352266] I [gsyncd(/gluster/backupvol):710:main_i] <top>: syncing: gluster://localhost:backupvol <gluster://localhost:backupvol> -> ssh://root at ks4:gluster://localhost:backupvol <ssh://root at ks4:gluster://localhost:backupvol> >>> [2016-07-18 16:35:28.352489] I [changelogagent(agent):73:__init__] ChangelogAgent: Agent listining... >>> [2016-07-18 16:35:28.492474] E [syncdutils(/gluster/backupvol):252:log_raise_exception] <top>: connection to peer is broken >>> [2016-07-18 16:35:28.492706] E [resource(/gluster/backupvol):226:errlog] Popen: command "ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-Fs2XND/b63292d563144e7818235d683516731d.sock root at ks4 /nonexistent/gsyncd --session-owner 3281242a-ab45-4a0d-99e5-2965b4ac5840 -N --listen --timeout 120 gluster://localhost:backupvol <gluster://localhost:backupvol>" returned with 255, saying: >>> [2016-07-18 16:35:28.492794] E [resource(/gluster/backupvol):230:logerr] Popen: ssh> key_load_public: invalid format >>> [2016-07-18 16:35:28.492863] E [resource(/gluster/backupvol):230:logerr] Popen: ssh> Permission denied (publickey,password). >>> [2016-07-18 16:35:28.493004] I [syncdutils(/gluster/backupvol):220:finalize] <top>: exiting. >>> [2016-07-18 16:35:28.494045] I [repce(agent):92:service_loop] RepceServer: terminating on reaching EOF. >>> [2016-07-18 16:35:28.494204] I [syncdutils(agent):220:finalize] <top>: exiting. >>> [2016-07-18 16:35:28.494143] I [monitor(monitor):333:monitor] Monitor: worker(/gluster/backupvol) died before establishing connection >>> >>> >>> I tried to fix them to the best of my knowledge but I am missing something. >>> >>> Can you help me to fix it? >>> >>> Thanks, >>> Alex >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >>> http://www.gluster.org/mailman/listinfo/gluster-users >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160724/28bbd0ab/attachment.html>
Saravanakumar Arumugam
2016-Jul-25 04:59 UTC
[Gluster-users] Geo-replication configuration issue
Hi, 1. Can you check /root/.ssh/authorized_keys (in master host) ? It should contain only entries starting with "command=" . If there is any duplicate entry without "command=" , delete the same. and check the geo-rep status again. 2. This is to confirm ssh connection between master and slave: When do run the following, you should get gsynd prompt(from master to slave). ssh -i /var/lib/glusterd/geo-replication/secret.pem root at slave 3. check your firewall settings Thanks, Saravana On 07/25/2016 02:24 AM, Alexandre Besnard wrote:> Anybody, any clue ? > >> On 20 Jul 2016, at 01:59, Alexandre Besnard >> <besnard.alexandre at gmail.com <mailto:besnard.alexandre at gmail.com>> wrote: >> >> Hello >> >> I deleted the content of /root/.ssh/authorized_keys on the slave (ks4) >> >> Then I configured passwordless authentication from the host (ks16): >> >> /ssh-copy-id root at ks4/ >> / >> / >> Also did from the host: >> >> /gluster system:: execute gsec_create/ >> >> which created a file >> in /var/lib/glusterd/geo-replication/common_secret.pem.pub >> >> Then created the geo-replicated volume successfully: >> >> /gluster volume geo-replication backupvol ks4::backupvol create >> push-pem force/ >> >> but still getting the same errors in the log after starting the volume: >> >> /[2016-07-19 23:52:14.900583] I [monitor(monitor):266:monitor] >> Monitor: ------------------------------------------------------------/ >> /[2016-07-19 23:52:14.900752] I [monitor(monitor):267:monitor] >> Monitor: starting gsyncd worker/ >> /[2016-07-19 23:52:14.958281] I >> [gsyncd(/gluster/backupvol):710:main_i] <top>: syncing: >> gluster://localhost:backupvol -> >> ssh://root at ks4:gluster://localhost:backupvol/ >> /[2016-07-19 23:52:14.958520] I [changelogagent(agent):73:__init__] >> ChangelogAgent: Agent listining.../ >> /[2016-07-19 23:52:15.81407] E >> [syncdutils(/gluster/backupvol):252:log_raise_exception] <top>: >> connection to peer is broken/ >> /[2016-07-19 23:52:15.81645] E >> [resource(/gluster/backupvol):226:errlog] Popen: command "ssh >> -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i >> */var/lib/glusterd/geo-replication/secret.pem *-p 22 >> -oControlMaster=auto -S >> /tmp/gsyncd-aux-ssh-Aux8JK/b63292d563144e7818235d683516731d.sock >> root at ks4 /nonexistent/gsyncd --session-owner >> 3281242a-ab45-4a0d-99e5-2965b4ac5840 -N --listen --timeout 120 >> gluster://localhost:backupvol" returned with 255, saying:/ >> /[2016-07-19 23:52:15.81733] E >> [resource(/gluster/backupvol):230:logerr] Popen: ssh> >> key_load_public: invalid format/ >> /[2016-07-19 23:52:15.81804] E >> [resource(/gluster/backupvol):230:logerr] Popen: ssh> Permission >> denied (publickey,password)./ >> /[2016-07-19 23:52:15.81947] I >> [syncdutils(/gluster/backupvol):220:finalize] <top>: exiting./ >> /[2016-07-19 23:52:15.82798] I [repce(agent):92:service_loop] >> RepceServer: terminating on reaching EOF./ >> /[2016-07-19 23:52:15.82946] I [syncdutils(agent):220:finalize] >> <top>: exiting./ >> /[2016-07-19 23:52:15.82858] I [monitor(monitor):333:monitor] >> Monitor: worker(/gluster/backupvol) died before establishing connection/ >> >>>> Any thoughts ? >> >> >>> On 19 Jul 2016, at 06:33, Aravinda <avishwan at redhat.com >>> <mailto:avishwan at redhat.com>> wrote: >>> >>> Hi, >>> >>> Looks like Master Pem keys are not copied to Slave nodes properly, >>> Please cleanup /root/.ssh/authorized_keys in Slave nodes and run >>> Geo-rep create force again. >>> >>> gluster volume geo-replication <MASTERVOL> <SLAVEHOST>::<SLAVEVOL> >>> create push-pem force >>> >>> Do you observe any errors related to hook scripts in glusterd log file? >>> >>> regards >>> Aravinda >>> >>> On 07/18/2016 10:11 PM, Alexandre Besnard wrote: >>>> Hello >>>> >>>> On a fresh Gluster 3.8 install, I am not able to configure a >>>> geo-replicated volume. Everything works fine up to starting of the >>>> volume however Gluster reports a faulty status. >>>> >>>> When looking at the logs (gluster_error): >>>> >>>> [2016-07-18 16:30:04.371686] I [cli.c:730:main] 0-cli: Started >>>> running gluster with version 3.8.0 >>>> [2016-07-18 16:30:04.435854] I [MSGID: 101190] >>>> [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started >>>> thread with index 1 >>>> [2016-07-18 16:30:04.435921] I [socket.c:2468:socket_event_handler] >>>> 0-transport: disconnecting now >>>> [2016-07-18 16:30:04.997986] I [input.c:31:cli_batch] 0-: Exiting >>>> with: 0 >>>> >>>> >>>> >>>> From the geo-replicated logs, it seems I have a SSH configuration >>>> issue: >>>> >>>> 2016-07-18 16:35:28.293524] I [monitor(monitor):266:monitor] >>>> Monitor: ------------------------------------------------------------ >>>> [2016-07-18 16:35:28.293740] I [monitor(monitor):267:monitor] >>>> Monitor: starting gsyncd worker >>>> [2016-07-18 16:35:28.352266] I >>>> [gsyncd(/gluster/backupvol):710:main_i] <top>: syncing: >>>> gluster://localhost:backupvol -> >>>> ssh://root at ks4:gluster://localhost:backupvol >>>> [2016-07-18 16:35:28.352489] I [changelogagent(agent):73:__init__] >>>> ChangelogAgent: Agent listining... >>>> [2016-07-18 16:35:28.492474] E >>>> [syncdutils(/gluster/backupvol):252:log_raise_exception] <top>: >>>> connection to peer is broken >>>> [2016-07-18 16:35:28.492706] E >>>> [resource(/gluster/backupvol):226:errlog] Popen: command "ssh >>>> -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i >>>> /var/lib/glusterd/geo-replication/secret.pem -p 22 >>>> -oControlMaster=auto -S >>>> /tmp/gsyncd-aux-ssh-Fs2XND/b63292d563144e7818235d683516731d.sock >>>> root at ks4 /nonexistent/gsyncd --session-owner >>>> 3281242a-ab45-4a0d-99e5-2965b4ac5840 -N --listen --timeout 120 >>>> gluster://localhost:backupvol" returned with 255, saying: >>>> [2016-07-18 16:35:28.492794] E >>>> [resource(/gluster/backupvol):230:logerr] Popen: ssh> >>>> key_load_public: invalid format >>>> [2016-07-18 16:35:28.492863] E >>>> [resource(/gluster/backupvol):230:logerr] Popen: ssh> Permission >>>> denied (publickey,password). >>>> [2016-07-18 16:35:28.493004] I >>>> [syncdutils(/gluster/backupvol):220:finalize] <top>: exiting. >>>> [2016-07-18 16:35:28.494045] I [repce(agent):92:service_loop] >>>> RepceServer: terminating on reaching EOF. >>>> [2016-07-18 16:35:28.494204] I [syncdutils(agent):220:finalize] >>>> <top>: exiting. >>>> [2016-07-18 16:35:28.494143] I [monitor(monitor):333:monitor] >>>> Monitor: worker(/gluster/backupvol) died before establishing connection >>>> >>>> >>>> I tried to fix them to the best of my knowledge but I am missing >>>> something. >>>> >>>> Can you help me to fix it? >>>> >>>> Thanks, >>>> Alex >>>> _______________________________________________ >>>> Gluster-users mailing list >>>> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >>>> http://www.gluster.org/mailman/listinfo/gluster-users >>> >> > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160725/0470d869/attachment.html>