similar to: BUG: After stop and start wrong port is advertised

We (finally) figured out the root cause, Jo! Patch https://review.gluster.org/#/c/18579 posted upstream for review. On Thu, Sep 21, 2017 at 2:08 PM, Jo Goossens <jo.goossens at hosted-power.com> wrote: > Hi, > > > > > > We use glusterfs 3.10.5 on Debian 9. > > > > When we stop or restart the service, e.g.: service glusterfs-server restart > > >

On Sat, 28 Oct 2017 at 02:36, Jo Goossens <jo.goossens at hosted-power.com> wrote: > Hello Atin, > > > > > > I just read it and very happy you found the issue. We really hope this > will be fixed in the next 3.10.7 version! > 3.10.7 - no I guess as the patch is still in review and 3.10.7 is getting tagged today. You?ll get this fix in 3.10.8. > > >

I've already replied to your earlier email. In case you've not seen it in your mailbox here it goes: This looks like a bug to me. For some reason glusterd's portmap is referring to a stale port (IMO) where as brick is still listening to the correct port. But ideally when glusterd service is restarted, all the portmap in-memory is rebuilt. I'd request for the following details from

For what it's worth, I just updated some CentOS 7 servers from GlusterFS 3.12.1 to 3.12.4 and hit this bug. Did the patch make it into 3.12.4? I had to use Mike Hulsman's script to check the daemon port against the port in the volume's brick info, update the port, and restart glusterd on each node. Luckily I only have four servers! Hoping I don't have to do this every time I

I encountered this once in the past, an additional symptom was peers were in disconnected state on the peers that were NOT using the wrong ports. Disconnected peers is how it detected it in the first place. It happened to me after rebooting, and I fixed it but wasn?t able to stop and gather debugging info on the time. The problem seemed to be that the volume files in

Hi, ? ? We use glusterfs 3.10.5 on Debian 9. ? When we stop or restart the service, e.g.:?service glusterfs-server restart ? We see that the wrong port get's advertised afterwards. For example: ? Before restart: ? Status of volume: public Gluster process ? ? ? ? ? ? ? ? ? ? ? ? ? ? TCP Port ?RDMA Port ?Online ?Pid

Hello Atin, ? ? Could you confirm this should have been fixed in 3.10.8? If so we'll test it for sure! Regards Jo ? ? -----Original message----- From:Atin Mukherjee <amukherj at redhat.com> Sent:Mon 30-10-2017 17:40 Subject:Re: [Gluster-users] BUG: After stop and start wrong port is advertised To:Jo Goossens <jo.goossens at hosted-power.com>; CC:gluster-users at

On Sat, 2 Dec 2017 at 19:29, Jo Goossens <jo.goossens at hosted-power.com> wrote: > Hello Atin, > > > > > > Could you confirm this should have been fixed in 3.10.8? If so we'll test > it for sure! > Fix should be part of 3.10.8 which is awaiting release announcement. > > Regards > > Jo > > > > > > > -----Original

Hello Atin, ? ? I just read it and very happy you found the issue. We really hope this will be fixed in the next 3.10.7 version! ? ? PS: Wow nice all that c code and those "goto out" statements (not always considered clean but the best way often I think). Can remember the days I wrote kernel drivers myself in c :) ? ? Regards Jo Goossens ? ? ? -----Original message----- From:Atin

Ouch! Yes, I see two port-related fixes in the GlusterFS 3.12.3 release notes[0][1][2]. I've attached a tarball of all yesterday's logs from /var/log/glusterd on one the affected nodes (called "wingu3"). I hope that's what you need. [0] https://github.com/gluster/glusterfs/blob/release-3.12/doc/release-notes/3.12.3.md [1] https://bugzilla.redhat.com/show_bug.cgi?id=1507747

Hello, ? ? Will we also suffer from this regression in any of the (previously) fixed 3.10 releases? We kept 3.10 and hope to stay stable :/ Regards Jo ? ? -----Original message----- From:Atin Mukherjee <amukherj at redhat.com> Sent:Tue 23-01-2018 05:15 Subject:Re: [Gluster-users] BUG: After stop and start wrong port is advertised To:Alan Orth <alan.orth at gmail.com>; CC:Jo

The patch was definitely there in 3.12.3. Do you have the glusterd and brick logs handy with you when this happened? On Sun, Jan 21, 2018 at 10:21 PM, Alan Orth <alan.orth at gmail.com> wrote: > For what it's worth, I just updated some CentOS 7 servers from GlusterFS > 3.12.1 to 3.12.4 and hit this bug. Did the patch make it into 3.12.4? I had > to use Mike Hulsman's

Hi Darrell, ? ? Thanks, for us it's really easy to reproduce atm. Each restart or stop/start is causing the issue atm over here. ? Atin will look into it on monday fortunately :) Regards Jo ? ? ? -----Original message----- From:Darrell Budic <budic at onholyground.com> Sent:Fri 22-09-2017 17:24 Subject:Re: [Gluster-users] BUG: After stop and start wrong port is advertised To:Atin

Hi, This bug is hitting me hard on two different clients. In RHGS 3.3 and on glusterfs 3.10.2 on Centos 7.4 in once case I had 59 differences in a total of 203 bricks. I wrote a quick and dirty script to check all ports against the brick file and the running process. #!/bin/bash Host=`uname -n| awk -F"." '{print $1}'` GlusterVol=`ps -eaf | grep /usr/sbin/glusterfsd|

We've a fix in release-3.10 branch which is merged and should be available in the next 3.10 update. On Wed, Nov 8, 2017 at 4:58 PM, Mike Hulsman <mike.hulsman at proxy.nl> wrote: > Hi, > > This bug is hitting me hard on two different clients. > In RHGS 3.3 and on glusterfs 3.10.2 on Centos 7.4 > in once case I had 59 differences in a total of 203 bricks. > > I

So from the logs what it looks to be a regression caused by commit 635c1c3 ( and the good news is that this is now fixed in release-3.12 branch and should be part of 3.12.5. Commit which fixes this issue: COMMIT: https://review.gluster.org/19146 committed in release-3.12 by \"Atin Mukherjee\" <amukherj at redhat.com> with a commit message- glusterd: connect to an existing brick

3.10 doesn't have this regression, so you're safe. On Tue, Jan 23, 2018 at 1:28 PM, Jo Goossens <jo.goossens at hosted-power.com> wrote: > Hello, > > > > > > Will we also suffer from this regression in any of the (previously) fixed > 3.10 releases? We kept 3.10 and hope to stay stable :/ > > > > Regards > > Jo > > > > > >

Hello Glusterites, I have a volume that will not mount read/write. V3.10.3 on Centos 7, this is a replica-3 volume, mounted with the fuse client. This is in support of an Ovirt installation, but I've isolated the problem to Gluster. `gluster peer status` looks normal, as does a `gluster v status`. Server.allow-insecure is set to on. Notable from the client volume logs are the 'connection

Hi all, ? ? One more thing, we have 3 apps servers with the gluster on it, replicated on 3 different gluster nodes. (So the gluster nodes are app servers at the same time). We could actually almost work locally if we wouldn't need to have the same files on the 3 nodes and redundancy :) ? Initial cluster was created like this: ? gluster volume create www replica 3 transport tcp

Hello, ? ? Here is the volume info as requested by soumya: ? #gluster volume info www ?Volume Name: www Type: Replicate Volume ID: 5d64ee36-828a-41fa-adbf-75718b954aff Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: 192.168.140.41:/gluster/www Brick2: 192.168.140.42:/gluster/www Brick3: 192.168.140.43:/gluster/www Options Reconfigured: