thr3ads.net - Gluster users - [Gluster-users] Geo-Replication not detecting changes [Mar 2017]

If this information is useful, please help other people find it:
Share via:

Kotresh Hiremath Ravishankar

2017-Mar-30 04:51 UTC

[Gluster-users] Geo-Replication not detecting changes

Hi Jeremiah,

That's really strange. Please enable DEBUG logs for geo-replication as below
and send
us the logs under
"/var/log/glusterfs/geo-replication/<mastervol>/*.log" from
master node

gluster vol geo-rep <mastervol> <slavehost>::<slavevol> config
log-level DEBUG

Geo-rep has two ways to detect changes.

1. changelog (Changelog Crawl)
2. xsync (Hybrid Crawl):
           This is good for initial sync. It has the limitation of not detecting
unlinks and renames.
   So the slave would end up having unlinked files and renamed src file if it is
used after initial sync.

Thanks and Regards,
Kotresh H R

----- Original Message -----> From: "Jeremiah Rothschild" <jeremiah at franz.com>
> To: gluster-users at gluster.org
> Sent: Wednesday, March 29, 2017 12:39:11 AM
> Subject: Re: [Gluster-users] Geo-Replication not detecting changes
> 
> Following up on my own thread...
> 
> I have spent hours and hours setting up, re-setting up, screwing with
> undocumented variables, upgrading from LTS to non-LTS, etc etc.
> 
> Nothing seems to give.
> 
> This is very much an out-of-the-box setup and core functionality just
isn't
> working.
> 
> Can anyone throw me a bone here? Please? Do I file a bug for such an
> open-ended issue? Is everyone assuming I've just screwed a step up? I
must
> say the documentation is pretty clear & simple. Do you want more logs?
> 
> If this is going to be a dead end then so be it but I at least need to make
> sure I've tried my hardest to get a working deployment.
> 
> Thanks for your time and understanding!
> 
> j
> 
> On Thu, Mar 23, 2017 at 11:47:03AM -0700, Jeremiah Rothschild wrote:
> > Hey all,
> > 
> > I have a vanilla geo-replication setup running. It is comprised of two
> > servers, both CentOS 7 and GlusterFS 3.8.10:
> > 
> > * server1: Local server. Master volume named "foo".
> > * server2: Remote server. Slave volume named "foo".
> > 
> > Everything went fine including the initial sync. However, no new
changes
> > are
> > being seen or synced.
> > 
> > Geo-rep status looks clean:
> > 
> > # gluster volume geo-replication foo server2.franz.com::foo status
> > MASTER NODE: server1.x.com
> > MASTER VOL: foo
> > MASTER BRICK: /gv0/foo
> > SLAVE USER: root
> > SLAVE NODE: server2.x.com::foo
> > STATUS: Active
> > CRAWL STATUS: Changelog Crawl
> > LAST_SYNCED: 2017-03-23 10:12:57
> > 
> > In the geo-rep master log, I see these being triggered:
> > 
> > # tail -n3
> > foo/ssh%3A%2F%2Froot%401.2.3.4%3Agluster%3A%2F%2F127.0.0.1%3Afoo.log
> > [2017-03-23 18:33:34.697525] I [master(/gv0/foo):534:crawlwrap]
_GMaster:
> > 20
> > crawls, 0 turns
> > [2017-03-23 18:34:37.441982] I [master(/gv0/foo):534:crawlwrap]
_GMaster:
> > 20
> > crawls, 0 turns
> > [2017-03-23 18:35:40.242851] I [master(/gv0/foo):534:crawlwrap]
_GMaster:
> > 20
> > crawls, 0 turns
> > 
> > I don't see any errors in any of the other logs.
> > 
> > Not sure what else to poke at here. What are the possible values for
the
> > "change_detector" config variable? Would it be worthwhile to
test with a
> > method other than "changelog"? Other thoughts/ideas?
> > 
> > Thanks in advance!
> > 
> > j
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://lists.gluster.org/mailman/listinfo/gluster-users
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>

Jeremiah Rothschild

2017-Mar-30 07:46 UTC

head link

[Gluster-users] Geo-Replication not detecting changes

On Thu, Mar 30, 2017 at 12:51:23AM -0400, Kotresh Hiremath Ravishankar
wrote:> Hi Jeremiah,
Hi Kotresh! Thanks for the follow-up!
> That's really strange. Please enable DEBUG logs for geo-replication as
below and send
> us the logs under
"/var/log/glusterfs/geo-replication/<mastervol>/*.log" from
master node
> 
> gluster vol geo-rep <mastervol> <slavehost>::<slavevol>
config log-level DEBUG
Ok.

I started from scratch & enabled debug level logging. The logs have been
attached to Bugzilla #1327244.
> Geo-rep has two ways to detect changes.
> 
> 1. changelog (Changelog Crawl)
> 2. xsync (Hybrid Crawl):
>            This is good for initial sync. It has the limitation of not
detecting unlinks and renames.
>    So the slave would end up having unlinked files and renamed src file if
it is used after initial sync.
FYI I did try changing the changelog_detector to xsync but it made no
difference. Note that I also detailed this in the "Additional Info"
section
of the Bugzilla bug.
> Thanks and Regards,
Thanks again!

j
> Kotresh H R
> 
> ----- Original Message -----
> > From: "Jeremiah Rothschild" <jeremiah at franz.com>
> > To: gluster-users at gluster.org
> > Sent: Wednesday, March 29, 2017 12:39:11 AM
> > Subject: Re: [Gluster-users] Geo-Replication not detecting changes
> > 
> > Following up on my own thread...
> > 
> > I have spent hours and hours setting up, re-setting up, screwing with
> > undocumented variables, upgrading from LTS to non-LTS, etc etc.
> > 
> > Nothing seems to give.
> > 
> > This is very much an out-of-the-box setup and core functionality just
isn't
> > working.
> > 
> > Can anyone throw me a bone here? Please? Do I file a bug for such an
> > open-ended issue? Is everyone assuming I've just screwed a step
up? I must
> > say the documentation is pretty clear & simple. Do you want more
logs?
> > 
> > If this is going to be a dead end then so be it but I at least need to
make
> > sure I've tried my hardest to get a working deployment.
> > 
> > Thanks for your time and understanding!
> > 
> > j
> > 
> > On Thu, Mar 23, 2017 at 11:47:03AM -0700, Jeremiah Rothschild wrote:
> > > Hey all,
> > > 
> > > I have a vanilla geo-replication setup running. It is comprised
of two
> > > servers, both CentOS 7 and GlusterFS 3.8.10:
> > > 
> > > * server1: Local server. Master volume named "foo".
> > > * server2: Remote server. Slave volume named "foo".
> > > 
> > > Everything went fine including the initial sync. However, no new
changes
> > > are
> > > being seen or synced.
> > > 
> > > Geo-rep status looks clean:
> > > 
> > > # gluster volume geo-replication foo server2.franz.com::foo
status
> > > MASTER NODE: server1.x.com
> > > MASTER VOL: foo
> > > MASTER BRICK: /gv0/foo
> > > SLAVE USER: root
> > > SLAVE NODE: server2.x.com::foo
> > > STATUS: Active
> > > CRAWL STATUS: Changelog Crawl
> > > LAST_SYNCED: 2017-03-23 10:12:57
> > > 
> > > In the geo-rep master log, I see these being triggered:
> > > 
> > > # tail -n3
> > >
foo/ssh%3A%2F%2Froot%401.2.3.4%3Agluster%3A%2F%2F127.0.0.1%3Afoo.log
> > > [2017-03-23 18:33:34.697525] I [master(/gv0/foo):534:crawlwrap]
_GMaster:
> > > 20
> > > crawls, 0 turns
> > > [2017-03-23 18:34:37.441982] I [master(/gv0/foo):534:crawlwrap]
_GMaster:
> > > 20
> > > crawls, 0 turns
> > > [2017-03-23 18:35:40.242851] I [master(/gv0/foo):534:crawlwrap]
_GMaster:
> > > 20
> > > crawls, 0 turns
> > > 
> > > I don't see any errors in any of the other logs.
> > > 
> > > Not sure what else to poke at here. What are the possible values
for the
> > > "change_detector" config variable? Would it be
worthwhile to test with a
> > > method other than "changelog"? Other thoughts/ideas?
> > > 
> > > Thanks in advance!
> > > 
> > > j
> > > _______________________________________________
> > > Gluster-users mailing list
> > > Gluster-users at gluster.org
> > > http://lists.gluster.org/mailman/listinfo/gluster-users
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://lists.gluster.org/mailman/listinfo/gluster-users
> >

Gluster users - Mar 2017 - Geo-Replication not detecting changes

[Gluster-users] Geo-Replication not detecting changes

[Gluster-users] Geo-Replication not detecting changes