Gluster users - Aug 2011 - Problems starting Geo-replication

I am trying to configure geo-replication between two gluster volumes.

The Master is:
[root at creator images]# gluster volume info
Volume Name: images
Type: Distributed-Replicate
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: stor-001:/glusterfs
Brick2: stor-002:/glusterfs
Brick3: stor-003:/glusterfs
Brick4: stor-004:/glusterfs
Options Reconfigured:
geo-replication.indexing: on

The slave is:
[root at arcus-0005 newimage]# gluster volume info
Volume Name: newimages
Type: Distribute
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: stor-005:/zpool01/glshare006-01
Brick2: stor-006:/zpool01/glshare006-01

The host named creator (also known as stor-000, where I am starting the
replication from) is a peer (glusterd and glusterfs running) but does not
provide bricks on the master group.
[root at creator images]# ps ax | grep gluster
 5958 ?        Ssl    5:44 /opt/glusterfs/3.2.2/sbin/glusterd
 6099 ?        Ssl  328:43 /opt/glusterfs/3.2.2/sbin/glusterfs --log-level=INFO
--volfile-id=/images --volfile-server=stor-000 /var/lib/xen/images



Here is what I get when trying to start geo-replication:

[root at creator ~]# gluster volume geo-replication images stor-005:/newimage
start
Starting geo-replication session between images & stor-005:/newimage has
been successful
[root at creator ~]# gluster volume geo-replication images stor-005:/newimage
status
MASTER               SLAVE                                              STATUS
--------------------------------------------------------------------------------
images               stor-005:/newimage                                 faulty
[root at creator ~]# gluster volume geo-replication images stor-005:/newimage
status
MASTER               SLAVE                                              STATUS
--------------------------------------------------------------------------------
images               stor-005:/newimage                                 faulty
[root at creator ~]# gluster volume geo-replication images stor-005:/newimage
config log-file
/var/log/glusterfs/geo-replication/images/ssh%3A%2F%2Froot%4010.10.10.26%3Afile%3A%2F%2F%2Fnewimage.log
[root at creator ~]# cat
/var/log/glusterfs/geo-replication/images/ssh%3A%2F%2Froot%4010.10.10.26%3Afile%3A%2F%2F%2Fnewimage.log
[2011-08-14 11:12:40.211878] I [monitor(monitor):19:set_state] Monitor: new
state: starting...
[2011-08-14 11:12:40.261898] I [monitor(monitor):42:monitor] Monitor:
------------------------------------------------------------
[2011-08-14 11:12:40.262080] I [monitor(monitor):43:monitor] Monitor: starting
gsyncd worker
[2011-08-14 11:12:40.342614] I [gsyncd:286:main_i] <top>: syncing:
gluster://localhost:images -> ssh://stor-005:/newimage
[2011-08-14 11:12:42.869029] E [syncdutils:131:exception] <top>: FAIL:
Traceback (most recent call last):
  File
"/opt/glusterfs/3.2.2/local/libexec//glusterfs/python/syncdaemon/gsyncd.py",
line 102, in main
    main_i()
  File
"/opt/glusterfs/3.2.2/local/libexec//glusterfs/python/syncdaemon/gsyncd.py",
line 296, in main_i
    local.service_loop(*[r for r in [remote] if r])
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/resource.py",
line 401, in service_loop
    GMaster(self, args[0]).crawl_loop()
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/master.py",
line 113, in crawl_loop
    self.crawl()
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/master.py",
line 173, in crawl
    volinfo_sys = self.get_sys_volinfo()
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/master.py",
line 22, in get_sys_volinfo
    fgn_vis, nat_vi = self.master.server.foreign_volume_infos(), \
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/resource.py",
line 330, in foreign_volume_infos
    xattr_list = Xattr.llistxattr_buf('.')
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/libcxattr.py",
line 57, in llistxattr_buf
    size = cls.llistxattr(path)
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/libcxattr.py",
line 38, in llistxattr
    ret = cls._query_xattr(path, siz, 'llistxattr')
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/libcxattr.py",
line 26, in _query_xattr
    cls.raise_oserr()
  File
"/opt/glusterfs/3.2.2/local/libexec/glusterfs/python/syncdaemon/libcxattr.py",
line 16, in raise_oserr
    raise OSError(errn, os.strerror(errn))
OSError: [Errno 107] Transport endpoint is not connected

Please help.

Thanks,
---
Jojo Colina
Managing Director/CTO
jojo.colina at arcusit.ph<mailto:jojo.colina at arcusit.ph>
ArcusIT, Inc.
"Your IT, made CloudEASY"
http://www.arcusit.ph/

1007 Antel Global Corporate Center
Julia Vargas cor. Meralco Avenue
Pasig City, Philippines 1605
Mobile: +639399371335
Landline: +6326870422 ext. 103


"entia non sunt multiplicanda praeter necessitatem" -- Ockhams razor
We must not make things more complicated than necessary.

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110814/ef758afa/attachment.html>

Hi Jojo Colina,

The above logs essentially means that listxattr is failing on
the auxiliary gluster mount-point which has died, Please post the contents
of the auxilary gluster mount-point which would be located at (based on the
info you have given):


/var/log/glusterfs/geo-replication/images/ssh%3A%2F%2Froot%4010.10.10.26%3Afile%3A%2F%2F%2Fnewimage.gluster.log


The log-file might throw more light as to why the gluster mount-point is
failing.

Regards,
Kaushik BV
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110816/6d3d465f/attachment.html>