Hi, We recently installed glusterfs 3.3.1. We have a 3 brick gluster system running that was being successfully mounted earlier. Yesterday we experienced a power outage and now after rebooting our systems, we are unable to mount this gluster file system. On the gluster client, a df -h command shows 41TB out of 55TB, while an ls command shows broken links for directories and missing files.>From the gluster server# gluster --version glusterfs 3.3.1 built on Oct 11 2012 22:01:05 # gluster volume info Volume Name: gdata Type: Distribute Volume ID: eccc3a90-212d-4563-ae8d-10a77758738d Status: Started Number of Bricks: 3 Transport-type: tcp Bricks: Brick1: gluster-0-0:/mseas-data-0-0 Brick2: gluster-0-1:/mseas-data-0-1 Brick3: gluster-data:/data [root at mseas-data ~]# ps -ef | grep gluster root 2783 1 0 Nov29 ? 00:01:15 /usr/sbin/glusterd -p /var/run/glusterd.pid root 2899 1 0 Nov29 ? 00:00:00 /usr/sbin/glusterfsd -s localhost --volfile-id gdata.gluster-data.data -p /var/lib/glusterd/vols/gdata/run/gluster-data-data.pid -S /tmp/e3eac7ce95e786a3d909b8fc65ed2059.socket --brick-name /data -l /var/log/glusterfs/bricks/data.log --xlator-option *-posix.glusterd-uuid=22f1102a-08e6-482d-ad23-d8e063cf32ed --brick-port 24009 --xlator-option gdata-server.listen-port=24009 root 2905 1 0 Nov29 ? 00:01:22 /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S /tmp/d5c892de43c28a1ee7481b780245b789.socket root 12329 12291 0 16:28 pts/1 00:00:00 grep gluster [root at nas-0-0 ~]# ps -ef | grep gluster root 2266 1 0 Nov29 ? 00:00:21 /usr/sbin/glusterd -p /var/run/glusterd.pid root 2461 1 0 Nov29 ? 00:00:00 /usr/sbin/glusterfsd -s localhost --volfile-id gdata.gluster-0-0.mseas-data-0-0 -p /var/lib/glusterd/vols/gdata/run/gluster-0-0-mseas-data-0-0.pid -S /tmp/f4391baa4f7cce7823362c6e0ad3d6de.socket --brick-name /mseas-data-0-0 -l /var/log/glusterfs/bricks/mseas-data-0-0.log --xlator-option *-posix.glusterd-uuid=3619440a-4ca3-4151-b62e-d4d6bf2e0c03 --brick-port 24009 --xlator-option gdata-server.listen-port=24009 root 2468 1 0 Nov29 ? 00:00:22 /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S /tmp/8f87e178e9707e4694ee7a2543c66db9.socket root 10007 9972 0 16:12 pts/2 00:00:00 grep gluster [root at nas-0-1 ~]# ps -ef | grep gluster root 2380 1 0 Nov29 ? 00:00:09 /usr/sbin/glusterd -p /var/run/glusterd.pid root 2550 1 0 Nov29 ? 00:00:00 /usr/sbin/glusterfsd -s localhost --volfile-id gdata.gluster-0-1.mseas-data-0-1 -p /var/lib/glusterd/vols/gdata/run/gluster-0-1-mseas-data-0-1.pid -S /tmp/a69a2b6a0de4366e53fc1584095eeedf.socket --brick-name /mseas-data-0-1 -l /var/log/glusterfs/bricks/mseas-data-0-1.log --xlator-option *-posix.glusterd-uuid=393fc4a6-1573-4564-971e-1b1aec434167 --brick-port 24009 --xlator-option gdata-server.listen-port=24009 root 2556 1 0 Nov29 ? 00:00:09 /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S /tmp/6054da6605d9f9d1c1e99252f1d235a6.socket root 6922 6888 0 16:14 pts/3 00:00:00 grep gluster -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Pat Haley Email: phaley at mit.edu Center for Ocean Engineering Phone: (617) 253-6824 Dept. of Mechanical Engineering Fax: (617) 253-8125 MIT, Room 5-213 http://web.mit.edu/phaley/www/ 77 Massachusetts Avenue Cambridge, MA 02139-4301 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20121130/5790224c/attachment.html>
Hi, I have some additional information. I have just installed gluster on a second client and tried to mount the same volume. The first client appeared to mount the gluster volume but many directories appear as broken lines and many files absent. The second client will not mount at all, giving the following error message: ERROR: Mount point does not exist. Usage: mount.glusterfs <volumeserver>:<volumeid/volumeport> -o <options> <mountpoint> Options: man 8 mount.glusterfs I don't know if this points the was to the problem or to debugging the problem? Thanks for any help you can give Pat> > Hi, > > We recently installed glusterfs 3.3.1. We have a 3 brick gluster system > running > that was being successfully mounted earlier. Yesterday we experienced a > power outage and now after rebooting our systems, we are unable to mount > this gluster file system. On the gluster client, a df -h command shows 41TB > out of 55TB, while an ls command shows broken links for directories and > missing files. > > From the gluster server > > # gluster --version > glusterfs 3.3.1 built on Oct 11 2012 22:01:05 > > # gluster volume info > > Volume Name: gdata > Type: Distribute > Volume ID: eccc3a90-212d-4563-ae8d-10a77758738d > Status: Started > Number of Bricks: 3 > Transport-type: tcp > Bricks: > Brick1: gluster-0-0:/mseas-data-0-0 > Brick2: gluster-0-1:/mseas-data-0-1 > Brick3: gluster-data:/data > > > [root at mseas-data ~]# ps -ef | grep gluster > root 2783 1 0 Nov29 ? 00:01:15 /usr/sbin/glusterd -p > /var/run/glusterd.pid > root 2899 1 0 Nov29 ? 00:00:00 /usr/sbin/glusterfsd -s > localhost --volfile-id gdata.gluster-data.data -p > /var/lib/glusterd/vols/gdata/run/gluster-data-data.pid -S > /tmp/e3eac7ce95e786a3d909b8fc65ed2059.socket --brick-name /data -l > /var/log/glusterfs/bricks/data.log --xlator-option > *-posix.glusterd-uuid=22f1102a-08e6-482d-ad23-d8e063cf32ed --brick-port > 24009 --xlator-option gdata-server.listen-port=24009 > root 2905 1 0 Nov29 ? 00:01:22 /usr/sbin/glusterfs -s > localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid > -l /var/log/glusterfs/nfs.log -S > /tmp/d5c892de43c28a1ee7481b780245b789.socket > root 12329 12291 0 16:28 pts/1 00:00:00 grep gluster > > > [root at nas-0-0 ~]# ps -ef | grep gluster > root 2266 1 0 Nov29 ? 00:00:21 /usr/sbin/glusterd -p > /var/run/glusterd.pid > root 2461 1 0 Nov29 ? 00:00:00 /usr/sbin/glusterfsd -s > localhost --volfile-id gdata.gluster-0-0.mseas-data-0-0 -p > /var/lib/glusterd/vols/gdata/run/gluster-0-0-mseas-data-0-0.pid -S > /tmp/f4391baa4f7cce7823362c6e0ad3d6de.socket --brick-name > /mseas-data-0-0 -l /var/log/glusterfs/bricks/mseas-data-0-0.log > --xlator-option > *-posix.glusterd-uuid=3619440a-4ca3-4151-b62e-d4d6bf2e0c03 --brick-port > 24009 --xlator-option gdata-server.listen-port=24009 > root 2468 1 0 Nov29 ? 00:00:22 /usr/sbin/glusterfs -s > localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid > -l /var/log/glusterfs/nfs.log -S > /tmp/8f87e178e9707e4694ee7a2543c66db9.socket > root 10007 9972 0 16:12 pts/2 00:00:00 grep gluster > > > [root at nas-0-1 ~]# ps -ef | grep gluster > root 2380 1 0 Nov29 ? 00:00:09 /usr/sbin/glusterd -p > /var/run/glusterd.pid > root 2550 1 0 Nov29 ? 00:00:00 /usr/sbin/glusterfsd -s > localhost --volfile-id gdata.gluster-0-1.mseas-data-0-1 -p > /var/lib/glusterd/vols/gdata/run/gluster-0-1-mseas-data-0-1.pid -S > /tmp/a69a2b6a0de4366e53fc1584095eeedf.socket --brick-name > /mseas-data-0-1 -l /var/log/glusterfs/bricks/mseas-data-0-1.log > --xlator-option > *-posix.glusterd-uuid=393fc4a6-1573-4564-971e-1b1aec434167 --brick-port > 24009 --xlator-option gdata-server.listen-port=24009 > root 2556 1 0 Nov29 ? 00:00:09 /usr/sbin/glusterfs -s > localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid > -l /var/log/glusterfs/nfs.log -S > /tmp/6054da6605d9f9d1c1e99252f1d235a6.socket > root 6922 6888 0 16:14 pts/3 00:00:00 grep gluster > > > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > Pat Haley Email: > phaley at mit.edu > Center for Ocean Engineering Phone: (617) 253-6824 > Dept. of Mechanical Engineering Fax: (617) 253-8125 > MIT, Room 5-213 > http://web.mit.edu/phaley/www/ > 77 Massachusetts Avenue > Cambridge, MA 02139-4301 > > > ------------------------------------------------------------------------ > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users-- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Pat Haley Email: phaley at mit.edu Center for Ocean Engineering Phone: (617) 253-6824 Dept. of Mechanical Engineering Fax: (617) 253-8125 MIT, Room 5-213 http://web.mit.edu/phaley/www/ 77 Massachusetts Avenue Cambridge, MA 02139-4301
On Fri, Nov 30, 2012 at 10:19 PM, Pat Haley <phaley at mit.edu> wrote:> > Hi,[...]> ERROR: Mount point does not exist. > Usage: mount.glusterfs <volumeserver>:<volumeid/volumeport> -o <options> <mountpoint>The obvious questions: can you ping <volumeserver> from the client? <mountpoint> does exist? -- Marcelo "?No ser? acaso que esta vida moderna est? teniendo m?s de moderna que de vida?" (Mafalda)
Hi, Yes, I can ping the volume server from both clients. That volume server also hosts another disk partition which is not part of the gluster volume and I can NFS mount that partition from both clients. I am not sure if this answers your question "<mountpoint> does exist?" -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Pat Haley Email: phaley at mit.edu Center for Ocean Engineering Phone: (617) 253-6824 Dept. of Mechanical Engineering Fax: (617) 253-8125 MIT, Room 5-213 http://web.mit.edu/phaley/www/ 77 Massachusetts Avenue Cambridge, MA 02139-4301 ________________________________________ From: Marcelo Roccasalva [roccas at gmail.com] Sent: Friday, November 30, 2012 9:01 PM To: Patrick Haley Cc: gluster-users at gluster.org Subject: Re: [Gluster-users] Cannot mount gluster volume On Fri, Nov 30, 2012 at 10:19 PM, Pat Haley <phaley at mit.edu> wrote:> > Hi,[...]> ERROR: Mount point does not exist. > Usage: mount.glusterfs <volumeserver>:<volumeid/volumeport> -o <options> <mountpoint>The obvious questions: can you ping <volumeserver> from the client? <mountpoint> does exist? -- Marcelo "?No ser? acaso que esta vida moderna est? teniendo m?s de moderna que de vida?" (Mafalda)