Hey guys, I am running gluster over infiniband, and I have a couple of questions. We have four servers, each with 1 disk that I am trying to access over infiniband using gluster. The servers look like they start okay, here are the last 10 or so lines of a client log (they are all identical): 2008-10-07 07:18:40 D [spec.y:196:section_sub] parser: child:stripe0->remote1 2008-10-07 07:18:40 D [spec.y:196:section_sub] parser: child:stripe0->remote2 2008-10-07 07:18:40 D [spec.y:196:section_sub] parser: child:stripe0->remote3 2008-10-07 07:18:40 D [spec.y:196:section_sub] parser: child:stripe0->remote4 2008-10-07 07:18:40 D [spec.y:209:section_end] parser: end:stripe0 2008-10-07 07:18:40 D [glusterfs.c:128:fuse_graph] glusterfs: setting option mount-point to /mnt/glusterfs 2008-10-07 07:18:40 D [xlator.c:115:xlator_set_type] xlator: attempt to load file /usr/local/lib/glusterfs/1.3.12/xlator/mount/fuse.so 2008-10-07 07:18:40 E [fuse-bridge.c:2699:init] glusterfs-fuse: fuse_mount failed (Transport endpoint is not connected) 2008-10-07 07:18:40 E [glusterfs.c:547:main] glusterfs: Initializing FUSE failed It says initializing FUSE failed, but lsmod shows fuse is running. Help? Here are my spec files: Server: # file: /etc/glusterfs/glusterfs-server.vol volume brick type storage/posix option directory /data/export end-volume volume server type protocol/server option transport-type ib-verbs/server option auth.ip.brick.allow * subvolumes brick end-volume Client: # file: /etc/glusterfs/glusterfs-client.vol volume remote1 type protocol/client option transport-type ib-verbs/client option remote-host 192.168.1.240 option remote-subvolume brick end-volume volume remote2 type protocol/client option transport-type ib-verbs/client option remote-host 192.168.1.241 option remote-subvolume brick end-volume volume remote3 type protocol/client option transport-type ib-verbs/client option remote-host 192.168.1.244 option remote-subvolume brick end-volume volume remote4 type protocol/client option transport-type ib-verbs/client option remote-host 192.168.1.248 option remote-subvolume brick end-volume volume stripe0 type cluster/stripe option block-size *:1MB subvolumes remote1 remote2 remote3 remote4 end-volume -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20081007/cac27f25/attachment.html>
> /usr/local/lib/glusterfs/1.3.12/xlator/mount/fuse.so > > 2008-10-07 07:18:40 E [fuse-bridge.c:2699:init] glusterfs-fuse: fuse_mount > failed (Transport endpoint is not connected) > > > > 2008-10-07 07:18:40 E [glusterfs.c:547:main] glusterfs: Initializing FUSE > failed >That line generally means a previous stale instance is not yet unmounted from the directory you are trying the new mount. umount the mountpoint once and try again. avati -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20081008/8bf2af99/attachment.html>
Interesting... What method is likely to give me the best throughput overall? I am getting wirespeed using striping but I am wondering if I can go faster. When you guys did the infiniband benchmarking, what method did you use? --Mark T. ________________________________________ From: anand.avati at gmail.com [anand.avati at gmail.com] On Behalf Of Anand Avati [avati at zresearch.com] Sent: Wednesday, October 08, 2008 12:39 AM To: Mark True Cc: gluster-users at gluster.org Subject: Re: [Gluster-users] gluster over infiniband.... /usr/local/lib/glusterfs/1.3.12/xlator/mount/fuse.so 2008-10-07 07:18:40 E [fuse-bridge.c:2699:init] glusterfs-fuse: fuse_mount failed (Transport endpoint is not connected) 2008-10-07 07:18:40 E [glusterfs.c:547:main] glusterfs: Initializing FUSE failed That line generally means a previous stale instance is not yet unmounted from the directory you are trying the new mount. umount the mountpoint once and try again. avati
Interesting... What method is likely to give me the best throughput overall? I am getting wirespeed using striping but I am wondering if I can go faster. When you guys did the infiniband benchmarking, what method did you use? --Mark T. ________________________________________ From: anand.avati at gmail.com [anand.avati at gmail.com] On Behalf Of Anand Avati [avati at zresearch.com] Sent: Wednesday, October 08, 2008 12:39 AM To: Mark True Cc: gluster-users at gluster.org Subject: Re: [Gluster-users] gluster over infiniband.... /usr/local/lib/glusterfs/1.3.12/xlator/mount/fuse.so 2008-10-07 07:18:40 E [fuse-bridge.c:2699:init] glusterfs-fuse: fuse_mount failed (Transport endpoint is not connected) 2008-10-07 07:18:40 E [glusterfs.c:547:main] glusterfs: Initializing FUSE failed That line generally means a previous stale instance is not yet unmounted from the directory you are trying the new mount. umount the mountpoint once and try again. avati ________________________________________ From: anand.avati at gmail.com [anand.avati at gmail.com] On Behalf Of Anand Avati [avati at zresearch.com] Sent: Wednesday, October 08, 2008 12:39 AM To: Mark True Cc: gluster-users at gluster.org Subject: Re: [Gluster-users] gluster over infiniband.... /usr/local/lib/glusterfs/1.3.12/xlator/mount/fuse.so 2008-10-07 07:18:40 E [fuse-bridge.c:2699:init] glusterfs-fuse: fuse_mount failed (Transport endpoint is not connected) 2008-10-07 07:18:40 E [glusterfs.c:547:main] glusterfs: Initializing FUSE failed That line generally means a previous stale instance is not yet unmounted from the directory you are trying the new mount. umount the mountpoint once and try again. avati
Hey guys, I am somewhat new to infiniband so this question is Gluster related but not entirely limited to gluster. When you have more than one infiniband card in your system, and you start gluster how do you tell it which fabric to participate in? Any help would be greatly appreciated. --Mark T. Best, Mark F. True Mark F. True RAID Inc. 5 Branch Street, 2nd Floor Methuen, MA 01844 Direct (800) 330-7335 x161 Cell (978) 886-2184 mtrue at raidinc.com<mailto:mtrue at raidinc.com> This e-mail and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to whom they are addressed. This communication may contain material protected by Corporate Trade Secrets. If you are not the intended recipient or the person responsible for delivering the e-mail for the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you believe you have received this e-mail in error, please immediately notify RAID, Inc. by telephone at 978-683-6444. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20081024/c458c2f8/attachment.html>