Gluster users - Jan 2013 - Configuration...

Hi have what might be some elementary questions.   Really, what I"d love
would be for someone who has had good success to publish his/her
configuration files and, maybe, the output from ifconfig.   At this point,
when I see the not-so-good performance I"m getting, I don't
realistically
know if I'm in the right ballfield.   It seems to me, that with so many
mellanox cards out there, and so many users of CentOS 6.x, there might be
someone who has gotten good performance and who might be willing to publish
the ifcfg-ib0 or whatever mechanism, combined with volume tunings.

As another question -- it looks to me that if you use those Mellanox VNICS,
you're limited to small MTU -- is that correct?

Thanks to everyone in this community.

--- still just getting started...

Matt Temple
------
Matt Temple
Director, Research Computing
Dana-Farber Cancer Institute.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130118/715228e6/attachment.html>

El dv 18 de 01 de 2013 a les 15:29 -0500, en/na Matthew Temple va
escriure:
> Hi have what might be some elementary questions.   Really, what I"d
> love would be for someone who has had good success to publish his/her
> configuration files and, maybe, the output from ifconfig.   
Hope it helps:

[root at computacion2 ~]# cat /etc/redhat-release 
CentOS release 6.3 (Final)

[root at computacion2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ib0 
DEVICE=ib0
BOOTPROTO=static
IPADDR=10.0.51.22
NETMASK=255.255.255.0
NETWORK=10.0.51.0
BROADCAST=10.0.51.255
ONBOOT=yes

[root at computacion2 ~]# ifconfig ib0
Ifconfig uses the ioctl access method to get the full address
information, which limits hardware addresses to 8 bytes.
Because Infiniband address has 20 bytes, only the first 8 bytes are
displayed correctly.
Ifconfig is obsolete! For replacement check ip.
ib0       Link encap:InfiniBand  HWaddr
80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00  
          inet addr:10.0.51.22  Bcast:10.0.51.255  Mask:255.255.255.0
          inet6 addr: fe80::202:c903:53:eeed/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:65520  Metric:1
          RX packets:228041357 errors:0 dropped:0 overruns:0 frame:0
          TX packets:216874432 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:256 
          RX bytes:616377554610 (574.0 GiB)  TX bytes:768025135100
(715.2 GiB)

[root at computacion2 ~]# lsmod | grep mlx
mlx4_ib                51754  0 
mlx4_en                70812  0 
mlx4_core             174465  2 mlx4_ib,mlx4_en
ib_mad                 38970  6
ib_cm,ib_sa,ib_umad,ib_qib,mlx4_ib,ib_mthca
ib_core                72903  15
rdma_ucm,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_sa,ib_uverbs,ib_umad,iw_nes,iw_cxgb4,iw_cxgb3,ib_qib,mlx4_ib,ib_mthca,ib_mad

We're running with OFED 1.5.4.1:

[root at computacion2 ~]# ibv_devinfo 
hca_id:	mlx4_0
	transport:			InfiniBand (0)
	fw_ver:				2.9.1000
	node_guid:			0002:c903:0053:eeec
	sys_image_guid:			0002:c903:0053:eeef
	vendor_id:			0x02c9
	vendor_part_id:			26428
	hw_ver:				0xB0
	board_id:			MT_0D9011XXXX
	phys_port_cnt:			1
		port:	1
			state:			PORT_ACTIVE (4)
			max_mtu:		2048 (4)
			active_mtu:		2048 (4)
			sm_lid:			1
			port_lid:		4
			port_lmc:		0x00
			link_layer:
-- 
Adri? Garc?a-Alz?rriz <adria.garcia-alzorriz at adam.es>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130121/f7d901db/attachment.sig>