?? ?
2006-Jul-11 22:04 UTC
[Lustre-devel] [lustre-discuss] About myrinet network configuration on lustre
Hi all: Before I setup lustre that uses tcp connection on five nodes with no problem. Recently I want to use Myrinet network to test Lustre''s performance. However, I could not sucess to setup the network part. I use gm-2.1.4 version. Now I try to setup MDS,OSS, and client into the same node. Below is my setting: test.sh ------ rm -f test-single.xml lmc -m test-single.xml --add net --node linux --nid 192.168.0.13 --nettype gm lmc -m test-single.xml --add mds --node linux --mds mds --fstype ldiskfs --dev /mds --size 300000 lmc -m test-single.xml --add lov --lov lov --mds mds --stripe_sz 100000 --stripe_cnt 0 --stripe_pattern 0 lmc -m test-single.xml --add ost --node linux --lov lov --ost ost --fstype ldiskfs --dev /ost --size 8000000 lmc -m test-single.xml --add mtpt --node linux --path /mnt/lustre --mds mds --lov lov ------ After I run the lconf command, I get an error message: loading module: libcfs srcdir /usr/src/packages/BUILD/lustre-1.4.6/lnet devdir libcfs loading module: lnet srcdir /usr/src/packages/BUILD/lustre-1.4.6/lnet devdir lnet loading module: ksocklnd srcdir /usr/src/packages/BUILD/lustre-1.4.6/lnet devdir klnds/socklnd loading module: lvfs srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir lvfs loading module: obdclass srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir obdclass loading module: ptlrpc srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir ptlrpc loading module: ost srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir ost loading module: ldiskfs srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir ldiskfs loading module: fsfilt_ldiskfs srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir lvfs loading module: obdfilter srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir obdfilter loading module: mdc srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir mdc loading module: osc srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir osc loading module: lov srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir lov loading module: mds srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir mds NETWORK: NET_linux_gm NET_linux_gm_UUID gm linux OSD: ost ost_UUID obdfilter /ost 8000000 ldiskfs no 0 0 OST mount options: errors=remount-ro MDSDEV: mds mds_UUID /mds ldiskfs no recording clients for filesystem: FS_fsname_UUID Recording log mds on mds LOV: lov_mds ffd0b_lov_mds_9df72514c0 mds_UUID 0 100000 0 0 [u''ost_UUID''] mds OSC: OSC_linux_ost_mds ffd0b_lov_mds_9df72514c0 ost_UUID Error preparing OSC ffd0b_lov_mds_9df72514c0 ! /usr/sbin/lctl (255): Can''t parse NID linux@gm Do I misconfigure something? Please help. Thanks. Chieh
Mc Carthy, Fergal
2006-Jul-12 03:51 UTC
[Lustre-devel] [lustre-discuss] About myrinet network configurationon lustre
First off I suspect that you may need to be using a newer version of the GM driver suite, 2.1.23 or later. I believe that Eric Barton from CFS worked with Myricom last year to get a fix into the gm-2.1.23 release which was required for correct operation of the more modern GMNAL/GMLND implementations. Have you setup your /etc/modprobe.conf with a valid networks specification that includes gm as an interconnect type? Also your config example below shows an IP address nid value for a nettype of gm, which I don''t believe to be correct. Fergal. -- Fergal.McCarthy@HP.com (The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender. To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as "HP CONFIDENTIAL".) -----Original Message----- From: lustre-devel-bounces@clusterfs.com [mailto:lustre-devel-bounces@clusterfs.com] On Behalf Of ?? ? Sent: 12 July 2006 05:04 To: lustre-devel@clusterfs.com Subject: [Lustre-devel] [lustre-discuss] About myrinet network configurationon lustre Hi all: Before I setup lustre that uses tcp connection on five nodes with no problem. Recently I want to use Myrinet network to test Lustre''s performance. However, I could not sucess to setup the network part. I use gm-2.1.4 version. Now I try to setup MDS,OSS, and client into the same node. Below is my setting: test.sh ------ rm -f test-single.xml lmc -m test-single.xml --add net --node linux --nid 192.168.0.13 --nettype gm lmc -m test-single.xml --add mds --node linux --mds mds --fstype ldiskfs --dev /mds --size 300000 lmc -m test-single.xml --add lov --lov lov --mds mds --stripe_sz 100000 --stripe_cnt 0 --stripe_pattern 0 lmc -m test-single.xml --add ost --node linux --lov lov --ost ost --fstype ldiskfs --dev /ost --size 8000000 lmc -m test-single.xml --add mtpt --node linux --path /mnt/lustre --mds mds --lov lov ------ After I run the lconf command, I get an error message: loading module: libcfs srcdir /usr/src/packages/BUILD/lustre-1.4.6/lnet devdir libcfs loading module: lnet srcdir /usr/src/packages/BUILD/lustre-1.4.6/lnet devdir lnet loading module: ksocklnd srcdir /usr/src/packages/BUILD/lustre-1.4.6/lnet devdir klnds/socklnd loading module: lvfs srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir lvfs loading module: obdclass srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir obdclass loading module: ptlrpc srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir ptlrpc loading module: ost srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir ost loading module: ldiskfs srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir ldiskfs loading module: fsfilt_ldiskfs srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir lvfs loading module: obdfilter srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir obdfilter loading module: mdc srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir mdc loading module: osc srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir osc loading module: lov srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir lov loading module: mds srcdir /usr/src/packages/BUILD/lustre-1.4.6/lustre devdir mds NETWORK: NET_linux_gm NET_linux_gm_UUID gm linux OSD: ost ost_UUID obdfilter /ost 8000000 ldiskfs no 0 0 OST mount options: errors=remount-ro MDSDEV: mds mds_UUID /mds ldiskfs no recording clients for filesystem: FS_fsname_UUID Recording log mds on mds LOV: lov_mds ffd0b_lov_mds_9df72514c0 mds_UUID 0 100000 0 0 [u''ost_UUID''] mds OSC: OSC_linux_ost_mds ffd0b_lov_mds_9df72514c0 ost_UUID Error preparing OSC ffd0b_lov_mds_9df72514c0 ! /usr/sbin/lctl (255): Can''t parse NID linux@gm Do I misconfigure something? Please help. Thanks. Chieh _______________________________________________ Lustre-devel mailing list Lustre-devel@clusterfs.com https://mail.clusterfs.com/mailman/listinfo/lustre-devel
Scott Atchley
2006-Jul-12 19:09 UTC
[Lustre-devel] [lustre-discuss] About myrinet network configurationon lustre
On Jul 12, 2006, at 5:50 AM, Mc Carthy, Fergal wrote:> Also your config example below shows an IP address nid value for a > nettype of gm, which I don''t believe to be correct. > > FergalI believe he will need to specify a hostname, not a IP address. Scott