Mc Carthy, Fergal
2006-May-19 07:36 UTC
[Lustre-discuss] Mellanox OpenIB 1.8.2 with Lustre 1.4.6
You need to configure an appropriate options lnet entry in your /etc/modprobe.conf file (or /etc/modules.conf if running on a 2.4 kernel based system) to tell LNET which networks to use. If not specified it defaults to using TCP. For example if you want to use both TCP and OpenIB, with the IB one preferred, then you would probably use a line like the following: options lnet networks=openib0,tcp0 This is explained in the Lustre 1.4.6 manual; do a search for options lnet and you will find more explanation. Fergal. -- Fergal.McCarthy@HP.com (The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender. To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as "HP CONFIDENTIAL".) -----Original Message----- From: lustre-discuss-bounces@clusterfs.com [mailto:lustre-discuss-bounces@clusterfs.com] On Behalf Of Harris Yu Sent: 02 April 2006 20:57 To: lustre-discuss@clusterfs.com Subject: [Lustre-discuss] Mellanox OpenIB 1.8.2 with Lustre 1.4.6 Hi everyone, I installed Lustre 1.4.6 with patched Linux kernel on SLES9SP3, and built Mellanox IB driver using IBGD 1.8.2 under Lustre kernel. The IPoIB is working fine. But when I tried to use the following command to configure the Lustre to use OpenIB to do the communication: lmc -m config.xml --add net --node oss1 --nettype openib --nid oss1 Then when I first time to start the server/client, I got the following error: Mar 30 23:58:07 oss1 kernel: LustreError: 10014:0:(events.c:397:ptlrpc_uuid_to_peer()) No NID found for oss1_UUID Mar 30 23:58:07 oss1 kernel: LustreError: 10014:0:(client.c:58:ptlrpc_uuid_to_connection()) cannot find peer oss1_UUID! Mar 30 23:58:07 oss1 kernel: LustreError: 10014:0:(ldlm_lib.c:297:client_obd_setup()) can''t add initial connection Mar 30 23:58:07 oss1 kernel: LustreError: 10014:0:(connection.c:136:ptlrpc_put_connection()) NULL connection Mar 30 23:58:07 oss1 kernel: LustreError: 10019:0:(obd_config.c:323:class_cleanup()) Device 4 not setup Then I tried the command ''lctl list_nids'', it only gave me the NID for TCP: oss1:/var/log # lctl list_nids 10.2.128.50@tcp How can I configure to add NID for OpenIB? Thanks in advance, Harris _______________________________________________ Lustre-discuss mailing list Lustre-discuss@clusterfs.com https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
Hi everyone, I installed Lustre 1.4.6 with patched Linux kernel on SLES9SP3, and built Mellanox IB driver using IBGD 1.8.2 under Lustre kernel. The IPoIB is working fine. But when I tried to use the following command to configure the Lustre to use OpenIB to do the communication: lmc -m config.xml --add net --node oss1 --nettype openib --nid oss1 Then when I first time to start the server/client, I got the following error: Mar 30 23:58:07 oss1 kernel: LustreError: 10014:0:(events.c:397:ptlrpc_uuid_to_peer()) No NID found for oss1_UUID Mar 30 23:58:07 oss1 kernel: LustreError: 10014:0:(client.c:58:ptlrpc_uuid_to_connection()) cannot find peer oss1_UUID! Mar 30 23:58:07 oss1 kernel: LustreError: 10014:0:(ldlm_lib.c:297:client_obd_setup()) can''t add initial connection Mar 30 23:58:07 oss1 kernel: LustreError: 10014:0:(connection.c:136:ptlrpc_put_connection()) NULL connection Mar 30 23:58:07 oss1 kernel: LustreError: 10019:0:(obd_config.c:323:class_cleanup()) Device 4 not setup Then I tried the command ''lctl list_nids'', it only gave me the NID for TCP: oss1:/var/log # lctl list_nids 10.2.128.50@tcp How can I configure to add NID for OpenIB? Thanks in advance, Harris