similar to: Multihomed question: want Lustre over IB andEthernet

on our cluster that has been running lustre for about 1 month. I have 1 MDT/MGS and 1 OSS with 2 OST''s. Our cluster uses all Gige and has about 608 nodes 1854 cores. We have allot of jobs that die, and/or go into high IO wait, strace shows processes stuck in fstat(). The big problem is (i think) I would like some feedback on it that of these 608 nodes 209 of them have in dmesg

This is the strangest problem I have seen. I have a lustre filesystem mounted on a linux server and its being exported to various alpha systems. The alphas mount it just fine however under heavy load the NFS server stops responding, as does the lustre mount on the export server. The weird thing is that if i mount the nfs export on another nfs server and run the same benchmark (bonnie) everything

Hello, Not sure if this is the right forum: I''m encountering difficulties with o2ib which prevents an LNET shutdown from proceeding: Unloading OpenIB kernel modules:NET: Unregistered protocal family 27 Failed to unload rdma_cm Failed to unload rdma_cm Failed to unload ib_cm Failed to unload ib_sa LustreError: 131-3: Received notification of device removal Please shutdown LNET

It seems that lustre puts its modules in /lib/modules/2.6.16.21-0.8-default despite the fact that my kernel is 2.6.16.21-0.8-smp ! uname -a Linux n32 2.6.16.21-0.8-smp #4 SMP Sun Sep 24 08:47:30 BST 2006 i686 i686 i386 GNU/Linux make[3]: Nothing to be done for `install-exec-am''. /bin/sh ../../mkinstalldirs /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre /usr/bin/install -c -m 644

Hi everyone, I have seen this question here before, but without a very satisfactory answer. One of our half a dozen clients has lost access to a set of OSTs: > lfs osts OBDS:: 0: lustre-OST0000_UUID ACTIVE 1: lustre-OST0001_UUID ACTIVE 2: lustre-OST0002_UUID INACTIVE 3: lustre-OST0003_UUID INACTIVE 4: lustre-OST0004_UUID INACTIVE 5: lustre-OST0005_UUID ACTIVE 6: lustre-OST0006_UUID ACTIVE

Hi, Can anyone guide me in building the lustre using pre-packaged lustre release.I''m using Ubuntu 7.10 I want to build lustre using RHEL2.6 rpms available on my system.I''m referring how_to in wiki. but in that no detailed step by step procedure is given for building lustre using pre-packed release. I''m in need of this. Thanks and Regards, Ashok Bharat -----Original

In 1.6.0, when creating a MDT, you could specify multiple --mgsnode options and it would failover between them. 1.6.3 only seems to take the last one and --mgsnode=192.168.1.252 at o2ib:192.168.1.253 at o2ib doesn''t seem to failover to the other node. Any ideas how to get around this? Robert Robert LeBlanc College of Life Sciences Computer Support Brigham Young University leblanc at

Hi! We''ve started to poke and prod at Lustre 1.6.4.1, and it seems to mostly work (we haven''t had it OOPS on us yet like the earlier 1.6-versions did). However, we had this weird incident where an active client (it was copying 4GB files and running ls at the time) got evicted by the MDS and all OST''s. After a while logs indicate that it did recover the connection

Hi, Our environment is: 2.6.9-55.0.9.EL_lustre.1.6.3smp I am getting following errors from two OSS''s ... Nov 7 10:39:51 storage09.beowulf.cluster kernel: LustreError: 23045:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID req at 00000100b410be00 x4190687/t0 o101->MGS at MGC10.143.245.201@tcp_0:26 lens 232/240 ref 1 fl Rpc:/0/0 rc 0/0 Nov 7 10:39:51

Hi all, I''m getting my feet wet in the infiniband lake and of course I run into some problems. It would seem I got the compilation part of sles11 kernel 2.6.27 + Lustre 1.8.3 + ofed 1.4.2 right, because it allows me to see and use the infiniband fabric, and because ko2iblnd loads without any complaints. In /etc/modprobe.d/lustre (this is a Debian system, hence this subdir of

I''m seeing this error message on one of my OSS''s but not the other three. Any idea what is causing it? Oct 25 13:58:56 oss2 kernel: LustreError: 3228:0:(client.c:519:ptlrpc_import_delay_req()) @@@ IMP_INVALID req at f6b13200 x18040/t0 o101->MGS at MGC192.168.0.200@tcp_0:26 lens 176/184 ref 1 fl Rpc:/0/0 rc 0/0 Oct 25 13:58:56 oss2 kernel: LustreError:

Hi, Our lustre environment is: 2.6.9-55.0.9.EL_lustre.1.6.3smp I would like to change recovery timeout from default value 250s to something longer I tried example from manual: set_timeout <secs> Sets the timeout (obd_timeout) for a server to wait before failing recovery. We performed that experiment on our test lustre installation with one OST. storage02 is our OSS [root at

Hello All, I have installed and configured Lustre 1.8.4 on SuSe 11.0 and everything works fine if i run modprobe lustre and when the lustre module is getting loaded. But when the server reboots it is not getting loaded. Kindly help. Lnet is configured in /etc/modprobe.conf.local as below. options lnet networks=tcp0(eth0) accept=all For loading lustre module i tried including lustre module in

Hi, On a few of the hpc cluster nodes, i am seeing a new lustre error that is pasted below. The volumes are working fine and there is nothing on the oss and mds to report. LustreError: 5080:0:(import.c:607:ptlrpc_connect_interpret()) data3-OST0000_UUID at 192.168.2.98@tcp changed handle from 0xfe51139158c64fae to 0xfe511392a35878b3; copying, but this may foreshadow disaster

Hi, i just want to know whether there are any alternative file systems for HP SFS. I heard that there is Cluster Gateway from Polyserve. Can anybody plz help me in finding more abt this Cluster Gateway. Thanks and Regards, Ashok Bharat -----Original Message----- From: lustre-discuss-bounces at lists.lustre.org on behalf of lustre-discuss-request at lists.lustre.org Sent: Tue 2/12/2008 3:18 AM

Hello, I am trying to mount a lustre filesystem thru an ib2ip gateway. The MDS''s have infiniband connections. The client nodes are tcp/ip connections. I am able to route between the client nodes and the MDS''s. I have the following in /etc/fstab: abe-mds1 at o2ib0,abe-mds2 at o2ib0:/home/client /abehome lustre _netdev,flock 0 0 I get the following when trying

Dear All, we are facing problem while connecting o2ib module. Lustre 2.2 with centos 6.3 gives problem while loading o2ib module for infiniband. Thanks in advance Regards, Faheem Patel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20130326/d0eb1e39/attachment.html

Dear All, I try to use our cluster though samba share. Everything work fine, but I think, we should have -o flock at lustre mount time. Great, it''s work. But when I want to save a file on the share, I get this on the logs: Jan 31 10:45:24 opteron-ren-11 kernel: LustreError: 24836:0:(file.c:2309:ll_file_flock()) unknown fcntl lock type: 32 Jan 31 10:45:24 opteron-ren-11 kernel:

Hi, list: I''m trying configure lustre with: 1 MGS -------------> 192.168.3.100 with mkfs.lustre --mgs /dev/md1 ; mount -t lustre ... 1 MDT ------------> 192.168.3.101 with mkfs.lustre --fsname=datafs00 --mdt --mgsnode=192.168.3.100 /dev/sda3 ; mount -t lustre ... 4 ost -----------> 192.168.3.102-104 with mkfs.lustre --fsname=datafs00 --ost --mgsnode=192.168.3.100 at tcp0

I am trying to compile lustre-1.6.6 against ofed-1.4 using following configure options : ./configure --disable-server --with-o2ib=/usr/src/ofa_kernel-1.4 --with-linux=/usr/src/kernels/2.6.18-92.el5-x86_64 I am getting following errors: configure: error: can''t compile with OpenIB gen2 headers under /usr/src/ofa_kernel-1.4 Is it that lustre-1.6.6 will only work with ofed-1.3. or I am