similar to: Lustre behaviour when multiple network paths are available?

Chris, Perhaps you need to perform some write_conf like command. I''m not sure if this is needed in 1.6 or not. Shane ----- Original Message ----- From: lustre-discuss-bounces at lists.lustre.org <lustre-discuss-bounces at lists.lustre.org> To: lustre-discuss <lustre-discuss at lists.lustre.org> Sent: Fri Mar 07 12:03:17 2008 Subject: Re: [Lustre-discuss] Multihomed

As any of you using version 1.6 of Lustre knows, Lustre servers can now be started simply my mounting the devices it is using. Even an /etc/fstab entry can be used if you can have the mount delayed until the network is started. Given this change, you have also notices that we have eliminated the initscript for Lustre that used to exist for releases prior to 1.6. I''d like to take a

I am running on CentOS 5 distribution without adding any updates from CentOS. I am using the lustre 1.6.4.1 kernel and software. I have two NICs that run though different switches. I have the lustre options in my modprobe.conf to look like this: options lnet networks=tcp0(eth1,eth0) My MGS seems to be only listening on the first interface however. When I try and ping the 1st interface (eth1)

Hi all, Here is the situation: I have 2 nodes MDS1 , MDS2 (10.0.0.22 , 10.0.0.23) I wish to use as failover MGS, active/active MDT with zfs. I have a jbod shelf with 12 disks, seen by both nodes as das (the shelf has 2 sas ports, connected to a sas hba on each node), and I am using lustre 2.4 on centos 6.4 x64 I have created 3 zfs pools: 1. mgs: # zpool

Hello All, I have installed and configured Lustre 1.8.4 on SuSe 11.0 and everything works fine if i run modprobe lustre and when the lustre module is getting loaded. But when the server reboots it is not getting loaded. Kindly help. Lnet is configured in /etc/modprobe.conf.local as below. options lnet networks=tcp0(eth0) accept=all For loading lustre module i tried including lustre module in

Hi all, I''m getting my feet wet in the infiniband lake and of course I run into some problems. It would seem I got the compilation part of sles11 kernel 2.6.27 + Lustre 1.8.3 + ofed 1.4.2 right, because it allows me to see and use the infiniband fabric, and because ko2iblnd loads without any complaints. In /etc/modprobe.d/lustre (this is a Debian system, hence this subdir of

Hi, I would like to know what TCP/UDP ports should i keep open in my firewall policies on my MGS server such that I can have my MGS server fire-walled. Also if in a event of loss of MGT would it be possible to recreate the MGT without loosing data or bringing the filesystem down (i.e. by using cached information from MDT''s and OST''s) Thanks Anand

Hello, Not sure if this is the right forum: I''m encountering difficulties with o2ib which prevents an LNET shutdown from proceeding: Unloading OpenIB kernel modules:NET: Unregistered protocal family 27 Failed to unload rdma_cm Failed to unload rdma_cm Failed to unload ib_cm Failed to unload ib_sa LustreError: 131-3: Received notification of device removal Please shutdown LNET

Hi, Lustre manual 1.6 v18 says that that in production lustre debug level should be set to fairly low. Manual also says that I can verify that level by running following commands: # sysctl portals.debug This gives ne following error error: ''portals.debug'' is an unknown key cat /proc/sys/lnet/debug gives output: ioctl neterror warning error emerg ha config console cat

Greetings! I am working with Lustre-2.1.2 on RHEL 6.2. First I configured it using the standard defaults over TCP/IP. Everything worked very nicely usnig a real, static --mgsnode=a.b.c.x value which was the actual IP of the MGS/MDS system1 node. I am now trying to integrate it with Pacemaker-1.1.7. I believe I have most of the set-up completed with a particular exception. The "lctl

Hi, With help from Oleg we got the right patches applied and NFS working well. Maximum performance was about 60 MB/sec. Last week that dropped to about 12.5 MB/sec and I cannot find a reason. Lustre clients all obtain 100+ MB/sec on GigE. Each OST is good for 270 MB/sec. When mounting the client on one of the OSSs I get 230 MB/sec. Seems the speed is there. How can NFS and Lustre be tuned

on our cluster that has been running lustre for about 1 month. I have 1 MDT/MGS and 1 OSS with 2 OST''s. Our cluster uses all Gige and has about 608 nodes 1854 cores. We have allot of jobs that die, and/or go into high IO wait, strace shows processes stuck in fstat(). The big problem is (i think) I would like some feedback on it that of these 608 nodes 209 of them have in dmesg

Hi, I''m seeing what can only be described as dismal striped write performance from lustre 1.6.3 clients :-/ 1.6.2 and 1.6.1 clients are fine. 1.6.4rc3 clients (from cvs a couple of days ago) are also terrible. the below shows that the OS (centos4.5/5) or fabric (gigE/IB) or lustre version on the servers doesn''t matter - the problem is with the 1.6.3 and 1.6.4rc3 client kernels

Hello, I''ve several problems with quota on our testcluster: When I set the quota for a person to a given value (e.g. the values which are provided in the operations manual), I''m able to write exact the amount which is set with setquota. But when I delete the files(file) I''m not able to use this space again. Here is what I''ve done in detail: lfs checkquota

Hello list members, I started to develop a kernel module which hooks into Lustre 2.3 for controlling data access based on nid and uid/gid. The background is the following: Here at GSI we have currently a reserved uid/gid space which partner institutes are using to access our exported Lustre mounts. However, we currently have no mechanism to control (guaranty) that the reserved uid/gid space are

Hi all, I used the method described below to build client rpms with the source kit lustre-1.8.5.tar.gz. There was only one error reported during the make rpms, relating to lustre-iolit-1.2-root, but the rpms were built under /usr/src/packages/RPMS/x86_64. The rpms lustre-modules, lustre and lustre-tests were then installed smoothly without any complaints. But the subsequent "modprobe

I have a kickstart file that I am using to install multiple machines. If I install with no %post script, everything runs great. When I add the following %post section, if fails. I have been working on this for a few days now without luck, Any help would be appreciated. Here is the error, the script follows. Traceback (most recent call first): File

Dear Lustre users, CFS is pleased to announce an updated document version (v1.7) of the Lustre? 1.6 Operations Manual, available in both PDF and HTML formats at http://www.manual.lustre.org. This edition of the Operations Manual includes the following enhancement: * Addition of mballoc3 content to the Lustre Proc chapter If you have any questions, suggestions, or recommended edits to the

Dear Lustre users, CFS is pleased to announce an updated document version (v1.7) of the Lustre? 1.6 Operations Manual, available in both PDF and HTML formats at http://www.manual.lustre.org. This edition of the Operations Manual includes the following enhancement: * Addition of mballoc3 content to the Lustre Proc chapter If you have any questions, suggestions, or recommended edits to the

Dear Lustre users, CFS is pleased to announce an updated document version (v1.7) of the Lustre? 1.6 Operations Manual, available in both PDF and HTML formats at http://www.manual.lustre.org. This edition of the Operations Manual includes the following enhancement: * Addition of mballoc3 content to the Lustre Proc chapter If you have any questions, suggestions, or recommended edits to the