similar to: gluster and rocks?

It seems that lustre puts its modules in /lib/modules/2.6.16.21-0.8-default despite the fact that my kernel is 2.6.16.21-0.8-smp ! uname -a Linux n32 2.6.16.21-0.8-smp #4 SMP Sun Sep 24 08:47:30 BST 2006 i686 i686 i386 GNU/Linux make[3]: Nothing to be done for `install-exec-am''. /bin/sh ../../mkinstalldirs /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre /usr/bin/install -c -m 644

We have been struggling with our Lustre performance for some time now especially with large directories. I recently did some informal benchmarking (on a live system so I know results are not scientifically valid) and noticed a huge drop in performance of reads(stat operations) past 20k files in a single directory. I''m using bonnie++, disabling IO testing (-s 0) and just creating, reading,

Hi have what might be some elementary questions. Really, what I"d love would be for someone who has had good success to publish his/her configuration files and, maybe, the output from ifconfig. At this point, when I see the not-so-good performance I"m getting, I don't realistically know if I'm in the right ballfield. It seems to me, that with so many mellanox cards out

Hello, Not sure if this is the right forum: I''m encountering difficulties with o2ib which prevents an LNET shutdown from proceeding: Unloading OpenIB kernel modules:NET: Unregistered protocal family 27 Failed to unload rdma_cm Failed to unload rdma_cm Failed to unload ib_cm Failed to unload ib_sa LustreError: 131-3: Received notification of device removal Please shutdown LNET

Hello, I'm trying to set up an experimental gluster storage cluster using virtual machines (VMs) on Xen. I want to be able to instruct the VMs to "forget" any previous gluster configuration (i.e. the server to forget about all previously configured peers, the peers to forget what storage volumes they are providing, etc). The use case is that the VMs will can be attached to

I am trying to track down a problem I reported on the list last week and discovered a new problem during my testing. If you have a four node setup with replicate/distribute and one of the nodes has a filesystem failure, the operating system will typically remount the filesystem read only. When this happens, the glusterfsd is still running on the failed machine, but i doesn't seem to

Hi, I am trying to compile the R-dev version on a unix Suse machine and got errors. Would someone be able to help me determine what to do to fix these errors: make[1]: Entering directory `/lustre/people/schaffer/R-devel/m4' make[1]: Nothing to be done for `R'. make[1]: Leaving directory `/lustre/people/schaffer/R-devel/m4' make[1]: Entering directory

We seemed to have it a stumbling block when building with the above (supported) versions. Our process... 1. Start with stock rhel5 2.6.18-8.1.14 source tree 2. Configure InfiniBand support out of the the kernel (we will build OFED separately). 3. Apply the 1.6.4.2 kernel patches to the kernel source. 4. Build the kernel. 5. Build OFED 1.2 against the patched kernel 6. Build Lustre using

Hello, I have a problem with the srptools to connect my Dom0 to the scst over IB ressources. *When i''m on the Debian kernel (without Dom0) * root@blade1:/# ibsrpdm -c id_ext=003048ffff9dd3b4,ioc_guid=003048ffff9dd3b4,dgid=fe80000000000000003048ffff9dd3b5,pkey=ffff,service_id=003048ffff9dd3b4

Hi all, I''m getting my feet wet in the infiniband lake and of course I run into some problems. It would seem I got the compilation part of sles11 kernel 2.6.27 + Lustre 1.8.3 + ofed 1.4.2 right, because it allows me to see and use the infiniband fabric, and because ko2iblnd loads without any complaints. In /etc/modprobe.d/lustre (this is a Debian system, hence this subdir of

Hi all, At long last, Lustre 1.6.5 is available on the Sun Download Center Site. http://www.sun.com/software/products/lustre/get.jsp The change log and release notes can be read here: http://wiki.lustre.org/index.php?title=Change_Log_1.6 Thank you for your assistance; as always, you can report issues via Bugzilla (https://bugzilla.lustre.org/) Happy downloading! -- The Lustre Team --

Hi all, At long last, Lustre 1.6.5 is available on the Sun Download Center Site. http://www.sun.com/software/products/lustre/get.jsp The change log and release notes can be read here: http://wiki.lustre.org/index.php?title=Change_Log_1.6 Thank you for your assistance; as always, you can report issues via Bugzilla (https://bugzilla.lustre.org/) Happy downloading! -- The Lustre Team --

This morning I''ve had both my infiniband and tcp lustre clients hiccup. They are evicted from the server presumably as a result of their high load and consequent timeouts. My question is- why don''t the clients re-connect. The infiniband and tcp clients both give the following message when I type "df" - Cannot send after transport endpoint shutdown (-108). I''ve

We have just discovered that a large buffer cache generated from traversing a lustre file system will cause a significant system overhead for applications with high memory demands. We have seen a 50% slowdown or worse for applications. Even High Performance Linpack, that have no file IO whatsoever is affected. The only remedy seems to be to empty the buffer cache from memory by running

Hello, I am having trouble installing the server modules for lustre 2.1.4 and use mellanox''s OFED distribution so we may use infiniband. Would you folks look at my procedure and results below and let me know what you think? Thanks very much! The mellanox ofed installation builds and installs some kernel modules too, so I used this method to ensure OFED compiled against the correct

To Harry Mangalam about Gluster/RDMA: make sure these modules are loaded # modprobe -v rdma_ucm # modprobe -v ib_uverbs # modprobe -v ib_ucm To run the subnet manager # modprobe -v ib_umad Make sure libibverbs and (libmlx4 or libmthca) RPMs are installed. I don't understand why they appropriate modules aren't loaded automatically. Could put something in /etc/modprobe.d/ to make this

Hello, I am trying to mount a lustre filesystem thru an ib2ip gateway. The MDS''s have infiniband connections. The client nodes are tcp/ip connections. I am able to route between the client nodes and the MDS''s. I have the following in /etc/fstab: abe-mds1 at o2ib0,abe-mds2 at o2ib0:/home/client /abehome lustre _netdev,flock 0 0 I get the following when trying

An user reported that he has hit an annoying deadlock while playing with ceph based on btrfs. Current updating device tree requires space from METADATA chunk, so we -may- need to do a recursive chunk allocation when adding/updating dev extent, that is where the deadlock comes from. If we use SYSTEM metadata to update device tree, we can avoid the recursive stuff. Reported-by: Jim Schutt

Dear All, we are facing problem while connecting o2ib module. Lustre 2.2 with centos 6.3 gives problem while loading o2ib module for infiniband. Thanks in advance Regards, Faheem Patel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20130326/d0eb1e39/attachment.html

Hi, I''m seeing what can only be described as dismal striped write performance from lustre 1.6.3 clients :-/ 1.6.2 and 1.6.1 clients are fine. 1.6.4rc3 clients (from cvs a couple of days ago) are also terrible. the below shows that the OS (centos4.5/5) or fabric (gigE/IB) or lustre version on the servers doesn''t matter - the problem is with the 1.6.3 and 1.6.4rc3 client kernels