similar to: Distributed storage and computing

Hello guys, does anyone of you have problems with chown and chmod operations taking extremely long time to complete. For example chown on 200 MB folder is taking minutes. The MDS is not loaded at all. All other operations (file read and write) seem to be doing fine. Any help is greatly appreciated. _______________________________________________ Lustre-discuss mailing list

I built and installed Lustre from source. When I installed built luster-modules- RPM, I got libcfs.ko from the kernel-2.6.18-prep has no modversions (the message is posted below), so it cannot be reused for kernel. Is there any procedure I need to do during the build? There is include/config/modversion.h under patched kernel, but the file does not have any contens and its size is 0. Thanks for

Hello, I have been reading http://wiki.lustre.org/images/1/1b/Hadoop_wp_v0.4.2.pdf for setting up Hadoop over lustre. Generally in hadoop setup, we have 1 Namenode and various number of datanodes. If I want to setup the same keeping Lustre as backend, in the document it is mentioned that: ".............Our experiments run on cluster with 8 nodes in total, one is mds/namenode, the rest are

I generally run TestDFSIO for Hadoop running on HDFS as shown: hadoop jar hadoop-*test*.jar TestDFSIO -write -nrFiles 10 -fileSize 1000 How to run it on lustre filesystem? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20130509/821dc4bb/attachment.html

Hi, I have 2 questions regarding building and installing Lustre server on Ubuntu. We have a running Lustre setup on RHEL and now trying to build and install on Ubuntu. 1. Whether latest Lustre (2.6+) server can be built on Ubuntu with the same steps as for RHEL? Or are there different steps for Ubuntu. 2. Has anyone tried this before and/or Whether Lustre2.6.x

Hello, I am a newbie to Lustre File system. In out data centre we have 2 Lustre file systems (small 40TB an larger 200 TB). We using our Lustre File system to perform I/O for Life sciences and bioinformatics applications. The vendor has decided to mount home directories on smaller lustre file system (40TB) and also installed bioinformatics applications on this smaller Lustre FS. The larger

I''m a Lustre newbie who just joined this list. I''d appreciate any help on the following Lustre 2.4 issue I''m running into: Every time I mount the MDT, the mount appears to succeed but /var/log/messages contains the message: "LustreError: 11-0: lustre-MDT0000-lwp-MDT0000: Communicating with 0@lo, operation mds_connect failed with -11". The MDT uses 4 local

Can anyone point me to a study or anything from some site that has a story re monitoring/tracking Lustre for: * Basic health (to catch problems before users call) * Performance régressions due to software, hardware, etc. * Bad actors from the user community. Are there simple probes of Lustre health? * Are there metrics available from internal Lustre monitoring? * Even

Hello, I have a production script that do read operations to a lot of small files. I read that one can gain performance boost with small files by using a loop device on top of Lustre. So a created 500 GB file striped across all of my OSTs(which are 8). I formatted the file with ext2 fs, and mounted it on a client. Just for the sake of testing a simple bash script finds all files with a given file

Hi all, Here is the situation: I have 2 nodes MDS1 , MDS2 (10.0.0.22 , 10.0.0.23) I wish to use as failover MGS, active/active MDT with zfs. I have a jbod shelf with 12 disks, seen by both nodes as das (the shelf has 2 sas ports, connected to a sas hba on each node), and I am using lustre 2.4 on centos 6.4 x64 I have created 3 zfs pools: 1. mgs: # zpool

We have just discovered that a large buffer cache generated from traversing a lustre file system will cause a significant system overhead for applications with high memory demands. We have seen a 50% slowdown or worse for applications. Even High Performance Linpack, that have no file IO whatsoever is affected. The only remedy seems to be to empty the buffer cache from memory by running

Since in Linux we are mostly a debian shop we''d like to stick with debian for our calculation nodes if possible. So I wanted to ask the lustre 2.2 instructions for Debian are they more or less relevant to lustre 2.4/2.5 or am I going headlong into a tall brick wall. Also are newer clients backwards compatible with older server software? I am currently just setting up a demo environment

Dear R community, I find R fantastic and use R whenever I can for my data analytic needs. Certain data sets, however, are so large that other tools seem to be needed to pre-process data such that it can be brought into R for further analysis. Questions I have for the many expert contributors on this list are: 1. How do others handle situations of large data sets (gigabytes, terabytes)

Hi Valeri On Fri, Apr 22, 2016 at 10:24 PM, Digimer <lists at alteeve.ca> wrote: > On 22/04/16 03:18 PM, Valeri Galtsev wrote: >> Dear Experts, >> >> I would like to ask everybody: what would you advise to use as a storage >> cluster, or as a distributed filesystem. >> >> I made my own research of what I can do, but I hit a snag with my >>

Hi, We run four-node Lustre 2.3, and I needed to both change hardware under MGS/MDS and reassign an OSS ip. Just the same, I added a brand new 10GE network to the system, which was the reason for MDS hardware change. I ran tunefs.lustre --writeconf as per chapter 14.4 in Lustre Manual, and everything mounts fine. Log regeneration apparently works, since it seems to do something, but

Hello Lustre Experts I am trying to solve a problem with very slow "ls" and other big amount of file operations but good overall read/write rates. We are running a small cluster of 3 OSSs with 9 OSTs, 1MDS (with SSD MDT) and currently two clients. All server nodes are centos 5.2 with lustre 1.8.1 while the clients are centos 5.4 with lustre 1.8.3. All components are networked with DDR

I have a question regarding storage volumes. I have uploaded a sparsed raw disk file to a storage volume, where virtual/max. size of the raw disk file is larger than size of the initial libvirt volume. Following is an example of commands: # Create storage volume of capacity 9G $ virsh-sys vol-create-as --pool pool-01 --name saturn.img --capacity 9G # Display volume info $ virsh-sys vol-info

What is the process for integrating a Lustre+SAMFS solution into an existing customer environment. The plan is to have CRS build the Lustre component, but Lustre and SAMFS will need to configured and integrated into the customer computing environment. I am very familiar with the SAMFS integration, but not Lustre integration. Do we have resources in PS to provide the integration? Is this

Dear All, is there a stratified version of the Wilcoxon test (also known as van Elteren test) available in R? I could find it in the survdiff function of the survival package for censored data. I think, it should be possible to use this function creating a dummy censoring indicator and setting it to not censored, but may be there is a better way to perform the test. Thanks, Heinz T??chler

Hello, I had been using R for text mining already. I wanted to use R for large scale text processing and for experiments with topic modeling. I started reading tutorials and working on some of those. I will now put down my understanding of each of the tools: 1) R text mining toolbox: Meant for local (client side) text processing and it uses the XML library 2) Hive: Hadoop interative, provides the