similar to: Unusual Block Allocations on OSTs

Hi everyone, I have seen this question here before, but without a very satisfactory answer. One of our half a dozen clients has lost access to a set of OSTs: > lfs osts OBDS:: 0: lustre-OST0000_UUID ACTIVE 1: lustre-OST0001_UUID ACTIVE 2: lustre-OST0002_UUID INACTIVE 3: lustre-OST0003_UUID INACTIVE 4: lustre-OST0004_UUID INACTIVE 5: lustre-OST0005_UUID ACTIVE 6: lustre-OST0006_UUID ACTIVE

Hello, We had a problem with our disk controller that required a reboot. 2 of our OSTs remounted and went through the recovery window but clients hang trying to access them. Also /proc/fs/lustre/obdfilter/$UUID/ is empty for that OST UUID. LDISKFS FS on dm-5, internal journal on dm-5:8 LDISKFS-fs: delayed allocation enabled LDISKFS-fs: file extents enabled LDISKFS-fs: mballoc enabled

Hi all, after installing a Lustre test file system consisting of 34 OSTs, I encountered a strange error when trying to set up quotas: lfs quotacheck gave me an "Input/Output error", while in /var/log/kern.log I found a Lustre error LustreError: 20807:0:(quota_check.c:227:lov_quota_check()) lov idx 32 inactive Indeed, in /proc/fs/lustre/lov/.../target_obd all 34 OSTs were listed

Hi Guys, I''m trying to figure out a way to get a list of files with objects present on an OST that is down. Normally one could do: lfs find -O <OST> dir but that is giving us Input/output errors (I assume because the OST is down). Is there a good way to get a list of objects (Maybe from the MDS?), what OSTs they are on, and correlate them with files? Thanks, Mark -- Mark

There are references to running fsck on the lustre OSTs after a crash or power failure. However, after downloading the ClusterFS e2fsprogs and building them, e2fsck does not recognize our ldiskfs- based OSTs. Is there a way to fsck the ldiskfs-based OSTs? Thanks, Charlie Taylor UF HPC Center

All, I''ve added a new 2.2 TB OST to my cluster easily enough, but this new disk array is meant to replace several smaller OSTs that I used to have of which were only 120 GB, 500 GB, and 700 GB. Adding an OST is easy, but how do I REMOVE the small OSTs that I no longer want to be part of my cluster? Is there a command to tell luster to move all the file stripes off one of the nodes?

Hello, we wounder whether is it possible to add OSTs to the Lustre with quota support without making it offline? We tried to do this but all quota information was lost. Despite the fact that OST was formatted with quota support we are receiving this error message: Lustre: 3743:0:(lproc_quota.c:447:lprocfs_quota_wr_type()) lustrefs-OST0016: quotaon failed because quota files

Hi all, we use SLES 11 and Lustre 1.8.1.1 + patches and like convert a lustre FS using external journals to one with async journals enabled. Question is whether the procedure: umount <filesystem> on all clients umount <osts> on all OSSes e2fsck <ost-device> on all OSSes for all all OSTs tune2fs -O ^has_journal <ost-device> on all

OK, so our old MDS had hardware issues so I configured a new MGS / MDS on a VM (this is a backup lustre filesystem and I wanted to separate the MGS / MDS from OSS of the previous), and then did this: For example: mount -t ldiskfs /dev/old /mnt/ost_old mount -t ldiskfs /dev/new /mnt/ost_new rsync -aSv /mnt/ost_old/ /mnt/ost_new # note trailing slash on ost_old/ If you are unable to connect both

Just checking to be sure this isn''t a known bug or problem. I couldn''t find a bz for this, but it would appear that tunefs.lustre --print fails on a lustre mdt or ost device if mounted with mmp. Is this expected behavior? TIA mds1-gps:~ # tunefs.lustre --print /dev/mapper/mdt1 checking for existing Lustre data: not found tunefs.lustre FATAL: Device /dev/mapper/mdt1 has not

Dear All, We have a small setup of lustre with 7 OSTs on 8gb FC . We have kept one OST per FC port. We have lustre 2.3 with CentOS 6.3. There are 32 clients which access this over FDR IB. We can achieve more than 1.3GB/s throughput using IOR, without cache. Which is roughly 185MB/s per OST. We wanted to know if this is normal. Should we expect more from 8gb FC port. OSTs are on 8+2 RAID6 .

We are seeing some disturbing (probably due to our ignorance) behavior from lustre 1.6.3 right now. We have 8 OSSs with 3 OSTs per OSS (24 physical LUNs). We just created a brand new lustre file system across this configuration using the default mkfs.lustre formatting options. We have this file system mounted across 400 clients. At the moment, we have 63 IOzone threads running

if i have a single oss with multiple ost''s do i get any parallelism when striping across the ost''s? i.e. would a client form multiple connections on whatever transport between the itself the oss?

Hi, everyone, I need some help in figuring out what may have happened here, as newly created files on an OST are being corrupted. I don''t know if this applies to all files written to this OST, or just to files of order 2GB size, but files are definitely being corrupted, with no errors reported by the OSS machine. Let me describe the situation. We had been running Lustre 1.8.4 for

Hi; One of our OSTs filled up. Once we realized this, we executed lctl --device 9 deactivate on our fs''s combo MDS/MGS machine. We saw in the syslog that the OST in question was deactivated: Lustre: setting import ufhpc-OST0008_UUID INACTIVE by administrator request However, ''lfs df'' on the clients does not show that the OST is deactivated there, unless we *also*

Hello, We have running lustre 1.8.1 and have met "No space lest on device" error when uploading 500 Gb small files (less then 100 Kb each). The problem seems to depends on the number of files. If we remove one file, we can create one new file, even with Gb size; but if we haven''t remove something we can''t create even very little file, as an example using touch

Hello, When I run a "df -i" in my clients I get 95% indes used or 5% inodes free: Filesystem Inodes IUsed IFree IUse% Mounted on lustre-mds-01:lustre-mds-02:/cetafs 22200087 20949839 1250248 95% /mnt/data But if I run lfs df -i i get: UUID Inodes IUsed IFree I

whoops, sent from wrong email address, form right address now: Hello, I am using Lustre 2.4.2 and have an OST that doesn't seem to be written to. When I check the MDS with 'lctl dl' I do not see that OST in the list. However when I check the OSS that OST belongs to I can see it is mounted and up; 0 UP osd-zfs l2-OST0003-osd l2-OST0003-osd_UUID 5 3 UP obdfilter l2-OST0003

Hi, With help from Oleg we got the right patches applied and NFS working well. Maximum performance was about 60 MB/sec. Last week that dropped to about 12.5 MB/sec and I cannot find a reason. Lustre clients all obtain 100+ MB/sec on GigE. Each OST is good for 270 MB/sec. When mounting the client on one of the OSSs I get 230 MB/sec. Seems the speed is there. How can NFS and Lustre be tuned

Hello everybody, at the following pages: http://www.rit.edu/~rc/docs/Survey_of_Clustered_Parallel_File_Systems_004_LANL.ppt http://www.intel.com/cd/ids/developer/asmo-na/eng/dc/tools/threading/238284.htm?page=2 I read: "[...] Currently, one additional drawback to Lustre is that a Lustre client cannot be on a server that is providing OSTs. This solution is being worked on and may be