Lustre discuss - May 2006 - bonnie++ (and performance in general)

Andras.Horvath@cern.ch wrote:
> 
> As you see, I have two OSTs, both using a file on ext3 as storage. In
> their dmesg, both have plenty of
> 
> ext3_map_inode_page: error -28 allocating block 0
> LustreError: 3676:(filter_io_24.c:116:filter_direct_io()) ino 1344665,
> blk 0 cr 4294967295 create 1: rc -28
> ext3_map_inode_page: error -28 allocating block 0
> LustreError: 3653:(filter_io_24.c:116:filter_direct_io()) ino 1357347,
> blk 0 cr 4294967295 create 1: rc -28
-28 is -ENOSPC.  You need to create larger file systems for your OSTs.

-Phil

Hi,

I''m new to both this list and Lustre so hello all :)

I''m experimenting with a 5-node cluster and Lustre 1.0.3 using RH7.3,
and the binary RPMS (kernel and utilities) downloaded from the clusterfs
site . Installation and basic functions went fine but now I''ve reached
the point where "I need some help" as the download page says.

I''m trying to run bonnie++ 1.03a on my Lustre filesystem as:

# bonnie++ -r 512m -s 2048 -f -n 4096:4096:512:768 -d /mnt/lustre/ -u0:0

and the first parts (serial write and rewrite) work OK. Then it tries to
create plenty of small files in 768 directories which, sometime during,
fails. On the MDS, I get

LustreError: 2545:(lov_obd.c:665:lov_create()) error creating objid
0x483fc3 sub-object on OST idx 0/2: rc = -28
LustreError: 2545:(lov_obd.c:739:lov_create()) reallocating LSM for
objid 0x483fc3: old 2 new 1

# lctl device_list
 0 UP mdt MDT MDT_UUID 2
 1 UP mds mds1 mds1_UUID 2
 2 UP osc OSC_lxs5012_ost1_mds1 fb4875c0-3b9d-45e7-9976-60a310c10d60 4
 3 UP osc OSC_lxs5012_ost2_mds1 fb4875c0-3b9d-45e7-9976-60a310c10d60 4
 4 UP lov lov_mds1 fb4875c0-3b9d-45e7-9976-60a310c10d60 2

As you see, I have two OSTs, both using a file on ext3 as storage. In
their dmesg, both have plenty of

ext3_map_inode_page: error -28 allocating block 0
LustreError: 3676:(filter_io_24.c:116:filter_direct_io()) ino 1344665,
blk 0 cr 4294967295 create 1: rc -28
ext3_map_inode_page: error -28 allocating block 0
LustreError: 3653:(filter_io_24.c:116:filter_direct_io()) ino 1357347,
blk 0 cr 4294967295 create 1: rc -28

What could be the problem?

Configuration:
http://cern.ch/ahorvath/lustre/config.xml
(two OSTs, two clients, one MDS, all connected via gbit)

Thanks in advance..

Andras Horvath