Hi All.
I''m trying to configure lustre CFS on 2 servers (1 MDS, 1 OST - both
SUSE enterprise 10).
Everything seems to be fine, it''s startup and I could see /mnt/lustre
mounted properly. However when I try to copy file after few seconds it
says "No space left on device", even so there is a bunch of free
space.
Here is what I have on those servers:
foma - MDS, susen - OST
--------------------------------------------------------------
susen:/mnt/lustre # ls
bigfile info2.txt test.info
susen:/mnt/lustre # ls -la
total 80280
drwxr-xr-x 2 root root 4096 2007-03-21 11:18 .
drwxr-xr-x 5 root root 120 2007-03-21 11:15 ..
-rw-r--r-- 1 root root 82108416 2007-03-21 11:18 bigfile
-rw-r--r-- 1 root root 14 2007-03-21 11:17 info2.txt
-rw-r--r-- 1 root root 197 2007-03-21 11:16 test.info
susen:/mnt/lustre #
susen:/mnt/lustre #
susen:/mnt/lustre # dd of=/mnt/lustre/bigfile2 if=/dev/zero bs=1048576
dd: writing /mnt/lustre/bigfile2'': No space left on device
81+0 records in
80+0 records out
84201472 bytes (84 MB) copied, 0.89681 seconds, 93.9 MB/s
susen:/mnt/lustre # df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 74G 3.7G 70G 5% /
udev 500M 108K 500M 1% /dev
foma:/mds-test/foma 275M 172M 89M 66% /mnt/lustre
susen:/mnt/lustre # dd of=/mnt/lustre/bigfile3 if=/dev/zero bs=1048576
dd: writing /mnt/lustre/bigfile3'': No space left on device
81+0 records in
80+0 records out
84205568 bytes (84 MB) copied, 0.930681 seconds, 90.5 MB/s
susen:/mnt/lustre # df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 74G 3.8G 70G 6% /
udev 500M 108K 500M 1% /dev
foma:/mds-test/foma 275M 252M 8.1M 97% /mnt/lustre
-----------------------------------------------------------------
>From the log file:
Mar 21 11:39:37 susen syslog-ng[1972]: STATS: dropped 0
Mar 21 11:40:29 susen kernel: LustreError:
3416:0:(client.c:576:ptlrpc_check_status()) @@@ type == PTL_RPC_MSG_ERR,
err == -28 req@f0513800 x1272/t0 o4->o
st1-test_UUID@susen_UUID:28 lens 328/288 ref 2 fl Rpc:R/0/0 rc 0/-28
Mar 21 11:40:29 susen kernel: LustreError:
3416:0:(client.c:576:ptlrpc_check_status()) Skipped 1 previous similar
message
Mar 21 11:40:29 susen kernel: LustreError:
3416:0:(client.c:576:ptlrpc_check_status()) @@@ type == PTL_RPC_MSG_ERR,
err == -28 req@f5b00c00 x1273/t0 o4->o
st1-test_UUID@susen_UUID:28 lens 328/288 ref 2 fl Rpc:R/0/0 rc 0/-28
Mar 21 11:40:40 susen kernel: LustreError:
3416:0:(client.c:576:ptlrpc_check_status()) @@@ type == PTL_RPC_MSG_ERR,
err == -28 req@f1364400 x1422/t0 o4->o
st2-test_UUID@susen_UUID:28 lens 328/288 ref 2 fl Rpc:R/0/0 rc 0/-28
Here is my config:
#!/bin/sh
# config.sh
# SuSen - MDS,OTS1 and client1
# Foma - OTS2 and client2
# Create nodes
rm -f config.xml
lmc -m config.xml --add node --node susen
lmc -m config.xml --add net --node susen --nid susen --nettype tcp
lmc -m config.xml --add node --node foma
lmc -m config.xml --add net --node foma --nid foma --nettype tcp
#lmc -m config.xml --add net --node client --nid ''*'' --nettype
tcp
# Cofigure MDS
lmc -m config.xml --add mds --node foma --mds mds-test --fstype ext3
--dev /tmp/mds-test --size 50000
# Configures OSTs
lmc -m config.xml --add lov --lov lov-test --mds mds-test --stripe_sz
1048576 --stripe_cnt 0 --stripe_pattern 0
lmc -m config.xml --add ost --node susen --lov lov-test --ost ost1-test
--fstype ext3 --dev /tmp/ost1-test --size 100000
lmc -m config.xml --add ost --node susen --lov lov-test --ost ost2-test
--fstype ext3 --dev /tmp/ost2-test --size 100000
lmc -m config.xml --add ost --node susen --lov lov-test --ost ost3-test
--fstype ext3 --dev /tmp/ost3-test --size 100000
# Configure client (this is a ''generic'' client used for all
client
mounts)
lmc -m config.xml --add mtpt --node foma --path /mnt/lustre --mds
mds-test --lov lov-test
Any suggestion ?