Lustre discuss - Apr 2008 - how to replace a ost

hi, all

i want to replace a ost, and i read the manual(820-3681.pdf from sun
site), i get this section 4.2.3.1 about how to do:

4.2.3.1 Removing an OST
To remove an OST:
1. Deactivate the OST (make it read-only) on the MDS so no new objects are
allocated to it.
2. Use lfs find to discover all files that have objects residing on
the deactivated
OST.
3. Copy these files to a new location and then move them back to their original
location to force their object re-creation on the active OSTs and the object
deletion on the OST to be removed.


and i do as what it say, deactivate the ost(cfs-OST0003_UUID) at my
MGS, and find the files reside in ost cfs-OST0003_UUID from my client,
and copy the file...ouch, i can''t copy the file, the error msg is:

cp /data//v/ff/b6/0b142400670.flv ./
cp: cannot stat `/data//v/ff/b6/0b142400670.flv '': Input/output error

because the ost is deactive, so i can''t stat and read the file? how to
do that?


thanks for you help.


-- 
Best regards
Felix New

hi,

sorry, i forgot to pasted the version:

[root at oss ~]# cat /proc/fs/lustre/version

lustre: 1.6.4.2
kernel: 47
build: 
1.6.4.2-19691231190000-PRISTINE-.cache.build.BUILD.lustre-kernel-2.6.9.lustre.linux-2.6.9-55.0.9.EL_lustre.1.6.4.2smp
[root at oss ~]# uname  -a
Linux oss 2.6.9-55.0.9.EL_lustre.1.6.4.2smp #1 SMP Wed Jan 16 19:52:57
EST 2008 x86_64 x86_64 x86_64 GNU/Linux
[root at oss ~]#

-- 
Best regards
Felix New

On Apr 29, 2008  18:14 +0800, Felix New wrote:
> i want to replace a ost, and i read the manual(820-3681.pdf from sun
> site), i get this section 4.2.3.1 about how to do:
> 
> 4.2.3.1 Removing an OST
> To remove an OST:
> 1. Deactivate the OST (make it read-only) on the MDS so no new objects are
> allocated to it.
> 2. Use lfs find to discover all files that have objects residing on
> the deactivated
> OST.
> 3. Copy these files to a new location and then move them back to their
original
> location to force their object re-creation on the active OSTs and the
object
> deletion on the OST to be removed.
> 
> 
> and i do as what it say, deactivate the ost(cfs-OST0003_UUID) at my
> MGS, and find the files reside in ost cfs-OST0003_UUID from my client,
> and copy the file...ouch, i can''t copy the file, the error msg is:
Did you deactivate it on MDS only?
> cp /data//v/ff/b6/0b142400670.flv ./
> cp: cannot stat `/data//v/ff/b6/0b142400670.flv '': Input/output
error
> 
> because the ost is deactive, so i can''t stat and read the file?
how to do that?
You shouldn''t deactivate the OST on the client.  If you are mounting
a client on the MDS node and doing this testing there, then you need
to be aware that there are 2 OST devices on such a setup - one for
the client and one for the MDS:

  9 UP mds myth-MDT0000 mds-myth-0_UUID 7
 10 UP osc myth-OST0000-osc myth-mdtlov_UUID 5
 11 UP osc myth-OST0001-osc myth-mdtlov_UUID 5
 12 UP osc myth-OST0002-osc myth-mdtlov_UUID 5
 13 UP osc myth-OST0003-osc myth-mdtlov_UUID 5
:
:
 17 UP mdc myth-MDT0000-mdc-ffff810027cd7138
34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 18 UP osc myth-OST0000-osc-ffff810027cd7138
34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 19 UP osc myth-OST0001-osc-ffff810027cd7138
34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 20 UP osc myth-OST0002-osc-ffff810027cd7138
34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 21 UP osc myth-OST0003-osc-ffff810027cd7138
34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5
 22 UP osc myth-OST0004-osc-ffff810027cd7138
34d9e3f0-6b7a-5412-f602-3cce5b5247f2 5

You need to deactivate ONLY the osc device for e.g. OST0003 for the MDS
(myth-OST0003-osc in my case) and not the client one.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.