Lustre discuss - Aug 2011 - Moving storage from one OSS to another

Hi all :)

Got a customer with:
     - 1 x S2A9900 (one couplet)
     - 500 x 2 TB drives
     - 4 OSS

This customer wants to add:
     - 1 x S2A9900 (one couplet)
     - 300 x 2 TB drives
     - 4 OSS

I know we could "just add" and restripe the existing files. But the
customer wants to physically move 100 drives from the existing S2A9900
to the new one in order to have 400 drives on one S2A9900 and 400
drives on the other one.

So my questions are:
     1.- Can this be done (move drives between OSTs) without losing data?
     2.- Can an OST be moved from one OSS to another without losing data?
     3.- Has anyone done this? How?

I imagine it can be done restriping files/migrating files within the
Lustre filesystem, removing "empty" OSTs, ...

TIA

   Rafa

Rafa Griman wrote:
> Hi all :)
>
> Got a customer
(It is quite bad form to ask for help for a commercial deal without 
using your work email address.  I assume you still work for Bull?)
> with:
>      - 1 x S2A9900 (one couplet)
>      - 500 x 2 TB drives
>      - 4 OSS
>
> This customer wants to add:
>      - 1 x S2A9900 (one couplet)
>      - 300 x 2 TB drives
>      - 4 OSS
>
> I know we could "just add" and restripe the existing files. But
the
> customer wants to physically move 100 drives from the existing S2A9900
> to the new one in order to have 400 drives on one S2A9900 and 400
> drives on the other one.
>
> So my questions are:
>      1.- Can this be done (move drives between OSTs) without losing data?
>   
First, make sure DDN supports moving drives _data intact_.  The drives 
in each tier will at least have to end up as an ordered tier for this to 
work, but I would guess it isn''t quite so simple.
>      2.- Can an OST be moved from one OSS to another without losing data?
>   
Yes, IF it is done properly.

Step 1, back up all essential data.

Step 2, add the _new server_ as a failover node for the OSTs being 
moved.  Note that this requires a "tunefs.lustre --writeconf" (with
new
parameters on the OSTs being moved) and re-mounting all the servers.

Step 3, move the drives

Step 4, update the server/failover NIDs, removing the old ones with 
another writeconf pass.


I _strongly_ recommend that you verify their Lustre support contract is 
current, and that you do a dry-run in a test environment before doing it 
live (if nothing else, you do have 4 servers and a new couplet to play 
with).
>      3.- Has anyone done this? How?
>
> I imagine it can be done restriping files/migrating files within the
> Lustre filesystem, removing "empty" OSTs, ...
>   
Well, yes, there is that option as well; use lfs_migrate (at least 2 
passes).
> TIA
>
>    Rafa
>   
Kevin

On 8 Aug 2011, at 04:57, Rafa Griman wrote:
> I know we could "just add" and restripe the existing files. But
the
> customer wants to physically move 100 drives from the existing S2A9900
> to the new one in order to have 400 drives on one S2A9900 and 400
> drives on the other one.
It would be more balanced if they do that.
> So my questions are:
>     1.- Can this be done (move drives between OSTs) without losing data?
Moving drives between OSTs does not make sense - the OST is the data stored on
the drives.  It is possible to move drives between controllers, you should
contact DDN support for advice on this.
>     2.- Can an OST be moved from one OSS to another without losing data?
Yes, you need to use tunefs.lustre for this.
>     3.- Has anyone done this? How?
We''ve done this, not frequently or as part of standard maintenance but
it''s not unusual when expanding filesystems in the way you are.

Ashley.