Lustre discuss - May 2006 - Multiple instances of lustre on same OST component filesystems?

Would this work given what Phil said?  I interpreted that a given OST 
block device
can only be assigned to one lov.

paul

Evan Felix wrote:
>What about running two lov/mounts that point to the same OST pool?
>
>ie lov1 attaches to ost1,ost2,ost3,ost4
>
>and lov2 attaches to ost1,ost2
>
>then you configure mount points for each lov.
>
>I''m not sure why I want to do this yet, but is it possible.
>
>I could have one lov do striping, and one doing mirroring I guess.
>
>from the OSD side it just serves up objects, so talking to two
lov''s
>should not be a problem.  now the MDS might not like that!
>
>Evan
>
>On Mon, 2004-09-20 at 23:01, Phil Schwan wrote:
>  
>
>>On Mon, 2004-09-20 at 12:27, Paul Nowoczynski wrote:
>>    
>>
>>>hello..
>>>i was wondering if it is possible (and/or supported) to run multiple
>>>lustre filesystems from the same OST disk partitions.
>>>      
>>>
>>No.  You can run multiple OSTs on a single OSS node, but they must use
>>separate partitions.  Likewise, you can connect multiple OSS nodes to a
>>single multi-port RAID device (very common for failover), but they must
>>use different partitions.
>>
>>If multiple nodes try to use the same physical disk blocks
>>simultaneously, catastrophic corruption will result.  Just imagine
>>mounting an ext3 file system from two nodes like that -- same thing.
>>
>>-Phil
>>
>>_______________________________________________
>>Lustre-discuss mailing list
>>Lustre-discuss@lists.clusterfs.com
>>https://lists.clusterfs.com/mailman/listinfo/lustre-discuss
>>    
>>

Phil Schwan wrote:
>On Tue, 2004-09-21 at 18:12, Evan Felix wrote:
>  
>
>>What about running two lov/mounts that point to the same OST pool?
>>
>>ie lov1 attaches to ost1,ost2,ost3,ost4
>>
>>and lov2 attaches to ost1,ost2
>>
>>then you configure mount points for each lov.
>>
>>I''m not sure why I want to do this yet, but is it possible.
>>
>>I could have one lov do striping, and one doing mirroring I guess.
>>
>>from the OSD side it just serves up objects, so talking to two
lov''s
>>should not be a problem.  now the MDS might not like that!
>>    
>>
>
>It used to be possible, for precisely the reason that you mentioned --
>it''s just an object store.
>
>But it''s no longer possible, because the MDS manages the allocation
of
>objects now, and the MDSs wouldn''t have any way to coordinate with
each
>other.  And even if they could, lfsck would need to be rewritten, or it
>would get very confused.
>  
>
>There are actually separate namespaces (object "groups" on the
OSTs)
>which could be used, but there is no way to configure an MDS to use a
>group other than group 0 today.  Not rocket science, perhaps, but also
>not trivial.
>  
>
I noticed group 0 when I mounted the OST filesystem and thought "why 
couldn''t have a 1 or 2 dir"?
So it is ''possible'' to have 2 MDSs if the MDS is made to use
an
alternate group.
Paul
>-Phil
>  
>

On Mon, 2004-09-20 at 12:27, Paul Nowoczynski wrote:
> hello..
> i was wondering if it is possible (and/or supported) to run multiple 
> lustre filesystems from the same OST disk partitions.
No.  You can run multiple OSTs on a single OSS node, but they must use
separate partitions.  Likewise, you can connect multiple OSS nodes to a
single multi-port RAID device (very common for failover), but they must
use different partitions.

If multiple nodes try to use the same physical disk blocks
simultaneously, catastrophic corruption will result.  Just imagine
mounting an ext3 file system from two nodes like that -- same thing.

-Phil

What about running two lov/mounts that point to the same OST pool?

ie lov1 attaches to ost1,ost2,ost3,ost4

and lov2 attaches to ost1,ost2

then you configure mount points for each lov.

I''m not sure why I want to do this yet, but is it possible.

I could have one lov do striping, and one doing mirroring I guess.

from the OSD side it just serves up objects, so talking to two lov''s
should not be a problem.  now the MDS might not like that!

Evan

On Mon, 2004-09-20 at 23:01, Phil Schwan wrote:
> On Mon, 2004-09-20 at 12:27, Paul Nowoczynski wrote:
> > hello..
> > i was wondering if it is possible (and/or supported) to run multiple 
> > lustre filesystems from the same OST disk partitions.
> 
> No.  You can run multiple OSTs on a single OSS node, but they must use
> separate partitions.  Likewise, you can connect multiple OSS nodes to a
> single multi-port RAID device (very common for failover), but they must
> use different partitions.
> 
> If multiple nodes try to use the same physical disk blocks
> simultaneously, catastrophic corruption will result.  Just imagine
> mounting an ext3 file system from two nodes like that -- same thing.
> 
> -Phil
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@lists.clusterfs.com
> https://lists.clusterfs.com/mailman/listinfo/lustre-discuss

On Tue, 2004-09-21 at 18:12, Evan Felix wrote:
> What about running two lov/mounts that point to the same OST pool?
> 
> ie lov1 attaches to ost1,ost2,ost3,ost4
> 
> and lov2 attaches to ost1,ost2
> 
> then you configure mount points for each lov.
> 
> I''m not sure why I want to do this yet, but is it possible.
> 
> I could have one lov do striping, and one doing mirroring I guess.
> 
> from the OSD side it just serves up objects, so talking to two
lov''s
> should not be a problem.  now the MDS might not like that!
It used to be possible, for precisely the reason that you mentioned --
it''s just an object store.

But it''s no longer possible, because the MDS manages the allocation of
objects now, and the MDSs wouldn''t have any way to coordinate with each
other.  And even if they could, lfsck would need to be rewritten, or it
would get very confused.

There are actually separate namespaces (object "groups" on the OSTs)
which could be used, but there is no way to configure an MDS to use a
group other than group 0 today.  Not rocket science, perhaps, but also
not trivial.

-Phil

hello..
i was wondering if it is possible (and/or supported) to run multiple 
lustre filesystems from the same OST disk partitions.

thanks
paul