Lustre discuss - May 2006 - LVS cluster with NFS export from Lustre

Another possibility: has anyone gotten liblustre to work on Solaris (and 
maybe Tru64 as well) in such a way that it could be used for someone''s 
home directory?

Is there still no guesstimate on when there might be a 1.6 (or beta 1.6) 
release and whether or not it might have kernel NFS support from the 
start?

I was hoping to adapt user-nfs-server to do what the Lustre kernel 2.4 
patch does with accessing the superblock of the Lustre filesystem itself 
to provide a true device id for export, but that is apparently not going 
to happen from user space (unless maybe I linked it with liblustre to 
query the FSID?).  Modifying it to handle an fsid option in /etc/exports 
is still a possibility, although I don''t really want to bang my head 
against that unless I know there''s no other option (and there
won''t be
one soon)...

Thanks,

Brent

On Thu, 9 Mar 2006, Brent A Nelson wrote:
> Has anyone done this with a 2.6 kernel? Unfs3 is not an option, as it
doesn''t
> re-export filesystems (and has other significant limitations), and Lustre 
> doesn''t yet interact with kernel NFS in 2.6.
>
> That leaves user-nfs-server, which can happily re-export a Lustre
filesystem.
> It has a 2GB file size limit and no locking, but I think we can live with 
> that.  However, when trying to put it behind an LVS director, it''s
clear that
> the device ids are different between nodes, even though they''re
exporting the
> same Lustre filesystem.
>
> So:
> 1) Has anyone already patched user-nfs-server to support an fsid= option
from
> exports (or something similar wher you can force it to use a particular 
> device id or can get it to look at the FSID from the Lustre filesystem
itself
> like the Lustre kernel NFS patch does)?
>
> 2) Is Lustre 1.6 likely imminent (if the roadmap is still accurate, it
would
> seem to be just around the corner) and will the first version perhaps
already
> have kernel NFS support for 2.6?
>
> 3) Has anyone gotten kernel NFS working with Lustre 1.4.x and a 2.6 kernel?
>
> 4) Is there some other tricky way such that the system can be told to
change
> the device id for a directory (then fstat/lstat in the user-nfs-server code
> would pick up on it and match across machines)?
>
> If none of the above, I''ll proceed with trying to do something
like option 1
> myself (the user-nfs-server code isn''t that complicated; I should
be able to
> figure out a workaround).
>
> Thanks,
>
> Brent Nelson
> Director of Computing
> Dept. of Physics
> University of Florida
>
> PS A lazy question (as in, I could just look myself, but hey, I''m
already
> asking the above, so why not toss this in as well): I noticed that there is
> some code in the new 1.4.6 release; are quotas implemented, now?
>

Has anyone done this with a 2.6 kernel? Unfs3 is not an option, as it 
doesn''t re-export filesystems (and has other significant limitations),
and
Lustre doesn''t yet interact with kernel NFS in 2.6.

That leaves user-nfs-server, which can happily re-export a Lustre 
filesystem.  It has a 2GB file size limit and no locking, but I think we 
can live with that.  However, when trying to put it behind an LVS 
director, it''s clear that the device ids are different between nodes,
even
though they''re exporting the same Lustre filesystem.

So:
1) Has anyone already patched user-nfs-server to support an fsid= option 
from exports (or something similar wher you can force it to use a 
particular device id or can get it to look at the FSID from the Lustre 
filesystem itself like the Lustre kernel NFS patch does)?

2) Is Lustre 1.6 likely imminent (if the roadmap is still accurate, it 
would seem to be just around the corner) and will the first version 
perhaps already have kernel NFS support for 2.6?

3) Has anyone gotten kernel NFS working with Lustre 1.4.x and a 2.6 
kernel?

4) Is there some other tricky way such that the system can be told to 
change the device id for a directory (then fstat/lstat in the 
user-nfs-server code would pick up on it and match across machines)?

If none of the above, I''ll proceed with trying to do something like
option
1 myself (the user-nfs-server code isn''t that complicated; I should be 
able to figure out a workaround).

Thanks,

Brent Nelson
Director of Computing
Dept. of Physics
University of Florida

PS A lazy question (as in, I could just look myself, but hey, I''m
already
asking the above, so why not toss this in as well): I noticed that there 
is some code in the new 1.4.6 release; are quotas implemented, now?