On Nov 01, 2004 16:00 -0800, John Chodera wrote:> I have a (perhaps rather naive) question regarding the use of Lustre with
> small clusters:
>
> Suppose we have a small cluster of, say, 30-50 dual-CPU nodes connected
> via a gigabit enthernet swtich, mostly running serial applications or
> parallel applications with minimal communication. For a small cluster
> like this which does not need to serve clients other than nodes within the
> cluster, is there any real barrier to using each node and using each node
> as both an OST and a compute client? I noticed in an earlier posting that
> there was a condition where the client was writing to an OST running on
> the same machine, and memory issues might cause a deadlock, but provided
> this case was extremely unlikely, would there be large performance issues
> with such a strategy?
There are a few issues with this sort of setup currently:
1) The memory deadlock problem you mention. If there isn''t only light
IO and low memory pressure, this scenario is relatively unlinkely,
but this isn''t something to depend on in a production environment.
We are currently working toward making this sort of configuration
more stable but as yet none of our customers are running in this mode.
2) The Lustre OSTs must be started before the MDS, and the clients need
to be started after the MDS, which makes theis configuration a bit
complex. There is the functionality for doing this in the lconf tool
(--maxlevel 50 would start only the OSTs). If the OSTs were started
first, and the client (mountpoint) was done with zeroconf after the
MDS had started that wouldn''t be so bad.
3) There is no mechanism for having clients write to their local OST
by default to avoid network traffic. This in itself isn''t a
showstopper,
but depending on your network it would likely slow things down.
Support for this is only in the design stage.
Cheers, Andreas
--
Andreas Dilger