Hello everybody, at the following pages: http://www.rit.edu/~rc/docs/Survey_of_Clustered_Parallel_File_Systems_004_LANL.ppt http://www.intel.com/cd/ids/developer/asmo-na/eng/dc/tools/threading/238284.htm?page=2 I read: "[...] Currently, one additional drawback to Lustre is that a Lustre client cannot be on a server that is providing OSTs. This solution is being worked on and may be available soon. [...]" Is that currently true? I mean, the client cannot be run on an OST server yet? Thank you, Marco
I believe this is still the case. It may work but your mileage will vary. On Dec 13, 2007, at 11:58 AM, Marco wrote:> Hello everybody, > > at the following pages: > > http://www.rit.edu/~rc/docs/Survey_of_Clustered_Parallel_File_Systems_004_LANL.ppt > http://www.intel.com/cd/ids/developer/asmo-na/eng/dc/tools/threading/238284.htm?page=2 > > I read: > > "[...] Currently, one additional drawback to Lustre is that a Lustre > client cannot be on a server that is providing OSTs. This solution is > being worked on and may be available soon. [...]" > > Is that currently true? I mean, the client cannot be run on an OST > server yet? > > Thank you, > Marco > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discussAaron Knister Associate Systems Administrator/Web Designer Center for Research on Environment and Water (301) 595-7001 aaron at iges.org
On Thu, 2007-12-13 at 17:58 +0100, Marco wrote:> > "[...] Currently, one additional drawback to Lustre is that a Lustre > client cannot be on a server that is providing OSTs.Well it can, it is just not (yet?) a supported configuration. Consider a machine with a Lustre client on it which is experiencing memory pressure. One way to deal with that is to start flushing memory pages to disk where for a Lustre client disk can mean to an OST. However if the OST it''s trying to flush to is on the same node, that OST will cause additional memory pressure in trying to allocate memory to receive the pages from the client. If the OST can''t allocate memory then the client can''t flush pages to free up some of it''s usage and you now have a deadlock. b.
* Brian J. Murrell <Brian.Murrell at Sun.COM> [2007 12 13, 16:16]:> Consider a machine with a Lustre client on it which is experiencing > memory pressure. One way to deal with that is to start flushing memory > pages to disk where for a Lustre client disk can mean to an OST.Ok, as you said, *can* mean to an OST. Let''s suppose a host is running the OST and the client, but it has a local (not managed by Lustre) swap. Let''s say Lustre is used to provide access to a large dataset on a SAN, but the system itself is stored on local disks on each node. In this case, if the client is experiencing memory pressure, it will just flush to the local disk, and it will continue doing so until there is swap available... or not? Am I missing something? :) Thanks for your answers! Marco
Hello! On Dec 14, 2007, at 4:31 AM, Marco wrote:> * Brian J. Murrell <Brian.Murrell at Sun.COM> [2007 12 13, 16:16]: >> Consider a machine with a Lustre client on it which is experiencing >> memory pressure. One way to deal with that is to start flushing >> memory >> pages to disk where for a Lustre client disk can mean to an OST. > Ok, as you said, *can* mean to an OST. Let''s suppose a host is running > the OST and the client, but it has a local (not managed by Lustre) > swap. > Let''s say Lustre is used to provide access to a large dataset on a > SAN, > but the system itself is stored on local disks on each node. > In this case, if the client is experiencing memory pressure, it will > just flush to the local disk, and it will continue doing so until > there > is swap available... or not?Only userspace pages could be swapped out. Dirty fs data is not swapped out, instead it is attempted to be written by calling FS. Pathologic scenarios like "mmap a file region entire memory in size, and dirty all of it" creates enormous amount of dirty pages in memory that could not be swapped out and lustre need memory to flush out dirty pages too. Of course there is some kernel reserve for those purposes, but in some cases it might be not enough (I heard reports that 2.6 kernels behave better in these conditions). Bye, Oleg