Does a /etc/passwd with all the filesystem users UID''s required only on the MDS ? Or does the OST''s need them also? Testing for me shows only the MDS, but I could be wrong. We don''t use LDAP or anything like that at the moment for UID GID mapping. Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985
On Fri, 2008-04-11 at 09:01 -0400, Brock Palen wrote:> Does a /etc/passwd with all the filesystem users UID''s required only > on the MDS ? Or does the OST''s need them also?MDS only. But this is not needed either if you mds# echo NONE > proc/fs/lustre/mds/<fsname>-MDT0000/group_upcall Then you only need uid/gid to be available on all clients. /Jakob
On Apr 11, 2008, at 6:07 AM, Jakob Goldbach wrote:> > On Fri, 2008-04-11 at 09:01 -0400, Brock Palen wrote: >> Does a /etc/passwd with all the filesystem users UID''s required only >> on the MDS ? Or does the OST''s need them also? > > MDS only. But this is not needed either if you > > mds# echo NONE > proc/fs/lustre/mds/<fsname>-MDT0000/group_upcall > > Then you only need uid/gid to be available on all clients. > > /JakobJakob is correct. The group_upcall is needed if you want to support large numbers of secondary groups. We have users at LLNL that belong to >16 groups, and the group_upcall is needed to support permissions access with all those groups. I think by default lustre will only check the first two groups you belong to. If your users don''t use additional groups, then you can do as Jakob suggested. -Marc ---- D. Marc Stearman LC Lustre Administration Lead marc at llnl.gov 925.423.9670 Pager: 1.888.203.0641
Hi, Do you see any problem in having each compute node, within a grid, acting as an OSS server via the separate IB channel on the fabric? My compute nodes have built-in raid controllers. Any feedback and comments are really appreciated. Regards, -Peter
On Fri, 2008-04-11 at 20:29 +0400, Peter Avakian wrote:> > Do you see any problem in having each compute node, within a grid, > acting as an OSS server via the separate IB channel on the fabric? My > compute nodes have built-in raid controllers.If by compute nodes you mean Lustre clients, then yes, this is a problem and an unsupported configuration. The reason is because memory pressures on a client/OSS machine can cause a deadlock. The client tries to flush pages to an OST to relieve memory pressure. An OST needs to allocate memory in order to process page flushes from a client. If a client trying to relieve memory pressure tries to flush pages to an OST on the same node, the OST will get failures trying to allocate memory (which is already under pressure) to fulfill the request from the client. Deadlock. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080411/59425d72/attachment-0002.bin
By compute node if you mean a node which is a part of a compute cluster(parallel computing) then it would be a very bad idea. I tried it on my 16 node cluster using both IB and ethernet.It always showed that nodes which were running OSS were over utilized and they were since user''s reported this fact. But in my case there was no raid controller.I was using a partition of the OS disk. You may very well try it in experimental environment and run some benchmarking test''s for both lustre(using compute node OSS and then other OSS) and lmbench , do it mix and match and compare the results.If you are successful then please repost the results. On Fri, Apr 11, 2008 at 9:59 PM, Peter Avakian <Peter.Avakian at sun.com> wrote:> Hi, > > > Do you see any problem in having each compute node, within a grid, > acting as an OSS server via the separate IB channel on the fabric? My > compute nodes have built-in raid controllers. > Any feedback and comments are really appreciated. > Regards, > -Peter > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-- Regards-- Rishi Pathak -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080414/677cc252/attachment-0002.html
On Fri, Apr 11, 2008 at 10:34 AM, Brian J. Murrell <Brian.Murrell at sun.com> wrote:> On Fri, 2008-04-11 at 20:29 +0400, Peter Avakian wrote: > > > > Do you see any problem in having each compute node, within a grid, > > acting as an OSS server via the separate IB channel on the fabric? My > > compute nodes have built-in raid controllers. > > If by compute nodes you mean Lustre clients, then yes, this is a problem > and an unsupported configuration. The reason is because memory > pressures on a client/OSS machine can cause a deadlock.What if either the compute node or (most likely) the OSS was in a VM (and make sure to not overcommit processors)? Chris> > The client tries to flush pages to an OST to relieve memory pressure. > An OST needs to allocate memory in order to process page flushes from a > client. If a client trying to relieve memory pressure tries to flush > pages to an OST on the same node, the OST will get failures trying to > allocate memory (which is already under pressure) to fulfill the request > from the client. Deadlock. > > b. > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > >