We are pre-Lustre right now and have some questions. Currently our cluster uses LDAP+automount to mount user''s home directories from our file server. Once we go Lustre, is there any sort of modification to LDAP or automount(besides the installation of the Lustre client programs) needed? -- Personally, I liked the university. They gave us money and facilities, we didn''t have to produce anything! You''ve never been out of college! You don''t know what it''s like out there! I''ve worked in the private sector. They expect results. -Ray Ghostbusters -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100527/a95f62d3/attachment.html
On Thu, 2010-05-27 at 13:24 -0500, David Noriega wrote:> We are pre-Lustre right now and have some questions. Currently our > cluster uses LDAP+automount to mount user''s home directories from our > file server. Once we go Lustre, is there any sort of modification to > LDAP or automount(besides the installation of the Lustre client > programs) needed?There {i|wa}s a known issue with Lustre and automount discussed on this list previously. I don''t know if it ever got fixed or if there was a bug filed on it. I suspect there was. You could search the list archives and/or bugzilla for details. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100527/534c0a99/attachment.bin
There have been some reports of problems with automount and Lustre that have never been tracked down. If someone with automount experience and config, and time to track this down could investigate I''m sure we could work it out. Cheers, Andreas On 2010-05-27, at 12:24, David Noriega <tsk133 at my.utsa.edu> wrote:> We are pre-Lustre right now and have some questions. Currently our > cluster uses LDAP+automount to mount user''s home directories from > our file server. Once we go Lustre, is there any sort of > modification to LDAP or automount(besides the installation of the > Lustre client programs) needed? > > -- > Personally, I liked the university. They gave us money and > facilities, we didn''t have to produce anything! You''ve never been > out of college! You don''t know what it''s like out there! I''ve worked > in the private sector. They expect results. -Ray Ghostbusters > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
On Thu, May 27, 2010 at 12:50:15PM -0600, Andreas Dilger wrote:> There have been some reports of problems with automount and Lustre > that have never been tracked down. If someone with automount > experience and config, and time to track this down could investigate > I''m sure we could work it out.The autofs that comes with RHEL5 won''t mount Lustre. We got it working with autofs-5.0.3-36, from some version of Fedora. Later versions of autofs should also work. The fix to autofs is almost trivial. A function scans the mount command to make sure it contains just "legal" characters. To get Lustre to automount, it needs "@" in it''s list of such. We did find that the automounter sometimes failed to remove the Lustre record from /etc/mtab on unmounting the file system. That would cause subsequent remounts to fail. Another easy fix: link /etc/mtab to /proc/mounts. David S.> > Cheers, Andreas > > On 2010-05-27, at 12:24, David Noriega <tsk133 at my.utsa.edu> wrote: > > > We are pre-Lustre right now and have some questions. Currently our > > cluster uses LDAP+automount to mount user''s home directories from > > our file server. Once we go Lustre, is there any sort of > > modification to LDAP or automount(besides the installation of the > > Lustre client programs) needed? > > > > -- > > Personally, I liked the university. They gave us money and > > facilities, we didn''t have to produce anything! You''ve never been > > out of college! You don''t know what it''s like out there! I''ve worked > > in the private sector. They expect results. -Ray Ghostbusters > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
On Thursday 27 May 2010, David Simas wrote:> On Thu, May 27, 2010 at 12:50:15PM -0600, Andreas Dilger wrote: > > There have been some reports of problems with automount and Lustre > > that have never been tracked down. If someone with automount > > experience and config, and time to track this down could investigate > > I''m sure we could work it out. > > The autofs that comes with RHEL5 won''t mount Lustre. We got > it working with autofs-5.0.3-36, from some version of Fedora. > Later versions of autofs should also work. The fix to autofs > is almost trivial. A function scans the mount command to > make sure it contains just "legal" characters. To get Lustre > to automount, it needs "@" in it''s list of such.I doubt that you cannot get it working. At my previous job we used NIS based automounter for almost everything including Lustre. All based on Debian, so I''m not absolutely sure about RedHat. However, it already did work with the very old Debian Sarge. The simple trick we had to do was to escape the "@", so to use "\@".> > We did find that the automounter sometimes failed to remove > the Lustre record from /etc/mtab on unmounting the file system. > That would cause subsequent remounts to fail. Another easy > fix: link /etc/mtab to /proc/mounts.That also happens sometimes without automounter. Cheers, Bernd -- Bernd Schubert DataDirect Networks
David Simas wrote:> On Thu, May 27, 2010 at 12:50:15PM -0600, Andreas Dilger wrote: > >> There have been some reports of problems with automount and Lustre >> that have never been tracked down. If someone with automount >> experience and config, and time to track this down could investigate >> I''m sure we could work it out. >> > > The autofs that comes with RHEL5 won''t mount Lustre. We got > it working with autofs-5.0.3-36, from some version of Fedora. > Later versions of autofs should also work. The fix to autofs > is almost trivial. A function scans the mount command to > make sure it contains just "legal" characters. To get Lustre > to automount, it needs "@" in it''s list of such. > > We did find that the automounter sometimes failed to remove > the Lustre record from /etc/mtab on unmounting the file system. > That would cause subsequent remounts to fail. Another easy > fix: link /etc/mtab to /proc/mounts. > >Also, remember that you can''t mount Lustre subdirectories. That is, you can mount your Lustre filesystem as, say, /home, but you can''t mount /home/username. An approach that we are testing (but haven''t tried in production yet) was suggested by an earlier post from Andreas Dilger, and involves two automounts. The first mounts the base Lustre filesystem(s) somewhere (say, /lustre) as a direct mount, the second is /etc/auto.home and looks like this: * -bind :/lustre/& In our case we have an executable map that generates the mount line based on the contents of the description field in LDAP (which indicates which Lustre FS contains the home - we have two), but the principle is the same.> David S. > > >> Cheers, Andreas >> >> On 2010-05-27, at 12:24, David Noriega <tsk133 at my.utsa.edu> wrote: >> >> >>> We are pre-Lustre right now and have some questions. Currently our >>> cluster uses LDAP+automount to mount user''s home directories from >>> our file server. Once we go Lustre, is there any sort of >>> modification to LDAP or automount(besides the installation of the >>> Lustre client programs) needed? >>> >>> -- >>> Personally, I liked the university. They gave us money and >>> facilities, we didn''t have to produce anything! You''ve never been >>> out of college! You don''t know what it''s like out there! I''ve worked >>> in the private sector. They expect results. -Ray Ghostbusters >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >>> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100527/d51a1164/attachment-0001.html
> Also, remember that you can''t mount Lustre subdirectories. That is, you > can mount your Lustre filesystem as, say, /home, but you can''t mount > /home/username.> An approach that we are testing (but haven''t tried in production yet) > was suggested by an earlier post from Andreas Dilger, and involves two > automounts. The first mounts the base Lustre filesystem(s) somewhere > (say, /lustre) as a direct mount, the second is /etc/auto.home and looks > like this: > * -bind :/lustre/&Or one could use ''amd'' from ''am-utils'' instead of ''autofs'' which is the easy-and-quick automounter. ''amd'' is what I''d use in most non-trivial cases as it can easily mount a filesystem and create a symlink to some part of it. Not sure latest versions have Lustre support.
On Thursday 27 May 2010, Fraser McCrossan wrote:> David Simas wrote: > > On Thu, May 27, 2010 at 12:50:15PM -0600, Andreas Dilger wrote: > >> There have been some reports of problems with automount and Lustre > >> that have never been tracked down. If someone with automount > >> experience and config, and time to track this down could investigate > >> I''m sure we could work it out. > > > > The autofs that comes with RHEL5 won''t mount Lustre. We got > > it working with autofs-5.0.3-36, from some version of Fedora. > > Later versions of autofs should also work. The fix to autofs > > is almost trivial. A function scans the mount command to > > make sure it contains just "legal" characters. To get Lustre > > to automount, it needs "@" in it''s list of such. > > > > We did find that the automounter sometimes failed to remove > > the Lustre record from /etc/mtab on unmounting the file system. > > That would cause subsequent remounts to fail. Another easy > > fix: link /etc/mtab to /proc/mounts. > > Also, remember that you can''t mount Lustre subdirectories. That is, you > can mount your Lustre filesystem as, say, /home, but you can''t mount > /home/username.Without having checked the code, I think it should be fairly simple to add support for that in mount.lustre: - Cut off the directory from the filesystem name - Mount Lustre into a temporary directory - Bind mount lustre into the target directory The most difficult part will be to write a single entry into /etc/mtab> > An approach that we are testing (but haven''t tried in production yet) > was suggested by an earlier post from Andreas Dilger, and involves two > automounts. The first mounts the base Lustre filesystem(s) somewhere > (say, /lustre) as a direct mount, the second is /etc/auto.home and looks > like this: > > * -bind :/lustre/& > > In our case we have an executable map that generates the mount line > based on the contents of the description field in LDAP (which indicates > which Lustre FS contains the home - we have two), but the principle is > the same.That should work. The disadvantage of bind mounts is that ''lfs'' does not recognize it as type lustre and therefore all those nice lfs subcommands will not work. Cheers, Bernd -- Bernd Schubert DataDirect Networks
On 2010-05-28, at 09:47, Bernd Schubert wrote:> Without having checked the code, I think it should be fairly simple to add > support for that in mount.lustre: > > - Cut off the directory from the filesystem name > > - Mount Lustre into a temporary directory > > - Bind mount lustre into the target directory > > The most difficult part will be to write a single entry into /etc/mtabAfter the bind mount is created, I think it should be possible to unmount the original mount. Of course, having built-in support for this would be even better. I always thought it would be possible to do this by changing the client mount to first get the Lustre ROOT (if it isn''t already mounted), and then walk the rest of the path and instantiate the specified directory. The other possibility is to do this at the server, but it would need a change to the network protocol (i.e. passing the subdirectory pathname as part of the GETINFO RPC) and have the MDS only return the FID of the subdirectory to the client.>> An approach that we are testing (but haven''t tried in production yet) >> was suggested by an earlier post from Andreas Dilger, and involves two >> automounts. The first mounts the base Lustre filesystem(s) somewhere >> (say, /lustre) as a direct mount, the second is /etc/auto.home and looks >> like this: >> >> * -bind :/lustre/& >> >> In our case we have an executable map that generates the mount line >> based on the contents of the description field in LDAP (which indicates >> which Lustre FS contains the home - we have two), but the principle is >> the same. > > That should work. The disadvantage of bind mounts is that ''lfs'' does not > recognize it as type lustre and therefore all those nice lfs subcommands will > not work.Another way to have lfs detect Lustre filesystems is to use statfs() on the filesystems and check the magic number against LL_SUPER_MAGIC instead of using the filesystem mount type. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc.