Hi I am a new luster user, trying to evaluate luster with few configuration. I am going through luster 1.6 Operation manual. But I am not able to understand which package should be installed on MDS, OSS, and client. Should I install all the packages on all three types of nodes? Please explain Best Regards Deval K. ==========================================================Privileged or confidential information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), please delete this message and kindly notify the sender by an emailed reply. Opinions, conclusions and other information in this message that do not relate to the official business of Progression and its associate entities shall be understood as neither given nor endorsed by them. ----------------------------------------------------------------------- Progression Infonet Private Limited, Gurgaon (Haryana), India Authorised dealer of PostMaster, by QuantumLink Communications Pvt. Ltd Get your free copy of PostMaster at http://www.postmaster.co.in/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20081008/ef8d426e/attachment.html
Deval kulshrestha wrote:> > Hi > > > > I am a new luster user, trying to evaluate luster with few > configuration. I am going through luster 1.6 Operation manual. But I > am not able to understand which package should be installed on MDS, > OSS , and client. > > Should I install all the packages on all three types of nodes? > > > > Please explain > > > > Best Regards > > Deval K >Lustre servers (MDS/OSS): kernel-lustre-smp // patched server kernel lustre-modules // Lustre kernel modules lustre // user space tools (server) lustre-ldiskfs // ldiskfs e2fsprogs // filesystem tools You can install all those RPMs on the client as well, but it is not necessary. Lustre clients (assuming you have the matching vendor kernel for the lustre-modules installed): lustre-client-modules // kernel modules for client lustre-client // user space (client) Kevin
Hi I have setup luster as following MGS/MDT/ost0/client- running on a x86_64 server with RHEL5.2 kaveri.hpc.com 2.6.18-53.1.14.el5_lustre.1.6.5smp OST1/client - running on x86_64 server with RHEL5.2 gangotri 2.6.18-53.1.14.el5_lustre.1.6.5smp Above setup is seems to be working fine, I have tested it with dd, it shows throughput as 54.5 MB/s Now I want to connect another Linux(RHEL4.4) client which is running on as 32 bit VM on ESX 3.5 I have installed following rpm on client lustre-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm lustre-modules-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm and modified NIS configuration(yp.conf, ypbind), hosts file and appended modprobe.conf with #Networking options, see /sys/module/lnet/parameters NO ../lnet/parameters dir options lnet networks=tcp0(eth0) But when I am trying to mount its giving following error [root at ganges lustre-client]# mount -t lustre kaveri at tcp0:/spfs /mnt/spfs mount.lustre: mount kaveri at tcp0:/spfs at /mnt/spfs failed: No such device Are the lustre modules loaded? Check /etc/modprobe.conf and /proc/filesystems Note ''alias lustre llite'' should be removed from modprobe.conf Is there any compatibility issue or any mis-configuration? I m not able to figure out what is wrong? Please comment Regards Deval -----Original Message----- From: Deval Kulshrestha [mailto:deval.kulshrestha at progression.com] Sent: Monday, October 13, 2008 1:57 PM To: ''lustre-discuss at lists.lustre.org'' Subject: HI Kevin Thanks for your reply. Now I can set it up. Regards Deval Message: 9 Date: Thu, 09 Oct 2008 05:46:27 -0600 From: Kevin Van Maren <Kevin.Vanmaren at Sun.COM> Subject: Re: [Lustre-discuss] Test setup configuration To: Deval kulshrestha <deval.kulshrestha at progression.com> Cc: lustre-discuss at lists.lustre.org Message-ID: <48EDEF13.60002 at sun.com> Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Deval kulshrestha wrote:> > Hi > > > > I am a new luster user, trying to evaluate luster with few > configuration. I am going through luster 1.6 Operation manual. But I > am not able to understand which package should be installed on MDS, > OSS , and client. > > Should I install all the packages on all three types of nodes? > > > > Please explain > > > > Best Regards > > Deval K >Lustre servers (MDS/OSS): kernel-lustre-smp // patched server kernel lustre-modules // Lustre kernel modules lustre // user space tools (server) lustre-ldiskfs // ldiskfs e2fsprogs // filesystem tools You can install all those RPMs on the client as well, but it is not necessary. Lustre clients (assuming you have the matching vendor kernel for the lustre-modules installed): lustre-client-modules // kernel modules for client lustre-client // user space (client) Kevin ------------------------------ _______________________________________________ Lustre-discuss mailing list Lustre-discuss at lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss End of Lustre-discuss Digest, Vol 33, Issue 12 ********************************************** No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.173 / Virus Database: 270.7.6/1715 - Release Date: 10/8/2008 7:19 PM ==========================================================Privileged or confidential information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), please delete this message and kindly notify the sender by an emailed reply. Opinions, conclusions and other information in this message that do not relate to the official business of Progression and its associate entities shall be understood as neither given nor endorsed by them. ----------------------------------------------------------------------- Progression Infonet Private Limited, Gurgaon (Haryana), India Authorised dealer of PostMaster, by QuantumLink Communications Pvt. Ltd Get your free copy of PostMaster at http://www.postmaster.co.in/
2008/10/14 Deval kulshrestha <deval.kulshrestha at progression.com>:> Is there any compatibility issue or any mis-configuration? I m not able to > figure out what is wrong?Like the messages from mount.lustre say, the lustre modules haven''t loaded/initialized properly. Look at dmesg, it should have some errors/info which will point you in the right direction
HI James Thanks for "dmesg" pointer, I checked dmesg but didn''t get any traces of luster. Do I also have to install patched kernel on luster client or just following rpm''s are sufficient. lustre-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm lustre-modules-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm Regards Deval Message: 5 Date: Tue, 14 Oct 2008 14:36:54 +0100 From: "James Braid" <jamesb at loreland.org> Subject: Re: [Lustre-discuss] Test setup configuration To: "Deval kulshrestha" <deval.kulshrestha at progression.com> Cc: lustre-discuss at lists.lustre.org Message-ID: <25446eb90810140636n569cf746o51899926cedb0e7d at mail.gmail.com> Content-Type: text/plain; charset=UTF-8 2008/10/14 Deval kulshrestha <deval.kulshrestha at progression.com>:> Is there any compatibility issue or any mis-configuration? I m not able to > figure out what is wrong?Like the messages from mount.lustre say, the lustre modules haven''t loaded/initialized properly. Look at dmesg, it should have some errors/info which will point you in the right direction ==========================================================Privileged or confidential information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), please delete this message and kindly notify the sender by an emailed reply. Opinions, conclusions and other information in this message that do not relate to the official business of Progression and its associate entities shall be understood as neither given nor endorsed by them. ----------------------------------------------------------------------- Progression Infonet Private Limited, Gurgaon (Haryana), India Authorised dealer of PostMaster, by QuantumLink Communications Pvt. Ltd Get your free copy of PostMaster at http://www.postmaster.co.in/
2008/10/15 Deval kulshrestha <deval.kulshrestha at progression.com>:> HI James > Thanks for "dmesg" pointer, I checked dmesg but didn''t get any traces of > luster.Does the command ''modprobe lustre'' work? It should load the lustre modules and print a bunch of messages into dmesg.
It seems that system does not find luster modules, it gives following error. [root at ganges ~]# modprobe lustre FATAL: Module lustre not found. RPM''s are installed successfully (checked using rpm -qil), Can you tell me what all rpm do I need to install on client only node Thanks and regards Deval K. -----Original Message----- From: James Braid [mailto:jamesb at loreland.org] Sent: Wednesday, October 15, 2008 2:48 PM To: Deval kulshrestha Cc: lustre-discuss at lists.lustre.org Subject: Re: Test setup configuration 2008/10/15 Deval kulshrestha <deval.kulshrestha at progression.com>:> HI James > Thanks for "dmesg" pointer, I checked dmesg but didn''t get any traces of > luster.Does the command ''modprobe lustre'' work? It should load the lustre modules and print a bunch of messages into dmesg. ==========================================================Privileged or confidential information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), please delete this message and kindly notify the sender by an emailed reply. Opinions, conclusions and other information in this message that do not relate to the official business of Progression and its associate entities shall be understood as neither given nor endorsed by them. ----------------------------------------------------------------------- Progression Infonet Private Limited, Gurgaon (Haryana), India Authorised dealer of PostMaster, by QuantumLink Communications Pvt. Ltd Get your free copy of PostMaster at http://www.postmaster.co.in/
Deval kulshrestha wrote:> It seems that system does not find luster modules, it gives following error. > > > > [root at ganges ~]# modprobe lustre > FATAL: Module lustre not found. > > RPM''s are installed successfully (checked using rpm -qil), > Can you tell me what all rpm do I need to install on client only node >Try depmod -a . tamas
I tried "depmod -a" but still same error even for modprobe and for original Error as following [root at ganges ~]# !mou mount -t lustre kaveri at tcp0:/spfs /mnt/spfs mount.lustre: mount kaveri at tcp0:/spfs at /mnt/spfs failed: No such device Are the lustre modules loaded? Check /etc/modprobe.conf and /proc/filesystems Note ''alias lustre llite'' should be removed from modprobe.conf Regards Deval -----Original Message----- From: Papp Tam?s [mailto:tompos at martos.bme.hu] Sent: Wednesday, October 15, 2008 3:42 PM To: Deval kulshrestha Cc: ''James Braid''; lustre-discuss at lists.lustre.org Subject: Re: [Lustre-discuss] Test setup configuration Deval kulshrestha wrote:> It seems that system does not find luster modules, it gives followingerror.> > > > [root at ganges ~]# modprobe lustre > FATAL: Module lustre not found. > > RPM''s are installed successfully (checked using rpm -qil), > Can you tell me what all rpm do I need to install on client only node >Try depmod -a . tamas ==========================================================Privileged or confidential information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), please delete this message and kindly notify the sender by an emailed reply. Opinions, conclusions and other information in this message that do not relate to the official business of Progression and its associate entities shall be understood as neither given nor endorsed by them. ----------------------------------------------------------------------- Progression Infonet Private Limited, Gurgaon (Haryana), India Authorised dealer of PostMaster, by QuantumLink Communications Pvt. Ltd Get your free copy of PostMaster at http://www.postmaster.co.in/
Deval kulshrestha wrote:> I tried "depmod -a" but still same error even for modprobe and for original > Error as following > > [root at ganges ~]# !mou > mount -t lustre kaveri at tcp0:/spfs /mnt/spfs > mount.lustre: mount kaveri at tcp0:/spfs at /mnt/spfs failed: No such device > Are the lustre modules loaded? > Check /etc/modprobe.conf and /proc/filesystems > Note ''alias lustre llite'' should be removed from modprobe.conf > >rpm -qa|grep lustre modprobe -l|grep lustre tamas
2008/10/14 Deval kulshrestha <deval.kulshrestha at progression.com>:> I have installed following rpm on client > lustre-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm > lustre-modules-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpmI just noticed you''re running 2.6.9. AFAIK Sun don''t ship patchless RHEL4 rpms. You will probably need to install the patched kernel or try building from source. There used to be some caveats with RHEL4 patchless clients, but I think they have been fixed in the later (RHEL4.5?) RHEL releases.
Error is often an indication of mis-matched Lustre modules and kernel version, so the modules are installed, but not where Linux can find them. (ie, kernel modules are in /lib/modules/`uname -r`, but lustre modules are in /lib/modules/<something_else>) I note that you installed the server kernel modules, so you need kernel 2.6.9_67.0.7.EL_lustre for them to work. See my original note below on patchless clients: use lustre-client-modules RPM, with the version matching the installed kernel. With the server module RPMs, you need the lustre kernel as well. Kevin Deval kulshrestha wrote:> Hi > > I have setup luster as following > > MGS/MDT/ost0/client- running on a x86_64 server with RHEL5.2 > kaveri.hpc.com 2.6.18-53.1.14.el5_lustre.1.6.5smp > > OST1/client - running on x86_64 server with RHEL5.2 > gangotri 2.6.18-53.1.14.el5_lustre.1.6.5smp > > Above setup is seems to be working fine, I have tested it with dd, it shows > throughput as 54.5 MB/s > > Now I want to connect another Linux(RHEL4.4) client which is running on as > 32 bit VM on ESX 3.5 > > I have installed following rpm on client > lustre-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm > lustre-modules-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm > > and modified NIS configuration(yp.conf, ypbind), hosts file and appended > modprobe.conf with > > #Networking options, see /sys/module/lnet/parameters NO ../lnet/parameters > dir > options lnet networks=tcp0(eth0) > > But when I am trying to mount its giving following error > > [root at ganges lustre-client]# mount -t lustre kaveri at tcp0:/spfs /mnt/spfs > mount.lustre: mount kaveri at tcp0:/spfs at /mnt/spfs failed: No such device > Are the lustre modules loaded? > Check /etc/modprobe.conf and /proc/filesystems > Note ''alias lustre llite'' should be removed from modprobe.conf > > Is there any compatibility issue or any mis-configuration? I m not able to > figure out what is wrong? > > Please comment > > Regards > Deval > > -----Original Message----- > From: Deval Kulshrestha [mailto:deval.kulshrestha at progression.com] > Sent: Monday, October 13, 2008 1:57 PM > To: ''lustre-discuss at lists.lustre.org'' > Subject: > > > HI Kevin > > Thanks for your reply. Now I can set it up. > > Regards > Deval > > Deval kulshrestha wrote: > >> Hi >> >> >> >> I am a new luster user, trying to evaluate luster with few >> configuration. I am going through luster 1.6 Operation manual. But I >> am not able to understand which package should be installed on MDS, >> OSS , and client. >> >> Should I install all the packages on all three types of nodes? >> >> >> >> Please explain >> >> >> >> Best Regards >> >> Deval K >> >> > > Lustre servers (MDS/OSS): > kernel-lustre-smp // patched server kernel > lustre-modules // Lustre kernel modules > lustre // user space tools (server) > lustre-ldiskfs // ldiskfs > e2fsprogs // filesystem tools > > You can install all those RPMs on the client as well, but it is not > necessary. > Lustre clients (assuming you have the matching vendor kernel for the > lustre-modules installed): > lustre-client-modules // kernel modules for client > lustre-client // user space (client) > > Kevin >
Hi Kevin You rightly said I am using RHEL4U4 (2.6.9-42.ELsmp) and installing luster rpm which are for 2.6.9-67. As James suggested I have installed kernel-lustre-smp-2.6.9-67.0.7.EL_lustre.1.6.5.i686.rpm lustre-modules-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm lustre-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm and rebooted the system with luster patched kernel [root at ganges lustre-client]# uname -a Linux ganges 2.6.9-67.0.7.EL_lustre.1.6.5smp #1 SMP Mon May 12 22:03:01 EDT 2008 i686 athlon i386 GNU/Linux And it''s working now(Thanks James). I am able to read/write/delete in Lustre based mount point But still need to validate one point I was using kernel 2.6.9-42(RHEL4U4) when I implemented patch which is for 2.6.9-67.0.7. it worked for different kernel. Is there anything like backward compatibility is working this for my setup? Ideally I should have used either 2.6.9-67.0.7. kernel based OS or Lustre source should be compiled to patched 2.6.9-42 kernel. Please help me to understand. Once again I appreciate your support. Thanks and regards Deval Kulshrestha -----Original Message----- From: Kevin.Vanmaren at Sun.COM [mailto:Kevin.Vanmaren at Sun.COM] Sent: Wednesday, October 15, 2008 4:59 PM To: Deval kulshrestha Cc: lustre-discuss at lists.lustre.org Subject: Re: [Lustre-discuss] Test setup configuration Error is often an indication of mis-matched Lustre modules and kernel version, so the modules are installed, but not where Linux can find them. (ie, kernel modules are in /lib/modules/`uname -r`, but lustre modules are in /lib/modules/<something_else>) I note that you installed the server kernel modules, so you need kernel 2.6.9_67.0.7.EL_lustre for them to work. See my original note below on patchless clients: use lustre-client-modules RPM, with the version matching the installed kernel. With the server module RPMs, you need the lustre kernel as well. Kevin Deval kulshrestha wrote:> Hi > > I have setup luster as following > > MGS/MDT/ost0/client- running on a x86_64 server with RHEL5.2 > kaveri.hpc.com 2.6.18-53.1.14.el5_lustre.1.6.5smp > > OST1/client - running on x86_64 server with RHEL5.2 > gangotri 2.6.18-53.1.14.el5_lustre.1.6.5smp > > Above setup is seems to be working fine, I have tested it with dd, itshows> throughput as 54.5 MB/s > > Now I want to connect another Linux(RHEL4.4) client which is running on as > 32 bit VM on ESX 3.5 > > I have installed following rpm on client > lustre-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm > lustre-modules-1.6.5-2.6.9_67.0.7.EL_lustre.1.6.5smp.i686.rpm > > and modified NIS configuration(yp.conf, ypbind), hosts file and appended > modprobe.conf with > > #Networking options, see /sys/module/lnet/parameters NO ../lnet/parameters > dir > options lnet networks=tcp0(eth0) > > But when I am trying to mount its giving following error > > [root at ganges lustre-client]# mount -t lustre kaveri at tcp0:/spfs /mnt/spfs > mount.lustre: mount kaveri at tcp0:/spfs at /mnt/spfs failed: No such device > Are the lustre modules loaded? > Check /etc/modprobe.conf and /proc/filesystems > Note ''alias lustre llite'' should be removed from modprobe.conf > > Is there any compatibility issue or any mis-configuration? I m not able to > figure out what is wrong? > > Please comment > > Regards > Deval > > -----Original Message----- > From: Deval Kulshrestha [mailto:deval.kulshrestha at progression.com] > Sent: Monday, October 13, 2008 1:57 PM > To: ''lustre-discuss at lists.lustre.org'' > Subject: > > > HI Kevin > > Thanks for your reply. Now I can set it up. > > Regards > Deval > > Deval kulshrestha wrote: > >> Hi >> >> >> >> I am a new luster user, trying to evaluate luster with few >> configuration. I am going through luster 1.6 Operation manual. But I >> am not able to understand which package should be installed on MDS, >> OSS , and client. >> >> Should I install all the packages on all three types of nodes? >> >> >> >> Please explain >> >> >> >> Best Regards >> >> Deval K >> >> > > Lustre servers (MDS/OSS): > kernel-lustre-smp // patched server kernel > lustre-modules // Lustre kernel modules > lustre // user space tools (server) > lustre-ldiskfs // ldiskfs > e2fsprogs // filesystem tools > > You can install all those RPMs on the client as well, but it is not > necessary. > Lustre clients (assuming you have the matching vendor kernel for the > lustre-modules installed): > lustre-client-modules // kernel modules for client > lustre-client // user space (client) > > Kevin >==========================================================Privileged or confidential information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), please delete this message and kindly notify the sender by an emailed reply. Opinions, conclusions and other information in this message that do not relate to the official business of Progression and its associate entities shall be understood as neither given nor endorsed by them. ----------------------------------------------------------------------- Progression Infonet Private Limited, Gurgaon (Haryana), India Authorised dealer of PostMaster, by QuantumLink Communications Pvt. Ltd Get your free copy of PostMaster at http://www.postmaster.co.in/