Alessandro Luccaroni - Diennea
2010-Jul-08 09:54 UTC
[Lustre-discuss] building lustre-client rpms
Hello, in the last few days i have installed a HA lustre cluster for testing purpouse. Everything goes smooth with the patched kernel (kernel-2.6.18-164.11.1.0.1.el5_lustre.1.8.3.x86_64.rpm) but i can''t find a way to mount the filsesystem with the patchless rpms: [root at lustreclient2 mnt]# mount -t lustre lustremds02 at tcp0:/lustre /mnt/lustre mount.lustre: mount lustremds02 at tcp0:/lustre at /mnt/lustre failed: No such device Are the lustre modules loaded? Check /etc/modprobe.conf and /proc/filesystems Note ''alias lustre llite'' should be removed from modprobe.conf In /proc/filesystems i can''t find the "lustre" row. [root at lustreclient2 src]# uname -r 2.6.18-164.el5 [root at lustreclient2 mnt]# ls -l /lib/modules/ drwxr-xr-x 6 root root 4096 Jul 7 17:57 2.6.18-164.11.1.0.1.el5 drwxr-xr-x 6 root root 4096 Jul 7 17:58 2.6.18-164.el5 I think the problem is the different kernel revision, but i can''t find on the Red Hat Network the 2.6.18_164.11.1.0.1.el5 kernel; the closest one is kernel-2.6.18-164.11.1.el5.x86_64.rpm. I have also tried to build the client rpm from source but in the /usr/src/redhat/RPMS/x86_64/ directory i can only find the server-side rpms (lustre, lustre-debuginfo, lustre-modules, lustre-source, lustre tests): cd /usr/src/lustre-1.8.3 ./configure --with-linux=/usr/src/kernels/2.6.18-194.8.1.el5-x86_64 make rpms Sure I''m missing something, can someone help me or point me to some documentation about building the patchless rpms from source?
This should all be documented in the manual. When you run "configure", it will determine whether to build the client or the server based on whether it detects a Lustre-patched kernel -- there is only one Lustre source. Installing the RPMs for the wrong kernel (2.6.18-164.el52.6.18-164.11.1.0.1.el5) won''t help. Building on the client should be as easy as: # cd /usr/src/lustre-1.8.3 # ./configure --with-linux=/lib/modules/`uname -r`/build # make rpms (but you may have to add a few RPMs required by the build process) The problem is you pointed your configure at 2.6.18-194.8.1.el5-x86_64, instead of what you are running, which is 2.6.18-164.el5 Alessandro Luccaroni - Diennea wrote:> Hello, > in the last few days i have installed a HA lustre cluster for testing > purpouse. > Everything goes smooth with the patched kernel > (kernel-2.6.18-164.11.1.0.1.el5_lustre.1.8.3.x86_64.rpm) but i can''t > find a way to mount the filsesystem with the patchless rpms: > > [root at lustreclient2 mnt]# mount -t lustre lustremds02 at tcp0:/lustre > /mnt/lustre > mount.lustre: mount lustremds02 at tcp0:/lustre at /mnt/lustre failed: No > such device > Are the lustre modules loaded? > Check /etc/modprobe.conf and /proc/filesystems > Note ''alias lustre llite'' should be removed from modprobe.conf > > In /proc/filesystems i can''t find the "lustre" row. > > [root at lustreclient2 src]# uname -r > 2.6.18-164.el5 > > [root at lustreclient2 mnt]# ls -l /lib/modules/ > drwxr-xr-x 6 root root 4096 Jul 7 17:57 2.6.18-164.11.1.0.1.el5 > drwxr-xr-x 6 root root 4096 Jul 7 17:58 2.6.18-164.el5 > > I think the problem is the different kernel revision, but i can''t find > on the Red Hat Network the 2.6.18_164.11.1.0.1.el5 kernel; the closest > one is kernel-2.6.18-164.11.1.el5.x86_64.rpm. > > > I have also tried to build the client rpm from source but in the > /usr/src/redhat/RPMS/x86_64/ directory i can only find the server-side > rpms (lustre, lustre-debuginfo, lustre-modules, lustre-source, lustre > tests): > > cd /usr/src/lustre-1.8.3 > ./configure --with-linux=/usr/src/kernels/2.6.18-194.8.1.el5-x86_64 > make rpms > > Sure I''m missing something, can someone help me or point me to some > documentation about building the patchless rpms from source? > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
On Thu, 2010-07-08 at 11:54 +0200, Alessandro Luccaroni - Diennea wrote:> > [root at lustreclient2 src]# uname -r > 2.6.18-164.el5We have never (directly[1]) supported this particular kernel revision. It looks rather old and likely we have supported an updated revision of it. Currently I see that we have packages for kernels: 2.6.18_164.6.1.0.1.el5 (1.8.2 on OEL5) 2.6.18_164.11.1.el5 (1.8.2 on RHEL5) 2.6.18_164.11.1.0.1.el5 (1.8.3 on OEL5) 2.6.18_164.11.1.el5 (1.8.3 on RHEL5)> [root at lustreclient2 mnt]# ls -l /lib/modules/ > drwxr-xr-x 6 root root 4096 Jul 7 17:57 2.6.18-164.11.1.0.1.el5 > drwxr-xr-x 6 root root 4096 Jul 7 17:58 2.6.18-164.el5 > > I think the problem is the different kernel revision, but i can''t find > on the Red Hat Network the 2.6.18_164.11.1.0.1.el5 kernel;You probably won''t. 2.6.18_164.11.1.0.1.el5 looks like an OEL5 kernel, not a RHEL5 kernel. I''d imagine you are actually using RHEL5 and you grabbed the OEL5 packages from the download site.> the closest > one is kernel-2.6.18-164.11.1.el5.x86_64.rpm.Which you can see above, I supported by our 1.8.3 packages.> I have also tried to build the client rpm from source but in the > /usr/src/redhat/RPMS/x86_64/ directory i can only find the server-side > rpms (lustre, lustre-debuginfo, lustre-modules, lustre-source, lustre > tests):Those are in fact client packages, despite the lack of "client" in their name. This is a defect in our RPM packaging which I hope will be fixed by the landing of some patches I am working on in another bug. It''s possible that I have not fixed that one though. Please feel free to file a bug in any case.> cd /usr/src/lustre-1.8.3 > ./configure --with-linux=/usr/src/kernels/2.6.18-194.8.1.el5-x86_64 > make rpmsWell, you only want to do this if your target kernel will be 2.6.18-194.8.1.el5. But if your target kernel is in fact 2.6.18-164.el5 as you wrote above, you want to install the kernel-devel for 2.6.18-164.el5 and point to /usr/src/kernels/2.6.18-164.el5. b. [1] If you are familiar with how weak modules works, you might be able to get the modules from the lustre client modules package for 2.6.18_164.11.1.el5 to work with 2.6.18-164.el5. Our packages don''t yet support weak modules, formally, but if you know how they work, you can do it manually. A future release will support weak modules, but that will depend on Redhat also adding more symbols to their kABI whitelist. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100708/76b23f2c/attachment.bin
Alessandro Luccaroni - Diennea
2010-Jul-08 14:48 UTC
[Lustre-discuss] building lustre-client rpms
First of all thank you for the help. The "configure string" was from another test server, sorry about the mess. It seems like the "configure" always detect a patched kernel. I tested it on a patched client node rebooted into the standard redhat kernel, on a RHEL 5.5 and on a fresh RHEL 5.4 installation: [root at lustreclient3 lustre-1.8.3]# ./configure --with-linux=/lib/modules/`uname -r`/build [root at lustreclient3 lustre-1.8.3]# make rpms [root at lustreclient3 lustre-1.8.3]# ls -l /usr/src/redhat/RPMS/x86_64/ total 38656 -rw-r--r-- 1 root root 4712160 Jul 8 16:09 lustre-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm -rw-r--r-- 1 root root 11097838 Jul 8 16:10 lustre-debuginfo-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm -rw-r--r-- 1 root root 15040050 Jul 8 16:10 lustre-modules-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm -rw-r--r-- 1 root root 5509072 Jul 8 16:10 lustre-source-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm -rw-r--r-- 1 root root 3152499 Jul 8 16:10 lustre-tests-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm [root at lustreclient3 lustre-1.8.3]# uname -a Linux lustreclient3.diennea.lan 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux The build process doesn''t request additional rpms.> This should all be documented in the manual. > > > When you run "configure", it will determine whether to build the > client or the server based on > whether it detects a Lustre-patched kernel -- there is only one Lustre > source. > > Installing the RPMs for the wrong kernel > (2.6.18-164.el52.6.18-164.11.1.0.1.el5) won''t help. > > Building on the client should be as easy as: > # cd /usr/src/lustre-1.8.3 > # ./configure --with-linux=/lib/modules/`uname -r`/build > # make rpms > (but you may have to add a few RPMs required by the build process) > > The problem is you pointed your configure at > 2.6.18-194.8.1.el5-x86_64, instead of what you are running, > which is 2.6.18-164.el5 > > > > Alessandro Luccaroni - Diennea wrote: >> Hello, >> in the last few days i have installed a HA lustre cluster for testing >> purpouse. >> Everything goes smooth with the patched kernel >> (kernel-2.6.18-164.11.1.0.1.el5_lustre.1.8.3.x86_64.rpm) but i can''t >> find a way to mount the filsesystem with the patchless rpms: >> >> [root at lustreclient2 mnt]# mount -t lustre lustremds02 at tcp0:/lustre >> /mnt/lustre >> mount.lustre: mount lustremds02 at tcp0:/lustre at /mnt/lustre failed: >> No such device >> Are the lustre modules loaded? >> Check /etc/modprobe.conf and /proc/filesystems >> Note ''alias lustre llite'' should be removed from modprobe.conf >> >> In /proc/filesystems i can''t find the "lustre" row. >> >> [root at lustreclient2 src]# uname -r >> 2.6.18-164.el5 >> >> [root at lustreclient2 mnt]# ls -l /lib/modules/ >> drwxr-xr-x 6 root root 4096 Jul 7 17:57 2.6.18-164.11.1.0.1.el5 >> drwxr-xr-x 6 root root 4096 Jul 7 17:58 2.6.18-164.el5 >> >> I think the problem is the different kernel revision, but i can''t >> find on the Red Hat Network the 2.6.18_164.11.1.0.1.el5 kernel; the >> closest one is kernel-2.6.18-164.11.1.el5.x86_64.rpm. >> >> >> I have also tried to build the client rpm from source but in the >> /usr/src/redhat/RPMS/x86_64/ directory i can only find the >> server-side rpms (lustre, lustre-debuginfo, lustre-modules, >> lustre-source, lustre tests): >> >> cd /usr/src/lustre-1.8.3 >> ./configure --with-linux=/usr/src/kernels/2.6.18-194.8.1.el5-x86_64 >> make rpms >> >> Sure I''m missing something, can someone help me or point me to some >> documentation about building the patchless rpms from source? >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss
On Thu, 2010-07-08 at 16:48 +0200, Alessandro Luccaroni - Diennea wrote:> > It seems like the "configure" always detect a patched kernel.It won''t if you point it at the kernel-devel for a non-patched kernel.> I tested it on a patched client node rebooted into the standard redhat > kernel, on a RHEL 5.5 and on a fresh RHEL 5.4 installation:Not really sure what you are saying there. Sorry.> [root at lustreclient3 lustre-1.8.3]# ./configure > --with-linux=/lib/modules/`uname -r`/build > [root at lustreclient3 lustre-1.8.3]# make rpms > [root at lustreclient3 lustre-1.8.3]# ls -l /usr/src/redhat/RPMS/x86_64/ > total 38656 > -rw-r--r-- 1 root root 4712160 Jul 8 16:09 lustre-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm > -rw-r--r-- 1 root root 11097838 Jul 8 16:10 lustre-debuginfo-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm > -rw-r--r-- 1 root root 15040050 Jul 8 16:10 lustre-modules-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm > -rw-r--r-- 1 root root 5509072 Jul 8 16:10 lustre-source-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm > -rw-r--r-- 1 root root 3152499 Jul 8 16:10 lustre-tests-1.8.3-2.6.18_164.el5_201007081603.x86_64.rpm > [root at lustreclient3 lustre-1.8.3]# uname -a > Linux lustreclient3.diennea.lan 2.6.18-164.el5 #1 SMP Tue Aug 18 > 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/LinuxThose are all patchless client RPMs despite the lack of the tag "client" in them, as I explained in my previous message. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100708/08d3cafd/attachment.bin
Alessandro Luccaroni - Diennea
2010-Jul-08 15:41 UTC
[Lustre-discuss] building lustre-client rpms
> You probably won''t. 2.6.18_164.11.1.0.1.el5 looks like an OEL5 kernel, > not a RHEL5 kernel. I''d imagine you are actually using RHEL5 and you > grabbed the OEL5 packages from the download site. >You are right. I''m feeling a bit dumb.> Those are in fact client packages, despite the lack of "client" in their > name. This is a defect in our RPM packaging which I hope will be fixed > by the landing of some patches I am working on in another bug. It''s > possible that I have not fixed that one though. Please feel free to > file a bug in any case. >Ok, this is a good news. I have just tested the RPMs and they work flawlessly. Thank you all for your time.