Arya Mazaheri
2011-Feb-19 14:34 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
Hi, I have installed lustre client packages on a client node. But it doesn''t mount the lustre file system from lustre server. It gets the following famous error: $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No such device Are the lustre modules loaded? Check /etc/modprobe.conf and /proc/filesystems Note ''alias lustre llite'' should be removed from modprobe.conf As I was searching through the mailing list, I have noticed that lustre.ko should be present in this directory: /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in 2.6.18-164.11.1.el5 instead. So I guessed that this may be the source of problem. Any ideas? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110219/d04d2bc6/attachment.html
Brian J. Murrell
2011-Feb-19 14:38 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
On 11-02-19 09:34 AM, Arya Mazaheri wrote:> > As I was searching through the mailing list, I have noticed that lustre.ko > should be present in this directory: > /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko > > My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in > 2.6.18-164.11.1.el5 instead.You need the client modules package that matches your kernel. If one is not available you will have to build it from the source. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110219/2d0b4af5/attachment.bin
Arya Mazaheri
2011-Feb-19 15:07 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
Well, I want to install client module on rocks 5.5 x86_64. there are some packages in source section of lustre download area. I am confused which one to choose? what are the differences between them? lustre-client-source-2.0.0.1-2.6.18_164.11.1.0.1.el5_lustre.2.0.0.1.i686.rpm lustre-client-source-2.0.0.1-2.6.16_60_0.42.8_lustre.2.0.0.1_smp.x86_64.rpm lustre-client-source-2.0.0.1-2.6.27_23_0.1_lustre.2.0.0.1_default.x86_64.rpm lustre-client-source-2.0.0.1-2.6.18_164.11.1.0.1.el5_lustre.2.0.0.1.x86_64.rpm lustre-client-source-2.0.0.1-2.6.16_60_0.42.8_lustre.2.0.0.1_bigsmp.i686.rpm lustre-client-source-2.0.0.1-2.6.27_23_0.1_lustre.2.0.0.1_default.i686.rpm lustre-client-source-2.0.0.1-2.6.18_164.11.1.el5_lustre.2.0.0.1.i686.rpm lustre-client-source-2.0.0.1-2.6.18_164.11.1.el5_lustre.2.0.0.1.x86_64.rpm and one other thing! where is the source for lustre-client-modules? On Sat, Feb 19, 2011 at 6:08 PM, Brian J. Murrell <brian at whamcloud.com>wrote:> On 11-02-19 09:34 AM, Arya Mazaheri wrote: > > > > As I was searching through the mailing list, I have noticed that > lustre.ko > > should be present in this directory: > > /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko > > > > My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in > > 2.6.18-164.11.1.el5 instead. > > You need the client modules package that matches your kernel. If one is > not available you will have to build it from the source. > > b. > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110219/e58d66ce/attachment.html
Albert Everett
2011-Feb-19 17:27 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
Our kernel is also 2.6.18_194.17.4.el5. We installed OFED 1.5.2 from source, following this guide: https://wiki.rocksclusters.org/wiki/index.php/Install_OFED_1.5.x_on_a_Rocks_5.3_cluster ... which left us, among other things, a folder /usr/src/ofa_kernel. Lustre on the server side is handled by our vendor, so all we needed to worry about is the client. To build a lustre client, we then installed lustre-1.8.5.tar.gz from source, not from rpms. Our first compile produced the error you show below. # ./configure --with-linux=/lib/modules/`uname -r`/build # make rpms To get the lustre installation to use our new OFED, we tried this and it worked. # ./configure --with-o2ib=/usr/src/ofa_kernel --with-linux=/lib/ modules/`uname -r`/build # make rpms RPMs showed up in /usr/src/redhat/RPMS/x86_64, and we are using lustre-1.8.5*.rpm and lustre-modules-*.rpm on our client machines. Albert On Feb 19, 2011, at 8:34 AM, Arya Mazaheri wrote:> Hi, > I have installed lustre client packages on a client node. But it > doesn''t mount the lustre file system from lustre server. It gets the > following famous error: > > $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre > mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No > such device > Are the lustre modules loaded? > Check /etc/modprobe.conf and /proc/filesystems > Note ''alias lustre llite'' should be removed from modprobe.conf > > > As I was searching through the mailing list, I have noticed that > lustre.ko should be present in this directory: > /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko > > My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in > 2.6.18-164.11.1.el5 instead. So I guessed that this may be the > source of problem. > > Any ideas? > > Thanks > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
Arya Mazaheri
2011-Feb-19 18:18 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
Hi Albert, It seems that you have made a new kernel in order to run lustre on clients. Am I right? I don''t want to change kernel on clients at all... On Sat, Feb 19, 2011 at 8:57 PM, Albert Everett <aeeverett at ualr.edu> wrote:> Our kernel is also 2.6.18_194.17.4.el5. > > We installed OFED 1.5.2 from source, following this guide: > > > https://wiki.rocksclusters.org/wiki/index.php/Install_OFED_1.5.x_on_a_Rocks_5.3_cluster > > ... which left us, among other things, a folder /usr/src/ofa_kernel. > > Lustre on the server side is handled by our vendor, so all we needed to > worry about is the client. > > To build a lustre client, we then installed lustre-1.8.5.tar.gz from > source, not from rpms. Our first compile produced the error you show below. > # ./configure --with-linux=/lib/modules/`uname -r`/build > # make rpms > > To get the lustre installation to use our new OFED, we tried this and it > worked. > > # ./configure --with-o2ib=/usr/src/ofa_kernel > --with-linux=/lib/modules/`uname -r`/build > # make rpms > > RPMs showed up in /usr/src/redhat/RPMS/x86_64, and we are using > lustre-1.8.5*.rpm and lustre-modules-*.rpm on our client machines. > > Albert > > > On Feb 19, 2011, at 8:34 AM, Arya Mazaheri wrote: > > Hi, >> I have installed lustre client packages on a client node. But it doesn''t >> mount the lustre file system from lustre server. It gets the following >> famous error: >> >> $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre >> mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No such >> device >> Are the lustre modules loaded? >> Check /etc/modprobe.conf and /proc/filesystems >> Note ''alias lustre llite'' should be removed from modprobe.conf >> >> >> As I was searching through the mailing list, I have noticed that lustre.ko >> should be present in this directory: >> /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko >> >> My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in >> 2.6.18-164.11.1.el5 instead. So I guessed that this may be the source of >> problem. >> >> Any ideas? >> >> Thanks >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110219/49592742/attachment.html
Albert Everett
2011-Feb-19 22:26 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
For lustre client, we did not need to alter our kernel at all. We just made and installed lustre-1.8.5 and lustre-modules-1.8.5 rpms. /etc/ modprobe.conf needs a tweak. For lustre server, I believe you will need to deal with a patched kernel. We have not been down this road yet since our vendor includes lustre server software with their hardware. Albert On Feb 19, 2011, at 12:18 PM, Arya Mazaheri wrote:> Hi Albert, > It seems that you have made a new kernel in order to run lustre on > clients. Am I right? > I don''t want to change kernel on clients at all... > > On Sat, Feb 19, 2011 at 8:57 PM, Albert Everett <aeeverett at ualr.edu> > wrote: > Our kernel is also 2.6.18_194.17.4.el5. > > We installed OFED 1.5.2 from source, following this guide: > > https://wiki.rocksclusters.org/wiki/index.php/Install_OFED_1.5.x_on_a_Rocks_5.3_cluster > > ... which left us, among other things, a folder /usr/src/ofa_kernel. > > Lustre on the server side is handled by our vendor, so all we needed > to worry about is the client. > > To build a lustre client, we then installed lustre-1.8.5.tar.gz from > source, not from rpms. Our first compile produced the error you show > below. > # ./configure --with-linux=/lib/modules/`uname -r`/build > # make rpms > > To get the lustre installation to use our new OFED, we tried this > and it worked. > > # ./configure --with-o2ib=/usr/src/ofa_kernel --with-linux=/lib/ > modules/`uname -r`/build > # make rpms > > RPMs showed up in /usr/src/redhat/RPMS/x86_64, and we are using > lustre-1.8.5*.rpm and lustre-modules-*.rpm on our client machines. > > Albert > > > On Feb 19, 2011, at 8:34 AM, Arya Mazaheri wrote: > > Hi, > I have installed lustre client packages on a client node. But it > doesn''t mount the lustre file system from lustre server. It gets the > following famous error: > > $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre > mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No > such device > Are the lustre modules loaded? > Check /etc/modprobe.conf and /proc/filesystems > Note ''alias lustre llite'' should be removed from modprobe.conf > > > As I was searching through the mailing list, I have noticed that > lustre.ko should be present in this directory: > /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko > > My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in > 2.6.18-164.11.1.el5 instead. So I guessed that this may be the > source of problem. > > Any ideas? > > Thanks > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > >
Arya Mazaheri
2011-Feb-20 20:20 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
I have done what you said. I will test my client to the server tomorrow. but would you tell me the tweaks you have done on /etc/modprobe.conf ? On Sun, Feb 20, 2011 at 1:56 AM, Albert Everett <aeeverett at ualr.edu> wrote:> For lustre client, we did not need to alter our kernel at all. We just made > and installed lustre-1.8.5 and lustre-modules-1.8.5 rpms. /etc/modprobe.conf > needs a tweak. > > For lustre server, I believe you will need to deal with a patched kernel. > We have not been down this road yet since our vendor includes lustre server > software with their hardware. > > Albert > > > On Feb 19, 2011, at 12:18 PM, Arya Mazaheri wrote: > > Hi Albert, >> It seems that you have made a new kernel in order to run lustre on >> clients. Am I right? >> I don''t want to change kernel on clients at all... >> >> On Sat, Feb 19, 2011 at 8:57 PM, Albert Everett <aeeverett at ualr.edu> >> wrote: >> Our kernel is also 2.6.18_194.17.4.el5. >> >> We installed OFED 1.5.2 from source, following this guide: >> >> >> https://wiki.rocksclusters.org/wiki/index.php/Install_OFED_1.5.x_on_a_Rocks_5.3_cluster >> >> ... which left us, among other things, a folder /usr/src/ofa_kernel. >> >> Lustre on the server side is handled by our vendor, so all we needed to >> worry about is the client. >> >> To build a lustre client, we then installed lustre-1.8.5.tar.gz from >> source, not from rpms. Our first compile produced the error you show below. >> # ./configure --with-linux=/lib/modules/`uname -r`/build >> # make rpms >> >> To get the lustre installation to use our new OFED, we tried this and it >> worked. >> >> # ./configure --with-o2ib=/usr/src/ofa_kernel >> --with-linux=/lib/modules/`uname -r`/build >> # make rpms >> >> RPMs showed up in /usr/src/redhat/RPMS/x86_64, and we are using >> lustre-1.8.5*.rpm and lustre-modules-*.rpm on our client machines. >> >> Albert >> >> >> On Feb 19, 2011, at 8:34 AM, Arya Mazaheri wrote: >> >> Hi, >> I have installed lustre client packages on a client node. But it doesn''t >> mount the lustre file system from lustre server. It gets the following >> famous error: >> >> $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre >> mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No such >> device >> Are the lustre modules loaded? >> Check /etc/modprobe.conf and /proc/filesystems >> Note ''alias lustre llite'' should be removed from modprobe.conf >> >> >> As I was searching through the mailing list, I have noticed that lustre.ko >> should be present in this directory: >> /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko >> >> My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in >> 2.6.18-164.11.1.el5 instead. So I guessed that this may be the source of >> problem. >> >> Any ideas? >> >> Thanks >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110220/f47805a1/attachment.html
Albert Everett
2011-Feb-21 16:14 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
Here''s what''s in our /etc/modprobe.conf related to IB and
lustre:
options ib_mthca msi_x=1
options lnet networks=o2ib0(ib0)
options ko2iblnd ipif_name=ib0
We have Mellanox Infinihost (III?) DDR cards and IPs defined for them.
$ /sbin/ifconfig ib0
ib0 Link encap:InfiniBand HWaddr 80:00:04:04:FE:
80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
inet addr:192.168.2.1 Bcast:192.168.2.255 Mask:
255.255.255.0
inet6 addr: fe80::202:c902:29:b341/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:65520 Metric:1
RX packets:1719 errors:0 dropped:0 overruns:0 frame:0
TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:256
RX bytes:96564 (94.3 KiB) TX bytes:2420 (2.3 KiB)
Albert
On Feb 20, 2011, at 2:20 PM, Arya Mazaheri wrote:
> I have done what you said. I will test my client to the server
> tomorrow. but would you tell me the tweaks you have done on /etc/
> modprobe.conf ?
>
> On Sun, Feb 20, 2011 at 1:56 AM, Albert Everett <aeeverett at
ualr.edu>
> wrote:
> For lustre client, we did not need to alter our kernel at all. We
> just made and installed lustre-1.8.5 and lustre-modules-1.8.5 rpms. /
> etc/modprobe.conf needs a tweak.
>
> For lustre server, I believe you will need to deal with a patched
> kernel. We have not been down this road yet since our vendor
> includes lustre server software with their hardware.
>
> Albert
>
>
> On Feb 19, 2011, at 12:18 PM, Arya Mazaheri wrote:
>
> Hi Albert,
> It seems that you have made a new kernel in order to run lustre on
> clients. Am I right?
> I don''t want to change kernel on clients at all...
>
> On Sat, Feb 19, 2011 at 8:57 PM, Albert Everett <aeeverett at
ualr.edu>
> wrote:
> Our kernel is also 2.6.18_194.17.4.el5.
>
> We installed OFED 1.5.2 from source, following this guide:
>
>
https://wiki.rocksclusters.org/wiki/index.php/Install_OFED_1.5.x_on_a_Rocks_5.3_cluster
>
> ... which left us, among other things, a folder /usr/src/ofa_kernel.
>
> Lustre on the server side is handled by our vendor, so all we needed
> to worry about is the client.
>
> To build a lustre client, we then installed lustre-1.8.5.tar.gz from
> source, not from rpms. Our first compile produced the error you show
> below.
> # ./configure --with-linux=/lib/modules/`uname -r`/build
> # make rpms
>
> To get the lustre installation to use our new OFED, we tried this
> and it worked.
>
> # ./configure --with-o2ib=/usr/src/ofa_kernel --with-linux=/lib/
> modules/`uname -r`/build
> # make rpms
>
> RPMs showed up in /usr/src/redhat/RPMS/x86_64, and we are using
> lustre-1.8.5*.rpm and lustre-modules-*.rpm on our client machines.
>
> Albert
>
>
> On Feb 19, 2011, at 8:34 AM, Arya Mazaheri wrote:
>
> Hi,
> I have installed lustre client packages on a client node. But it
> doesn''t mount the lustre file system from lustre server. It gets
the
> following famous error:
>
> $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre
> mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No
> such device
> Are the lustre modules loaded?
> Check /etc/modprobe.conf and /proc/filesystems
> Note ''alias lustre llite'' should be removed from
modprobe.conf
>
>
> As I was searching through the mailing list, I have noticed that
> lustre.ko should be present in this directory:
> /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko
>
> My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in
> 2.6.18-164.11.1.el5 instead. So I guessed that this may be the
> source of problem.
>
> Any ideas?
>
> Thanks
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>
>
>
Arya Mazaheri
2011-Feb-21 18:09 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
Thanks Albert, I really appreciate you... Now everything is working... On Mon, Feb 21, 2011 at 7:44 PM, Albert Everett <aeeverett at ualr.edu> wrote:> Here''s what''s in our /etc/modprobe.conf related to IB and lustre: > > options ib_mthca msi_x=1 > options lnet networks=o2ib0(ib0) > options ko2iblnd ipif_name=ib0 > > We have Mellanox Infinihost (III?) DDR cards and IPs defined for them. > > $ /sbin/ifconfig ib0 > ib0 Link encap:InfiniBand HWaddr > 80:00:04:04:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 > inet addr:192.168.2.1 Bcast:192.168.2.255 Mask:255.255.255.0 > inet6 addr: fe80::202:c902:29:b341/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:65520 Metric:1 > RX packets:1719 errors:0 dropped:0 overruns:0 frame:0 > TX packets:30 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:256 > RX bytes:96564 (94.3 KiB) TX bytes:2420 (2.3 KiB) > > Albert > > > On Feb 20, 2011, at 2:20 PM, Arya Mazaheri wrote: > > I have done what you said. I will test my client to the server tomorrow. >> but would you tell me the tweaks you have done on /etc/modprobe.conf ? >> >> On Sun, Feb 20, 2011 at 1:56 AM, Albert Everett <aeeverett at ualr.edu> >> wrote: >> For lustre client, we did not need to alter our kernel at all. We just >> made and installed lustre-1.8.5 and lustre-modules-1.8.5 rpms. >> /etc/modprobe.conf needs a tweak. >> >> For lustre server, I believe you will need to deal with a patched kernel. >> We have not been down this road yet since our vendor includes lustre server >> software with their hardware. >> >> Albert >> >> >> On Feb 19, 2011, at 12:18 PM, Arya Mazaheri wrote: >> >> Hi Albert, >> It seems that you have made a new kernel in order to run lustre on >> clients. Am I right? >> I don''t want to change kernel on clients at all... >> >> On Sat, Feb 19, 2011 at 8:57 PM, Albert Everett <aeeverett at ualr.edu> >> wrote: >> Our kernel is also 2.6.18_194.17.4.el5. >> >> We installed OFED 1.5.2 from source, following this guide: >> >> >> https://wiki.rocksclusters.org/wiki/index.php/Install_OFED_1.5.x_on_a_Rocks_5.3_cluster >> >> ... which left us, among other things, a folder /usr/src/ofa_kernel. >> >> Lustre on the server side is handled by our vendor, so all we needed to >> worry about is the client. >> >> To build a lustre client, we then installed lustre-1.8.5.tar.gz from >> source, not from rpms. Our first compile produced the error you show below. >> # ./configure --with-linux=/lib/modules/`uname -r`/build >> # make rpms >> >> To get the lustre installation to use our new OFED, we tried this and it >> worked. >> >> # ./configure --with-o2ib=/usr/src/ofa_kernel >> --with-linux=/lib/modules/`uname -r`/build >> # make rpms >> >> RPMs showed up in /usr/src/redhat/RPMS/x86_64, and we are using >> lustre-1.8.5*.rpm and lustre-modules-*.rpm on our client machines. >> >> Albert >> >> >> On Feb 19, 2011, at 8:34 AM, Arya Mazaheri wrote: >> >> Hi, >> I have installed lustre client packages on a client node. But it doesn''t >> mount the lustre file system from lustre server. It gets the following >> famous error: >> >> $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre >> mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No such >> device >> Are the lustre modules loaded? >> Check /etc/modprobe.conf and /proc/filesystems >> Note ''alias lustre llite'' should be removed from modprobe.conf >> >> >> As I was searching through the mailing list, I have noticed that lustre.ko >> should be present in this directory: >> /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko >> >> My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in >> 2.6.18-164.11.1.el5 instead. So I guessed that this may be the source of >> problem. >> >> Any ideas? >> >> Thanks >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> >> >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110221/3c2b8639/attachment.html
Albert Everett
2011-Feb-21 18:22 UTC
[Lustre-discuss] Installing Lustre client on 2.6.18-194 kernel
Excellent - thanks for letting us know. Thanks again to Tim Carlson for his OFED wiki, and to Terascala for help with building up the lustre client. Albert On Feb 21, 2011, at 12:09 PM, Arya Mazaheri wrote:> Thanks Albert, I really appreciate you... > Now everything is working... > > > On Mon, Feb 21, 2011 at 7:44 PM, Albert Everett <aeeverett at ualr.edu> > wrote: > Here''s what''s in our /etc/modprobe.conf related to IB and lustre: > > options ib_mthca msi_x=1 > options lnet networks=o2ib0(ib0) > options ko2iblnd ipif_name=ib0 > > We have Mellanox Infinihost (III?) DDR cards and IPs defined for them. > > $ /sbin/ifconfig ib0 > ib0 Link encap:InfiniBand HWaddr 80:00:04:04:FE: > 80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 > inet addr:192.168.2.1 Bcast:192.168.2.255 Mask: > 255.255.255.0 > inet6 addr: fe80::202:c902:29:b341/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:65520 Metric:1 > RX packets:1719 errors:0 dropped:0 overruns:0 frame:0 > TX packets:30 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:256 > RX bytes:96564 (94.3 KiB) TX bytes:2420 (2.3 KiB) > > Albert > > > On Feb 20, 2011, at 2:20 PM, Arya Mazaheri wrote: > > I have done what you said. I will test my client to the server > tomorrow. but would you tell me the tweaks you have done on /etc/ > modprobe.conf ? > > On Sun, Feb 20, 2011 at 1:56 AM, Albert Everett <aeeverett at ualr.edu> > wrote: > For lustre client, we did not need to alter our kernel at all. We > just made and installed lustre-1.8.5 and lustre-modules-1.8.5 rpms. / > etc/modprobe.conf needs a tweak. > > For lustre server, I believe you will need to deal with a patched > kernel. We have not been down this road yet since our vendor > includes lustre server software with their hardware. > > Albert > > > On Feb 19, 2011, at 12:18 PM, Arya Mazaheri wrote: > > Hi Albert, > It seems that you have made a new kernel in order to run lustre on > clients. Am I right? > I don''t want to change kernel on clients at all... > > On Sat, Feb 19, 2011 at 8:57 PM, Albert Everett <aeeverett at ualr.edu> > wrote: > Our kernel is also 2.6.18_194.17.4.el5. > > We installed OFED 1.5.2 from source, following this guide: > > https://wiki.rocksclusters.org/wiki/index.php/Install_OFED_1.5.x_on_a_Rocks_5.3_cluster > > ... which left us, among other things, a folder /usr/src/ofa_kernel. > > Lustre on the server side is handled by our vendor, so all we needed > to worry about is the client. > > To build a lustre client, we then installed lustre-1.8.5.tar.gz from > source, not from rpms. Our first compile produced the error you show > below. > # ./configure --with-linux=/lib/modules/`uname -r`/build > # make rpms > > To get the lustre installation to use our new OFED, we tried this > and it worked. > > # ./configure --with-o2ib=/usr/src/ofa_kernel --with-linux=/lib/ > modules/`uname -r`/build > # make rpms > > RPMs showed up in /usr/src/redhat/RPMS/x86_64, and we are using > lustre-1.8.5*.rpm and lustre-modules-*.rpm on our client machines. > > Albert > > > On Feb 19, 2011, at 8:34 AM, Arya Mazaheri wrote: > > Hi, > I have installed lustre client packages on a client node. But it > doesn''t mount the lustre file system from lustre server. It gets the > following famous error: > > $ mount -t lustre 192.168.0.1:/lustre /mnt/lustre > mount.lustre: mount 172.16.113.232:/lustre at /mnt/lustre failed: No > such device > Are the lustre modules loaded? > Check /etc/modprobe.conf and /proc/filesystems > Note ''alias lustre llite'' should be removed from modprobe.conf > > > As I was searching through the mailing list, I have noticed that > lustre.ko should be present in this directory: > /lib/modules/2.6.18-194.17.4.el5/kernel/fs/lustre/lustre.ko > > My current kernel is 2.6.18-194.17.4.el5. but lustre.ko is in > 2.6.18-164.11.1.el5 instead. So I guessed that this may be the > source of problem. > > Any ideas? > > Thanks > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > > > >