Hello All, I am trying to use luster with qlogic ofed 1.4.2 . After building and installing the kernel when i try modprobe lustre i get the following errors # modprobe lustre WARNING: Error inserting osc (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/osc.ko): Input/output error WARNING: Error inserting mdc (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/mdc.ko): Input/output error WARNING: Error inserting lov (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lov.ko): Input/output error FATAL: Error inserting lustre (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lustre.ko): Input/output error dmesg shows errors of the type Lustre: OBD class driver, http://www.lustre.org/ Lustre: Lustre Version: 1.8.3 Lustre: Build Version: 1.8.3-20100409182943-PRISTINE-2.6.18-164.11.1.el5_lustre.1.8.3 ko2iblnd: disagrees about version of symbol ib_fmr_pool_unmap ko2iblnd: Unknown symbol ib_fmr_pool_unmap ko2iblnd: disagrees about version of symbol ib_create_cq ko2iblnd: Unknown symbol ib_create_cq ko2iblnd: disagrees about version of symbol rdma_resolve_addr ko2iblnd: Unknown symbol rdma_resolve_addr Following are the rpm''s installed # rpm -qa | grep kernel kernel-doc-2.6.18-164.6.1.el5 kernel-2.6.18-164.6.1.el5 kernel-headers-2.6.18-164.6.1.el5 kernel-ib-devel-1.4.2.1-2.6.18_164.11.1.el5_lustre.1.8.3 kernel-2.6.18-164.11.1.el5_lustre.1.8.3 kernel-ib-1.4.2.1-2.6.18_164.11.1.el5_lustre.1.8.3 kernel-devel-2.6.18-164.6.1.el5 kernel-devel-2.6.18-164.11.1.el5_lustre.1.8.3 What am i missing. Ady -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100609/22cf4f9c/attachment.html
Looks like a mis-match on the OFED modules Lustre is expecting. If not using the included OFED, you need these steps: 1) Build (optional) and install the Lustre kernel 2) Build OFED against the Lustre kernel and install it 3) Build Lustre against the kernel and the OFED you are using Lustre defaults to using the in-kernel OFED unless you point configure at a different set of headers. Kevin ----- Original Message ----- From: srirangam.addepalli at gmail.com To: lustre-discuss at lists.lustre.org Sent: Wednesday, June 9, 2010 1:33:04 PM GMT -07:00 US/Canada Mountain Subject: [Lustre-discuss] Luster 1.8.3 Qlogic OFED 1.4.2 Hello All, I am trying to use luster with qlogic ofed 1.4.2 . After building and installing the kernel when i try modprobe lustre i get the following errors # modprobe lustre WARNING: Error inserting osc (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/osc.ko): Input/output error WARNING: Error inserting mdc (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/mdc.ko): Input/output error WARNING: Error inserting lov (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lov.ko): Input/output error FATAL: Error inserting lustre (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lustre.ko): Input/output error dmesg shows errors of the type Lustre: OBD class driver, http://www.lustre.org/ Lustre: Lustre Version: 1.8.3 Lustre: Build Version: 1.8.3-20100409182943-PRISTINE-2.6.18-164.11.1.el5_lustre.1.8.3 ko2iblnd: disagrees about version of symbol ib_fmr_pool_unmap ko2iblnd: Unknown symbol ib_fmr_pool_unmap ko2iblnd: disagrees about version of symbol ib_create_cq ko2iblnd: Unknown symbol ib_create_cq ko2iblnd: disagrees about version of symbol rdma_resolve_addr ko2iblnd: Unknown symbol rdma_resolve_addr Following are the rpm''s installed # rpm -qa | grep kernel kernel-doc-2.6.18-164.6.1.el5 kernel-2.6.18-164.6.1.el5 kernel-headers-2.6.18-164.6.1.el5 kernel-ib-devel-1.4.2.1-2.6.18_164.11.1.el5_lustre.1.8.3 kernel-2.6.18-164.11.1.el5_lustre.1.8.3 kernel-ib-1.4.2.1-2.6.18_164.11.1.el5_lustre.1.8.3 kernel-devel-2.6.18-164.6.1.el5 kernel-devel-2.6.18-164.11.1.el5_lustre.1.8.3 What am i missing. Ady -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100609/0c0858aa/attachment.html
Hi, first of all explain my problem, I would have a information, I have read many articles but I could not reach a conclusion. I also had the problem mis-match on the OFED modules Lustre is expecting, then I did as suggested: 1. I installed kernel-2.6.18-164.11.1.el5_lustre.1.8.3 2. Then I installed the drivers Infiniband QLogig QLogicIB-Basic.RHEL5- x86_64.6.0.0.0.54.tgz 3. now I should compile Lustre 1.8.3 and here my questions: I have used the option --with-iib or --with-o2ib ?? In both cases, what path should I enter ?? Thanks Stefano Date: Wed, 9 Jun 2010 12:48:55 -0700 (PDT) From: Kevin Van Maren <kevin.van.maren at oracle.com> Subject: Re: [Lustre-discuss] Luster 1.8.3 Qlogic OFED 1.4.2 To: <srirangam.addepalli at gmail.com> Cc: lustre-discuss at lists.lustre.org Message-ID: <6cbdafd2-d871-4fb2-8ba5-1726dc133847 at default> Content-Type: text/plain; charset="utf-8" Looks like a mis-match on the OFED modules Lustre is expecting. If not using the included OFED, you need these steps: 1) Build (optional) and install the Lustre kernel 2) Build OFED against the Lustre kernel and install it 3) Build Lustre against the kernel and the OFED you are using Lustre defaults to using the in-kernel OFED unless you point configure at a different set of headers. Kevin ----- Original Message ----- From: srirangam.addepalli at gmail.com To: lustre-discuss at lists.lustre.org Sent: Wednesday, June 9, 2010 1:33:04 PM GMT -07:00 US/Canada Mountain Subject: [Lustre-discuss] Luster 1.8.3 Qlogic OFED 1.4.2 Hello All, I am trying to use luster with qlogic ofed 1.4.2 . After building and installing the kernel when i try modprobe lustre i get the following errors # modprobe lustre WARNING: Error inserting osc (/lib/modules/2.6.18-164.11.1.el5_lustre. 1.8.3/kernel/fs/lustre/osc.ko): Input/output error WARNING: Error inserting mdc (/lib/modules/2.6.18-164.11.1.el5_lustre. 1.8.3/kernel/fs/lustre/mdc.ko): Input/output error WARNING: Error inserting lov (/lib/modules/2.6.18-164.11.1.el5_lustre. 1.8.3/kernel/fs/lustre/lov.ko): Input/output error FATAL: Error inserting lustre (/lib/modules/2.6.18-164.11.1.el5_lustre. 1.8.3/kernel/fs/lustre/lustre.ko): Input/output error dmesg shows errors of the type Ing. Stefano Elmopi Gruppo Darco - Resp. ICT Sistemi Via Ostiense 131/L Corpo B, 00154 Roma cell. 3466147165 tel. 0657060500 email:stefano.elmopi at sociale.it "Ai sensi e per effetti della legge sulla tutela della riservatezza personale (D.lgs n. 196/2003), questa @mail e'' destinata unicamente alle persone sopra indicate e le informazioni in essa contenute sono da considerarsi strettamente riservate. E'' proibito leggere, copiare, usare o diffondere il contenuto della presente @mail senza autorizzazione. Se avete ricevuto questo messaggio per errore, siete pregati di rispedire la stessa al mittente. Grazie" -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100611/e866278f/attachment.html
With OFED, you use o2ib. iib was for the old InfiniCon/SilverStorm InfiniServ IB software (pre-OFED) that QLogic acquired (as opposed to the InfiniPath software they acquired from PathScale). With an stock OFED install it is "--with-o2ib=/usr/src/ofa_kernel", but the path may be different for the qlogic version. Kevin Stefano Elmopi wrote:> > > > Hi, > > first of all explain my problem, I would have a information, > I have read many articles but I could not reach a conclusion. > I also had the problem mis-match on the OFED modules Lustre is expecting, > then I did as suggested: > > 1. I installed kernel-2.6.18-164.11.1.el5_lustre.1.8.3 > 2. Then I installed the drivers Infiniband QLogig > QLogicIB-Basic.RHEL5-x86_64.6.0.0.0.54.tgz > 3. now I should compile Lustre 1.8.3 and here my questions: > > I have used the option --with-iib or --with-o2ib ?? > In both cases, what path should I enter ?? > > Thanks > > Stefano > > Date: Wed, 9 Jun 2010 12:48:55 -0700 (PDT) > From: Kevin Van Maren <kevin.van.maren at oracle.com > <mailto:kevin.van.maren at oracle.com>> > Subject: Re: [Lustre-discuss] Luster 1.8.3 Qlogic OFED 1.4.2 > To: <srirangam.addepalli at gmail.com <mailto:srirangam.addepalli at gmail.com>> > Cc: lustre-discuss at lists.lustre.org > <mailto:lustre-discuss at lists.lustre.org> > Message-ID: <6cbdafd2-d871-4fb2-8ba5-1726dc133847 at default> > Content-Type: text/plain; charset="utf-8" > > Looks like a mis-match on the OFED modules Lustre is expecting. > > If not using the included OFED, you need these steps: > > 1) Build (optional) and install the Lustre kernel > 2) Build OFED against the Lustre kernel and install it > 3) Build Lustre against the kernel and the OFED you are using > > Lustre defaults to using the in-kernel OFED unless you point configure > at a different set of headers. > > Kevin > > > ----- Original Message ----- > From: srirangam.addepalli at gmail.com > <mailto:srirangam.addepalli at gmail.com> > To: lustre-discuss at lists.lustre.org > <mailto:lustre-discuss at lists.lustre.org> > Sent: Wednesday, June 9, 2010 1:33:04 PM GMT -07:00 US/Canada Mountain > Subject: [Lustre-discuss] Luster 1.8.3 Qlogic OFED 1.4.2 > > Hello All, > I am trying to use luster with qlogic ofed 1.4.2 . After building and > installing the kernel when i try modprobe lustre > i get the following errors > > > # modprobe lustre > WARNING: Error inserting osc > (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/osc.ko): > Input/output error > WARNING: Error inserting mdc > (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/mdc.ko): > Input/output error > WARNING: Error inserting lov > (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lov.ko): > Input/output error > FATAL: Error inserting lustre > (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lustre.ko): > Input/output error > > > dmesg shows errors of the type > > > Ing. Stefano Elmopi > Gruppo Darco - Resp. ICT Sistemi > Via Ostiense 131/L Corpo B, 00154 Roma > > cell. 3466147165 > tel. 0657060500 > email:stefano.elmopi at sociale.it <mailto:stefano.elmopi at sociale.it> > > "Ai sensi e per effetti della legge sulla tutela della riservatezza > personale > (D.lgs n. 196/2003), questa @mail e'' destinata unicamente alle > persone sopra > indicate e le informazioni in essa contenute sono da considerarsi > strettamente > riservate. E'' proibito leggere, copiare, usare o diffondere il > contenuto della > presente @mail senza autorizzazione. Se avete ricevuto questo > messaggio per > errore, siete pregati di rispedire la stessa al mittente. Grazie" >
If you replace OFED, you do not need to rebuild the kernel (unless you want to patch/change it), so you can install the binary Lustre kernel. You do need to rebuild Lustre (or at least the kernel modules) (step #3), as o2ib must be built against the OFED you are running. Kevin Marco Aurelio L Gomes wrote:> Hi, > > I saw the post above on the lustre-discuss list and would like to know > if in case I install OFED 1.5 on Lustre 1.8.3 I''ll need build the lustre > kernel, or only install the available kernel from lustre download page. > When I see that the build in (1) is optional I thought that is possible > to use that available kernel. > > Thanks in advance. > > Best regards, > > Marco Gomes > Systems/HPC-Cluster > Numerical Offshore Tank > Naval and Ocean Engineering Department''s Laboratory > Escola Polit?cnica > University of S?o Paulo > +55 11 3777 4142 ext. 250 > > On Wed, 2010-06-09 at 12:48 -0700, Kevin Van Maren wrote: > >> Looks like a mis-match on the OFED modules Lustre is expecting. >> >> If not using the included OFED, you need these steps: >> >> 1) Build (optional) and install the Lustre kernel >> 2) Build OFED against the Lustre kernel and install it >> 3) Build Lustre against the kernel and the OFED you are using >> >> Lustre defaults to using the in-kernel OFED unless you point configure >> at a different set of headers. >> >> Kevin >> >> >> ----- Original Message ----- >> From: srirangam.addepalli at gmail.com >> To: lustre-discuss at lists.lustre.org >> Sent: Wednesday, June 9, 2010 1:33:04 PM GMT -07:00 US/Canada Mountain >> Subject: [Lustre-discuss] Luster 1.8.3 Qlogic OFED 1.4.2 >> >> Hello All, >> I am trying to use luster with qlogic ofed 1.4.2 . After building and >> installing the kernel when i try modprobe lustre >> i get the following ! errors >> >> >> # modprobe lustre >> WARNING: Error inserting osc >> (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/osc.ko): Input/output error >> WARNING: Error inserting mdc >> (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/mdc.ko): Input/output error >> WARNING: Error inserting lov >> (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lov.ko): Input/output error >> FATAL: Error inserting lustre >> (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lustre.ko): Input/output error >> >> >> dmesg shows errors of the type >> >> Lustre: OBD class driver, >> http://www.lustre.org/ >> Lustre: Lustre Version: >> 1.8.3 >> Lustre: Build Version: >> 1.8.3-20100409182943-PRISTINE-2.6.18-164.11.1.el5_lustre.1.8.3 ! >> ko2iblnd: disagrees about version of symbol >> ib_fmr_pool_unmap >> ko2iblnd: Unknown symbol >> ib_fmr_pool_unmap  ! ; >> ko2iblnd: disagrees about version of symbol >> ib_create_cq >> ko2iblnd: Unknown symbol >> ib_create_cq &nb! sp; >> ko2iblnd: disagrees about version of symbol >> rdma_resolve_addr >> ko2iblnd: Unknown symbol rdma_resolve_addr >> >> Following are the rpm''s installed >> >> # rpm -qa | grep kernel >> kernel-doc-2.6.18-164.6.1.el5 >> kernel-2.6.18-164.6.1.el5 >> kernel-headers-2.6.18-164.6.1.el5 >> kernel-ib-devel-1.4.2.1-2.6.18_164.11.1.el5_lustre.1.8.3 >> kernel-2.6.18-164.11.1.el5_lustre.1.8.3 >> kernel-ib-1.4.2.1-2.6.18_164.11.1.el5_lustre.1.8.3 >> kernel-devel-2.6.18-164.6.1.el5 >> kernel-devel-2.6.18-164.11.1.el5_lustre.1.8.3 >> >> What am i missing. >> >> Ady >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > >
Marco Aurelio L Gomes
2010-Jul-19 18:21 UTC
[Lustre-discuss] Luster 1.8.3 Qlogic OFED 1.4.2
I tested without rebuilding the kernel and it worked! Thanks so much for helping. Regards, Marco Gomes On Fri, 2010-07-16 at 17:43 -0600, Kevin Van Maren wrote:> If you replace OFED, you do not need to rebuild the kernel (unless you > want to patch/change it), so you can install the binary Lustre kernel. > You do need to rebuild Lustre (or at least the kernel modules) (step > #3), as o2ib must be built against the OFED you are running. > > Kevin > > > Marco Aurelio L Gomes wrote: > > Hi, > > > > I saw the post above on the lustre-discuss list and would like to know > > if in case I install OFED 1.5 on Lustre 1.8.3 I''ll need build the lustre > > kernel, or only install the available kernel from lustre download page. > > When I see that the build in (1) is optional I thought that is possible > > to use that available kernel. > > > > Thanks in advance. > > > > Best regards, > > > > Marco Gomes > > Systems/HPC-Cluster > > Numerical Offshore Tank > > Naval and Ocean Engineering Department''s Laboratory > > Escola Polit?cnica > > University of S?o Paulo > > +55 11 3777 4142 ext. 250 > > > > On Wed, 2010-06-09 at 12:48 -0700, Kevin Van Maren wrote: > > > >> Looks like a mis-match on the OFED modules Lustre is expecting. > >> > >> If not using the included OFED, you need these steps: > >> > >> 1) Build (optional) and install the Lustre kernel > >> 2) Build OFED against the Lustre kernel and install it > >> 3) Build Lustre against the kernel and the OFED you are using > >> > >> Lustre defaults to using the in-kernel OFED unless you point configure > >> at a different set of headers. > >> > >> Kevin > >> > >> > >> ----- Original Message ----- > >> From: srirangam.addepalli at gmail.com > >> To: lustre-discuss at lists.lustre.org > >> Sent: Wednesday, June 9, 2010 1:33:04 PM GMT -07:00 US/Canada Mountain > >> Subject: [Lustre-discuss] Luster 1.8.3 Qlogic OFED 1.4.2 > >> > >> Hello All, > >> I am trying to use luster with qlogic ofed 1.4.2 . After building and > >> installing the kernel when i try modprobe lustre > >> i get the following ! errors > >> > >> > >> # modprobe lustre > >> WARNING: Error inserting osc > >> (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/osc.ko): Input/output error > >> WARNING: Error inserting mdc > >> (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/mdc.ko): Input/output error > >> WARNING: Error inserting lov > >> (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lov.ko): Input/output error > >> FATAL: Error inserting lustre > >> (/lib/modules/2.6.18-164.11.1.el5_lustre.1.8.3/kernel/fs/lustre/lustre.ko): Input/output error > >> > >> > >> dmesg shows errors of the type > >> > >> Lustre: OBD class driver, > >> http://www.lustre.org/ > >> Lustre: Lustre Version: > >> 1.8.3 > >> Lustre: Build Version: > >> 1.8.3-20100409182943-PRISTINE-2.6.18-164.11.1.el5_lustre.1.8.3 ! > >> ko2iblnd: disagrees about version of symbol > >> ib_fmr_pool_unmap > >> ko2iblnd: Unknown symbol > >> ib_fmr_pool_unmap  ! ; > >> ko2iblnd: disagrees about version of symbol > >> ib_create_cq > >> ko2iblnd: Unknown symbol > >> ib_create_cq &nb! sp; > >> ko2iblnd: disagrees about version of symbol > >> rdma_resolve_addr > >> ko2iblnd: Unknown symbol rdma_resolve_addr > >> > >> Following are the rpm''s installed > >> > >> # rpm -qa | grep kernel > >> kernel-doc-2.6.18-164.6.1.el5 > >> kernel-2.6.18-164.6.1.el5 > >> kernel-headers-2.6.18-164.6.1.el5 > >> kernel-ib-devel-1.4.2.1-2.6.18_164.11.1.el5_lustre.1.8.3 > >> kernel-2.6.18-164.11.1.el5_lustre.1.8.3 > >> kernel-ib-1.4.2.1-2.6.18_164.11.1.el5_lustre.1.8.3 > >> kernel-devel-2.6.18-164.6.1.el5 > >> kernel-devel-2.6.18-164.11.1.el5_lustre.1.8.3 > >> > >> What am i missing. > >> > >> Ady > >> > >> _______________________________________________ > >> Lustre-discuss mailing list > >> Lustre-discuss at lists.lustre.org > >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > >> > > > > >