Charles Taylor
2008-Feb-18 16:20 UTC
[Lustre-discuss] kernel-2.6.18-8.1.14 + lustre 1.6.4.2 + OFED 1.2
We seemed to have it a stumbling block when building with the above (supported) versions. Our process... 1. Start with stock rhel5 2.6.18-8.1.14 source tree 2. Configure InfiniBand support out of the the kernel (we will build OFED separately). 3. Apply the 1.6.4.2 kernel patches to the kernel source. 4. Build the kernel. 5. Build OFED 1.2 against the patched kernel 6. Build Lustre using the patched Kernel+OFED 1.2 This worked fine for 1.6.3 but for 1.6.4.2, after patching the kernel and trying to build OFED 1.2, we get the error below. Now, we find references to this problem all over the place but nothing specific to building lustre. Since Lustre 1.6.4.2 is supposed to support OFED 1.2, it seems that OFED should build against a kernel patched for lustre 1.6.4.2 but apparently it does not. If we use the un- patched kernel (i.e. do not apply the lustre 1.6.4.2 kernel patches to it), OFED 1.2 builds without a problem. We are looking for an easy #define way out of this hole right now but if there is a known solution to this, we would love to know about it. If there is a better way to do this, we are open to that as well. Thanks, charlie taylor uf hpc center ======================================================================== ========================================================== gcc -Wp,-MD,/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ ulp/ipoib/.ipoib_fs.o.d -nostdinc -isystem /usr/lib/gcc/x86_64- redhat-linux/4.1.1/include -D__KERNEL__ \ -I/root/lustre/1.6.4.2/ofa_kernel-1.2/kernel_addons/backport/2.6.18/ include/ \ -I/root/lustre/1.6.4.2/ofa_kernel-1.2/include \ -I/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/include \ -Iinclude \ \ -include include/linux/autoconf.h \ -include /root/lustre/1.6.4.2/ofa_kernel-1.2/include/linux/autoconf.h \ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict- aliasing -fno-common -Wstrict-prototypes -Wundef -Werror-implicit- function-declaration -Os -mtune=generic -m64 -mno-red-zone - mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno- asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno- sse2 -mno-3dnow -fomit-frame-pointer -g -fno-stack-protector - Wdeclaration-after-statement -Wno-pointer-sign -I/root/lustre/ 1.6.4.2/ofa_kernel-1.2/include -I/root/lustre/1.6.4.2/ofa_kernel-1.2/ drivers/infiniband/include -I/root/lustre/1.6.4.2/ ofa_kernel-1.2/drivers/infiniband/ulp/ipoib -I/root/lustre/ 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/debug -I/root/lustre/ 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/hw/cxgb3/core -I/root/ lustre/1.6.4.2/ofa_kernel-1.2/drivers/net/cxgb3 -I/root/lustre/ 1.6.4.2/ofa_kernel-1.2/drivers/net/rds -DMODULE -D"KBUILD_STR(s) =#s" -D"KBUILD_BASENAME=KBUILD_STR(ipoib_fs)" - D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -c -o /root/lustre/1.6.4.2/ ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/.tmp_ipoib_fs.o /root/ lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ipoib_fs.c /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ ipoib_fs.c: In function ?ipoib_mcg_open?: /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ ipoib_fs.c:144: error: ?struct inode? has no member named ?u? /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ ipoib_fs.c: In function ?ipoib_path_open?: /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ ipoib_fs.c:250: error: ?struct inode? has no member named ?u? make[4]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ ulp/ipoib/ipoib_fs.o] Error 1 make[3]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ ulp/ipoib] Error 2 make[2]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband] Error 2 make[1]: *** [_module_/root/lustre/1.6.4.2/ofa_kernel-1.2] Error 2 make[1]: Leaving directory `/root/lustre/1.6.4.2/linux-2.6.18'' make: *** [kernel] Error 2
Aaron Knister
2008-Feb-18 16:35 UTC
[Lustre-discuss] kernel-2.6.18-8.1.14 + lustre 1.6.4.2 + OFED 1.2
Just out of curiosity, why not use the ofed stack in the kernel itself? On Feb 18, 2008, at 11:20 AM, Charles Taylor wrote:> > We seemed to have it a stumbling block when building with the above > (supported) versions. Our process... > > 1. Start with stock rhel5 2.6.18-8.1.14 source tree > 2. Configure InfiniBand support out of the the kernel (we will build > OFED separately). > 3. Apply the 1.6.4.2 kernel patches to the kernel source. > 4. Build the kernel. > 5. Build OFED 1.2 against the patched kernel > 6. Build Lustre using the patched Kernel+OFED 1.2 > > This worked fine for 1.6.3 but for 1.6.4.2, after patching the kernel > and trying to build OFED 1.2, we get the error below. Now, we find > references to this problem all over the place but nothing specific to > building lustre. Since Lustre 1.6.4.2 is supposed to support OFED > 1.2, it seems that OFED should build against a kernel patched for > lustre 1.6.4.2 but apparently it does not. If we use the un- > patched kernel (i.e. do not apply the lustre 1.6.4.2 kernel patches > to it), OFED 1.2 builds without a problem. We are looking for an > easy #define way out of this hole right now but if there is a known > solution to this, we would love to know about it. > > If there is a better way to do this, we are open to that as well. > > Thanks, > > charlie taylor > uf hpc center > > > > > =====================================================================> ==========================================================> > gcc -Wp,-MD,/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ > ulp/ipoib/.ipoib_fs.o.d -nostdinc -isystem /usr/lib/gcc/x86_64- > redhat-linux/4.1.1/include -D__KERNEL__ \ > -I/root/lustre/1.6.4.2/ofa_kernel-1.2/kernel_addons/backport/2.6.18/ > include/ \ > -I/root/lustre/1.6.4.2/ofa_kernel-1.2/include \ > -I/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/include \ > -Iinclude \ > \ > -include include/linux/autoconf.h \ > -include /root/lustre/1.6.4.2/ofa_kernel-1.2/include/linux/ > autoconf.h \ > -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict- > aliasing -fno-common -Wstrict-prototypes -Wundef -Werror-implicit- > function-declaration -Os -mtune=generic -m64 -mno-red-zone - > mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno- > asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno- > sse2 -mno-3dnow -fomit-frame-pointer -g -fno-stack-protector - > Wdeclaration-after-statement -Wno-pointer-sign -I/root/lustre/ > 1.6.4.2/ofa_kernel-1.2/include -I/root/lustre/1.6.4.2/ofa_kernel-1.2/ > drivers/infiniband/include -I/root/lustre/1.6.4.2/ > ofa_kernel-1.2/drivers/infiniband/ulp/ipoib -I/root/lustre/ > 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/debug -I/root/lustre/ > 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/hw/cxgb3/core -I/root/ > lustre/1.6.4.2/ofa_kernel-1.2/drivers/net/cxgb3 -I/root/lustre/ > 1.6.4.2/ofa_kernel-1.2/drivers/net/rds -DMODULE -D"KBUILD_STR(s) > =#s" -D"KBUILD_BASENAME=KBUILD_STR(ipoib_fs)" - > D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -c -o /root/lustre/1.6.4.2/ > ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/.tmp_ipoib_fs.o /root/ > lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ipoib_fs.c > /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > ipoib_fs.c: In function ?ipoib_mcg_open?: > /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > ipoib_fs.c:144: error: ?struct inode? has no member named ?u? > /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > ipoib_fs.c: In function ?ipoib_path_open?: > /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > ipoib_fs.c:250: error: ?struct inode? has no member named ?u? > make[4]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ > ulp/ipoib/ipoib_fs.o] Error 1 > make[3]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ > ulp/ipoib] Error 2 > make[2]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband] > Error 2 > make[1]: *** [_module_/root/lustre/1.6.4.2/ofa_kernel-1.2] Error 2 > make[1]: Leaving directory `/root/lustre/1.6.4.2/linux-2.6.18'' > make: *** [kernel] Error 2 > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discussAaron Knister Associate Systems Analyst Center for Ocean-Land-Atmosphere Studies (301) 595-7000 aaron at iges.org
Charles Taylor
2008-Feb-18 18:36 UTC
[Lustre-discuss] kernel-2.6.18-8.1.14 + lustre 1.6.4.2 + OFED 1.2
Turns out that when you build your patched kernel, you DO NOT want to change the EXTRAVERSION string in the Makefile such that it does not include ".el5." somewhere in the string. If you do that, some of the OFED configure scripts pick some patch files that are for a different (vanilla) kernel. So once we put ".el5." back into our version string, things worked as expected. It sure seems like we should be able to set "EXTRAVERSION" to anything we want without breaking builds i.e., there should be a more reliable way of figuring out which kernel you are building against. Sigh, Charlie Taylor UF HPC Center On Feb 18, 2008, at 11:35 AM, Aaron Knister wrote:> > On Feb 18, 2008, at 11:20 AM, Charles Taylor wrote: > >> >> We seemed to have it a stumbling block when building with the above >> (supported) versions. Our process... >> >> 1. Start with stock rhel5 2.6.18-8.1.14 source tree >> 2. Configure InfiniBand support out of the the kernel (we will build >> OFED separately). >> 3. Apply the 1.6.4.2 kernel patches to the kernel source. >> 4. Build the kernel. >> 5. Build OFED 1.2 against the patched kernel >> 6. Build Lustre using the patched Kernel+OFED 1.2 >> >> This worked fine for 1.6.3 but for 1.6.4.2, after patching the kernel >> and trying to build OFED 1.2, we get the error below. Now, we find >> references to this problem all over the place but nothing specific to >> building lustre. Since Lustre 1.6.4.2 is supposed to support OFED >> 1.2, it seems that OFED should build against a kernel patched for >> lustre 1.6.4.2 but apparently it does not. If we use the un- >> patched kernel (i.e. do not apply the lustre 1.6.4.2 kernel patches >> to it), OFED 1.2 builds without a problem. We are looking for an >> easy #define way out of this hole right now but if there is a known >> solution to this, we would love to know about it. >> >> If there is a better way to do this, we are open to that as well. >> >> Thanks, >> >> charlie taylor >> uf hpc center >> >> >> >> >> ===================================================================== >> >> ==========================================================>> >> gcc -Wp,-MD,/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ >> ulp/ipoib/.ipoib_fs.o.d -nostdinc -isystem /usr/lib/gcc/x86_64- >> redhat-linux/4.1.1/include -D__KERNEL__ \ >> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/kernel_addons/backport/2.6.18/ >> include/ \ >> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/include \ >> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/include \ >> -Iinclude \ >> \ >> -include include/linux/autoconf.h \ >> -include /root/lustre/1.6.4.2/ofa_kernel-1.2/include/linux/ >> autoconf.h \ >> -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict- >> aliasing -fno-common -Wstrict-prototypes -Wundef -Werror-implicit- >> function-declaration -Os -mtune=generic -m64 -mno-red-zone - >> mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno- >> asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno- >> sse2 -mno-3dnow -fomit-frame-pointer -g -fno-stack-protector - >> Wdeclaration-after-statement -Wno-pointer-sign -I/root/lustre/ >> 1.6.4.2/ofa_kernel-1.2/include -I/root/lustre/1.6.4.2/ofa_kernel-1.2/ >> drivers/infiniband/include -I/root/lustre/1.6.4.2/ >> ofa_kernel-1.2/drivers/infiniband/ulp/ipoib -I/root/lustre/ >> 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/debug -I/root/lustre/ >> 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/hw/cxgb3/core -I/root/ >> lustre/1.6.4.2/ofa_kernel-1.2/drivers/net/cxgb3 -I/root/lustre/ >> 1.6.4.2/ofa_kernel-1.2/drivers/net/rds -DMODULE -D"KBUILD_STR(s) >> =#s" -D"KBUILD_BASENAME=KBUILD_STR(ipoib_fs)" - >> D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -c -o /root/lustre/1.6.4.2/ >> ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/.tmp_ipoib_fs.o /root/ >> lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ipoib_fs.c >> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >> ipoib_fs.c: In function ?ipoib_mcg_open?: >> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >> ipoib_fs.c:144: error: ?struct inode? has no member named ?u? >> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >> ipoib_fs.c: In function ?ipoib_path_open?: >> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >> ipoib_fs.c:250: error: ?struct inode? has no member named ?u? >> make[4]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ >> ulp/ipoib/ipoib_fs.o] Error 1 >> make[3]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ >> ulp/ipoib] Error 2 >> make[2]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband] >> Error 2 >> make[1]: *** [_module_/root/lustre/1.6.4.2/ofa_kernel-1.2] Error 2 >> make[1]: Leaving directory `/root/lustre/1.6.4.2/linux-2.6.18'' >> make: *** [kernel] Error 2 >> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > > Aaron Knister > Associate Systems Analyst > Center for Ocean-Land-Atmosphere Studies > > (301) 595-7000 > aaron at iges.org > > > >
Charles Taylor
2008-Feb-18 19:11 UTC
[Lustre-discuss] kernel-2.6.18-8.1.14 + lustre 1.6.4.2 + OFED 1.2
We needed OFED 1.2 for another driver we were using (rapidscale). In reality we don''t need that driver any more and perhaps we could use the kernel''s stack but OFED 1.1 is pretty old and there is no real reason to go back - despite the evidence to the contrary. :) ct On Feb 18, 2008, at 11:35 AM, Aaron Knister wrote:> Just out of curiosity, why not use the ofed stack in the kernel > itself? > > On Feb 18, 2008, at 11:20 AM, Charles Taylor wrote: > >> >> We seemed to have it a stumbling block when building with the above >> (supported) versions. Our process... >> >> 1. Start with stock rhel5 2.6.18-8.1.14 source tree >> 2. Configure InfiniBand support out of the the kernel (we will build >> OFED separately). >> 3. Apply the 1.6.4.2 kernel patches to the kernel source. >> 4. Build the kernel. >> 5. Build OFED 1.2 against the patched kernel >> 6. Build Lustre using the patched Kernel+OFED 1.2 >> >> This worked fine for 1.6.3 but for 1.6.4.2, after patching the kernel >> and trying to build OFED 1.2, we get the error below. Now, we find >> references to this problem all over the place but nothing specific to >> building lustre. Since Lustre 1.6.4.2 is supposed to support OFED >> 1.2, it seems that OFED should build against a kernel patched for >> lustre 1.6.4.2 but apparently it does not. If we use the un- >> patched kernel (i.e. do not apply the lustre 1.6.4.2 kernel patches >> to it), OFED 1.2 builds without a problem. We are looking for an >> easy #define way out of this hole right now but if there is a known >> solution to this, we would love to know about it. >> >> If there is a better way to do this, we are open to that as well. >> >> Thanks, >> >> charlie taylor >> uf hpc center >> >> >> >> >> ===================================================================== >> >> ==========================================================>> >> gcc -Wp,-MD,/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ >> ulp/ipoib/.ipoib_fs.o.d -nostdinc -isystem /usr/lib/gcc/x86_64- >> redhat-linux/4.1.1/include -D__KERNEL__ \ >> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/kernel_addons/backport/2.6.18/ >> include/ \ >> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/include \ >> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/include \ >> -Iinclude \ >> \ >> -include include/linux/autoconf.h \ >> -include /root/lustre/1.6.4.2/ofa_kernel-1.2/include/linux/ >> autoconf.h \ >> -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict- >> aliasing -fno-common -Wstrict-prototypes -Wundef -Werror-implicit- >> function-declaration -Os -mtune=generic -m64 -mno-red-zone - >> mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno- >> asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno- >> sse2 -mno-3dnow -fomit-frame-pointer -g -fno-stack-protector - >> Wdeclaration-after-statement -Wno-pointer-sign -I/root/lustre/ >> 1.6.4.2/ofa_kernel-1.2/include -I/root/lustre/1.6.4.2/ofa_kernel-1.2/ >> drivers/infiniband/include -I/root/lustre/1.6.4.2/ >> ofa_kernel-1.2/drivers/infiniband/ulp/ipoib -I/root/lustre/ >> 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/debug -I/root/lustre/ >> 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/hw/cxgb3/core -I/root/ >> lustre/1.6.4.2/ofa_kernel-1.2/drivers/net/cxgb3 -I/root/lustre/ >> 1.6.4.2/ofa_kernel-1.2/drivers/net/rds -DMODULE -D"KBUILD_STR(s) >> =#s" -D"KBUILD_BASENAME=KBUILD_STR(ipoib_fs)" - >> D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -c -o /root/lustre/1.6.4.2/ >> ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/.tmp_ipoib_fs.o /root/ >> lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ipoib_fs.c >> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >> ipoib_fs.c: In function ?ipoib_mcg_open?: >> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >> ipoib_fs.c:144: error: ?struct inode? has no member named ?u? >> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >> ipoib_fs.c: In function ?ipoib_path_open?: >> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >> ipoib_fs.c:250: error: ?struct inode? has no member named ?u? >> make[4]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ >> ulp/ipoib/ipoib_fs.o] Error 1 >> make[3]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ >> ulp/ipoib] Error 2 >> make[2]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband] >> Error 2 >> make[1]: *** [_module_/root/lustre/1.6.4.2/ofa_kernel-1.2] Error 2 >> make[1]: Leaving directory `/root/lustre/1.6.4.2/linux-2.6.18'' >> make: *** [kernel] Error 2 >> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > > Aaron Knister > Associate Systems Analyst > Center for Ocean-Land-Atmosphere Studies > > (301) 595-7000 > aaron at iges.org > > > >
Aaron Knister
2008-Feb-18 20:18 UTC
[Lustre-discuss] kernel-2.6.18-8.1.14 + lustre 1.6.4.2 + OFED 1.2
I could have sworn the rhel2.6.18 kernel has ofed 1.2. that''s weird. On Feb 18, 2008, at 2:11 PM, Charles Taylor wrote:> We needed OFED 1.2 for another driver we were using (rapidscale). > In reality we don''t need that driver any more and perhaps we could > use the kernel''s stack but OFED 1.1 is pretty old and there is no > real reason to go back - despite the evidence to the contrary. :) > > ct > > On Feb 18, 2008, at 11:35 AM, Aaron Knister wrote: > >> Just out of curiosity, why not use the ofed stack in the kernel >> itself? >> >> On Feb 18, 2008, at 11:20 AM, Charles Taylor wrote: >> >>> >>> We seemed to have it a stumbling block when building with the above >>> (supported) versions. Our process... >>> >>> 1. Start with stock rhel5 2.6.18-8.1.14 source tree >>> 2. Configure InfiniBand support out of the the kernel (we will build >>> OFED separately). >>> 3. Apply the 1.6.4.2 kernel patches to the kernel source. >>> 4. Build the kernel. >>> 5. Build OFED 1.2 against the patched kernel >>> 6. Build Lustre using the patched Kernel+OFED 1.2 >>> >>> This worked fine for 1.6.3 but for 1.6.4.2, after patching the >>> kernel >>> and trying to build OFED 1.2, we get the error below. Now, we >>> find >>> references to this problem all over the place but nothing specific >>> to >>> building lustre. Since Lustre 1.6.4.2 is supposed to support OFED >>> 1.2, it seems that OFED should build against a kernel patched for >>> lustre 1.6.4.2 but apparently it does not. If we use the un- >>> patched kernel (i.e. do not apply the lustre 1.6.4.2 kernel patches >>> to it), OFED 1.2 builds without a problem. We are looking for an >>> easy #define way out of this hole right now but if there is a known >>> solution to this, we would love to know about it. >>> >>> If there is a better way to do this, we are open to that as well. >>> >>> Thanks, >>> >>> charlie taylor >>> uf hpc center >>> >>> >>> >>> >>> = >>> = >>> ===================================================================>>> ==========================================================>>> >>> gcc -Wp,-MD,/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ >>> ulp/ipoib/.ipoib_fs.o.d -nostdinc -isystem /usr/lib/gcc/x86_64- >>> redhat-linux/4.1.1/include -D__KERNEL__ \ >>> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/kernel_addons/backport/2.6.18/ >>> include/ \ >>> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/include \ >>> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/include \ >>> -Iinclude \ >>> \ >>> -include include/linux/autoconf.h \ >>> -include /root/lustre/1.6.4.2/ofa_kernel-1.2/include/linux/ >>> autoconf.h \ >>> -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict- >>> aliasing -fno-common -Wstrict-prototypes -Wundef -Werror-implicit- >>> function-declaration -Os -mtune=generic -m64 -mno-red-zone - >>> mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno- >>> asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno- >>> sse2 -mno-3dnow -fomit-frame-pointer -g -fno-stack-protector - >>> Wdeclaration-after-statement -Wno-pointer-sign -I/root/lustre/ >>> 1.6.4.2/ofa_kernel-1.2/include -I/root/lustre/1.6.4.2/ >>> ofa_kernel-1.2/ >>> drivers/infiniband/include -I/root/lustre/1.6.4.2/ >>> ofa_kernel-1.2/drivers/infiniband/ulp/ipoib -I/root/lustre/ >>> 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/debug -I/root/lustre/ >>> 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/hw/cxgb3/core -I/root/ >>> lustre/1.6.4.2/ofa_kernel-1.2/drivers/net/cxgb3 -I/root/lustre/ >>> 1.6.4.2/ofa_kernel-1.2/drivers/net/rds -DMODULE -D"KBUILD_STR(s) >>> =#s" -D"KBUILD_BASENAME=KBUILD_STR(ipoib_fs)" - >>> D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -c -o /root/lustre/1.6.4.2/ >>> ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/.tmp_ipoib_fs.o /root/ >>> lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >>> ipoib_fs.c >>> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >>> ipoib_fs.c: In function ?ipoib_mcg_open?: >>> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >>> ipoib_fs.c:144: error: ?struct inode? has no member named ?u? >>> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >>> ipoib_fs.c: In function ?ipoib_path_open?: >>> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ >>> ipoib_fs.c:250: error: ?struct inode? has no member named ?u? >>> make[4]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/ >>> infiniband/ >>> ulp/ipoib/ipoib_fs.o] Error 1 >>> make[3]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/ >>> infiniband/ >>> ulp/ipoib] Error 2 >>> make[2]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/ >>> infiniband] >>> Error 2 >>> make[1]: *** [_module_/root/lustre/1.6.4.2/ofa_kernel-1.2] Error 2 >>> make[1]: Leaving directory `/root/lustre/1.6.4.2/linux-2.6.18'' >>> make: *** [kernel] Error 2 >>> >>> >>> >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> Aaron Knister >> Associate Systems Analyst >> Center for Ocean-Land-Atmosphere Studies >> >> (301) 595-7000 >> aaron at iges.org >> >> >> >> >Aaron Knister Associate Systems Analyst Center for Ocean-Land-Atmosphere Studies (301) 595-7000 aaron at iges.org
Jim Garlick
2008-Feb-19 16:06 UTC
[Lustre-discuss] kernel-2.6.18-8.1.14 + lustre 1.6.4.2 + OFED 1.2
I think OFED was updated to 1.2 at the RHEL 5.1 update. Jim On Mon, Feb 18, 2008 at 03:18:34PM -0500, Aaron Knister wrote:> I could have sworn the rhel2.6.18 kernel has ofed 1.2. that''s weird. > > On Feb 18, 2008, at 2:11 PM, Charles Taylor wrote: > > > We needed OFED 1.2 for another driver we were using (rapidscale). > > In reality we don''t need that driver any more and perhaps we could > > use the kernel''s stack but OFED 1.1 is pretty old and there is no > > real reason to go back - despite the evidence to the contrary. :) > > > > ct > > > > On Feb 18, 2008, at 11:35 AM, Aaron Knister wrote: > > > >> Just out of curiosity, why not use the ofed stack in the kernel > >> itself? > >> > >> On Feb 18, 2008, at 11:20 AM, Charles Taylor wrote: > >> > >>> > >>> We seemed to have it a stumbling block when building with the above > >>> (supported) versions. Our process... > >>> > >>> 1. Start with stock rhel5 2.6.18-8.1.14 source tree > >>> 2. Configure InfiniBand support out of the the kernel (we will build > >>> OFED separately). > >>> 3. Apply the 1.6.4.2 kernel patches to the kernel source. > >>> 4. Build the kernel. > >>> 5. Build OFED 1.2 against the patched kernel > >>> 6. Build Lustre using the patched Kernel+OFED 1.2 > >>> > >>> This worked fine for 1.6.3 but for 1.6.4.2, after patching the > >>> kernel > >>> and trying to build OFED 1.2, we get the error below. Now, we > >>> find > >>> references to this problem all over the place but nothing specific > >>> to > >>> building lustre. Since Lustre 1.6.4.2 is supposed to support OFED > >>> 1.2, it seems that OFED should build against a kernel patched for > >>> lustre 1.6.4.2 but apparently it does not. If we use the un- > >>> patched kernel (i.e. do not apply the lustre 1.6.4.2 kernel patches > >>> to it), OFED 1.2 builds without a problem. We are looking for an > >>> easy #define way out of this hole right now but if there is a known > >>> solution to this, we would love to know about it. > >>> > >>> If there is a better way to do this, we are open to that as well. > >>> > >>> Thanks, > >>> > >>> charlie taylor > >>> uf hpc center > >>> > >>> > >>> > >>> > >>> = > >>> = > >>> ===================================================================> >>> ==========================================================> >>> > >>> gcc -Wp,-MD,/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ > >>> ulp/ipoib/.ipoib_fs.o.d -nostdinc -isystem /usr/lib/gcc/x86_64- > >>> redhat-linux/4.1.1/include -D__KERNEL__ \ > >>> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/kernel_addons/backport/2.6.18/ > >>> include/ \ > >>> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/include \ > >>> -I/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/include \ > >>> -Iinclude \ > >>> \ > >>> -include include/linux/autoconf.h \ > >>> -include /root/lustre/1.6.4.2/ofa_kernel-1.2/include/linux/ > >>> autoconf.h \ > >>> -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict- > >>> aliasing -fno-common -Wstrict-prototypes -Wundef -Werror-implicit- > >>> function-declaration -Os -mtune=generic -m64 -mno-red-zone - > >>> mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare -fno- > >>> asynchronous-unwind-tables -funit-at-a-time -mno-sse -mno-mmx -mno- > >>> sse2 -mno-3dnow -fomit-frame-pointer -g -fno-stack-protector - > >>> Wdeclaration-after-statement -Wno-pointer-sign -I/root/lustre/ > >>> 1.6.4.2/ofa_kernel-1.2/include -I/root/lustre/1.6.4.2/ > >>> ofa_kernel-1.2/ > >>> drivers/infiniband/include -I/root/lustre/1.6.4.2/ > >>> ofa_kernel-1.2/drivers/infiniband/ulp/ipoib -I/root/lustre/ > >>> 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/debug -I/root/lustre/ > >>> 1.6.4.2/ofa_kernel-1.2/drivers/infiniband/hw/cxgb3/core -I/root/ > >>> lustre/1.6.4.2/ofa_kernel-1.2/drivers/net/cxgb3 -I/root/lustre/ > >>> 1.6.4.2/ofa_kernel-1.2/drivers/net/rds -DMODULE -D"KBUILD_STR(s) > >>> =#s" -D"KBUILD_BASENAME=KBUILD_STR(ipoib_fs)" - > >>> D"KBUILD_MODNAME=KBUILD_STR(ib_ipoib)" -c -o /root/lustre/1.6.4.2/ > >>> ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/.tmp_ipoib_fs.o /root/ > >>> lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > >>> ipoib_fs.c > >>> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > >>> ipoib_fs.c: In function ?ipoib_mcg_open?: > >>> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > >>> ipoib_fs.c:144: error: ?struct inode? has no member named ?u? > >>> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > >>> ipoib_fs.c: In function ?ipoib_path_open?: > >>> /root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib/ > >>> ipoib_fs.c:250: error: ?struct inode? has no member named ?u? > >>> make[4]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/ > >>> infiniband/ > >>> ulp/ipoib/ipoib_fs.o] Error 1 > >>> make[3]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/ > >>> infiniband/ > >>> ulp/ipoib] Error 2 > >>> make[2]: *** [/root/lustre/1.6.4.2/ofa_kernel-1.2/drivers/ > >>> infiniband] > >>> Error 2 > >>> make[1]: *** [_module_/root/lustre/1.6.4.2/ofa_kernel-1.2] Error 2 > >>> make[1]: Leaving directory `/root/lustre/1.6.4.2/linux-2.6.18'' > >>> make: *** [kernel] Error 2 > >>> > >>> > >>> > >>> _______________________________________________ > >>> Lustre-discuss mailing list > >>> Lustre-discuss at lists.lustre.org > >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss > >> > >> Aaron Knister > >> Associate Systems Analyst > >> Center for Ocean-Land-Atmosphere Studies > >> > >> (301) 595-7000 > >> aaron at iges.org > >> > >> > >> > >> > > > > Aaron Knister > Associate Systems Analyst > Center for Ocean-Land-Atmosphere Studies > > (301) 595-7000 > aaron at iges.org > > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss