Thierry Delaitre
2006-Sep-25 05:51 UTC
[Lustre-discuss] Re: [openib-general] problems with lustre o2ib module & ofed
It seems that lustre puts its modules in /lib/modules/2.6.16.21-0.8-default despite the fact that my kernel is 2.6.16.21-0.8-smp ! uname -a Linux n32 2.6.16.21-0.8-smp #4 SMP Sun Sep 24 08:47:30 BST 2006 i686 i686 i386 GNU/Linux make[3]: Nothing to be done for `install-exec-am''. /bin/sh ../../mkinstalldirs /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre /usr/bin/install -c -m 644 lquota.ko /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre/lquota I therefore ends up with a /lib/modules/2.6.16.21-0.8-smp and /lib/modules/2.6.16.21-0.8-default i''m now searching why lustre thinks my kernel is 2.6.16.21-0.8-default and not 2.6.16.21-0.8-smp Thierry. On Mon, 25 Sep 2006, Thierry Delaitre wrote:> > On Mon, 25 Sep 2006, Michael S. Tsirkin wrote: > > > Quoting r. Thierry Delaitre <delaitt@cpc.wmin.ac.uk>: > > > > > > I''ve set the o2ib path to /usr/local/ofed/src/openib-1.1 as shown in the > > > lustre''s configure line below. Lustre''s configure script looks for a > > > driver/infiniband directory which only seems to exist under > > > /usr/local/ofed/src/openib-1.1 > > > > > > ./configure --with-o2ib=/usr/local/ofed/src/openib-1.1/ > > > > > > Thierry. > > > > > > > replace /usr/local/ofed with the prefix you specified. > > > > This looks wrong - openib-1.1 is the pristine sources. > > openib/include is the exported interface and is what you should use > > for dependent modules. > > No idea why would lustre need drivers/infiniband. > > Try creating a softlink: > > > > mkdir /usr/local/ofed/src/openib/drivers/infiniband > > ln -s /usr/local/ofed/src/openib/include /usr/local/ofed/src/openib/drivers/infiniband > > I untarred lustre 1.5.95, compiled it (./configure > --with-o2ib=/usr/local/ofed/src/openib) . did a make install, depmod -a > and still get the following: > > my modprobe.conf is the following > > options lnet ip2nets="o2ib0 161.74.83.[0-255]" > > lctl network up > LNET configure error 100: Network is down > > ko2iblnd: disagrees about version of symbol ib_create_cq > ko2iblnd: Unknown symbol ib_create_cq > ko2iblnd: disagrees about version of symbol ib_dereg_mr > ko2iblnd: Unknown symbol ib_dereg_mr > ko2iblnd: disagrees about version of symbol ib_destroy_cq > ko2iblnd: Unknown symbol ib_destroy_cq > ko2iblnd: disagrees about version of symbol ib_get_dma_mr > ko2iblnd: Unknown symbol ib_get_dma_mr > ko2iblnd: disagrees about version of symbol ib_alloc_pd > ko2iblnd: Unknown symbol ib_alloc_pd > ko2iblnd: disagrees about version of symbol ib_modify_qp > ko2iblnd: Unknown symbol ib_modify_qp > ko2iblnd: disagrees about version of symbol ib_dealloc_pd > ko2iblnd: Unknown symbol ib_dealloc_pd > LustreError: 4177:0:(api-ni.c:1002:lnet_startup_lndnis()) Can''t load LND > o2ib, module ko2iblnd, rc=256 > > lsmod | grep ib > libcfs 103060 1 lnet > ib_ucm 19332 0 > ib_addr 10756 1 rdma_cm > ib_cm 31968 2 ib_ucm,rdma_cm > ib_ipoib 48400 0 > ib_sa 16652 3 rdma_cm,ib_cm,ib_ipoib > ib_uverbs 38312 2 rdma_ucm,ib_ucm > ib_umad 17968 0 > ib_mthca 116240 0 > ib_mad 36116 4 ib_cm,ib_sa,ib_umad,ib_mthca > ib_core 49024 9 > ib_ucm,rdma_cm,ib_cm,ib_ipoib,ib_sa,ib_uverbs,ib_umad,ib_mthca,ib_mad > > nm /lib/modules/2.6.16.21-0.8-smp/kernel/drivers/infiniband/core/ib_core.ko | grep ib_alloc_pd > d5dcb698 A __crc_ib_alloc_pd > 0000001c r __kcrctab_ib_alloc_pd > 0000006a r __kstrtab_ib_alloc_pd > 00000038 r __ksymtab_ib_alloc_pd > 00000c65 T ib_alloc_pd > > from lustre''s config.log: > > configure:6500: checking whether to enable OpenIB gen2 support > configure:6586: cp conftest.c build && make modules CC=gcc -f > /root/lustre-1.5.95/build/Makefile LUSTRE_LINUX > _CONFIG=/usr/src/linux/.config -o tmp_include_depends -o scripts -o include/config/MARKER -C /usr/src/linux EXTRA_CFLAGS=-Werror-implicit-function-declaration -g -I/root/lustre-1.5.95/lnet/include -I/root/lustre-1.5.95/lustre/include -I/usr/local/ofed/src/openib/include M=/root/lustre-1.5.95/build > /root/lustre-1.5.95/build/conftest.c:42: warning: function declaration > isn''t a prototype > /root/lustre-1.5.95/build/conftest.c: In function ''main'': > /root/lustre-1.5.95/build/conftest.c:49: warning: unused variable ''rej_reason'' > /root/lustre-1.5.95/build/conftest.c:48: warning: unused variable ''pool_fmr'' > /root/lustre-1.5.95/build/conftest.c:47: warning: unused variable ''qp_attr'' > /root/lustre-1.5.95/build/conftest.c:46: warning: unused variable ''device_attr'' > /root/lustre-1.5.95/build/conftest.c:45: warning: unused variable ''conn_param'' > WARNING: "rdma_create_id" [/root/lustre-1.5.95/build/conftest.ko] undefined! > configure:6589: $? = 0 > configure:6591: test -s build/conftest.o > configure:6594: $? = 0 > configure:6597: result: yes > > > Thierry. > > _______________________________________________ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > >---------------------------------------- Dr Thierry DELAITRE Systems and Services Manager, CSCS University of Westminster 115 New Cavendish Street, London W1W 6UW Tel: 020 7911 5000 ext: 3586 Fax: 020 7911 5089 Mobile short dial code 1788 http://www.cscs.wmin.ac.uk/~delaitt ---------------------------------------- This e-mail and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must not copy or show them to anyone, nor should you take any action based on them, other than to notify the error by replying to the sender.
Thierry Delaitre
2006-Sep-25 08:03 UTC
[Lustre-discuss] Re: [openib-general] problems with lustre o2ib module & ofed
On Mon, 25 Sep 2006, Thierry Delaitre wrote:> > It seems that lustre puts its modules in /lib/modules/2.6.16.21-0.8-default > despite the fact that my kernel is 2.6.16.21-0.8-smp ! > > uname -a > Linux n32 2.6.16.21-0.8-smp #4 SMP Sun Sep 24 08:47:30 BST 2006 i686 i686 i386 GNU/Linux > > make[3]: Nothing to be done for `install-exec-am''. > /bin/sh ../../mkinstalldirs /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre > /usr/bin/install -c -m 644 lquota.ko /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre/lquota > > I therefore ends up with a /lib/modules/2.6.16.21-0.8-smp and > /lib/modules/2.6.16.21-0.8-default > > i''m now searching why lustre thinks my kernel is 2.6.16.21-0.8-default and > not 2.6.16.21-0.8-smpI''ve updated the UTS_RELEASE string in /usr/src/linux-2.6.16.21-0.8/include/linux/version.h from default to smp and deleted my /lib/modules/ lustre now installs in /lib/modules/2.6.16.21-0.8-smp/kernel along with ofed ib drivers. i recompiled the kernel, ofed and lustre and still gets this: ko2iblnd: disagrees about version of symbol ib_create_cq ko2iblnd: Unknown symbol ib_create_cq ko2iblnd: disagrees about version of symbol ib_dereg_mr ko2iblnd: Unknown symbol ib_dereg_mr ko2iblnd: disagrees about version of symbol ib_destroy_cq ko2iblnd: Unknown symbol ib_destroy_cq ko2iblnd: disagrees about version of symbol ib_get_dma_mr ko2iblnd: Unknown symbol ib_get_dma_mr ko2iblnd: disagrees about version of symbol ib_alloc_pd ko2iblnd: Unknown symbol ib_alloc_pd ko2iblnd: disagrees about version of symbol ib_modify_qp ko2iblnd: Unknown symbol ib_modify_qp ko2iblnd: disagrees about version of symbol ib_dealloc_pd ko2iblnd: Unknown symbol ib_dealloc_pd LustreError: 7430:0:(api-ni.c:1002:lnet_startup_lndnis()) Can''t load LND o2ib, module ko2iblnd, rc=256 nm /lib/modules/2.6.16.21-0.8-smp/kernel/drivers/infiniband/core/ib_core.ko | grep ib_create_cq 3cfe7afa A __crc_ib_create_cq 00000060 r __kcrctab_ib_create_cq 0000015f r __kstrtab_ib_create_cq 000000c0 r __ksymtab_ib_create_cq 00000d50 T ib_create_cq i''m a bit stuck! Thierry.> Thierry. > > On Mon, 25 Sep 2006, Thierry Delaitre wrote: > > > > > On Mon, 25 Sep 2006, Michael S. Tsirkin wrote: > > > > > Quoting r. Thierry Delaitre <delaitt@cpc.wmin.ac.uk>: > > > > > > > > I''ve set the o2ib path to /usr/local/ofed/src/openib-1.1 as shown in the > > > > lustre''s configure line below. Lustre''s configure script looks for a > > > > driver/infiniband directory which only seems to exist under > > > > /usr/local/ofed/src/openib-1.1 > > > > > > > > ./configure --with-o2ib=/usr/local/ofed/src/openib-1.1/ > > > > > > > > Thierry. > > > > > > > > > replace /usr/local/ofed with the prefix you specified. > > > > > > This looks wrong - openib-1.1 is the pristine sources. > > > openib/include is the exported interface and is what you should use > > > for dependent modules. > > > No idea why would lustre need drivers/infiniband. > > > Try creating a softlink: > > > > > > mkdir /usr/local/ofed/src/openib/drivers/infiniband > > > ln -s /usr/local/ofed/src/openib/include /usr/local/ofed/src/openib/drivers/infiniband > > > > I untarred lustre 1.5.95, compiled it (./configure > > --with-o2ib=/usr/local/ofed/src/openib) . did a make install, depmod -a > > and still get the following: > > > > my modprobe.conf is the following > > > > options lnet ip2nets="o2ib0 161.74.83.[0-255]" > > > > lctl network up > > LNET configure error 100: Network is down > > > > ko2iblnd: disagrees about version of symbol ib_create_cq > > ko2iblnd: Unknown symbol ib_create_cq > > ko2iblnd: disagrees about version of symbol ib_dereg_mr > > ko2iblnd: Unknown symbol ib_dereg_mr > > ko2iblnd: disagrees about version of symbol ib_destroy_cq > > ko2iblnd: Unknown symbol ib_destroy_cq > > ko2iblnd: disagrees about version of symbol ib_get_dma_mr > > ko2iblnd: Unknown symbol ib_get_dma_mr > > ko2iblnd: disagrees about version of symbol ib_alloc_pd > > ko2iblnd: Unknown symbol ib_alloc_pd > > ko2iblnd: disagrees about version of symbol ib_modify_qp > > ko2iblnd: Unknown symbol ib_modify_qp > > ko2iblnd: disagrees about version of symbol ib_dealloc_pd > > ko2iblnd: Unknown symbol ib_dealloc_pd > > LustreError: 4177:0:(api-ni.c:1002:lnet_startup_lndnis()) Can''t load LND > > o2ib, module ko2iblnd, rc=256 > > > > lsmod | grep ib > > libcfs 103060 1 lnet > > ib_ucm 19332 0 > > ib_addr 10756 1 rdma_cm > > ib_cm 31968 2 ib_ucm,rdma_cm > > ib_ipoib 48400 0 > > ib_sa 16652 3 rdma_cm,ib_cm,ib_ipoib > > ib_uverbs 38312 2 rdma_ucm,ib_ucm > > ib_umad 17968 0 > > ib_mthca 116240 0 > > ib_mad 36116 4 ib_cm,ib_sa,ib_umad,ib_mthca > > ib_core 49024 9 > > ib_ucm,rdma_cm,ib_cm,ib_ipoib,ib_sa,ib_uverbs,ib_umad,ib_mthca,ib_mad > > > > nm /lib/modules/2.6.16.21-0.8-smp/kernel/drivers/infiniband/core/ib_core.ko | grep ib_alloc_pd > > d5dcb698 A __crc_ib_alloc_pd > > 0000001c r __kcrctab_ib_alloc_pd > > 0000006a r __kstrtab_ib_alloc_pd > > 00000038 r __ksymtab_ib_alloc_pd > > 00000c65 T ib_alloc_pd > > > > from lustre''s config.log: > > > > configure:6500: checking whether to enable OpenIB gen2 support > > configure:6586: cp conftest.c build && make modules CC=gcc -f > > /root/lustre-1.5.95/build/Makefile LUSTRE_LINUX > > _CONFIG=/usr/src/linux/.config -o tmp_include_depends -o scripts -o include/config/MARKER -C /usr/src/linux EXTRA_CFLAGS=-Werror-implicit-function-declaration -g -I/root/lustre-1.5.95/lnet/include -I/root/lustre-1.5.95/lustre/include -I/usr/local/ofed/src/openib/include M=/root/lustre-1.5.95/build > > /root/lustre-1.5.95/build/conftest.c:42: warning: function declaration > > isn''t a prototype > > /root/lustre-1.5.95/build/conftest.c: In function ''main'': > > /root/lustre-1.5.95/build/conftest.c:49: warning: unused variable ''rej_reason'' > > /root/lustre-1.5.95/build/conftest.c:48: warning: unused variable ''pool_fmr'' > > /root/lustre-1.5.95/build/conftest.c:47: warning: unused variable ''qp_attr'' > > /root/lustre-1.5.95/build/conftest.c:46: warning: unused variable ''device_attr'' > > /root/lustre-1.5.95/build/conftest.c:45: warning: unused variable ''conn_param'' > > WARNING: "rdma_create_id" [/root/lustre-1.5.95/build/conftest.ko] undefined! > > configure:6589: $? = 0 > > configure:6591: test -s build/conftest.o > > configure:6594: $? = 0 > > configure:6597: result: yes > > > > > > Thierry. > > > > _______________________________________________ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > > > ---------------------------------------- > Dr Thierry DELAITRE > Systems and Services Manager, CSCS > University of Westminster > 115 New Cavendish Street, London W1W 6UW > > Tel: 020 7911 5000 ext: 3586 > Fax: 020 7911 5089 > Mobile short dial code 1788 > > http://www.cscs.wmin.ac.uk/~delaitt > ---------------------------------------- > > This e-mail and its attachments are intended for the above named only > and may be confidential. If they have come to you in error you must > not copy or show them to anyone, nor should you take any action based > on them, other than to notify the error by replying to the sender. > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > >---------------------------------------- Dr Thierry DELAITRE Systems and Services Manager, CSCS University of Westminster 115 New Cavendish Street, London W1W 6UW Tel: 020 7911 5000 ext: 3586 Fax: 020 7911 5089 Mobile short dial code 1788 http://www.cscs.wmin.ac.uk/~delaitt ---------------------------------------- This e-mail and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must not copy or show them to anyone, nor should you take any action based on them, other than to notify the error by replying to the sender.
Thierry Delaitre
2006-Sep-26 11:42 UTC
[openib-general] [Lustre-discuss] Re: problems with lustre o2ib module & ofed
On Tue, 26 Sep 2006, Jack Morgenstein wrote:> On Monday 25 September 2006 17:01, Thierry Delaitre wrote: > > I noticed in the Lustre configure file the following > --with-linux=path set path to Linux source (default=/usr/src/linux) > > Where does /usr/src/linux link to? > > You might consider explicitly specifying the following options as well in the > Lustre ./configure step: > > --with-linux=path set path to Linux source (default=/usr/src/linux) > --with-linux-obj=path set path to Linux objects dir (default=$LINUX) > --with-linux-config=path > set path to Linux .conf (default=$LINUX_OBJ/.config)I specified the whole string and still the same. ./configure --with-o2ib=/usr/local/ofed/src/openib --with-linux=/usr/src/linux-2.6.16.21-0.8 --with-linux-obj=/usr/src/linux-2.6.16.21-0.8 --with-linux-config=/usr/src/linux-2.6.16.21-0.8/.config Thierry.> - Jack > > > > On Mon, 25 Sep 2006, Thierry Delaitre wrote: > > > > > > > > It seems that lustre puts its modules in /lib/modules/2.6.16.21-0.8-default > > > despite the fact that my kernel is 2.6.16.21-0.8-smp ! > > > > > > uname -a > > > Linux n32 2.6.16.21-0.8-smp #4 SMP Sun Sep 24 08:47:30 BST 2006 i686 i686 i386 GNU/Linux > > > > > > make[3]: Nothing to be done for `install-exec-am''. > > > /bin/sh ../../mkinstalldirs /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre > > > /usr/bin/install -c -m 644 lquota.ko /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre/lquota > > > > > > I therefore ends up with a /lib/modules/2.6.16.21-0.8-smp and > > > /lib/modules/2.6.16.21-0.8-default > > > > > > i''m now searching why lustre thinks my kernel is 2.6.16.21-0.8-default and > > > not 2.6.16.21-0.8-smp > > > > I''ve updated the UTS_RELEASE string in > > /usr/src/linux-2.6.16.21-0.8/include/linux/version.h from default to smp > > and deleted my /lib/modules/ > > lustre now installs in /lib/modules/2.6.16.21-0.8-smp/kernel along with > > ofed ib drivers. i recompiled the kernel, ofed and lustre and still gets > > this: > > > > ko2iblnd: disagrees about version of symbol ib_create_cq > > ko2iblnd: Unknown symbol ib_create_cq > > ko2iblnd: disagrees about version of symbol ib_dereg_mr > > ko2iblnd: Unknown symbol ib_dereg_mr > > ko2iblnd: disagrees about version of symbol ib_destroy_cq > > ko2iblnd: Unknown symbol ib_destroy_cq > > ko2iblnd: disagrees about version of symbol ib_get_dma_mr > > ko2iblnd: Unknown symbol ib_get_dma_mr > > ko2iblnd: disagrees about version of symbol ib_alloc_pd > > ko2iblnd: Unknown symbol ib_alloc_pd > > ko2iblnd: disagrees about version of symbol ib_modify_qp > > ko2iblnd: Unknown symbol ib_modify_qp > > ko2iblnd: disagrees about version of symbol ib_dealloc_pd > > ko2iblnd: Unknown symbol ib_dealloc_pd > > LustreError: 7430:0:(api-ni.c:1002:lnet_startup_lndnis()) Can''t load LND > > o2ib, module ko2iblnd, rc=256 > > > > nm /lib/modules/2.6.16.21-0.8-smp/kernel/drivers/infiniband/core/ib_core.ko | grep ib_create_cq > > 3cfe7afa A __crc_ib_create_cq > > 00000060 r __kcrctab_ib_create_cq > > 0000015f r __kstrtab_ib_create_cq > > 000000c0 r __ksymtab_ib_create_cq > > 00000d50 T ib_create_cq > > > > i''m a bit stuck! > > > > Thierry. > > > > > Thierry. > > > > > > On Mon, 25 Sep 2006, Thierry Delaitre wrote: > > > > > > > > > > > On Mon, 25 Sep 2006, Michael S. Tsirkin wrote: > > > > > > > > > Quoting r. Thierry Delaitre <delaitt@cpc.wmin.ac.uk>: > > > > > > > > > > > > I''ve set the o2ib path to /usr/local/ofed/src/openib-1.1 as shown in the > > > > > > lustre''s configure line below. Lustre''s configure script looks for a > > > > > > driver/infiniband directory which only seems to exist under > > > > > > /usr/local/ofed/src/openib-1.1 > > > > > > > > > > > > ./configure --with-o2ib=/usr/local/ofed/src/openib-1.1/ > > > > > > > > > > > > Thierry. > > > > > > > > > > > > > replace /usr/local/ofed with the prefix you specified. > > > > > > > > > > This looks wrong - openib-1.1 is the pristine sources. > > > > > openib/include is the exported interface and is what you should use > > > > > for dependent modules. > > > > > No idea why would lustre need drivers/infiniband. > > > > > Try creating a softlink: > > > > > > > > > > mkdir /usr/local/ofed/src/openib/drivers/infiniband > > > > > ln -s /usr/local/ofed/src/openib/include /usr/local/ofed/src/openib/drivers/infiniband > > > > > > > > I untarred lustre 1.5.95, compiled it (./configure > > > > --with-o2ib=/usr/local/ofed/src/openib) . did a make install, depmod -a > > > > and still get the following: > > > > > > > > my modprobe.conf is the following > > > > > > > > options lnet ip2nets="o2ib0 161.74.83.[0-255]" > > > > > > > > lctl network up > > > > LNET configure error 100: Network is down > > > > > > > > ko2iblnd: disagrees about version of symbol ib_create_cq > > > > ko2iblnd: Unknown symbol ib_create_cq > > > > ko2iblnd: disagrees about version of symbol ib_dereg_mr > > > > ko2iblnd: Unknown symbol ib_dereg_mr > > > > ko2iblnd: disagrees about version of symbol ib_destroy_cq > > > > ko2iblnd: Unknown symbol ib_destroy_cq > > > > ko2iblnd: disagrees about version of symbol ib_get_dma_mr > > > > ko2iblnd: Unknown symbol ib_get_dma_mr > > > > ko2iblnd: disagrees about version of symbol ib_alloc_pd > > > > ko2iblnd: Unknown symbol ib_alloc_pd > > > > ko2iblnd: disagrees about version of symbol ib_modify_qp > > > > ko2iblnd: Unknown symbol ib_modify_qp > > > > ko2iblnd: disagrees about version of symbol ib_dealloc_pd > > > > ko2iblnd: Unknown symbol ib_dealloc_pd > > > > LustreError: 4177:0:(api-ni.c:1002:lnet_startup_lndnis()) Can''t load LND > > > > o2ib, module ko2iblnd, rc=256 > > > > > > > > lsmod | grep ib > > > > libcfs 103060 1 lnet > > > > ib_ucm 19332 0 > > > > ib_addr 10756 1 rdma_cm > > > > ib_cm 31968 2 ib_ucm,rdma_cm > > > > ib_ipoib 48400 0 > > > > ib_sa 16652 3 rdma_cm,ib_cm,ib_ipoib > > > > ib_uverbs 38312 2 rdma_ucm,ib_ucm > > > > ib_umad 17968 0 > > > > ib_mthca 116240 0 > > > > ib_mad 36116 4 ib_cm,ib_sa,ib_umad,ib_mthca > > > > ib_core 49024 9 > > > > ib_ucm,rdma_cm,ib_cm,ib_ipoib,ib_sa,ib_uverbs,ib_umad,ib_mthca,ib_mad > > > > > > > > nm /lib/modules/2.6.16.21-0.8-smp/kernel/drivers/infiniband/core/ib_core.ko | grep ib_alloc_pd > > > > d5dcb698 A __crc_ib_alloc_pd > > > > 0000001c r __kcrctab_ib_alloc_pd > > > > 0000006a r __kstrtab_ib_alloc_pd > > > > 00000038 r __ksymtab_ib_alloc_pd > > > > 00000c65 T ib_alloc_pd > > > > > > > > from lustre''s config.log: > > > > > > > > configure:6500: checking whether to enable OpenIB gen2 support > > > > configure:6586: cp conftest.c build && make modules CC=gcc -f > > > > /root/lustre-1.5.95/build/Makefile LUSTRE_LINUX > > > > _CONFIG=/usr/src/linux/.config -o tmp_include_depends -o scripts -o include/config/MARKER -C /usr/src/linux EXTRA_CFLAGS=-Werror-implicit-function-declaration -g -I/root/lustre-1.5.95/lnet/include -I/root/lustre-1.5.95/lustre/include -I/usr/local/ofed/src/openib/include M=/root/lustre-1.5.95/build > > > > /root/lustre-1.5.95/build/conftest.c:42: warning: function declaration > > > > isn''t a prototype > > > > /root/lustre-1.5.95/build/conftest.c: In function ''main'': > > > > /root/lustre-1.5.95/build/conftest.c:49: warning: unused variable ''rej_reason'' > > > > /root/lustre-1.5.95/build/conftest.c:48: warning: unused variable ''pool_fmr'' > > > > /root/lustre-1.5.95/build/conftest.c:47: warning: unused variable ''qp_attr'' > > > > /root/lustre-1.5.95/build/conftest.c:46: warning: unused variable ''device_attr'' > > > > /root/lustre-1.5.95/build/conftest.c:45: warning: unused variable ''conn_param'' > > > > WARNING: "rdma_create_id" [/root/lustre-1.5.95/build/conftest.ko] undefined! > > > > configure:6589: $? = 0 > > > > configure:6591: test -s build/conftest.o > > > > configure:6594: $? = 0 > > > > configure:6597: result: yes > > > > > > > > > > > > Thierry. > > > > > > > > _______________________________________________ > > > > openib-general mailing list > > > > openib-general@openib.org > > > > http://openib.org/mailman/listinfo/openib-general > > > > > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > > > > > > > > > > > > > ---------------------------------------- > > > Dr Thierry DELAITRE > > > Systems and Services Manager, CSCS > > > University of Westminster > > > 115 New Cavendish Street, London W1W 6UW > > > > > > Tel: 020 7911 5000 ext: 3586 > > > Fax: 020 7911 5089 > > > Mobile short dial code 1788 > > > > > > http://www.cscs.wmin.ac.uk/~delaitt > > > ---------------------------------------- > > > > > > This e-mail and its attachments are intended for the above named only > > > and may be confidential. If they have come to you in error you must > > > not copy or show them to anyone, nor should you take any action based > > > on them, other than to notify the error by replying to the sender. > > > > > > _______________________________________________ > > > Lustre-discuss mailing list > > > Lustre-discuss@clusterfs.com > > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > > > > > > > > > ---------------------------------------- > > Dr Thierry DELAITRE > > Systems and Services Manager, CSCS > > University of Westminster > > 115 New Cavendish Street, London W1W 6UW > > > > Tel: 020 7911 5000 ext: 3586 > > Fax: 020 7911 5089 > > Mobile short dial code 1788 > > > > http://www.cscs.wmin.ac.uk/~delaitt > > ---------------------------------------- > > > > This e-mail and its attachments are intended for the above named only > > and may be confidential. If they have come to you in error you must > > not copy or show them to anyone, nor should you take any action based > > on them, other than to notify the error by replying to the sender. > > > > _______________________________________________ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > >---------------------------------------- Dr Thierry DELAITRE Systems and Services Manager, CSCS University of Westminster 115 New Cavendish Street, London W1W 6UW Tel: 020 7911 5000 ext: 3586 Fax: 020 7911 5089 Mobile short dial code 1788 http://www.cscs.wmin.ac.uk/~delaitt ---------------------------------------- This e-mail and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must not copy or show them to anyone, nor should you take any action based on them, other than to notify the error by replying to the sender.
Michael S. Tsirkin
2006-Sep-26 21:29 UTC
[openib-general] [Lustre-discuss] Re: problems with lustre o2ib module & ofed
Quoting r. Thierry Delaitre <delaitt@cpc.wmin.ac.uk>:> Subject: Re: [Lustre-discuss] Re: problems with lustre o2ib module & ofed > > > On Tue, 26 Sep 2006, Jack Morgenstein wrote: > > > On Monday 25 September 2006 17:01, Thierry Delaitre wrote: > > > > I noticed in the Lustre configure file the following > > --with-linux=path set path to Linux source (default=/usr/src/linux) > > > > Where does /usr/src/linux link to? > > > > You might consider explicitly specifying the following options as well in the > > Lustre ./configure step: > > > > --with-linux=path set path to Linux source (default=/usr/src/linux) > > --with-linux-obj=path set path to Linux objects dir (default=$LINUX) > > --with-linux-config=path > > set path to Linux .conf (default=$LINUX_OBJ/.config) > > I specified the whole string and still the same. > > ./configure --with-o2ib=/usr/local/ofed/src/openib --with-linux=/usr/src/linux-2.6.16.21-0.8 --with-linux-obj=/usr/src/linux-2.6.16.21-0.8 --with-linux-config=/usr/src/linux-2.6.16.21-0.8/.config > > Thierry.1. Did you reboot after rebuilding everything? 2. Try to check the compiler command line used for building lustre. You must make sure gen2 is before linux kernel in -I flag list. -- MST _______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Thierry Delaitre
2006-Sep-27 00:17 UTC
[Lustre-discuss] Re: problems with lustre o2ib module & ofed
On Tue, 26 Sep 2006, Michael S. Tsirkin wrote:> Quoting r. Thierry Delaitre <delaitt@cpc.wmin.ac.uk>: > > Subject: Re: [Lustre-discuss] Re: problems with lustre o2ib module & ofed > > > > > > On Tue, 26 Sep 2006, Jack Morgenstein wrote: > > > > > On Monday 25 September 2006 17:01, Thierry Delaitre wrote: > > > > > > I noticed in the Lustre configure file the following > > > --with-linux=path set path to Linux source (default=/usr/src/linux) > > > > > > Where does /usr/src/linux link to? > > > > > > You might consider explicitly specifying the following options as well in the > > > Lustre ./configure step: > > > > > > --with-linux=path set path to Linux source (default=/usr/src/linux) > > > --with-linux-obj=path set path to Linux objects dir (default=$LINUX) > > > --with-linux-config=path > > > set path to Linux .conf (default=$LINUX_OBJ/.config) > > > > I specified the whole string and still the same. > > > > ./configure --with-o2ib=/usr/local/ofed/src/openib --with-linux=/usr/src/linux-2.6.16.21-0.8 --with-linux-obj=/usr/src/linux-2.6.16.21-0.8 --with-linux-config=/usr/src/linux-2.6.16.21-0.8/.config > > > > Thierry. > > 1. Did you reboot after rebuilding everything? > > 2. Try to check the compiler command line used for building lustre. > You must make sure gen2 is before linux kernel in -I flag list.I''ve managed to solve the problem by deleting the /usr/src/linux/include/rdma and pointing /usr/src/linux/driver/infiniband to /usr/local/ofed/src/openib/driver/infiniband Thanks to all for your help. Thierry.