Hello,
I''m attempting to get Lustre set up on my machine. I am running CentOS
and
I''ve patched, recompiled and booted into my new kernel using the rhel5
patches. I then compiled Lustre, created the RPMs and installed them.
I''ve
attached as much info below as I could. I ran into this same issue using the
rhel5 rpms from the lustre download site.
[root at catapult ~]# yum list lustre*
Installed Packages
lustre.i386 1.6.4.3-2.6.18_53.1.14 installed
lustre-modules.i386 1.6.4.3-2.6.18_53.1.14 installed
[root at catapult ~]# uname -a
Linux catapult 2.6.18-53.1.14.el5.lustre #1 SMP Sun Mar 9 23:49:12 PDT 2008
i686 i686 i386 GNU/Linux
[root at catapult ~]# modinfo lustre
filename: /lib/modules/2.6.18-53.1.14.el5.lustre
/kernel/fs/lustre/lustre.ko
license: GPL
description: Lustre Lite Client File System
author: Cluster File Systems, Inc. <info at clusterfs.com>
srcversion: 39D35F54FE0ABEDEB6EE0A6
depends: obdclass,mdc,ptlrpc,libcfs,lvfs,lov,lnet
vermagic: 2.6.18-53.1.14.el5.lustre SMP mod_unload 686 REGPARM
4KSTACKS gcc-4.1
[root at catapult ~]# modprobe lustre
WARNING: Error inserting ptlrpc
(/lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/ptlrpc.ko):
Input/output error
WARNING: Error inserting mdc
(/lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/mdc.ko):
Unknown symbol in module, or unknown parameter (see dmesg)
WARNING: Error inserting lov
(/lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lov.ko):
Unknown symbol in module, or unknown parameter (see dmesg)
FATAL: Error inserting lustre
(/lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lustre.ko):
Unknown symbol in module, or unknown parameter (see dmesg)
[root at catapult ~]# ls /lib/modules/2.6.18-53.1.14.el5.lustre
/kernel/fs/lustre
llite_lloop.ko lov.ko lquota.ko lustre.ko lvfs.ko mdc.ko mgc.ko
obdclass.ko obdecho.ko osc.ko ptlrpc.ko
[root at catapult ~]# dmesg
Lustre: OBD class driver, info at clusterfs.com
Lustre Version: 1.6.4.3
Build Version:
1.6.4.3-19691231160000-PRISTINE-.usr.src.linux-2.6.18-53.1.14.el5.lustre
Lustre: Added LNI 192.168.0.3 at tcp [8/256]
LustreError: 2359:0:(api-ni.c:1025:lnet_startup_lndnis()) Can''t load
LND
elan, module kqswlnd, rc=256
Lustre: Removed LNI 192.168.0.3 at tcp
LustreError: 2359:0:(events.c:654:ptlrpc_init_portals()) network
initialisation failed
mdc: Unknown symbol ldlm_prep_enqueue_req
mdc: Unknown symbol ldlm_resource_get
mdc: Unknown symbol lustre_msg_get_last_xid
mdc: Unknown symbol _ldlm_lock_debug
mdc: Unknown symbol ptlrpcd_addref
mdc: Unknown symbol lustre_msg_get_magic
mdc: Unknown symbol ptlrpc_check_set
mdc: Unknown symbol lustre_msg_get_last_committed
mdc: Unknown symbol ptlrpc_queue_wait
mdc: Unknown symbol client_import_del_conn
mdc: Unknown symbol ptlrpc_request_addref
mdc: Unknown symbol lustre_msg_bufcount
mdc: Unknown symbol ptlrpc_invalidate_import
mdc: Unknown symbol ldlm_completion_ast
mdc: Unknown symbol lustre_msg_add_flags
mdc: Unknown symbol client_obd_setup
mdc: Unknown symbol ldlm_cancel_resource_local
mdc: Unknown symbol ldlm_lock_match
mdc: Unknown symbol ptlrpc_set_import_active
mdc: Unknown symbol client_obd_cleanup
mdc: Unknown symbol ptlrpc_prep_bulk_imp
mdc: Unknown symbol ptlrpc_set_add_req
mdc: Unknown symbol lustre_swab_mds_body
mdc: Unknown symbol __ldlm_handle2lock
mdc: Unknown symbol ldlm_cli_cancel_list
mdc: Unknown symbol ptlrpcd_add_req
mdc: Unknown symbol llog_client_ops
mdc: Unknown symbol ptlrpc_prep_bulk_page
mdc: Unknown symbol lustre_msg_buf
mdc: Unknown symbol lustre_msg_buflen
mdc: Unknown symbol ldlm_lock_put
mdc: Unknown symbol lustre_swab_obd_statfs
mdc: Unknown symbol client_import_add_conn
mdc: Unknown symbol ldlm_lock_addref
mdc: Unknown symbol ldlm_lock_decref_and_cancel
mdc: Unknown symbol ldlm_resource_iterate
mdc: Unknown symbol unlock_res_and_lock
mdc: Unknown symbol ldlm_cli_enqueue
mdc: Unknown symbol lock_res_and_lock
mdc: Unknown symbol client_disconnect_export
mdc: Unknown symbol ptlrpc_free_rq_pool
mdc: Unknown symbol lustre_msg_get_opc
mdc: Unknown symbol ptlrpc_import_setasync
mdc: Unknown symbol lprocfs_wr_ping
mdc: Unknown symbol ldlm_namespace_cleanup
mdc: Unknown symbol lustre_msg_get_status
mdc: Unknown symbol ldlm_resource_putref
mdc: Unknown symbol lustre_msg_size
mdc: Unknown symbol ldlm_it2str
mdc: Unknown symbol _debug_req
mdc: Unknown symbol lustre_msg_get_type
mdc: Unknown symbol lustre_swab_repbuf
mdc: Unknown symbol ptlrpc_recover_import
mdc: Unknown symbol ptlrpc_prep_req
mdc: Unknown symbol client_connect_import
mdc: Unknown symbol ptlrpcd_decref
mdc: Unknown symbol ldlm_cli_enqueue_fini
mdc: Unknown symbol lustre_msg_set_buflen
mdc: Unknown symbol ptlrpc_req_finished
mdc: Unknown symbol ldlm_lock_decref
lov: Unknown symbol ptlrpc_set_destroy
lov: Unknown symbol ptlrpc_prep_set
lov: Unknown symbol _ldlm_lock_debug
lov: Unknown symbol lustre_swab_lov_user_md
lov: Unknown symbol __ldlm_handle2lock
lov: Unknown symbol ldlm_lock_put
lov: Unknown symbol ptlrpc_activate_import
lov: Unknown symbol ldlm_lock_allow_match
lov: Unknown symbol lustre_swab_lov_desc
lov: Unknown symbol ptlrpc_set_wait
lustre: Unknown symbol ptlrpc_set_destroy
lustre: Unknown symbol ldlm_lock_dump_handle
lustre: Unknown symbol lov_stripe_lock
lustre: Unknown symbol mdc_set_open_replay_data
lustre: Unknown symbol ptlrpc_prep_set
lustre: Unknown symbol lustre_msg_string
lustre: Unknown symbol ldlm_cli_cancel_unused
lustre: Unknown symbol _ldlm_lock_debug
lustre: Unknown symbol mdc_revalidate_lock
lustre: Unknown symbol mdc_clear_open_replay_data
lustre: Unknown symbol mdc_setxattr
lustre: Unknown symbol mdc_getattr
lustre: Unknown symbol ptlrpc_queue_wait
lustre: Unknown symbol mdc_sync
lustre: Unknown symbol ldlm_extent_shift_kms
lustre: Unknown symbol ldlm_flock_completion_ast
lustre: Unknown symbol lov_test_and_clear_async_rc
lustre: Unknown symbol mdc_req2lustre_md
lustre: Unknown symbol mdc_change_cbdata
lustre: Unknown symbol lustre_swab_lov_user_md
lustre: Unknown symbol lustre_msg_bufcount
lustre: Unknown symbol mdc_intent_lock
lustre: Unknown symbol ldlm_completion_ast
lustre: Unknown symbol ldlm_lock_match
lustre: Unknown symbol mdc_set_lock_data
lustre: Unknown symbol ptlrpc_set_import_active
lustre: Unknown symbol lustre_swab_lov_user_md_join
lustre: Unknown symbol mdc_rename
lustre: Unknown symbol mdc_getstatus
lustre: Unknown symbol mdc_getattr_name
lustre: Unknown symbol lustre_pack_reply
lustre: Unknown symbol lustre_msg_buf
lustre: Unknown symbol ldlm_cli_cancel
lustre: Unknown symbol lustre_msg_buflen
lustre: Unknown symbol it_open_error
lustre: Unknown symbol mdc_intent_getattr_async
lustre: Unknown symbol mdc_create
lustre: Unknown symbol mdc_readpage
lustre: Unknown symbol unlock_res_and_lock
lustre: Unknown symbol ldlm_cli_enqueue
lustre: Unknown symbol lock_res_and_lock
lustre: Unknown symbol ldlm_lock2handle
lustre: Unknown symbol mdc_link
lustre: Unknown symbol mdc_setattr
lustre: Unknown symbol it_clear_disposition
lustre: Unknown symbol mdc_free_lustre_md
lustre: Unknown symbol mdc_getxattr
lustre: Unknown symbol mdc_store_inode_generation
lustre: Unknown symbol lustre_msg_size
lustre: Unknown symbol ldlm_it2str
lustre: Unknown symbol lustre_swab_repbuf
lustre: Unknown symbol lustre_swab_lov_user_md_objects
lustre: Unknown symbol ptlrpc_prep_req
lustre: Unknown symbol lov_stripe_unlock
lustre: Unknown symbol ptlrpc_set_wait
lustre: Unknown symbol mdc_unlink
lustre: Unknown symbol it_disposition
lustre: Unknown symbol mdc_init_ea_size
lustre: Unknown symbol mdc_enqueue
lustre: Unknown symbol ptlrpc_req_finished
lustre: Unknown symbol ldlm_lock_decref
lustre: Unknown symbol mdc_close
Thanks in advance for any help!
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080310/c693a92c/attachment-0002.html
On Mon, Mar 10, 2008 at 10:04:50AM -0500, mitcheloc wrote:> > [root at catapult ~]# dmesg > Lustre: OBD class driver, [2]info at clusterfs.com > Lustre Version: [3]1.6.4.3 > Build Version: > 1.6.4.3-19691231160000-PRISTINE-.usr.src.linux-2.6.18-53.1.14.el5.lustr > e > Lustre: Added LNI 192.168.0.3 at tcp [8/256] > LustreError: 2359:0:(api-ni.c:1025:lnet_startup_lndnis()) Can''t load > LND elan, module kqswlnd, rc=256LNet couldn''t load the driver module (kqswlnd) for elan. What''s your lnet module parameters? Please also run: ls /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre rpm -ql lustre-modules Thanks, Isaac> Lustre: Removed LNI [4]192.168.0.3 at tcp > LustreError: 2359:0:(events.c:654:ptlrpc_init_portals()) network > initialisation failed
Isaac, Thanks for the quick response. A quick google search didn''t tell me how I can check the module parameters. What command or file should I check for this? And as you requested: [root at catapult ~]# ls /lib/modules/2.6.18-53.1.14.el5.lustre /kernel/net/lustre ksocklnd.ko libcfs.ko lnet.ko lnet_selftest.ko [root at catapult ~]# rpm -ql lustre-modules /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/llite_lloop.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lov.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lquota.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lustre.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lvfs.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/mdc.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/mgc.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/obdclass.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/obdecho.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/osc.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/ptlrpc.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/ksocklnd.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/libcfs.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/lnet.ko /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/lnet_selftest.ko /usr/share/doc/lustre-modules-1.6.4.3 /usr/share/doc/lustre-modules-1.6.4.3/COPYING Thank you! On Mon, Mar 10, 2008 at 11:02 AM, Isaac Huang <He.Huang at sun.com> wrote:> On Mon, Mar 10, 2008 at 10:04:50AM -0500, mitcheloc wrote: > > > > [root at catapult ~]# dmesg > > Lustre: OBD class driver, [2]info at clusterfs.com > > Lustre Version: [3]1.6.4.3 > > Build Version: > > > 1.6.4.3-19691231160000-PRISTINE-.usr.src.linux-2.6.18-53.1.14.el5.lustr > > e > > Lustre: Added LNI 192.168.0.3 at tcp [8/256] > > LustreError: 2359:0:(api-ni.c:1025:lnet_startup_lndnis()) Can''t load > > LND elan, module kqswlnd, rc=256 > > LNet couldn''t load the driver module (kqswlnd) for elan. What''s your > lnet module parameters? > > Please also run: > ls /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre > rpm -ql lustre-modules > > Thanks, > Isaac > > > Lustre: Removed LNI [4]192.168.0.3 at tcp > > LustreError: 2359:0:(events.c:654:ptlrpc_init_portals()) network > > initialisation failed >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080310/304fd2f0/attachment-0002.html
On Mon, Mar 10, 2008 at 11:19:54AM -0500, mitcheloc wrote:> Isaac, > > Thanks for the quick response. A quick google search didn''t tell me how > I can check the module parameters. What command or file should I check > for this? >It shall be in /etc/modprobe.conf or some file under /etc/modprobe.d. Exact location depends on your distribution. Look for a line that starts with "options lnet ".> > And as you requested: > > [root at catapult ~]# ls > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre > ksocklnd.ko libcfs.ko lnet.ko lnet_selftest.ko >The kqswlnd.ko is missing. Isaac
>From modprobe.conf:options lnet networks=tcp0,elan0 Where should kqswlnd.ko be coming from? On Mon, Mar 10, 2008 at 11:35 AM, Isaac Huang <He.Huang at sun.com> wrote:> On Mon, Mar 10, 2008 at 11:19:54AM -0500, mitcheloc wrote: > > Isaac, > > > > Thanks for the quick response. A quick google search didn''t tell me > how > > I can check the module parameters. What command or file should I > check > > for this? > > > > It shall be in /etc/modprobe.conf or some file under /etc/modprobe.d. > Exact location depends on your distribution. Look for a line that > starts with "options lnet ". > > > > > And as you requested: > > > > [root at catapult ~]# ls > > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre > > ksocklnd.ko libcfs.ko lnet.ko lnet_selftest.ko > > > > The kqswlnd.ko is missing. > > Isaac >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080310/2a3bbe42/attachment-0002.html
On Mon, Mar 10, 2008 at 11:38:33AM -0500, mitcheloc wrote:> From modprobe.conf: > > options lnet networks=tcp0,elan0If you don''t have Quadrics Elan hardware, you can change it to: options lnet networks=tcp0 Otherwise,> Where should kqswlnd.ko be coming from?you need to compile lustre with proper QsNet support. Isaac
Isaac,
I checked my ethernet card and it didn''t look like Quadrics hardware.
[root at catapult ~]# lspci | grep Ethernet
00:19.0 Ethernet controller: Intel Corporation 82566DM Gigabit Network
Connection (rev 02)
So I removed the parameter, rebooted and it worked like a charm! I wonder
how that setting got into my modules.conf file. I checked on another CentOS
system I set up and it is not there. It was probably inserted by some other
DFS I was trying out.
After changing modules.conf and rebooting:
[root at catapult ~]# modprobe lustre
[root at catapult ~]# dmesg
Lustre: OBD class driver, info at clusterfs.com
Lustre Version: 1.6.4.3
Build Version:
1.6.4.3-19691231160000-PRISTINE-.usr.src.linux-2.6.18-53.1.14.el5.lustre
Lustre: Added LNI 192.168.0.3 at tcp [8/256]
Lustre: Accept secure, port 988
Lustre: Lustre Client File System; info at clusterfs.com
Thanks & hopefully I don''t run into any other issues.
Cheers,
Mitchel
On Mon, Mar 10, 2008 at 12:48 PM, Isaac Huang <He.Huang at sun.com> wrote:
> On Mon, Mar 10, 2008 at 11:38:33AM -0500, mitcheloc wrote:
> > From modprobe.conf:
> >
> > options lnet networks=tcp0,elan0
>
> If you don''t have Quadrics Elan hardware, you can change it to:
> options lnet networks=tcp0
>
> Otherwise,
>
> > Where should kqswlnd.ko be coming from?
>
> you need to compile lustre with proper QsNet support.
>
> Isaac
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080310/0d7ba230/attachment-0002.html
Hmm. I did run into this while trying llmount.sh. [root at catapult tests]# pwd /usr/src/lustre-1.6.4.3/lustre/tests [root at catapult tests]# sh llmount.sh Loading modules from /usr/src/lustre-1.6.4.3/lustre/tests/.. lnet options: ''networks=tcp0'' FATAL: Module mgs not found. [root at catapult tests]# dmesg -c [root at catapult tests]# Does this mean I should add a ",mgs" to "networks=tcp0"? On Mon, Mar 10, 2008 at 1:32 PM, mitcheloc <mitcheloc at gmail.com> wrote:> Isaac, > > I checked my ethernet card and it didn''t look like Quadrics hardware. > > [root at catapult ~]# lspci | grep Ethernet > 00:19.0 Ethernet controller: Intel Corporation 82566DM Gigabit Network > Connection (rev 02) > > So I removed the parameter, rebooted and it worked like a charm! I wonder > how that setting got into my modules.conf file. I checked on another > CentOS system I set up and it is not there. It was probably inserted by some > other DFS I was trying out. > > After changing modules.conf and rebooting: > > [root at catapult ~]# modprobe lustre > [root at catapult ~]# dmesg > Lustre: OBD class driver, info at clusterfs.com > Lustre Version: 1.6.4.3 > Build Version: > 1.6.4.3-19691231160000-PRISTINE-.usr.src.linux-2.6.18-53.1.14.el5.lustre > Lustre: Added LNI 192.168.0.3 at tcp [8/256] > Lustre: Accept secure, port 988 > Lustre: Lustre Client File System; info at clusterfs.com > > Thanks & hopefully I don''t run into any other issues. > > Cheers, > Mitchel > > On Mon, Mar 10, 2008 at 12:48 PM, Isaac Huang <He.Huang at sun.com> wrote: > > > On Mon, Mar 10, 2008 at 11:38:33AM -0500, mitcheloc wrote: > > > From modprobe.conf: > > > > > > options lnet networks=tcp0,elan0 > > > > If you don''t have Quadrics Elan hardware, you can change it to: > > options lnet networks=tcp0 > > > > Otherwise, > > > > > Where should kqswlnd.ko be coming from? > > > > you need to compile lustre with proper QsNet support. > > > > Isaac > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080310/71f87237/attachment-0002.html
> Hmm. I did run into this while trying llmount.sh. > [root at catapult tests]# pwd > /usr/src/lustre-1.6.4.3/lustre/tests <http://1.6.4.3/lustre/tests> > [root at catapult tests]# sh llmount.sh > Loading modules from /usr/src/lustre-1.6.4.3/lustre/tests/ > <http://1.6.4.3/lustre/tests/>.. > lnet options: ''networks=tcp0'' > FATAL: Module mgs not found. > [root at catapult tests]# dmesg -c > [root at catapult tests]# > Does this mean I should add a ",mgs" to "networks=tcp0"?Can you verify if mgs module is exist? Run command by Isaac mentioned: ls /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre rpm -ql lustre-modules If so, please try to "modprobe mgs" manually to see if any messages displayed. Jack> ------------------------------------------------------------------------ > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
> Jack, > > Thanks, I had copied the output from that in a previous e-mail and > here it is again: > > [root at catapult ~]# ls > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre > llite_lloop.ko lov.ko lquota.ko lustre.ko lvfs.ko mdc.ko mgc.ko > obdclass.ko obdecho.ko osc.ko ptlrpc.ko > > and: > > [root at catapult ~]# rpm -ql lustre-modules > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/llite_lloop.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lov.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lquota.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lustre.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lvfs.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/mdc.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/mgc.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/obdclass.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/obdecho.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/osc.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/ptlrpc.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/ksocklnd.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/libcfs.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/lnet.ko > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/lnet_selftest.ko > /usr/share/doc/lustre-modules-1.6.4.3 > /usr/share/doc/lustre-modules-1.6.4.3/COPYING > And here is the output for modprobing mgs: > > [root at catapult ~]# modprobe mgs > FATAL: Module mgs not found. > [root at catapult ~]# dmesg > [root at catapult ~]# > >Seems you just built patchless lustre modules, for patched lustre modues as follows: #ls /lib/modules/2.6.18-53.1.13.el5_lustre.1.6.4smp/kernel/fs/lustre/ fsfilt_ldiskfs.ko lov.ko lvfs.ko mgc.ko obdecho.ko ost.ko llite_lloop.ko lquota.ko mdc.ko mgs.ko obdfilter.ko ptlrpc.ko llog_test.ko lustre.ko mds.ko obdclass.ko osc.ko I''m not sure what commands you use to compile lustre(configure/lbuild), I''d recommend you re-compile lustre with patched lustre modules. Normally, you don''t specify any flags, but try to the following commands or by steps as Lustre_manual document described: ./configure --disable-modules --disable-utils --disable-liblustre --disable-tests --disable-doc make clean make dist sh -x $CUR_LUSTRE/build/lbuild --target=2.6-rhel5 --tag=b1_6 --kerneltree=/path/to/kernel --target-arch=$ARCH --lustre=$CUR_LUSTRE/lustre-$VERSION.tar.gz --release ...> Apparently it is not on my system. Looking in the lustre source, it > looks like mgs should have been compiled with everything else. Is > there a special flag to enable it? > > Thank you, > Mitchel > >
I just realized my previous messages were being sent direct and not to the list. Sorry about that Jack. Does anyone have an idea what this message means? /usr/src/lustre-1.6.4.3/build/lbuild: line 1095: syntax error near unexpected token `<'' /usr/src/lustre-1.6.4.3/build/lbuild: line 1095: ` read a b < <(echo $(pwd)/usr/src/linux-*)'' I''m still trying to get the mgs module to build on my system. I don''t understand why the other modules build without issue. This is the latest build command I''ve used and it results in the above error: [root at catapult lustre-1.6.4.3]# sh -x /usr/src/lustre-1.6.4.3/build/lbuild--target2.6-rhel5 --tag=b1_6 --kerneltree=/home/mitchel/rpmbuild/BUILD/kernel-2.6.18 /linux-2.6.18.i686/ --lustre=/usr/src/lustre-1.6.4.3.tar.gz On Mon, Mar 10, 2008 at 4:47 PM, mitcheloc <mitcheloc at gmail.com> wrote:> Jack, > > I did issue a ./configure and make before trying that command. Here is my > second go at the command: > > [root at catapult lustre-1.6.4.3]# sh -x /usr/src/lustre-1.6.4.3/build/lbuild--target> 2.6-rhel5 --tag=b1_6 --kerneltree=/home/mitchel/rpmbuild/BUILD/kernel- > 2.6.18/linux-2.6.18.i686 --lustre=/usr/src/lustre-1.6.4.3.tar.gz > > I''m not sure where $CUR_LUSTRE is supposed to come from? > I get the exact same error running that command: > > ....... > + export CC=gcc > + CC=gcc > /usr/src/lustre-1.6.4.3/build/lbuild: line 1095: syntax error near > unexpected token `<'' > /usr/src/lustre-1.6.4.3/build/lbuild: line 1095: ` read a b < > <(echo $(pwd)/usr/src/linux-*)'' > > > Sorry if I''m overlooking something obvious.... > > > On Mon, Mar 10, 2008 at 4:08 PM, Jack Chen <Jian.Chen at sun.com> wrote: > > > Hi Mitcheloc, > > > > I''m also not sure if the parameters I used is suitable for you, I just > > want to let you know what ''s the lustre build process.but you must run > > ./configure and make before using lbuild command. And it''s necessary to > > add --lustre=$CUR_LUSTRE/lustre-$VERSION.tar.gz.to lbuild command. > > > > I think the steps in quick start page is correct to build patch lustre > > kernel, recompile lustre by quick start doc and reference to my > > method, let''s see if the mgs missed issue can be fixed. > > > > Regards, > > Jack > > > > > > mitcheloc wrote: > > > Jack, > > > > > > I mostly followed the quick start page on the wiki with a combination > > > of other sources to figure out how to patch the CentOS kernel > > properly. > > > > > > Where did you get that lbuild command? I''ve not seen instructions to > > > use it anywhere. > > > > > > I guessed at where the kerneltree param needed to point. Here is what > > > happens: > > > > > > [root at catapult lustre-1.6.4.3]# sh -x > > > /usr/src/lustre-1.6.4.3/build/lbuild <http://1.6.4.3/build/lbuild> > > > --target=2.6-rhel5 --tag=b1_6 > > > --kerneltree=/home/mitchel/rpmbuild/BUILD/kernel-2.6.18/linux- > > 2.6.18.i686/ > > > + TOPDIR=/usr/src/lustre-1.6.4.3 <http://1.6.4.3> > > > + KERNELDIR> > > + LINUX> > > + LUSTRE> > > + RELEASE=false > > > + DO_SRC=0 > > > + DOWNLOAD=1 > > > + TAG> > > + CANONICAL_TARGET> > > + TARGET> > > ++ uname -m > > > + TARGET_ARCH=i686 > > > + TARGET_ARCHS> > > + TARGET_ARCHS_ALL=i686 > > > + ''['' i686 = i686 '']'' > > > + TARGET_ARCHS_ALL=''i686 i586 i386'' > > > + CONFIGURE_FLAGS> > > + EXTERNAL_PATCHES> > > + EXTRA_VERSION> > > + LUSTRE_EXTRA_VERSION> > > + STAGEDIR> > > + TMPDIR=/var/tmp > > > + TIMESTAMP> > > + REUSERPM> > > + REUSEBUILD> > > + NORPM=false > > > + LDISKFSRPM=true > > > + SKIPLDISKFSRPM=''v1_4_* b1_4'' > > > + SMPTYPES=''smp bigsmp default ''\''''''\'''''' > > > + KERNCONFSMPTYPE> > > + PATCHLESS=false > > > + LINUXOBJ> > > + REUSEDKERNELMASK> > > + DISTRO> > > + KERNELTREE> > > + KERNELRPMSBASE> > > + KERNELRPM> > > + KERNELSOURCERPM> > > + RPMSMPTYPE> > > + KERNELRPMCONFIG> > > + KERNELRPMRELEASE> > > + KERNELCOMPILEDIR> > > + KERNEL> > > + SERIES> > > + CONFIG> > > + VERSION> > > + RHBUILD=0 > > > + SUSEBUILD=0 > > > + LINUX26=0 > > > + SUSEBUILD=0 > > > + BASE_ARCHS> > > + BIGMEM_ARCHS> > > + BOOT_ARCHS> > > + JENSEN_ARCHS> > > + SMP_ARCHS> > > + BIGSMP_ARCHS> > > + PSERIES64_ARCHS> > > + UP_ARCHS> > > ++ date > > > + DATE=''Mon Mar 10 13:37:28 PDT 2008'' > > > + USE_DATESTAMP=1 > > > + RPMBUILD> > > + export CC=gcc > > > + CC=gcc > > > /usr/src/lustre-1.6.4.3/build/lbuild <http://1.6.4.3/build/lbuild>: > > > line 1095: syntax error near unexpected token `<'' > > > /usr/src/lustre-1.6.4.3/build/lbuild <http://1.6.4.3/build/lbuild>: > > > line 1095: ` read a b < <(echo $(pwd)/usr/src/linux-*)'' > > > > > > > > > Thanks, > > > Mitchel > > > > > > On Mon, Mar 10, 2008 at 3:32 PM, Jack Chen <Jian.Chen at sun.com > > > <mailto:Jian.Chen at sun.com>> wrote: > > > > > > > > > > Jack, > > > > > > > > Thanks, I had copied the output from that in a previous e-mail > > and > > > > here it is again: > > > > > > > > [root at catapult ~]# ls > > > > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre > > > > llite_lloop.ko lov.ko lquota.ko lustre.ko lvfs.ko mdc.ko > > > mgc.ko > > > > obdclass.ko obdecho.ko osc.ko ptlrpc.ko > > > > > > > > and: > > > > > > > > [root at catapult ~]# rpm -ql lustre-modules > > > > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/fs/lustre/llite_lloop.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lov.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/fs/lustre/lquota.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/fs/lustre/lustre.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/lvfs.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/mdc.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/mgc.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/fs/lustre/obdclass.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/fs/lustre/obdecho.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/fs/lustre/osc.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/fs/lustre/ptlrpc.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/net/lustre/ksocklnd.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/net/lustre/libcfs.ko > > > > /lib/modules/2.6.18-53.1.14.el5.lustre/kernel/net/lustre/lnet.ko > > > > > > > /lib/modules/2.6.18-53.1.14.el5.lustre > > /kernel/net/lustre/lnet_selftest.ko > > > > /usr/share/doc/lustre-modules-1.6.4.3 > > > > /usr/share/doc/lustre-modules-1.6.4.3/COPYING > > > > And here is the output for modprobing mgs: > > > > > > > > [root at catapult ~]# modprobe mgs > > > > FATAL: Module mgs not found. > > > > [root at catapult ~]# dmesg > > > > [root at catapult ~]# > > > > > > > > > > > Seems you just built patchless lustre modules, for patched lustre > > > modues > > > as follows: > > > > > > #ls /lib/modules/2.6.18- > > 53.1.13.el5_lustre.1.6.4smp/kernel/fs/lustre/ > > > fsfilt_ldiskfs.ko lov.ko lvfs.ko mgc.ko obdecho.ko > > > ost.ko > > > llite_lloop.ko lquota.ko mdc.ko mgs.ko obdfilter.ko > > > ptlrpc.ko > > > llog_test.ko lustre.ko mds.ko obdclass.ko osc.ko > > > > > > I''m not sure what commands you use to compile > > > lustre(configure/lbuild), > > > I''d recommend you re-compile lustre with patched lustre modules. > > > Normally, you don''t specify any flags, but try to the following > > > commands > > > or by steps as Lustre_manual document described: > > > > > > ./configure --disable-modules --disable-utils --disable-liblustre > > > --disable-tests --disable-doc > > > make clean > > > make dist > > > sh -x $CUR_LUSTRE/build/lbuild --target=2.6-rhel5 --tag=b1_6 > > > --kerneltree=/path/to/kernel > > > --target-arch=$ARCH --lustre=$CUR_LUSTRE/lustre-$VERSION.tar.gz > > > --release ... > > > > > > > > > > Apparently it is not on my system. Looking in the lustre source, > > it > > > > looks like mgs should have been compiled with everything else. > > Is > > > > there a special flag to enable it? > > > > > > > > Thank you, > > > > Mitchel > > > > > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080313/6ab282cd/attachment-0002.html
> Does anyone have an idea what this message means? > > /usr/src/lustre-1.6.4.3/build/lbuild <http://1.6.4.3/build/lbuild>: > line 1095: syntax error near unexpected token `<'' > /usr/src/lustre-1.6.4.3/build/lbuild <http://1.6.4.3/build/lbuild>: > line 1095: ` read a b < <(echo $(pwd)/usr/src/linux-*)'' > I''m still trying to get the mgs module to build on my system. I don''t > understand why the other modules build without issue. > > This is the latest build command I''ve used and it results in the above > error: > [root at catapult lustre-1.6.4.3]# sh -x > /usr/src/lustre-1.6.4.3/build/lbuild <http://1.6.4.3/build/lbuild> > --target=2.6-rhel5 --tag=b1_6 > --kerneltree=/home/mitchel/rpmbuild/BUILD/kernel-2.6.18/linux-2.6.18.i686/ > --lustre=/usr/src/lustre-1.6.4.3.tar.gz > >Do not use sh to execute lbuild command, please retry "./path/to/lbuild $parameters" Cheers, Jack
On Wed, 2008-03-19 at 11:10 +0800, Jack_Chen wrote:> > Does anyone have an idea what this message means?> > [root at catapult lustre-1.6.4.3]# sh -x> Do not use sh to execute lbuild command, please retry "./path/to/lbuild > $parameters"Or alternatively, if you do want to set flags on execution of lbuild, use "bash" not "sh": bash -x .../lbuild. lbuild contains bashisms that are not supported in the "sh" (i.e. strictly posix compliant) variant of the bourne shell. Cheers, b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080324/091d2bd0/attachment-0002.bin