Lustre discuss - Jun 2006 - Problem in running Lustre 1.4.6.2 with a 2.6.12 vanilla kernel

Hi,

I successfully compiled lustre 1.4.6.2 with a patched 2.6.12 vanilla 
kernel configured for User Mode Linux.

Now, I''m trying to run the llmount.sh script with the configuration 
script local.sh on User Mode Linux.
All is ok until the end of the MDS configuration. But at this point, UML 
freezes and i don''t know why.


Here, are some informations :

standard ouput :
%--------------------------------------------------------------
# NAME=local LCONF=lconf\ -v sh llmount.sh
using objdir module paths
config.portals ./../utils/../../lnet
configuring for host:  [''host-A'']
setting /proc/sys/net/core/rmem_max to at least 16777216
setting /proc/sys/net/core/wmem_max to at least 16777216
Service: network NET_host-A_tcp NET_host-A_tcp_UUID
loading module: libcfs srcdir ./../utils/../../lnet devdir libcfs
+ /sbin/insmod ./../utils/../../lnet/libcfs/libcfs.ko
loading module: lnet srcdir ./../utils/../../lnet devdir lnet
+ /sbin/modprobe lnet
+ /sbin/insmod ./../utils/../../lnet/lnet/lnet.ko accept=all
loading module: ksocklnd srcdir ./../utils/../../lnet devdir klnds/socklnd
+ /sbin/insmod ./../utils/../../lnet/klnds/socklnd/ksocklnd.ko
Service: ldlm ldlm ldlm_UUID
loading module: lvfs srcdir ./../utils/.. devdir lvfs
+ /sbin/insmod ./../utils/../lvfs/lvfs.ko
loading module: obdclass srcdir ./../utils/.. devdir obdclass
+ /sbin/insmod ./../utils/../obdclass/obdclass.ko
loading module: ptlrpc srcdir ./../utils/.. devdir ptlrpc
+ /sbin/insmod ./../utils/../ptlrpc/ptlrpc.ko
Service: osd OSD_OST_host-A_host-A OSD_OST_host-A_host-A_UUID
loading module: ost srcdir ./../utils/.. devdir ost
+ /sbin/insmod ./../utils/../ost/ost.ko
loading module: ldiskfs srcdir ./../utils/.. devdir ldiskfs
+ /sbin/insmod ./../utils/../ldiskfs/ldiskfs.ko
loading module: fsfilt_ldiskfs srcdir ./../utils/.. devdir lvfs
+ /sbin/insmod ./../utils/../lvfs/fsfilt_ldiskfs.ko
loading module: obdfilter srcdir ./../utils/.. devdir obdfilter
+ /sbin/insmod ./../utils/../obdfilter/obdfilter.ko
Service: mdsdev MDD_mds1_host-A MDD_mds1_host-A_UUID
original inode_size  0
stripe_count  1  inode_size  512
loading module: mdc srcdir ./../utils/.. devdir mdc
+ /sbin/insmod ./../utils/../mdc/mdc.ko
loading module: osc srcdir ./../utils/.. devdir osc
+ /sbin/insmod ./../utils/../osc/osc.ko
loading module: lov srcdir ./../utils/.. devdir lov
+ /sbin/insmod ./../utils/../lov/lov.ko
loading module: mds srcdir ./../utils/.. devdir mds
+ /sbin/insmod ./../utils/../mds/mds.ko
Service: mountpoint MNT_host-A MNT_host-A_UUID
get_lov_tgts failed, using get_refs
dbg LOV __init__: [(<__main__.OSC instance at 0x402c97ac>, 0, 1, 1)] 
[u''OST_host-A_UUID''] 1
loading module: llite srcdir ./../utils/.. devdir llite
+ /sbin/insmod ./../utils/../llite/llite.ko
+ sysctl lnet/debug_path /tmp/lustre-log-host-A
+ ./../utils/lctl  modules > /tmp/ogdb-host-A
Service: network NET_host-A_tcp NET_host-A_tcp_UUID
NETWORK: NET_host-A_tcp NET_host-A_tcp_UUID tcp host-A
Service: ldlm ldlm ldlm_UUID
Service: osd OSD_OST_host-A_host-A OSD_OST_host-A_host-A_UUID
OSD: OST_host-A OST_host-A_UUID obdfilter /tmp/ost1-host-A 400000 
ldiskfs no 0 0
+ losetup /dev/loop0
+ losetup /dev/loop1
+ losetup /dev/loop2
+ losetup /dev/loop3
+ losetup /dev/loop4
+ losetup /dev/loop5
+ losetup /dev/loop6
+ losetup /dev/loop7
+ dd if=/dev/zero bs=1k count=0 seek=400000 of=/tmp/ost1-host-A
+ mkfs.ext2 -j -b 4096  -F     /tmp/ost1-host-A 100000
+ tune2fs -O dir_index /tmp/ost1-host-A
+ losetup /dev/loop0
+ losetup /dev/loop0 /tmp/ost1-host-A
+ dumpe2fs -f -h /dev/loop0
no external journal found for /dev/loop0
OST mount options: errors=remount-ro
+ ./../utils/lctl
  attach obdfilter OST_host-A OST_host-A_UUID
  quit
+ ./../utils/lctl
  cfg_device OST_host-A
  setup /dev/loop0 ldiskfs f errors=remount-ro
  quit
+ ./../utils/lctl
  attach ost OSS OSS_UUID
  quit
+ ./../utils/lctl
  cfg_device OSS
  setup
  quit
Service: mdsdev MDD_mds1_host-A MDD_mds1_host-A_UUID
original inode_size  0
stripe_count  1  inode_size  512
MDSDEV: mds1 mds1_UUID /tmp/mds1-host-A ldiskfs no
+ losetup /dev/loop0
+ losetup /dev/loop1
+ losetup /dev/loop2
+ losetup /dev/loop3
+ losetup /dev/loop4
+ losetup /dev/loop5
+ losetup /dev/loop6
+ losetup /dev/loop7
+ dd if=/dev/zero bs=1k count=0 seek=400000 of=/tmp/mds1-host-A
+ mkfs.ext2 -j -b 4096  -F  -i 4096   -I 512 /tmp/mds1-host-A 100000
+ tune2fs -O dir_index /tmp/mds1-host-A
+ losetup /dev/loop0
+ losetup /dev/loop1
+ losetup /dev/loop1 /tmp/mds1-host-A
+ ./../utils/lctl
  attach mds mds1 mds1_UUID
  quit
+ ./../utils/lctl
  cfg_device mds1
  setup /dev/loop1 ldiskfs
  quit

%--------------------------------------------------------------

log output :
%--------------------------------------------------------------
libcfs: no version for "struct_module" found: kernel tainted.
Lustre: OBD class driver Build Version: 
1.4.6.2-19700101010000-PRISTINE-.home.rbois.linux-2.6.12.6-lustre-1.4.6.-2.6.12.6,
info@clusterfs.com
Lustre: Added LNI 172.16.1.2@tcp [8/256]
Lustre: Accept all, port 988
Lustre: Filtering OBD driver; info@clusterfs.com
Lustre: Lustre Lite Client File System; info@clusterfs.com
kjournald starting.  Commit interval 5 seconds
LDISKFS FS on loop0, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
Lustre: 1294:0:(lvfs_linux.c:278:simple_mkdir()) maximum lustre stack 12376
Lustre: 1294:0:(service.c:51:ptlrpc_alloc_request_buffer()) maximum 
lustre stack 12504
Lustre: 1294:0:(lib-lnet.h:261:lnet_me_alloc()) maximum lustre stack 12808
kjournald starting.  Commit interval 5 seconds
LDISKFS FS on loop1, internal journal
LDISKFS-fs: mounted filesystem with ordered data mode.
Lustre: 1361:0:(handler.c:1708:mds_update_server_data()) maximum lustre 
stack 13416

%--------------------------------------------------------------

Thanks for your help,

Rapha?l

On Jun 26, 2006  18:38 +0200, Raphael BOIS wrote:
> I successfully compiled lustre 1.4.6.2 with a patched 2.6.12 vanilla 
> kernel configured for User Mode Linux.
> 
> Now, I''m trying to run the llmount.sh script with the
configuration
> script local.sh on User Mode Linux.
> All is ok until the end of the MDS configuration. But at this point, UML 
> freezes and i don''t know why.
> 
> 
> Lustre: 1294:0:(lvfs_linux.c:278:simple_mkdir()) maximum lustre stack 12376
> Lustre: 1294:0:(service.c:51:ptlrpc_alloc_request_buffer()) maximum 
> lustre stack 12504
> Lustre: 1294:0:(lib-lnet.h:261:lnet_me_alloc()) maximum lustre stack 12808
> kjournald starting.  Commit interval 5 seconds
> LDISKFS FS on loop1, internal journal
> LDISKFS-fs: mounted filesystem with ordered data mode.
> Lustre: 1361:0:(handler.c:1708:mds_update_server_data()) maximum lustre 
> stack 13416
What is the stack size configured on your UML?  It should be:
CONFIG_KERNEL_STACK_ORDER=3

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Andreas Dilger a ?crit :
> On Jun 26, 2006  18:38 +0200, Raphael BOIS wrote:
>   
>> I successfully compiled lustre 1.4.6.2 with a patched 2.6.12 vanilla 
>> kernel configured for User Mode Linux.
>>
>> Now, I''m trying to run the llmount.sh script with the
configuration
>> script local.sh on User Mode Linux.
>> All is ok until the end of the MDS configuration. But at this point,
UML
>> freezes and i don''t know why.
>>
>>
>> Lustre: 1294:0:(lvfs_linux.c:278:simple_mkdir()) maximum lustre stack
12376
>> Lustre: 1294:0:(service.c:51:ptlrpc_alloc_request_buffer()) maximum 
>> lustre stack 12504
>> Lustre: 1294:0:(lib-lnet.h:261:lnet_me_alloc()) maximum lustre stack
12808
>> kjournald starting.  Commit interval 5 seconds
>> LDISKFS FS on loop1, internal journal
>> LDISKFS-fs: mounted filesystem with ordered data mode.
>> Lustre: 1361:0:(handler.c:1708:mds_update_server_data()) maximum lustre
>> stack 13416
>>     
>
> What is the stack size configured on your UML?  It should be:
> CONFIG_KERNEL_STACK_ORDER=3
Ok ! It works now.

Thanks,

Rapha?l