Chris Hunter
2008-Dec-16 16:41 UTC
[Lustre-discuss] tunefs.lustre --mgsnode non-destructive ?
We want to reconfigure our lustre for multi-homed network. However the mgs only appears to respond to clients using the original network (02ib). Here is the kerenel module (modprobe.conf) line > options lnet ''networks="o2ib(ib0),tcp(eth0)"'' When mounting lustre on the new (tcp) network, we see the following syslog messages: > kernel: LustreError: 10899:0:(events.c:454:ptlrpc_uuid_to_peer()) No NID found for 10.2.1.30 at o2ib > kernel: LustreError: 10899:0:(client.c:58:ptlrpc_uuid_to_connection()) cannot find peer 10.2.1.30 at o2 ib! > kernel: LustreError: 10899:0:(ldlm_lib.c:321:client_obd_setup()) can''t add initial connection > kernel: LustreError: 8819:0:(connection.c:144:ptlrpc_put_connection()) NULL connection > kernel: LustreError: 10899:0:(obd_config.c:332:class_setup()) setup lustre0-MDT0000-mdc-ffff81022ea1 cc00 failed (-2) > kernel: LustreError: 10899:0:(obd_config.c:1071:class_config_llog_handler()) Err -2 on cfg command: > kernel: Lustre: cmd=cf003 0:lustre0-MDT0000-mdc 1:lustre0-MDT0000_UUID 2:10.2.1.30 at o2ib > kernel: LustreError: 15c-8: MGC10.1.1.30 at tcp: The configuration from log ''lustre0-client'' failed (-2). After viewing syslogs and searching mailing lists, it appears the mdt needs to be reformatted to include the new network (tcp) routing to the mgs. We have a combined mgs+mdt, My best guess, the upgrade command on the mds is: tunefs.lustre --mgsnode="IP1 at o2ib0,IP2 at tcp0" dev_mdt The original command to make the mdt did not include the mgsnode flag: tunefs --dryrun -l dev_mdt >Reading CONFIGS/mountdata > > Read previous values: >Target: lustre0-MDT0000 >Index: 0 >Lustre FS: lustre0 >Mount type: ldiskfs >Flags: 0x5 > (MDT MGS ) >Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr >Parameters: mdt.group_upcall=/usr/sbin/l_getgroups Q. Is tunefs.lustre command this non-destructive ? tunefs.lustre --mgsnode="IP1 at o2ib0,IP2 at tcp0" dev_mdt Q. Do I need to update config on the OSTs for our new multi-homed network ? Thanks, Chris Hunter
Chris Hunter
2008-Dec-16 22:12 UTC
[Lustre-discuss] tunefs.lustre --mgsnode non-destructive ?
We want to reconfigure our lustre for multi-homed network. However the mgs only appears to respond to clients using the original network (02ib). Here is the kerenel module (modprobe.conf) line> options lnet ''networks="o2ib(ib0),tcp(eth0)"''When mounting lustre on the new (tcp) network, we see the following syslog messages:> kernel: LustreError: 10899:0:(events.c:454:ptlrpc_uuid_to_peer()) NoNID found for 10.2.1.30 at o2ib> kernel: LustreError:10899:0:(client.c:58:ptlrpc_uuid_to_connection()) cannot find peer 10.2.1.30 at o2 ib!> kernel: LustreError: 10899:0:(ldlm_lib.c:321:client_obd_setup())can''t add initial connection> kernel: LustreError:8819:0:(connection.c:144:ptlrpc_put_connection()) NULL connection> kernel: LustreError: 10899:0:(obd_config.c:332:class_setup()) setuplustre0-MDT0000-mdc-ffff81022ea1 cc00 failed (-2)> kernel: LustreError:10899:0:(obd_config.c:1071:class_config_llog_handler()) Err -2 on cfg command:> kernel: Lustre: cmd=cf003 0:lustre0-MDT0000-mdc1:lustre0-MDT0000_UUID 2:10.2.1.30 at o2ib> kernel: LustreError: 15c-8: MGC10.1.1.30 at tcp: The configuration fromlog ''lustre0-client'' failed (-2). After viewing syslogs and searching mailing lists, it appears the mdt needs to be reformatted to include the new network (tcp) routing to the mgs. We have a combined mgs+mdt, My best guess, the upgrade command on the mds is: tunefs.lustre --mgsnode="IP1 at o2ib0,IP2 at tcp0" dev_mdt The original command to make the mdt did not include the mgsnode flag: tunefs --dryrun -l dev_mdt>Reading CONFIGS/mountdata > > Read previous values: >Target: lustre0-MDT0000 >Index: 0 >Lustre FS: lustre0 >Mount type: ldiskfs >Flags: 0x5 > (MDT MGS ) >Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr >Parameters: mdt.group_upcall=/usr/sbin/l_getgroupsQ. Is tunefs.lustre command this non-destructive ? tunefs.lustre --mgsnode="IP1 at o2ib0,IP2 at tcp0" dev_mdt Q. Do I need to update config on the OSTs for our new multi-homed network ? Thanks, Chris Hunter
Chris Hunter
2008-Dec-17 04:23 UTC
[Lustre-discuss] tunefs.lustre --mgsnode non-destructive ?
We want to reconfigure our lustre for multi-homed network. However the mgs only appears to respond to clients using the original network (o2ib). Here is the kernel module (modprobe.conf) line> options lnet ''networks="o2ib(ib0),tcp(eth0)"''When mounting lustre on the new (tcp) network, we see the following syslog messages:> kernel: LustreError: 10899:0:(events.c:454:ptlrpc_uuid_to_peer()) NoNID found for 10.2.1.30 at o2ib> kernel: LustreError:10899:0:(client.c:58:ptlrpc_uuid_to_connection()) cannot find peer 10.2.1.30 at o2 ib!> kernel: LustreError: 10899:0:(ldlm_lib.c:321:client_obd_setup())can''t add initial connection> kernel: LustreError:8819:0:(connection.c:144:ptlrpc_put_connection()) NULL connection> kernel: LustreError: 10899:0:(obd_config.c:332:class_setup()) setuplustre0-MDT0000-mdc-ffff81022ea1 cc00 failed (-2)> kernel: LustreError:10899:0:(obd_config.c:1071:class_config_llog_handler()) Err -2 on cfg command:> kernel: Lustre: cmd=cf003 0:lustre0-MDT0000-mdc1:lustre0-MDT0000_UUID 2:10.2.1.30 at o2ib> kernel: LustreError: 15c-8: MGC10.1.1.30 at tcp: The configuration fromlog ''lustre0-client'' failed (-2). After viewing syslogs and searching mailing lists, it appears the mdt needs to be reformatted to include the new network (tcp) routing to the mgs. We have a combined mgs+mdt. My best guess, the needed command on the mds is: tunefs.lustre --mgsnode="IP1 at o2ib0,IP2 at tcp0" dev_mdt The original command to make the mdt did not include the mgsnode flag: tunefs --dryrun -l dev_mdt> Reading CONFIGS/mountdata > > Read previous values: > Target: lustre0-MDT0000 > Index: 0 > Lustre FS: lustre0 > Mount type: ldiskfs > Flags: 0x5 > (MDT MGS ) > Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr > Parameters: mdt.group_upcall=/usr/sbin/l_getgroupsQ. Is tunefs.lustre command this non-destructive ? tunefs.lustre --mgsnode="IP1 at o2ib0,IP2 at tcp0" dev_mdt Q. Do I need to update config on the OSTs for our new multi-homed network ? Thanks, Chris Hunter