thr3ads.net - Lustre discuss - [Lustre-discuss] no default mmp set for MDT with --failnode [May 2008]

If this information is useful, please help other people find it:
Share via:

jrs

2008-May-05 20:34 UTC

[Lustre-discuss] no default mmp set for MDT with --failnode

The docs say that specifying --failnode should automatically
set mmp on the filesystem.  I''ve seen it work and not work ...
It doesn''t in this case ...


mds01:/net/lmd01/space/lustre # mkfs.lustre --reformat --fsname i3_lfs4 --mdt
--mgs --failnode mds02 /dev/mapper/mdt_lustre03-04_00_dm_8_mds01

    Permanent disk data:
Target:     i3_lfs4-MDTffff
Index:      unassigned
Lustre FS:  i3_lfs4
Mount type: ldiskfs
Flags:      0x75
               (MDT MGS needs_index first_time update )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters: failover.node=10.200.20.58 at tcp mgsnode=10.200.20.58 at tcp
mdt.group_upcall=/usr/sbin/l_getgroups

device size = 5722881MB
formatting backing filesystem ldiskfs on
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01
         target name  i3_lfs4-MDTffff
         4k blocks     0
         options        -J size=400 -i 4096 -I 512 -q -O dir_index -F
mkfs_cmd = mkfs.ext2 -j -b 4096 -L i3_lfs4-MDTffff  -J size=400 -i 4096 -I 512
-q -O dir_index -F /dev/mapper/mdt_lustre03-04_00_dm_8_mds01
Writing CONFIGS/mountdata

mds01:/net/lmd01/space/lustre # mount -t lustre
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01 /mnt/lustre/mdt

mds01:/net/lmd01/space/lustre # lctl dl
   0 UP mgs MGS MGS 5
   1 UP mgc MGC10.200.20.58 at tcp 713e72db-26b3-58d7-5424-ad0c1520c915 5
   2 UP mdt MDS MDS_uuid 3
   3 UP lov i3_lfs4-mdtlov i3_lfs4-mdtlov_UUID 4
   4 UP mds i3_lfs4-MDT0000 i3_lfs4-MDT0000_UUID 3
mds01:/net/lmd01/space/lustre # umount -t lustre -a
mds01:/net/lmd01/space/lustre # dumpe2fs -h
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01 |grep feat
dumpe2fs 1.40.4.cfs1 (31-Dec-2007)
Filesystem features:      has_journal resize_inode dir_index filetype
needs_recovery sparse_super large_file

Kalpak Shah

2008-May-06 10:59 UTC

head link

[Lustre-discuss] no default mmp set for MDT with --failnode

On Mon, 2008-05-05 at 16:34 -0400, jrs wrote:> The docs say that specifying --failnode should automatically
> set mmp on the filesystem.  I''ve seen it work and not work ...
> It doesn''t in this case ...
What is the kernel version you are using? Actually MMP is supported only
for kernel versions >= 2.6.9.

Thanks,
Kalpak
> 
> 
> mds01:/net/lmd01/space/lustre # mkfs.lustre --reformat --fsname i3_lfs4
--mdt --mgs --failnode mds02 /dev/mapper/mdt_lustre03-04_00_dm_8_mds01
> 
>     Permanent disk data:
> Target:     i3_lfs4-MDTffff
> Index:      unassigned
> Lustre FS:  i3_lfs4
> Mount type: ldiskfs
> Flags:      0x75
>                (MDT MGS needs_index first_time update )
> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
> Parameters: failover.node=10.200.20.58 at tcp mgsnode=10.200.20.58 at tcp
mdt.group_upcall=/usr/sbin/l_getgroups
> 
> device size = 5722881MB
> formatting backing filesystem ldiskfs on
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01
>          target name  i3_lfs4-MDTffff
>          4k blocks     0
>          options        -J size=400 -i 4096 -I 512 -q -O dir_index -F
> mkfs_cmd = mkfs.ext2 -j -b 4096 -L i3_lfs4-MDTffff  -J size=400 -i 4096 -I
512 -q -O dir_index -F /dev/mapper/mdt_lustre03-04_00_dm_8_mds01
> Writing CONFIGS/mountdata
> 
> mds01:/net/lmd01/space/lustre # mount -t lustre
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01 /mnt/lustre/mdt
> 
> mds01:/net/lmd01/space/lustre # lctl dl
>    0 UP mgs MGS MGS 5
>    1 UP mgc MGC10.200.20.58 at tcp 713e72db-26b3-58d7-5424-ad0c1520c915 5
>    2 UP mdt MDS MDS_uuid 3
>    3 UP lov i3_lfs4-mdtlov i3_lfs4-mdtlov_UUID 4
>    4 UP mds i3_lfs4-MDT0000 i3_lfs4-MDT0000_UUID 3
> mds01:/net/lmd01/space/lustre # umount -t lustre -a
> mds01:/net/lmd01/space/lustre # dumpe2fs -h
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01 |grep feat
> dumpe2fs 1.40.4.cfs1 (31-Dec-2007)
> Filesystem features:      has_journal resize_inode dir_index filetype
needs_recovery sparse_super large_file
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

jrs

2008-May-06 13:28 UTC

head link

[Lustre-discuss] no default mmp set for MDT with --failnode

I''m using the kernel provided with the latest lustre rpms (1.6.4.3), so
it''s
mds02:~ # uname -a
Linux mds02 2.6.16-54-0.2.5_lustre.1.6.4.3smp #1 SMP Sun Feb 17 10:24:52 EST
2008 x86_64 x86_64 x86_64 GNU/Linux

John

Kalpak Shah wrote:> On Mon, 2008-05-05 at 16:34 -0400, jrs wrote:
>> The docs say that specifying --failnode should automatically
>> set mmp on the filesystem.  I''ve seen it work and not work ...
>> It doesn''t in this case ...
> 
> What is the kernel version you are using? Actually MMP is supported only
> for kernel versions >= 2.6.9.
> 
> Thanks,
> Kalpak
> 
>>
>> mds01:/net/lmd01/space/lustre # mkfs.lustre --reformat --fsname i3_lfs4
--mdt --mgs --failnode mds02 /dev/mapper/mdt_lustre03-04_00_dm_8_mds01
>>
>>     Permanent disk data:
>> Target:     i3_lfs4-MDTffff
>> Index:      unassigned
>> Lustre FS:  i3_lfs4
>> Mount type: ldiskfs
>> Flags:      0x75
>>                (MDT MGS needs_index first_time update )
>> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
>> Parameters: failover.node=10.200.20.58 at tcp mgsnode=10.200.20.58 at
tcp mdt.group_upcall=/usr/sbin/l_getgroups
>>
>> device size = 5722881MB
>> formatting backing filesystem ldiskfs on
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01
>>          target name  i3_lfs4-MDTffff
>>          4k blocks     0
>>          options        -J size=400 -i 4096 -I 512 -q -O dir_index -F
>> mkfs_cmd = mkfs.ext2 -j -b 4096 -L i3_lfs4-MDTffff  -J size=400 -i 4096
-I 512 -q -O dir_index -F /dev/mapper/mdt_lustre03-04_00_dm_8_mds01
>> Writing CONFIGS/mountdata
>>
>> mds01:/net/lmd01/space/lustre # mount -t lustre
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01 /mnt/lustre/mdt
>>
>> mds01:/net/lmd01/space/lustre # lctl dl
>>    0 UP mgs MGS MGS 5
>>    1 UP mgc MGC10.200.20.58 at tcp 713e72db-26b3-58d7-5424-ad0c1520c915
5
>>    2 UP mdt MDS MDS_uuid 3
>>    3 UP lov i3_lfs4-mdtlov i3_lfs4-mdtlov_UUID 4
>>    4 UP mds i3_lfs4-MDT0000 i3_lfs4-MDT0000_UUID 3
>> mds01:/net/lmd01/space/lustre # umount -t lustre -a
>> mds01:/net/lmd01/space/lustre # dumpe2fs -h
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01 |grep feat
>> dumpe2fs 1.40.4.cfs1 (31-Dec-2007)
>> Filesystem features:      has_journal resize_inode dir_index filetype
needs_recovery sparse_super large_file
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>

jrs

2008-May-12 16:31 UTC

head link

[Lustre-discuss] no default mmp set for MDT with --failnode

Hi Kalpak,

Here''s that command on one of my OSSes.

oss01:/net/mds01/space # debugfs -c -R "supported_features mmp"
debugfs 1.40.4.cfs1 (31-Dec-2007)
Supported feature: mmp
oss01:/net/mds01/space #

Here''s an OST that I just made using:
oss01:/net/mds01/space # mkfs.lustre --ost --reformat --fsname i3_lfs4 --mgsnode
mds01 --mgsnode mds02 --failnode oss02 /dev/mapper/ost_lustr\
e01-02_00_oss01_dm_9_mds01

    Permanent disk data:
Target:     i3_lfs4-OSTffff
Index:      unassigned
Lustre FS:  i3_lfs4
Mount type: ldiskfs
Flags:      0x72
               (OST needs_index first_time update )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.200.20.59 at tcp mgsnode=10.200.20.58 at tcp
failover.node=10.200.20.56 at tcp

device size = 5722808MB
formatting backing filesystem ldiskfs on
/dev/mapper/ost_lustre01-02_00_oss01_dm_9_mds01
         target name  i3_lfs4-OSTffff
         4k blocks     0
         options        -J size=400 -i 16384 -I 256 -q -O dir_index -F
mkfs_cmd = mkfs.ext2 -j -b 4096 -L i3_lfs4-OSTffff  -J size=400 -i 16384 -I 256
-q -O dir_index -F /dev/mapper/ost_lustre01-02_00_oss01_dm_9_\
mds01
Writing CONFIGS/mountdata
oss01:/net/mds01/space #


I then mounted and umounted the OST on both the home OSS and the failover.

oss01:/net/mds01/space # dumpe2fs -h
/dev/mapper/ost_lustre01-02_00_oss01_dm_9_mds01 |grep feat
dumpe2fs 1.40.4.cfs1 (31-Dec-2007)
Filesystem features:      has_journal resize_inode dir_index filetype
needs_recovery extents sparse_super large_file
oss01:/net/mds01/space #

You''ll see that after using --failnode, mmp is not enabled.

I can enable it by hand with tune2fs but after having done that on my MDT, my
heartbeat failover no longer works: trying to hb_takeover after the master MDS
is
up the failover won''t release the mount.  I''ll file a bug on
that when I can get
time to reproduce it.

Thanks
John

Kalpak Shah wrote:> On Mon, 2008-05-05 at 16:34 -0400, jrs wrote:
>> The docs say that specifying --failnode should automatically
>> set mmp on the filesystem.  I''ve seen it work and not work ...
>> It doesn''t in this case ...
>>
>> mds01:/net/lmd01/space/lustre # mkfs.lustre --reformat --fsname i3_lfs4
--mdt --mgs --failnode mds02 /dev/mapper/mdt_lustre03-04_00_dm_8_mds01
>>
>>     Permanent disk data:
>> Target:     i3_lfs4-MDTffff
>> Index:      unassigned
>> Lustre FS:  i3_lfs4
>> Mount type: ldiskfs
>> Flags:      0x75
>>                (MDT MGS needs_index first_time update )
>> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
>> Parameters: failover.node=10.200.20.58 at tcp mgsnode=10.200.20.58 at
tcp mdt.group_upcall=/usr/sbin/l_getgroups
>>
>> device size = 5722881MB
>> formatting backing filesystem ldiskfs on
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01
>>          target name  i3_lfs4-MDTffff
>>          4k blocks     0
>>          options        -J size=400 -i 4096 -I 512 -q -O dir_index -F
>> mkfs_cmd = mkfs.ext2 -j -b 4096 -L i3_lfs4-MDTffff  -J size=400 -i 4096
-I 512 -q -O dir_index -F /dev/mapper/mdt_lustre03-04_00_dm_8_mds01
>> Writing CONFIGS/mountdata
> 
> The fact that both uninit_groups and mmp have not been enabled indicate
> that this is not a problem related to just the MMP feature.
> 
> Could you please post the output of the following commands?
> 1) debugfs -c -R "supported_features mmp"
> 2) debugfs -c -R "supported_features mmp"
> 
> Is this problem reproducible for you?
> 
> Thanks,
> Kalpak
> 
>> mds01:/net/lmd01/space/lustre # mount -t lustre
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01 /mnt/lustre/mdt
>>
>> mds01:/net/lmd01/space/lustre # lctl dl
>>    0 UP mgs MGS MGS 5
>>    1 UP mgc MGC10.200.20.58 at tcp 713e72db-26b3-58d7-5424-ad0c1520c915
5
>>    2 UP mdt MDS MDS_uuid 3
>>    3 UP lov i3_lfs4-mdtlov i3_lfs4-mdtlov_UUID 4
>>    4 UP mds i3_lfs4-MDT0000 i3_lfs4-MDT0000_UUID 3
>> mds01:/net/lmd01/space/lustre # umount -t lustre -a
>> mds01:/net/lmd01/space/lustre # dumpe2fs -h
/dev/mapper/mdt_lustre03-04_00_dm_8_mds01 |grep feat
>> dumpe2fs 1.40.4.cfs1 (31-Dec-2007)
>> Filesystem features:      has_journal resize_inode dir_index filetype
needs_recovery sparse_super large_file
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>

Lustre discuss - May 2008 - no default mmp set for MDT with --failnode

[Lustre-discuss] no default mmp set for MDT with --failnode

[Lustre-discuss] no default mmp set for MDT with --failnode

[Lustre-discuss] no default mmp set for MDT with --failnode

[Lustre-discuss] no default mmp set for MDT with --failnode