Hello, Sorry for this trivial question, but I did not find the solution myself : Everyone in the ldiskfs/ext4 community is happy since the OST formatting is quicker with the uninit_bg and lazy_itable_init features. And I appreciate a lot this feature too ! However, today for a particular test, When I format my OSTs, I would like this task to be completely finished in order to be sure to have the maximum OSTs bandwidth when I run my benchmarks. So, I tried the following : --mkfsoptions="-E lazy_itable_init=0,lazy_journal_init=0" but no success : formatting is still really quick and as soon as I mount my OST, iostats show me the formatting activity on it. Is it still possible to format like in the old ext3 age ? Thanks for your help Cédric _______________________________________________ Lustre-discuss mailing list Lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
You can try mounting the OST with "-o noinit_itable". This doesn''t fix the formatting (not sure why that isn''t being recognized), but it prevents the kernel thread from zeroing the inode table. This could cause problems for e2fsck in the distant future if certain types of corruption are hit, but does not affect testing in any way Cheers, Andreas On 2013-10-14, at 7:35, "Lambert, Cedric" <cedric.lambert-VXdhtT5mjnY@public.gmane.org<mailto:cedric.lambert-VXdhtT5mjnY@public.gmane.org>> wrote: Hello, Sorry for this trivial question, but I did not find the solution myself : Everyone in the ldiskfs/ext4 community is happy since the OST formatting is quicker with the uninit_bg and lazy_itable_init features. And I appreciate a lot this feature too ! However, today for a particular test, When I format my OSTs, I would like this task to be completely finished in order to be sure to have the maximum OSTs bandwidth when I run my benchmarks. So, I tried the following : --mkfsoptions="-E lazy_itable_init=0,lazy_journal_init=0" but no success : formatting is still really quick and as soon as I mount my OST, iostats show me the formatting activity on it. Is it still possible to format like in the old ext3 age ? Thanks for your help Cédric _______________________________________________ Lustre-discuss mailing list Lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org<mailto:Lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org> http://lists.lustre.org/mailman/listinfo/lustre-discuss
OK, this mounting options does perfectly the trick for my performance tests. Thanks for your help. However, I do not understand why I fall into this problem, and maybe some of you could help me : I think there is something I did not understand in the mkfs options since my MDT behaves perfectly and its formatting takes a long time. It is not the case for my OSTs. Here are the mkfs_cmd used for my Lustre targets : Mkfs_cmd for my MDT : mke2fs -j -b 4096 -L scratch:MDT0000 -J size=400 -I 512 -i 2048 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,huge_file,flex_bg -E lazy_journal_init -F /dev/sdb2 97177856 Mkfs_cmd for my OSTs : mke2fs -j -b 4096 -L scratch:OST0000 -J size=400 -I 256 -i 524288 -q -O extents,uninit_bg,dir_nlink,quota,huge_file,flex_bg -G 256 -E stride=64,lazy_itable_init=0,lazy_journal_init=0,resize=4290772992 -F /dev/sdc 2441822902 When I format my OSTs with lazy_itable_init=0,lazy_journal_init=0 , it takes approximatively 12 seconds for the formatting. Without these options it is instantaneous. Thus, there is a difference in the behavior of the command, but 12 seconds is not a lot for formatting 10 TB ! Do you have any idea what''s wrong in my OST formatting ? Thank you Cédric -----Original Message----- From: Dilger, Andreas [mailto:andreas.dilger-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org] Sent: lundi 14 octobre 2013 19:13 To: Lambert, Cedric Cc: lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org Subject: Re: [Lustre-discuss] Ldiskfs formatting You can try mounting the OST with "-o noinit_itable". This doesn''t fix the formatting (not sure why that isn''t being recognized), but it prevents the kernel thread from zeroing the inode table. This could cause problems for e2fsck in the distant future if certain types of corruption are hit, but does not affect testing in any way Cheers, Andreas On 2013-10-14, at 7:35, "Lambert, Cedric" <cedric.lambert-VXdhtT5mjnY@public.gmane.org<mailto:cedric.lambert-VXdhtT5mjnY@public.gmane.org>> wrote: Hello, Sorry for this trivial question, but I did not find the solution myself : Everyone in the ldiskfs/ext4 community is happy since the OST formatting is quicker with the uninit_bg and lazy_itable_init features. And I appreciate a lot this feature too ! However, today for a particular test, When I format my OSTs, I would like this task to be completely finished in order to be sure to have the maximum OSTs bandwidth when I run my benchmarks. So, I tried the following : --mkfsoptions="-E lazy_itable_init=0,lazy_journal_init=0" but no success : formatting is still really quick and as soon as I mount my OST, iostats show me the formatting activity on it. Is it still possible to format like in the old ext3 age ? Thanks for your help Cédric _______________________________________________ Lustre-discuss mailing list Lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org<mailto:Lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org> http://lists.lustre.org/mailman/listinfo/lustre-discuss
On 2013/10/14 12:31 PM, "Lambert, Cedric" <cedric.lambert-VXdhtT5mjnY@public.gmane.org> wrote:>OK, this mounting options does perfectly the trick for my performance >tests. Thanks for your help. >However, I do not understand why I fall into this problem, and maybe some >of you could help me : >I think there is something I did not understand in the mkfs options since >my MDT behaves perfectly and its formatting takes a long time. >It is not the case for my OSTs. >Here are the mkfs_cmd used for my Lustre targets : >Mkfs_cmd for my MDT : mke2fs -j -b 4096 -L scratch:MDT0000 -J size=400 >-I 512 -i 2048 -q -O >dirdata,uninit_bg,^extents,dir_nlink,quota,huge_file,flex_bg -E >lazy_journal_init -F /dev/sdb2 97177856 >Mkfs_cmd for my OSTs : mke2fs -j -b 4096 -L scratch:OST0000 -J size=400 >-I 256 -i 524288 -q -O >extents,uninit_bg,dir_nlink,quota,huge_file,flex_bg -G 256 -E >stride=64,lazy_itable_init=0,lazy_journal_init=0,resize=4290772992 -F >/dev/sdc 2441822902 > >When I format my OSTs with lazy_itable_init=0,lazy_journal_init=0 , it >takes approximatively 12 seconds for the formatting. Without these >options it is instantaneous. Thus, there is a difference in the behavior >of the command, but 12 seconds is not a lot for formatting 10 TB ! > >Do you have any idea what''s wrong in my OST formatting ?The OSTs have far fewer inodes (one inode per 1MB of space on large OSTs) compared to the MDT (one inode per 2kB). Also, the OSTs use a flex_bg factor of 256 instead of 32, so it also reduces seeking for bitmap writes by a factor 8. So it may well be that the OST formatting is actually being completed that much more quickly than the MDT formatting. Cheers, Andreas>Thank you >Cédric > > >-----Original Message----- >From: Dilger, Andreas [mailto:andreas.dilger-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org] >Sent: lundi 14 octobre 2013 19:13 >To: Lambert, Cedric >Cc: lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org >Subject: Re: [Lustre-discuss] Ldiskfs formatting > >You can try mounting the OST with "-o noinit_itable". This doesn''t fix >the formatting (not sure why that isn''t being recognized), but it >prevents the kernel thread from zeroing the inode table. > >This could cause problems for e2fsck in the distant future if certain >types of corruption are hit, but does not affect testing in any way > >Cheers, Andreas > >On 2013-10-14, at 7:35, "Lambert, Cedric" ><cedric.lambert-VXdhtT5mjnY@public.gmane.org<mailto:cedric.lambert-VXdhtT5mjnY@public.gmane.org>> wrote: > >Hello, > >Sorry for this trivial question, but I did not find the solution myself : >Everyone in the ldiskfs/ext4 community is happy since the OST formatting >is quicker with the uninit_bg and lazy_itable_init features. And I >appreciate a lot this feature too ! >However, today for a particular test, When I format my OSTs, I would >like this task to be completely finished in order to be sure to have the >maximum OSTs bandwidth when I run my benchmarks. > >So, I tried the following : >--mkfsoptions="-E lazy_itable_init=0,lazy_journal_init=0" but no >success : formatting is still really quick and as soon as I mount my OST, >iostats show me the formatting activity on it. > >Is it still possible to format like in the old ext3 age ? > >Thanks for your help >Cédric >_______________________________________________ >Lustre-discuss mailing list >Lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org<mailto:Lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org> >http://lists.lustre.org/mailman/listinfo/lustre-discuss >Cheers, Andreas -- Andreas Dilger Lustre Software Architect Intel High Performance Data Division