jan taubert (RRes-Roth)
2006-Nov-21 04:08 UTC
[Lustre-discuss] Increasing default_stripe_size
I tired to change to default_stripe_size from previously 1048576 to 65536 in my setup.sh . After that I run "lconf --write_conf --node mds1 config.xml" on the primary MDS and after it was finished I run "lconf --node mds1 config.xml". I got the following output in /var/log/messages: Nov 21 10:41:45 mds1 kernel: Lustre: OBD class driver Build Version: 1.4.7.3-19691231170000-PRISTINE-.testsuite.tmp.lbuild-boulder.lbuild-v1_ 4_7_3-2.6-rhel4-x86_64.lbuild.BUILD.l ustre-kernel-2.6.9.lustre.linux-2.6.9-42.0.2.EL_lustre.1.4.7.3smp, info@clusterfs.com Nov 21 10:41:45 mds1 kernel: Lustre: Added LNI 192.168.1.1@o2ib [8/64] Nov 21 10:41:45 mds1 kernel: kjournald starting. Commit interval 5 seconds Nov 21 10:41:45 mds1 kernel: LDISKFS FS on sdb, internal journal Nov 21 10:41:45 mds1 kernel: LDISKFS-fs: mounted filesystem with ordered data mode. Nov 21 10:41:45 mds1 kernel: Lustre: MDT mds-bab now serving /dev/sdb (aa5428e6-d07a-42b6-b95f-f0f88f9676ce) with recovery enabled Nov 21 10:41:46 mds1 kernel: Lustre: MDT mds-bab has stopped. Nov 21 10:57:30 mds1 kernel: Lustre: Removed LNI 192.168.1.1@o2ib Nov 21 10:58:51 mds1 kernel: Lustre: OBD class driver Build Version: 1.4.7.3-19691231170000-PRISTINE-.testsuite.tmp.lbuild-boulder.lbuild-v1_ 4_7_3-2.6-rhel4-x86_64.lbuild.BUILD.l ustre-kernel-2.6.9.lustre.linux-2.6.9-42.0.2.EL_lustre.1.4.7.3smp, info@clusterfs.com Nov 21 10:58:51 mds1 kernel: Lustre: Added LNI 192.168.1.1@o2ib [8/64] Nov 21 10:58:51 mds1 kernel: kjournald starting. Commit interval 5 seconds Nov 21 10:58:51 mds1 kernel: LDISKFS FS on sdb, internal journal Nov 21 10:58:51 mds1 kernel: LDISKFS-fs: mounted filesystem with ordered data mode. Nov 21 10:58:51 mds1 kernel: Lustre: 6326:0:(lov_obd.c:638:lov_setup()) Increasing default_stripe_size 65536 to 1048576 Nov 21 10:58:51 mds1 kernel: Lustre: 6326:0:(lov_obd.c:639:lov_setup()) Please update config and run --write-conf on MDS Nov 21 10:58:51 mds1 kernel: Lustre: Changing connection for OSC_mds1.rothamsted.bbsrc.ac.uk_ost1-bab_mds-bab to oss1_UUID/192.168.1.3@o2ib Nov 21 10:58:51 mds1 kernel: Lustre: MDT mds-bab now serving /dev/sdb (aa5428e6-d07a-42b6-b95f-f0f88f9676ce) with recovery enabled Nov 21 10:58:51 mds1 kernel: Lustre: MDS mds-bab: All OSCs now active, resetting orphans Why does the log say that the default_stripe_size was increased to its previous value? Do I have to reformat all the OSS? I thought that new files that will be created would use 65536 and old file will be kept with 1048576. Regards, Jan Jan Taubert Biomathematics and Bioinformatics Division Rothamsted Research West Common, Harpenden, Hertfordshire. AL5 2JQ, UK tel: 01582 763133 ext 2108 fax: 01582 760981 email: jan.taubert@bbsrc.ac.uk Rothamsted Research is a company limited by guarantee, registered in England under the registration number 2393175 and a not for profit charity number 802038. -------------- next part -------------- A non-text attachment was scrubbed... Name: setup.sh Type: application/octet-stream Size: 1895 bytes Desc: setup.sh Url : http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20061121/c90ef7b1/setup-0001.obj
Nov 21 10:58:51 mds1 kernel: Lustre: 6326:0:(lov_obd.c:638:lov_setup()) Increasing default_stripe_size 65536 to 1048576 Nov 21 10:58:51 mds1 kernel: Lustre: 6326:0:(lov_obd.c:639:lov_setup()) Please update config and run --write-conf on MDS Nov 21 10:58:51 mds1 kernel: Lustre: Changing connection for OSC_mds1.rothamsted.bbsrc.ac.uk_ost1-bab_mds-bab to oss1_UUID/192.168.1.3@o2ib Nov 21 10:58:51 mds1 kernel: Lustre: MDT mds-bab now serving /dev/sdb (aa5428e6-d07a-42b6-b95f-f0f88f9676ce) with recovery enabled Nov 21 10:58:51 mds1 kernel: Lustre: MDS mds-bab: All OSCs now active, resetting orphans> $ lfs setstripe <directory> 65536 -1 -1I think the recommendation was to set the default stripe on a directory, lustre won''t let one set the default stripe of the <filesystem> below 1MB - Tom --- Tomas Hancock, SWD, Hewlett Packard, Galway. Ireland +353-91-754765 The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender. To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as "HP CONFIDENTIAL".
jan taubert (RRes-Roth)
2006-Nov-21 05:06 UTC
[Lustre-discuss] Increasing default_stripe_size
I see. I also tried "lfs setstripe <directory> 65536 -1 -1" on the directory, but the speed gain was less than 10%. Anything else what I could try (except rewritting formatdb)? Many thanks, Jan -----Original Message----- From: Hancock, Tom [mailto:Tom.Hancock@hp.com] Sent: 21 November 2006 12:01 To: jan taubert (RRes-Roth); lustre-discuss@clusterfs.com Subject: RE: [Lustre-discuss] Increasing default_stripe_size Nov 21 10:58:51 mds1 kernel: Lustre: 6326:0:(lov_obd.c:638:lov_setup()) Increasing default_stripe_size 65536 to 1048576 Nov 21 10:58:51 mds1 kernel: Lustre: 6326:0:(lov_obd.c:639:lov_setup()) Please update config and run --write-conf on MDS Nov 21 10:58:51 mds1 kernel: Lustre: Changing connection for OSC_mds1.rothamsted.bbsrc.ac.uk_ost1-bab_mds-bab to oss1_UUID/192.168.1.3@o2ib Nov 21 10:58:51 mds1 kernel: Lustre: MDT mds-bab now serving /dev/sdb (aa5428e6-d07a-42b6-b95f-f0f88f9676ce) with recovery enabled Nov 21 10:58:51 mds1 kernel: Lustre: MDS mds-bab: All OSCs now active, resetting orphans> $ lfs setstripe <directory> 65536 -1 -1I think the recommendation was to set the default stripe on a directory, lustre won''t let one set the default stripe of the <filesystem> below 1MB - Tom --- Tomas Hancock, SWD, Hewlett Packard, Galway. Ireland +353-91-754765 The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender. To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as "HP CONFIDENTIAL".