ashok bharat bayana
2008-Mar-10 12:00 UTC
[Lustre-discuss] Lustre-discuss Digest, Vol 26, Issue 25
Can''t we build patched lustre server for 2.6.20.15vanilla kernel ? Regards, Ashok Bharat -----Original Message----- From: lustre-discuss-bounces at lists.lustre.org on behalf of lustre-discuss-request at lists.lustre.org Sent: Mon 3/10/2008 4:58 PM To: lustre-discuss at lists.lustre.org Subject: Lustre-discuss Digest, Vol 26, Issue 25 Send Lustre-discuss mailing list submissions to lustre-discuss at lists.lustre.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.lustre.org/mailman/listinfo/lustre-discuss or, via email, send a message with subject or body ''help'' to lustre-discuss-request at lists.lustre.org You can reach the person managing the list at lustre-discuss-owner at lists.lustre.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Lustre-discuss digest..." Today''s Topics: 1. Re: Kernel Patch Management (Andreas Dilger) 2. Re: yet another lustre error (Alex Lyashkov) 3. Re: lustre and small files overhead (Joe Barjo) ---------------------------------------------------------------------- Message: 1 Date: Mon, 10 Mar 2008 00:36:12 -0600 From: Andreas Dilger <adilger at sun.com> Subject: Re: [Lustre-discuss] Kernel Patch Management To: ashok bharat bayana <ashok.bharat.bayana at iiitb.ac.in> Cc: lustre-discuss at lists.lustre.org Message-ID: <20080310063612.GF5851 at webber.adilger.int> Content-Type: text/plain; charset=us-ascii On Mar 10, 2008 11:21 +0530, ashok bharat bayana wrote:> I''m trying to build lustre(1.6.4.2) version on 2.6.20.15 kernel on an x86 based machine.I''m using Ubuntu 7.10 flavour. > > While applying patches to the kernel in the series which_patch file,under the supported kernels, vanilla-2.6.20.15.series is not menioned but vanilla-2.6.18-.series is mentioned. > > So, can I proceed applying the series file of 2.6.18 version on a 2.6.20.15 version.No, making Lustre kernel patches is not so easy. You should instead use a supported kernel version for the server, and use patchless Lustre clients for any systems which cannot run on a supported kernel.> -----Original Message----- > From: lustre-discuss-bounces at lists.lustre.org on behalf of lustre-discuss-request at lists.lustre.org > Sent: Fri 2/29/2008 12:17 PM > To: lustre-discuss at lists.lustre.org > Subject: Lustre-discuss Digest, Vol 25, Issue 58 > > Send Lustre-discuss mailing list submissions to > lustre-discuss at lists.lustre.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.lustre.org/mailman/listinfo/lustre-discuss > or, via email, send a message with subject or body ''help'' to > lustre-discuss-request at lists.lustre.org > > You can reach the person managing the list at > lustre-discuss-owner at lists.lustre.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Lustre-discuss digest..." > > > Today''s Topics: > > 1. Re: Configuration of lustre FS on single node > (ashok bharat bayana) > 2. Re: Configuration of lustre FS on single node (Amit Sharma) > 3. Re: Configuration of lustre FS on single node > (ashok bharat bayana) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 29 Feb 2008 12:07:18 +0530 > From: "ashok bharat bayana" <ashok.bharat.bayana at iiitb.ac.in> > Subject: Re: [Lustre-discuss] Configuration of lustre FS on single > node > To: <lustre-discuss at lists.lustre.org>, > <lustre-discuss at lists.lustre.org> > Message-ID: <8626C1B7EB748940BCDD7596134632BE850229 at jal.iiitb.ac.in> > Content-Type: text/plain; charset="iso-8859-1" > > > Hello, > I successfully build lustre(1.6.4.2) on my system for a patchless client But I dont know how to proceed in configuring the file system. > > I''m trying to build all MDS,client and MDT''s on a single node. > >From the tutorials I came to know that > > First, create an MDT for the "spfs" file system that uses the /dev/sda disk. This MDT also acts as the MGS for the site. > > $ mkfs.lustre --fsname spfs --mdt --mgs /dev/sda > > But by triggering this command my system crashed in which the whole data is lost and again I need to install the OS. > I want help in proceeding of mounting a lustre file system. > > Thanks and Regards, > Ashok Bharat > > -----Original Message----- > From: lustre-discuss-bounces at lists.lustre.org on behalf of lustre-discuss-request at lists.lustre.org > Sent: Wed 2/27/2008 10:30 PM > To: lustre-discuss at lists.lustre.org > Subject: Lustre-discuss Digest, Vol 25, Issue 56 > > Send Lustre-discuss mailing list submissions to > lustre-discuss at lists.lustre.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.lustre.org/mailman/listinfo/lustre-discuss > or, via email, send a message with subject or body ''help'' to > lustre-discuss-request at lists.lustre.org > > You can reach the person managing the list at > lustre-discuss-owner at lists.lustre.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Lustre-discuss digest..." > > > Today''s Topics: > > 1. Multiple NICs per OST (Joshua Bower-Cooley) > 2. obd error on MGS/MDT node (Jeremy Mann) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 26 Feb 2008 11:55:19 -0800 > From: Joshua Bower-Cooley <jbowercooley at lcogt.net> > Subject: [Lustre-discuss] Multiple NICs per OST > To: lustre-discuss at lists.lustre.org > Message-ID: <200802261155.19907.jbowercooley at lcogt.net> > Content-Type: text/plain; charset="utf-8" > > Hi- > I''ve had lustre running happily for some time over a single 10g ethernet NIC > per node. After switching to dual 10g and creating a new filesystem > (1.6.4.2), I''m seeing nothing but keep-alive packets with bad checksums. > > What is the current "correct" way to do this now? The manual suggest not using > bonding, but several list postings now reccommend it. Without bonding, do I > need to have my 2 switches stacked, or will Lustre recognize the division in > my subnet? > > LNET module options I''ve tried are: > 1) networks="tcp0(eth2,eth3)" > 2) ip2nets="tcp(eth2,eth3); tcp(eth2) 10.9.[1-4].*; tcp(eth3) 10.9.[5-8].*;" > and many other variations > > All other services are functional on both interfaces. > > Thanks in advance, > J Bower-Cooley > Scientific Systems Engineer > Las Cumbres Observatory Global Telescope > > > ------------------------------ > > Message: 2 > Date: Wed, 27 Feb 2008 09:50:08 -0600 (CST) > From: "Jeremy Mann" <jeremy at biochem.uthscsa.edu> > Subject: [Lustre-discuss] obd error on MGS/MDT node > To: lustre-discuss at lists.lustre.org > Message-ID: > <62901.24.173.62.147.1204127408.squirrel at biochem.uthscsa.edu> > Content-Type: text/plain;charset=iso-8859-1 > > Today we are starting to see obd_change_cbdata errors on our MGS/MDT node. > What does this error mean? > > Version of lustre and kernel is: > > 2.6.9-42.0.10.EL_lustre-1.6.0.1smp X86_64 > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > previous similar message > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > obd_change_cbdata: NULL export > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > previous similar message > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > obd_change_cbdata: NULL export > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 2 > previous similar messages > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > obd_change_cbdata: NULL export > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 4 > previous similar messages > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > obd_change_cbdata: NULL export > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 5 > previous similar messages > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > obd_change_cbdata: NULL export > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > previous similar message > > -- > Jeremy Mann > jeremy at biochem.uthscsa.edu > > University of Texas Health Science Center > Bioinformatics Core Facility > http://www.bioinformatics.uthscsa.edu > Phone: (210) 567-2672 > > > > ------------------------------ > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > End of Lustre-discuss Digest, Vol 25, Issue 56 > ********************************************** > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080229/887613dd/attachment-0001.html > > ------------------------------ > > Message: 2 > Date: Fri, 29 Feb 2008 12:03:09 +0530 > From: Amit Sharma <Amit.Sharma at Sun.COM> > Subject: Re: [Lustre-discuss] Configuration of lustre FS on single > node > To: ashok bharat bayana <ashok.bharat.bayana at iiitb.ac.in> > Cc: lustre-discuss at lists.lustre.org > Message-ID: <47C7A725.5050309 at sun.com> > Content-Type: text/plain; format=flowed; charset=ISO-8859-1 > > Ashok, > > Could you let us know what are the modules that you have installed on > the system. The linux version on your system. And the steps that you > have done so far. > > thanks, > Amit > ashok bharat bayana wrote: > > > > Hello, > > I successfully build lustre(1.6.4.2) on my system for a patchless client > > But I dont know how to proceed in configuring the file system. > > > > I''m trying to build all MDS,client and MDT''s on a single node. > > >From the tutorials I came to know that > > > > First, create an MDT for the "spfs" file system that uses the /dev/sda > > disk. This MDT also acts as the MGS for the site. > > > > $ mkfs.lustre --fsname spfs --mdt --mgs /dev/sda > > > > But by triggering this command my system crashed in which the whole data > > is lost and again I need to install the OS. > > I want help in proceeding of mounting a lustre file system. > > > > Thanks and Regards, > > Ashok Bharat > > > > -----Original Message----- > > From: lustre-discuss-bounces at lists.lustre.org on behalf of > > lustre-discuss-request at lists.lustre.org > > Sent: Wed 2/27/2008 10:30 PM > > To: lustre-discuss at lists.lustre.org > > Subject: Lustre-discuss Digest, Vol 25, Issue 56 > > > > Send Lustre-discuss mailing list submissions to > > lustre-discuss at lists.lustre.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > or, via email, send a message with subject or body ''help'' to > > lustre-discuss-request at lists.lustre.org > > > > You can reach the person managing the list at > > lustre-discuss-owner at lists.lustre.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of Lustre-discuss digest..." > > > > > > Today''s Topics: > > > > 1. Multiple NICs per OST (Joshua Bower-Cooley) > > 2. obd error on MGS/MDT node (Jeremy Mann) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Tue, 26 Feb 2008 11:55:19 -0800 > > From: Joshua Bower-Cooley <jbowercooley at lcogt.net> > > Subject: [Lustre-discuss] Multiple NICs per OST > > To: lustre-discuss at lists.lustre.org > > Message-ID: <200802261155.19907.jbowercooley at lcogt.net> > > Content-Type: text/plain; charset="utf-8" > > > > Hi- > > I''ve had lustre running happily for some time over a single 10g ethernet NIC > > per node. After switching to dual 10g and creating a new filesystem > > (1.6.4.2), I''m seeing nothing but keep-alive packets with bad checksums. > > > > What is the current "correct" way to do this now? The manual suggest not > > using > > bonding, but several list postings now reccommend it. Without bonding, do I > > need to have my 2 switches stacked, or will Lustre recognize the division in > > my subnet? > > > > LNET module options I''ve tried are: > > 1) networks="tcp0(eth2,eth3)" > > 2) ip2nets="tcp(eth2,eth3); tcp(eth2) 10.9.[1-4].*; tcp(eth3) 10.9.[5-8].*;" > > and many other variations > > > > All other services are functional on both interfaces. > > > > Thanks in advance, > > J Bower-Cooley > > Scientific Systems Engineer > > Las Cumbres Observatory Global Telescope > > > > > > ------------------------------ > > > > Message: 2 > > Date: Wed, 27 Feb 2008 09:50:08 -0600 (CST) > > From: "Jeremy Mann" <jeremy at biochem.uthscsa.edu> > > Subject: [Lustre-discuss] obd error on MGS/MDT node > > To: lustre-discuss at lists.lustre.org > > Message-ID: > > <62901.24.173.62.147.1204127408.squirrel at biochem.uthscsa.edu> > > Content-Type: text/plain;charset=iso-8859-1 > > > > Today we are starting to see obd_change_cbdata errors on our MGS/MDT node. > > What does this error mean? > > > > Version of lustre and kernel is: > > > > 2.6.9-42.0.10.EL_lustre-1.6.0.1smp X86_64 > > > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > > previous similar message > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > > previous similar message > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 2 > > previous similar messages > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 4 > > previous similar messages > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 5 > > previous similar messages > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > > previous similar message > > > > -- > > Jeremy Mann > > jeremy at biochem.uthscsa.edu > > > > University of Texas Health Science Center > > Bioinformatics Core Facility > > http://www.bioinformatics.uthscsa.edu > > Phone: (210) 567-2672 > > > > > > > > ------------------------------ > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > > > > End of Lustre-discuss Digest, Vol 25, Issue 56 > > ********************************************** > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > -- > Amit Sharma > Lustre Engineering > Sun Microsystems, Bangalore. > > > ------------------------------ > > Message: 3 > Date: Fri, 29 Feb 2008 12:27:53 +0530 > From: "ashok bharat bayana" <ashok.bharat.bayana at iiitb.ac.in> > Subject: Re: [Lustre-discuss] Configuration of lustre FS on single > node > To: "Amit Sharma" <Amit.Sharma at Sun.COM> > Cc: lustre-discuss at lists.lustre.org > Message-ID: <8626C1B7EB748940BCDD7596134632BE85022A at jal.iiitb.ac.in> > Content-Type: text/plain; charset="iso-8859-1" > > > Hi, > the kernel version I used is 2.6.20.15 and the distro is Ubuntu 7.10 > The steps so far I have done are: > 1)I have downloaded the sources lustre(1.6.4.2) from CFS > 2)I tried to build lustre for a patchless client in which i triggered the command, > [lustre]$ ./configure --with-linux=/unpatched/kernel/source > 3)then I build it using make and then installed the modules using make install commands > (I''m not sure whether the modules are installed correctly or not.Isn''t the modules are installed using make install cmd) > 4)Now in /usr/lib there is a lustre folder created. > (Can we setup all MDS,client and OST''s on a single system? my system configuration is Intel Pentium 512 MB RAM 1.6 GHz) > Could you please tell me how to mount lustre fs in my laptop. > > Thanks and Regards, > Ashok Bharat > -----Original Message----- > From: Amit Sharma [mailto:Amit.Sharma at Sun.COM] > Sent: Fri 2/29/2008 12:03 PM > To: ashok bharat bayana > Cc: lustre-discuss at lists.lustre.org > Subject: Re: [Lustre-discuss] Configuration of lustre FS on single node > > Ashok, > > Could you let us know what are the modules that you have installed on > the system. The linux version on your system. And the steps that you > have done so far. > > thanks, > Amit > ashok bharat bayana wrote: > > > > Hello, > > I successfully build lustre(1.6.4.2) on my system for a patchless client > > But I dont know how to proceed in configuring the file system. > > > > I''m trying to build all MDS,client and MDT''s on a single node. > > >From the tutorials I came to know that > > > > First, create an MDT for the "spfs" file system that uses the /dev/sda > > disk. This MDT also acts as the MGS for the site. > > > > $ mkfs.lustre --fsname spfs --mdt --mgs /dev/sda > > > > But by triggering this command my system crashed in which the whole data > > is lost and again I need to install the OS. > > I want help in proceeding of mounting a lustre file system. > > > > Thanks and Regards, > > Ashok Bharat > > > > -----Original Message----- > > From: lustre-discuss-bounces at lists.lustre.org on behalf of > > lustre-discuss-request at lists.lustre.org > > Sent: Wed 2/27/2008 10:30 PM > > To: lustre-discuss at lists.lustre.org > > Subject: Lustre-discuss Digest, Vol 25, Issue 56 > > > > Send Lustre-discuss mailing list submissions to > > lustre-discuss at lists.lustre.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > or, via email, send a message with subject or body ''help'' to > > lustre-discuss-request at lists.lustre.org > > > > You can reach the person managing the list at > > lustre-discuss-owner at lists.lustre.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of Lustre-discuss digest..." > > > > > > Today''s Topics: > > > > 1. Multiple NICs per OST (Joshua Bower-Cooley) > > 2. obd error on MGS/MDT node (Jeremy Mann) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Tue, 26 Feb 2008 11:55:19 -0800 > > From: Joshua Bower-Cooley <jbowercooley at lcogt.net> > > Subject: [Lustre-discuss] Multiple NICs per OST > > To: lustre-discuss at lists.lustre.org > > Message-ID: <200802261155.19907.jbowercooley at lcogt.net> > > Content-Type: text/plain; charset="utf-8" > > > > Hi- > > I''ve had lustre running happily for some time over a single 10g ethernet NIC > > per node. After switching to dual 10g and creating a new filesystem > > (1.6.4.2), I''m seeing nothing but keep-alive packets with bad checksums. > > > > What is the current "correct" way to do this now? The manual suggest not > > using > > bonding, but several list postings now reccommend it. Without bonding, do I > > need to have my 2 switches stacked, or will Lustre recognize the division in > > my subnet? > > > > LNET module options I''ve tried are: > > 1) networks="tcp0(eth2,eth3)" > > 2) ip2nets="tcp(eth2,eth3); tcp(eth2) 10.9.[1-4].*; tcp(eth3) 10.9.[5-8].*;" > > and many other variations > > > > All other services are functional on both interfaces. > > > > Thanks in advance, > > J Bower-Cooley > > Scientific Systems Engineer > > Las Cumbres Observatory Global Telescope > > > > > > ------------------------------ > > > > Message: 2 > > Date: Wed, 27 Feb 2008 09:50:08 -0600 (CST) > > From: "Jeremy Mann" <jeremy at biochem.uthscsa.edu> > > Subject: [Lustre-discuss] obd error on MGS/MDT node > > To: lustre-discuss at lists.lustre.org > > Message-ID: > > <62901.24.173.62.147.1204127408.squirrel at biochem.uthscsa.edu> > > Content-Type: text/plain;charset=iso-8859-1 > > > > Today we are starting to see obd_change_cbdata errors on our MGS/MDT node. > > What does this error mean? > > > > Version of lustre and kernel is: > > > > 2.6.9-42.0.10.EL_lustre-1.6.0.1smp X86_64 > > > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > > previous similar message > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > > previous similar message > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 2 > > previous similar messages > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 4 > > previous similar messages > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 5 > > previous similar messages > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) > > obd_change_cbdata: NULL export > > LustreError: 65:0:(obd_class.h:1171:obd_change_cbdata()) Skipped 1 > > previous similar message > > > > -- > > Jeremy Mann > > jeremy at biochem.uthscsa.edu > > > > University of Texas Health Science Center > > Bioinformatics Core Facility > > http://www.bioinformatics.uthscsa.edu > > Phone: (210) 567-2672 > > > > > > > > ------------------------------ > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > > > > End of Lustre-discuss Digest, Vol 25, Issue 56 > > ********************************************** > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > -- > Amit Sharma > Lustre Engineering > Sun Microsystems, Bangalore. > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080229/39e94cf4/attachment.html > > ------------------------------ > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > End of Lustre-discuss Digest, Vol 25, Issue 58 > ********************************************** >> _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discussCheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. ------------------------------ Message: 2 Date: Mon, 10 Mar 2008 11:08:18 +0200 From: Alex Lyashkov <Alexey.Lyashkov at Sun.COM> Subject: Re: [Lustre-discuss] yet another lustre error To: Brock Palen <brockp at umich.edu> Cc: lustre-discuss at lists.lustre.org Message-ID: <1205140098.1359.270.camel at bear.shadowland> Content-Type: text/plain On Fri, 2008-03-07 at 18:45 -0500, Brock Palen wrote:> On a file system thats been up for only 57 days, I have: >> target_handle_reconnect > nobackup-MDT0000: 34b4fbea-200b-1f7c-dac0-516b8ce786fc reconnecting > ldlm_lib.c > target_handle_connect > nobackup-MDT0000: refuse reconnection from 34b4fbea-200b-1f7c- > dac0-516b8ce786fc at 10.164.0.111@tcp to 0x00000100069a7000; still busy > with 2 active RPCs > ldlm_lib.c > target_send_reply_msg > @@@ processing error (-16) req at 0000010019159a00 x11199816/t0 o38- > >34b4fbea-200b-1f7c-dac0-516b8ce786fc at NET_0x200000aa4006f_UUID:-1 > lens 304/200 ref 0 fl Interpret:/0/0 rc -16/0 > > > What I see messages about active rpc''s in other logs. What would > this mean? Is something suck someplace ? >-16 = EBUSY. This say client reconnected to server which already work on different request from this client. After old rpc from this client will be finished - client will be reconnected. -- Alex Lyashkov <Alexey.lyashkov at sun.com> Lustre Group, Sun Microsystems ------------------------------ Message: 3 Date: Mon, 10 Mar 2008 12:27:51 +0100 From: Joe Barjo <jobarjo78 at yahoo.fr> Subject: Re: [Lustre-discuss] lustre and small files overhead Cc: lustre-discuss at lists.lustre.org Message-ID: <47D51B37.5070703 at yahoo.fr> Content-Type: text/plain; charset="iso-8859-1" Andreas Dilger a ?crit :> On Mar 07, 2008 12:49 +0100, Joe Barjo wrote: > >> I made some more tests, and have setup a micro lustre cluster on lvm >> volumes. >> node a: MDS >> node b and c: OST >> node a,b,c,d,e,f: clients >> Gigabit ethernet network. >> Made the optimizations: lnet.debug=0, lru_size to 10000, max_dirty_mb to >> 1024 >> > > For high RPC-rate operations using an interconnect like Infiniband is > better than ethernet. > >infiniband is not in our budget...>> The svn checkout takes 50s ( 15s on a localdisk, 25s on a local lustre >> demo (with debug=0)) >> Launching gkrellm, a single svn checkout consumes about 20% of the MDS >> system cpu with about 2.4mbyte/sec ethernet communication. >> > > >> About 6MByte/s disk bandwidth on OST1, up to 12-16MB/s on OST2 disk >> bandwidth, network bandwidth on OST is about 10 to 20 times under disk >> bandwidth. >> Why so much disk bandwidth on OSTs, is it a readahead problem? >> > > That does seem strange, I can''t really say why. There is some metadata > overhead, and that is higher with small files, but I don''t think it > would be 10-20x overhead. > >The checkouted source is only 65 megabytes. So much OST disk bandwidth is probably not normal. Maybe you should verify this point. Are you sure there isn''t an optimazation for this? This looks like readahead or something like that.>> I launched a compilation distributed on the 6 clients: >> MDS system cpu goes up to 60% system ressource (athlon 64 3500+) >> 12MByte/s on the ethernet, OST goes up to the same level as previous test. >> >> How come is there so much network communications on the MDT? >> > > Because every metadata operation has to be done on the MDS currently. > We are working toward having metadata writeback cache operations on > the client, but it doesn''t happen currently. For operations like > compilation it is basically entirely metadata overhead. > > >> As I understood that the MDS can not be load balanced, I don''t see how >> lustre is scalable to thousands of clients... >> > > Because in many HPC environments there are very few metadata operations > in comparison to the amount of data being read/written. Average file > sizes are 20-30MB instead of 20-30kB. > > >> It looks like lustre is not made for this kind of application >> > > No, it definitely isn''t tuned for small files. >Could it be tuned one day for small files? Which filesystem would you suggest for me? I already tried nfs, afs I will now try glusterfs Thanks for your support -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080310/11e3dfa0/attachment.html ------------------------------ _______________________________________________ Lustre-discuss mailing list Lustre-discuss at lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss End of Lustre-discuss Digest, Vol 26, Issue 25 ********************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080310/f22f67b3/attachment-0002.html