Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3? === Joe Mervini Sandia National Laboratories High Performance Computing 505.844.6770 jamervi at sandia.gov
On Thu, Feb 24, 2011 at 10:48:32AM -0700, Mervini, Joseph A wrote:> Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3?Yes, see bugzilla ticket 18688. It was landed in 1.8.1. Cheers, Johann -- Johann Lombardi Whamcloud, Inc. www.whamcloud.com
Cool! Thank you Johann. === Joe Mervini Sandia National Laboratories High Performance Computing 505.844.6770 jamervi at sandia.gov On Feb 24, 2011, at 11:05 AM, Johann Lombardi wrote:> On Thu, Feb 24, 2011 at 10:48:32AM -0700, Mervini, Joseph A wrote: >> Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3? > > Yes, see bugzilla ticket 18688. It was landed in 1.8.1. > > Cheers, > Johann > > -- > Johann Lombardi > Whamcloud, Inc. > www.whamcloud.com >
However, I don''t think you can decrease the number of running threads. See https://bugzilla.lustre.org/show_bug.cgi?id=22417 (and also https://bugzilla.lustre.org/show_bug.cgi?id=22516 ) Kevin Mervini, Joseph A wrote:> Cool! Thank you Johann. > ===> > Joe Mervini > Sandia National Laboratories > High Performance Computing > 505.844.6770 > jamervi at sandia.gov > > > > On Feb 24, 2011, at 11:05 AM, Johann Lombardi wrote: > > >> On Thu, Feb 24, 2011 at 10:48:32AM -0700, Mervini, Joseph A wrote: >> >>> Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3? >>> >> Yes, see bugzilla ticket 18688. It was landed in 1.8.1. >> >> Cheers, >> Johann >> >> -- >> Johann Lombardi >> Whamcloud, Inc. >> www.whamcloud.com >> >> > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
I''m inclined to agree. So apparently the only time that modifying the runtime max values has a benefit is while the threads_started is low? Joe === Joe Mervini Sandia National Laboratories High Performance Computing 505.844.6770 jamervi at sandia.gov On Feb 24, 2011, at 1:09 PM, Kevin Van Maren wrote:> However, I don''t think you can decrease the number of running threads. > See https://bugzilla.lustre.org/show_bug.cgi?id=22417 (and also > https://bugzilla.lustre.org/show_bug.cgi?id=22516 ) > > Kevin > > > Mervini, Joseph A wrote: >> Cool! Thank you Johann. >> ===>> >> Joe Mervini >> Sandia National Laboratories >> High Performance Computing >> 505.844.6770 >> jamervi at sandia.gov >> >> >> >> On Feb 24, 2011, at 11:05 AM, Johann Lombardi wrote: >> >> >>> On Thu, Feb 24, 2011 at 10:48:32AM -0700, Mervini, Joseph A wrote: >>> >>>> Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3? >>>> >>> Yes, see bugzilla ticket 18688. It was landed in 1.8.1. >>> >>> Cheers, >>> Johann >>> >>> -- >>> Johann Lombardi >>> Whamcloud, Inc. >>> www.whamcloud.com >>> >>> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > >
Yes, this can be set at startup time to limit the number of started threads. There is a patch I wrote to also reduce the number of running treads but it wasn''t landed yet. Cheers, Andreas On 2011-02-24, at 14:04, "Mervini, Joseph A" <jamervi at sandia.gov> wrote:> I''m inclined to agree. So apparently the only time that modifying the runtime max values has a benefit is while the threads_started is low? > > Joe > ===> > Joe Mervini > Sandia National Laboratories > High Performance Computing > 505.844.6770 > jamervi at sandia.gov > > > > On Feb 24, 2011, at 1:09 PM, Kevin Van Maren wrote: > >> However, I don''t think you can decrease the number of running threads. >> See https://bugzilla.lustre.org/show_bug.cgi?id=22417 (and also >> https://bugzilla.lustre.org/show_bug.cgi?id=22516 ) >> >> Kevin >> >> >> Mervini, Joseph A wrote: >>> Cool! Thank you Johann. >>> ===>>> >>> Joe Mervini >>> Sandia National Laboratories >>> High Performance Computing >>> 505.844.6770 >>> jamervi at sandia.gov >>> >>> >>> >>> On Feb 24, 2011, at 11:05 AM, Johann Lombardi wrote: >>> >>> >>>> On Thu, Feb 24, 2011 at 10:48:32AM -0700, Mervini, Joseph A wrote: >>>> >>>>> Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3? >>>>> >>>> Yes, see bugzilla ticket 18688. It was landed in 1.8.1. >>>> >>>> Cheers, >>>> Johann >>>> >>>> -- >>>> Johann Lombardi >>>> Whamcloud, Inc. >>>> www.whamcloud.com >>>> >>>> >>> >>> >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >>> >> >> > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
That could be awful handy - especially when trying to tune a live file system for performance. Is that going to be a 2.0 only enhancement or can it be applied to existing 1.8 versions? On Feb 24, 2011, at 9:19 PM, Andreas Dilger wrote:> Yes, this can be set at startup time to limit the number of started threads. There is a patch I wrote to also reduce the number of running treads but it wasn''t landed yet. > > Cheers, Andreas > > On 2011-02-24, at 14:04, "Mervini, Joseph A" <jamervi at sandia.gov> wrote: > >> I''m inclined to agree. So apparently the only time that modifying the runtime max values has a benefit is while the threads_started is low? >> >> Joe >> ===>> >> Joe Mervini >> Sandia National Laboratories >> High Performance Computing >> 505.844.6770 >> jamervi at sandia.gov >> >> >> >> On Feb 24, 2011, at 1:09 PM, Kevin Van Maren wrote: >> >>> However, I don''t think you can decrease the number of running threads. >>> See https://bugzilla.lustre.org/show_bug.cgi?id=22417 (and also >>> https://bugzilla.lustre.org/show_bug.cgi?id=22516 ) >>> >>> Kevin >>> >>> >>> Mervini, Joseph A wrote: >>>> Cool! Thank you Johann. >>>> ===>>>> >>>> Joe Mervini >>>> Sandia National Laboratories >>>> High Performance Computing >>>> 505.844.6770 >>>> jamervi at sandia.gov >>>> >>>> >>>> >>>> On Feb 24, 2011, at 11:05 AM, Johann Lombardi wrote: >>>> >>>> >>>>> On Thu, Feb 24, 2011 at 10:48:32AM -0700, Mervini, Joseph A wrote: >>>>> >>>>>> Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3? >>>>>> >>>>> Yes, see bugzilla ticket 18688. It was landed in 1.8.1. >>>>> >>>>> Cheers, >>>>> Johann >>>>> >>>>> -- >>>>> Johann Lombardi >>>>> Whamcloud, Inc. >>>>> www.whamcloud.com >>>>> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Lustre-discuss mailing list >>>> Lustre-discuss at lists.lustre.org >>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >>>> >>> >>> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >= Joe Mervini Sandia National Laboratories Dept 09326 PO Box 5800 MS-0823 Albuquerque NM 87185-0823
On 2011-02-25, at 4:37 PM, Mervini, Joseph A wrote:> That could be awful handy - especially when trying to tune a live file system for performance. Is that going to be a 2.0 only enhancement or can it be applied to existing 1.8 versions?The patch was originally developed for 1.8, and ported to 2.1. That said, last time I tested it there were a few problems (crashing variety) so it isn''t ready for prime time yet. Testing/debugging would be appreciated, patch for 1.8 and 2.1 are at: https://bugzilla.lustre.org/show_bug.cgi?id=22516> On Feb 24, 2011, at 9:19 PM, Andreas Dilger wrote: >> Yes, this can be set at startup time to limit the number of started threads. There is a patch I wrote to also reduce the number of running treads but it wasn''t landed yet. >> >> Cheers, Andreas >> >> On 2011-02-24, at 14:04, "Mervini, Joseph A" <jamervi at sandia.gov> wrote: >> >>> I''m inclined to agree. So apparently the only time that modifying the runtime max values has a benefit is while the threads_started is low? >>> >>> Joe >>> ===>>> >>> Joe Mervini >>> Sandia National Laboratories >>> High Performance Computing >>> 505.844.6770 >>> jamervi at sandia.gov >>> >>> >>> >>> On Feb 24, 2011, at 1:09 PM, Kevin Van Maren wrote: >>> >>>> However, I don''t think you can decrease the number of running threads. >>>> See https://bugzilla.lustre.org/show_bug.cgi?id=22417 (and also >>>> https://bugzilla.lustre.org/show_bug.cgi?id=22516 ) >>>> >>>> Kevin >>>> >>>> >>>> Mervini, Joseph A wrote: >>>>> Cool! Thank you Johann. >>>>> ===>>>>> >>>>> Joe Mervini >>>>> Sandia National Laboratories >>>>> High Performance Computing >>>>> 505.844.6770 >>>>> jamervi at sandia.gov >>>>> >>>>> >>>>> >>>>> On Feb 24, 2011, at 11:05 AM, Johann Lombardi wrote: >>>>> >>>>> >>>>>> On Thu, Feb 24, 2011 at 10:48:32AM -0700, Mervini, Joseph A wrote: >>>>>> >>>>>>> Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3? >>>>>>> >>>>>> Yes, see bugzilla ticket 18688. It was landed in 1.8.1. >Cheers, Andreas -- Andreas Dilger Principal Engineer Whamcloud, Inc.
In the long term, I would think we can add a common library for threads-pool, we have several modules having their own implementation of threads-pool(ptlrpc service has two threads pool, LNDs...), so if we have such a library (create threads, kill threads, grow threads, shrink threads) then we can get rid of a bunch of duplicated code, though these APIs have to be designed carefully so can satisfy all current use-cases at least. Regards Liang On Feb 26, 2011, at 12:16 PM, Andreas Dilger wrote:> On 2011-02-25, at 4:37 PM, Mervini, Joseph A wrote: >> That could be awful handy - especially when trying to tune a live file system for performance. Is that going to be a 2.0 only enhancement or can it be applied to existing 1.8 versions? > > The patch was originally developed for 1.8, and ported to 2.1. That said, last time I tested it there were a few problems (crashing variety) so it isn''t ready for prime time yet. > > Testing/debugging would be appreciated, patch for 1.8 and 2.1 are at: > https://bugzilla.lustre.org/show_bug.cgi?id=22516 > > >> On Feb 24, 2011, at 9:19 PM, Andreas Dilger wrote: >>> Yes, this can be set at startup time to limit the number of started threads. There is a patch I wrote to also reduce the number of running treads but it wasn''t landed yet. >>> >>> Cheers, Andreas >>> >>> On 2011-02-24, at 14:04, "Mervini, Joseph A" <jamervi at sandia.gov> wrote: >>> >>>> I''m inclined to agree. So apparently the only time that modifying the runtime max values has a benefit is while the threads_started is low? >>>> >>>> Joe >>>> ===>>>> >>>> Joe Mervini >>>> Sandia National Laboratories >>>> High Performance Computing >>>> 505.844.6770 >>>> jamervi at sandia.gov >>>> >>>> >>>> >>>> On Feb 24, 2011, at 1:09 PM, Kevin Van Maren wrote: >>>> >>>>> However, I don''t think you can decrease the number of running threads. >>>>> See https://bugzilla.lustre.org/show_bug.cgi?id=22417 (and also >>>>> https://bugzilla.lustre.org/show_bug.cgi?id=22516 ) >>>>> >>>>> Kevin >>>>> >>>>> >>>>> Mervini, Joseph A wrote: >>>>>> Cool! Thank you Johann. >>>>>> ===>>>>>> >>>>>> Joe Mervini >>>>>> Sandia National Laboratories >>>>>> High Performance Computing >>>>>> 505.844.6770 >>>>>> jamervi at sandia.gov >>>>>> >>>>>> >>>>>> >>>>>> On Feb 24, 2011, at 11:05 AM, Johann Lombardi wrote: >>>>>> >>>>>> >>>>>>> On Thu, Feb 24, 2011 at 10:48:32AM -0700, Mervini, Joseph A wrote: >>>>>>> >>>>>>>> Quick question: Has runtime modification of the number of OST threads been implemented in Lustre-1.8.3? >>>>>>>> >>>>>>> Yes, see bugzilla ticket 18688. It was landed in 1.8.1. >> > > > Cheers, Andreas > -- > Andreas Dilger > Principal Engineer > Whamcloud, Inc. > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss