Hello. when trying to build the lustre client kernel modules configure aborts with the message "configure: error: Lustre does not support kernels with preempt enabled." I''m using kernel 2.6.22.19-0.12-rt with CONFIG_PREEMPT_RT as provided by SLERT 10. Is there a quick solution to this issue ? Regards, Ralph
Brian J. Murrell
2008-May-29 16:35 UTC
[Lustre-discuss] Lustre clients: SLERT 10 support ?
On Thu, 2008-05-29 at 18:24 +0200, Ralph Krotz wrote:> Hello. > > when trying to build the lustre client kernel modules configure aborts with > the message "configure: error: Lustre does not support kernels with preempt > enabled." > > I''m using kernel 2.6.22.19-0.12-rt with CONFIG_PREEMPT_RT as provided by SLERT > 10. > > Is there a quick solution to this issue ?Not really. Maybe somebody with more knowledge about why we don''t like CONFIG_PREEMPT_RT on the client can explain, but as far as addressing your failure directly, the only option would be to disable CONFIG_PREEMPT_RT and rebuild your client kernel. Presumably this would conflict with your use of the -rt SLERT kernel though. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080529/08adb192/attachment.bin
On Thu, May 29, 2008 at 12:35:15PM -0400, Brian J. Murrell wrote:> > when trying to build the lustre client kernel modules configure aborts with > > the message "configure: error: Lustre does not support kernels with preempt > > enabled." > > > > I''m using kernel 2.6.22.19-0.12-rt with CONFIG_PREEMPT_RT as provided by SLERT > > 10. > > > > Is there a quick solution to this issue ? > > Not really. Maybe somebody with more knowledge about why we don''t like > CONFIG_PREEMPT_RT on the client can explain, but as far as addressingWith CONFIG_PREEMPT_RT enabled, critical sections protected by standard spinlocks (i.e. spinlock_t/rwlock_t) become preemptible. This has several drawbacks and is not at all a harmless change for us. For instance, since you can now block while acquiring a spinlock, we are no longer allowed to acquire a spinlock with interrupts disabled. It is still possible to define non-preemptible sections by using raw_spinlock_t instead of spinlock_t, but Lustre doesn''t support this currently and just replacing spinlock_t by raw_spinlock_t everywhere in the code would probably defeat the purpose of CONFIG_PREEMPT_RT (i.e. hard realtime). Johann
On Fri, May 30, 2008 at 01:10:18PM +0200, Johann Lombardi wrote:> On Thu, May 29, 2008 at 12:35:15PM -0400, Brian J. Murrell wrote: > > > when trying to build the lustre client kernel modules configure aborts with > > > the message "configure: error: Lustre does not support kernels with preempt > > > enabled." > > > > > > I''m using kernel 2.6.22.19-0.12-rt with CONFIG_PREEMPT_RT as provided by SLERT > > > 10.As a side note, we don''t support CONFIG_PREEMPT too. See bug 14685 for more information (more particularly Andreas'' comment). Johann
On May 29, 2008 12:35 -0400, Brian J. Murrell wrote:> On Thu, 2008-05-29 at 18:24 +0200, Ralph Krotz wrote: > > when trying to build the lustre client kernel modules configure aborts with > > the message "configure: error: Lustre does not support kernels with preempt > > enabled." > > > > I''m using kernel 2.6.22.19-0.12-rt with CONFIG_PREEMPT_RT as provided by > > SLERT 10. > > > > Is there a quick solution to this issue ?The "quick" solution is to just remove this configure check from build/autoconf/lustre-build-linux.m4.> Not really. Maybe somebody with more knowledge about why we don''t like > CONFIG_PREEMPT_RT on the client can explain, but as far as addressing > your failure directly, the only option would be to disable > CONFIG_PREEMPT_RT and rebuild your client kernel. Presumably this would > conflict with your use of the -rt SLERT kernel though.There were problems with PREEMPT in the past, but I don''t know if they were resolved or not as we test without this feature enabled. I seem to recall an issue with performance loss on HPC computing jobs due to PREEMPT being enabled, but I don''t recall the details. If you are running in a non-production environment (i.e. terrible things will not happen if client hangs), then I''d suggest to run with PREEMPT enabled and see where it gets you. Please feed back any experience you have. I''ve been running with PREEMPT enabled on my home test cluster for some time without ill effects, but it doesn''t get a very high load so it may not expose the kind of problems that might be seen under heavy load. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.