Jagga Soorma
2010-Jan-26 23:10 UTC
[Lustre-discuss] /etc/init.d/openibd hanging system shutdown
Hi Guys, My SLES clients are hanging during a system shutdown right after giving me a message of Shutting down D-Bus Daemon done: /etc/init.d/rc3.d: ls K08* K08dbus K08openibd Has anyone noticed this on any of their clients if you are running sles? I have to kill the power on my client every time I need to shutdown. Any help would be greatly appreciated. Thanks, -J -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100126/7b1757e2/attachment.html
Jagga Soorma
2010-Jan-26 23:56 UTC
[Lustre-discuss] /etc/init.d/openibd hanging system shutdown
Did some more digging around and the mlx4_ib module is not unloading: ..snip.. + ''['' ''!'' -z ''ERROR: Removing ''\''''mlx4_ib''\'''': Device or resource busy'' '']'' ..snip.. # rmmod mlx4_ib ERROR: Removing ''mlx4_ib'': Device or resource busy #lctl net down LNET busy How can I shutdown lctl network? I do not have any lustre fs''s mounted on my client anymore. Thanks, -J On Tue, Jan 26, 2010 at 3:10 PM, Jagga Soorma <jagga13 at gmail.com> wrote:> Hi Guys, > > My SLES clients are hanging during a system shutdown right after giving me > a message of Shutting down D-Bus Daemon done: > > /etc/init.d/rc3.d: ls K08* > K08dbus K08openibd > > Has anyone noticed this on any of their clients if you are running sles? I > have to kill the power on my client every time I need to shutdown. > > Any help would be greatly appreciated. > > Thanks, > -J >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100126/1b96a75a/attachment.html
Erik Froese
2010-Jan-27 04:19 UTC
[Lustre-discuss] /etc/init.d/openibd hanging system shutdown
try lctl net down or lctl net unconfigure If those fail run lustre_rmmod Erik On Tue, Jan 26, 2010 at 6:56 PM, Jagga Soorma <jagga13 at gmail.com> wrote:> Did some more digging around and the mlx4_ib module is not unloading: > > ..snip.. > + ''['' ''!'' -z ''ERROR: Removing ''\''''mlx4_ib''\'''': Device or resource busy'' '']'' > ..snip.. > > # rmmod mlx4_ib > ERROR: Removing ''mlx4_ib'': Device or resource busy > > #lctl net down > LNET busy > > How can I shutdown lctl network? I do not have any lustre fs''s mounted on > my client anymore. > > Thanks, > -J > > > On Tue, Jan 26, 2010 at 3:10 PM, Jagga Soorma <jagga13 at gmail.com> wrote: > >> Hi Guys, >> >> My SLES clients are hanging during a system shutdown right after giving me >> a message of Shutting down D-Bus Daemon done: >> >> /etc/init.d/rc3.d: ls K08* >> K08dbus K08openibd >> >> Has anyone noticed this on any of their clients if you are running sles? >> I have to kill the power on my client every time I need to shutdown. >> >> Any help would be greatly appreciated. >> >> Thanks, >> -J >> > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100126/a844f6d2/attachment.html
Christopher J. Morrone
2010-Jan-27 04:40 UTC
[Lustre-discuss] /etc/init.d/openibd hanging system shutdown
Here we use an init script that does that. We use it on RHEL based systems, but it shouldn''t take much work to modify it for your needs: http://github.com/morrone/lustre/raw/1.8.2.0-5chaos/lustre/scripts/lnet It is also in a patch attached to bug 20165 (/etc/init.d/lnet) along with heartbeat scripts. Here''s the link: https://bugzilla.lustre.org/show_bug.cgi?id=20165 Chris Erik Froese wrote:> try lctl net down or lctl net unconfigure > > If those fail run lustre_rmmod > > Erik > > On Tue, Jan 26, 2010 at 6:56 PM, Jagga Soorma <jagga13 at gmail.com> wrote: > >> Did some more digging around and the mlx4_ib module is not unloading: >> >> ..snip.. >> + ''['' ''!'' -z ''ERROR: Removing ''\''''mlx4_ib''\'''': Device or resource busy'' '']'' >> ..snip.. >> >> # rmmod mlx4_ib >> ERROR: Removing ''mlx4_ib'': Device or resource busy >> >> #lctl net down >> LNET busy >> >> How can I shutdown lctl network? I do not have any lustre fs''s mounted on >> my client anymore. >> >> Thanks, >> -J >> >> >> On Tue, Jan 26, 2010 at 3:10 PM, Jagga Soorma <jagga13 at gmail.com> wrote: >> >>> Hi Guys, >>> >>> My SLES clients are hanging during a system shutdown right after giving me >>> a message of Shutting down D-Bus Daemon done: >>> >>> /etc/init.d/rc3.d: ls K08* >>> K08dbus K08openibd >>> >>> Has anyone noticed this on any of their clients if you are running sles? >>> I have to kill the power on my client every time I need to shutdown. >>> >>> Any help would be greatly appreciated. >>> >>> Thanks, >>> -J >>> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://*lists.lustre.org/mailman/listinfo/lustre-discuss >> >> > > > ------------------------------------------------------------------------ > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://*lists.lustre.org/mailman/listinfo/lustre-discuss
Jagga Soorma
2010-Jan-27 18:01 UTC
[Lustre-discuss] /etc/init.d/openibd hanging system shutdown
Thanks Erik, The lustre_rmmod took care of stopping and unloading the lnet module. That is what I guess needed to happen for openibd to not hang during a reboot. Thanks again for your assistance. -J On Tue, Jan 26, 2010 at 8:19 PM, Erik Froese <erik.froese at gmail.com> wrote:> try lctl net down or lctl net unconfigure > > If those fail run lustre_rmmod > > Erik > > On Tue, Jan 26, 2010 at 6:56 PM, Jagga Soorma <jagga13 at gmail.com> wrote: > >> Did some more digging around and the mlx4_ib module is not unloading: >> >> ..snip.. >> + ''['' ''!'' -z ''ERROR: Removing ''\''''mlx4_ib''\'''': Device or resource busy'' >> '']'' >> ..snip.. >> >> # rmmod mlx4_ib >> ERROR: Removing ''mlx4_ib'': Device or resource busy >> >> #lctl net down >> LNET busy >> >> How can I shutdown lctl network? I do not have any lustre fs''s mounted on >> my client anymore. >> >> Thanks, >> -J >> >> >> On Tue, Jan 26, 2010 at 3:10 PM, Jagga Soorma <jagga13 at gmail.com> wrote: >> >>> Hi Guys, >>> >>> My SLES clients are hanging during a system shutdown right after giving >>> me a message of Shutting down D-Bus Daemon done: >>> >>> /etc/init.d/rc3.d: ls K08* >>> K08dbus K08openibd >>> >>> Has anyone noticed this on any of their clients if you are running sles? >>> I have to kill the power on my client every time I need to shutdown. >>> >>> Any help would be greatly appreciated. >>> >>> Thanks, >>> -J >>> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100127/80bcd881/attachment.html