I have a machine here that resets itself every one hour (without my intention, of course): # cat /var/log/messages | grep "sith kernel: Linux version 2.6.18-128.1.16.el5" Jul 14 22:29:41 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 14 23:30:09 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 00:30:36 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 01:31:04 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 02:31:31 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 03:32:01 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 04:32:30 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 05:32:58 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 06:33:26 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 07:33:56 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 08:34:21 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 09:34:52 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 10:38:48 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 11:35:47 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 12:36:17 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 13:36:46 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 The machine is supposed to be up 24/7 and after each reset the system reboots again, and works normally only to reset again one hour later. However, the only unusual thing I am able to recognize in the logs is winbind daemon whining about something I don't understand: # tail -f /var/log/messages Jul 15 13:47:39 sith winbindd[3353]: [2009/07/15 13:47:39, 0] nsswitch/idmap.c:idmap_alloc_init(820) Jul 15 13:47:39 sith winbindd[3353]: ERROR: Initialization failed for alloc backend, deferred! Jul 15 13:47:39 sith smbd[3373]: [2009/07/15 13:47:39, 0] auth/auth_util.c:create_builtin_users(810) Jul 15 13:47:39 sith smbd[3373]: create_builtin_users: Failed to create Users Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] nsswitch/winbindd_passdb.c:sid_to_name(126) Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to lookup SID S-1-22-1-99 with passdb backend Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] nsswitch/winbindd_passdb.c:sid_to_name(126) Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to lookup SID S-1-1-0 with passdb backend Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] nsswitch/winbindd_passdb.c:sid_to_name(126) Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to lookup SID S-1-5-2 with passdb backend Nevertheless, samba server seems to be running ok, so I am not sure this is related to reboots, and there is nothing else suspicious in the logs AFAICT. How do I troubleshoot these restarts? I suspected hardware failure (power supply, cooling fans, etc.) but somehow the restarts happen way too periodically, so I have second thoughts on software as well. CentOS 5.3, fully updated. Btw, I don't have physical access to the machine until September, so diagnosing hardware is pretty limited atm. Only remote ssh available. I would really appreciate any advice on this! Best, :-) Marko
Hi I am really not sure if this is related but I had a case sometime ago with a server that rebooted really reguarly and the problem was the nic that was faulty causing a kernel panic, but in these cases when there is really not much to go on from the logs For your possible deadlocks you could add into smb.conf strict locking = no Regards Per Qvindesland E-mail: per at norhex.com [1] http://www.linkedin.com/in/perqvindesland [2] --- Original message follows --- SUBJECT:?[CentOS] My server reboots every hour! Help please! FROM: ?Marko Vojinovic TO:?"CentOS mailing list" DATE:?15-07-2009 14:16 I have a machine here that resets itself every one hour (without my intention, of course): # cat /var/log/messages | grep "sith kernel: Linux version 2.6.18-128.1.16.el5" Jul 14 22:29:41 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 14 23:30:09 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 00:30:36 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 01:31:04 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 02:31:31 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 03:32:01 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 04:32:30 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 05:32:58 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 06:33:26 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 07:33:56 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 08:34:21 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 09:34:52 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 10:38:48 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 11:35:47 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 12:36:17 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 Jul 15 13:36:46 sith kernel: Linux version 2.6.18-128.1.16.el5 (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 The machine is supposed to be up 24/7 and after each reset the system reboots again, and works normally only to reset again one hour later. However, the only unusual thing I am able to recognize in the logs is winbind daemon whining about something I don't understand: # tail -f /var/log/messages Jul 15 13:47:39 sith winbindd[3353]: [2009/07/15 13:47:39, 0] nsswitch/idmap.c:idmap_alloc_init(820) Jul 15 13:47:39 sith winbindd[3353]: ERROR: Initialization failed for alloc backend, deferred! Jul 15 13:47:39 sith smbd[3373]: [2009/07/15 13:47:39, 0] auth/auth_util.c:create_builtin_users(810) Jul 15 13:47:39 sith smbd[3373]: create_builtin_users: Failed to create Users Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] nsswitch/winbindd_passdb.c:sid_to_name(126) Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to lookup SID S-1-22-1-99 with passdb backend Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] nsswitch/winbindd_passdb.c:sid_to_name(126) Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to lookup SID S-1-1-0 with passdb backend Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] nsswitch/winbindd_passdb.c:sid_to_name(126) Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to lookup SID S-1-5-2 with passdb backend Nevertheless, samba server seems to be running ok, so I am not sure this is related to reboots, and there is nothing else suspicious in the logs AFAICT. How do I troubleshoot these restarts? I suspected hardware failure (power supply, cooling fans, etc.) but somehow the restarts happen way too periodically, so I have second thoughts on software as well. CentOS 5.3, fully updated. Btw, I don't have physical access to the machine until September, so diagnosing hardware is pretty limited atm. Only remote ssh available. I would really appreciate any advice on this! Best, :-) Marko _______________________________________________ CentOS mailing list CentOS at centos.org http://lists.centos.org/mailman/listinfo/centos Links: ------ [1] http://webmail.norhex.com/# [2] http://www.linkedin.com/in/perqvindesland -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.centos.org/pipermail/centos/attachments/20090715/ab8ed7ea/attachment-0003.html>
On Wed, Jul 15, 2009 at 8:16 AM, Marko Vojinovic <vvmarko at gmail.com> wrote:> I have a machine here that resets itself every one hour (without my > intention, of course): > > # cat /var/log/messages | grep "sith kernel: Linux version > 2.6.18-128.1.16.el5" > Jul 14 22:29:41 sith kernel: Linux version 2.6.18-128.1.16.el5 > (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat > 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 > Jul 14 23:30:09 sith kernel: Linux version 2.6.18-128.1.16.el5 >Check the root crontab and the cron.hourly directory for a scheduled job?? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.centos.org/pipermail/centos/attachments/20090715/184ceb8b/attachment-0003.html>
On Wed, Jul 15, 2009 at 1:23 PM, Kwan Lowe<kwan.lowe at gmail.com> wrote:> On Wed, Jul 15, 2009 at 8:16 AM, Marko Vojinovic <vvmarko at gmail.com> wrote: >> >> I have a machine here that resets itself every one hour (without my >> intention, of course): >> >> # cat /var/log/messages | grep "sith kernel: Linux version >> 2.6.18-128.1.16.el5" >> Jul 14 22:29:41 sith kernel: Linux version 2.6.18-128.1.16.el5 >> (mockbuild at builder16.centos.org) (gcc version 4.1.2 20080704 (Red Hat >> 4.1.2-44)) #1 SMP Tue Jun 30 06:10:28 EDT 2009 >> Jul 14 23:30:09 sith kernel: Linux version 2.6.18-128.1.16.el5 > > Check the root crontab and the cron.hourly directory for a scheduled job??Nothing suspicious there: # cat /etc/crontab SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root HOME=/ # run-parts 01 * * * * root run-parts /etc/cron.hourly 02 4 * * * root run-parts /etc/cron.daily 22 4 * * 0 root run-parts /etc/cron.weekly 42 4 1 * * root run-parts /etc/cron.monthly # cd /etc/cron.hourly/ # ll -a total 40 drwxr-xr-x 2 root root 4096 Apr 1 14:04 . drwxr-xr-x 115 root root 12288 Jul 15 14:04 .. -rwxr-xr-x 1 root root 118 Feb 26 23:01 inn-cron-nntpsend -rwxr-xr-x 1 root root 118 Feb 26 23:01 inn-cron-rnews # chkconfig innd --list innd 0:off 1:off 2:off 3:off 4:off 5:off 6:off I should also note that resets are abrupt, the system doesn't seem to go through shutdown phase. Thanks for the suggestion, though! Best, :-) Marko
Marko Vojinovic wrote:> I have a machine here that resets itself every one hour (without my > intention, of course):<cut> I had a similar problem, it turned to be "wake on LAN" option in BIOS. Can you check if this option is ON on your side? Regards,
On Wed, 2009-07-15 at 13:16 +0100, Marko Vojinovic wrote:> > The machine is supposed to be up 24/7 and after each reset the system > reboots again, and works normally only to reset again one hour later. > However, the only unusual thing I am able to recognize in the logs is > winbind daemon whining about something I don't understand: > > # tail -f /var/log/messages > Jul 15 13:47:39 sith winbindd[3353]: [2009/07/15 13:47:39, 0] > nsswitch/idmap.c:idmap_alloc_init(820) > Jul 15 13:47:39 sith winbindd[3353]: ERROR: Initialization failed > for alloc backend, deferred! > Jul 15 13:47:39 sith smbd[3373]: [2009/07/15 13:47:39, 0] > auth/auth_util.c:create_builtin_users(810) > Jul 15 13:47:39 sith smbd[3373]: create_builtin_users: Failed to create Users > Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] > nsswitch/winbindd_passdb.c:sid_to_name(126) > Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to > lookup SID S-1-22-1-99 with passdb backend > Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] > nsswitch/winbindd_passdb.c:sid_to_name(126) > Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to > lookup SID S-1-1-0 with passdb backend > Jul 15 13:47:39 sith winbindd[2927]: [2009/07/15 13:47:39, 0] > nsswitch/winbindd_passdb.c:sid_to_name(126) > Jul 15 13:47:39 sith winbindd[2927]: Possible deadlock: Trying to > lookup SID S-1-5-2 with passdb backend > > Nevertheless, samba server seems to be running ok, so I am not sure > this is related to reboots, and there is nothing else suspicious in > the logs AFAICT. >--- Almost positive the winbind and samba errors are not related to the reboots. Looks like you had samba configured for LDAP type auth then switched to user auth and winbind is complaining about lookups. Thus if that is that case turn off winbindd. John
Apparently Analagous Threads
- Gathering information about RAM in sockets
- winbind filling up log with "Possible deadlock: Trying to lookup SID xxx with passdb backend"
- Power failure diagnosis?
- Two cleanly installed CentOS 5.6 servers but with different Xen kernel versions
- r-devel fails tests for parallel