freebsd stable - Jan 2011 - 8-STABLE/amd64 semi-regular crash with "kernel trap 12 with interrupts disabled" in "process 12 (swi4: clock)"

Hello, Freebsd-stable.


  One of my servers crashes about once a week, with always same
diagnostics: "kernel trap 12 with interrupts disabled" and in same
process: "swi4: clock"

  It doesn't look as memory failure, as memtest86+ can not find any
errors in 8 passes.

  Also, after this crash server refuse to auto-reboot, last message on
console is "cpu_reset: Stopping other CPUs", and it hangs.

  Kernel config, booting dmesg & results of "savecore" are
attached
(bzipped).

-- 
// Black Lion AKA Lev Serebryakov <lev@FreeBSD.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: core.txt.0.bz2
Type: application/octet-stream
Size: 37857 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20110118/a4be568d/core.txt.0.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dmesg.boot.bz2
Type: application/octet-stream
Size: 4155 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20110118/a4be568d/dmesg.boot.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: KERNEL.CONFIG.bz2
Type: application/octet-stream
Size: 1701 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20110118/a4be568d/KERNEL.CONFIG.obj

On 19.01.2011 03:12, Lev Serebryakov wrote:
> Hello, Freebsd-stable.
> 
> 
>   One of my servers crashes about once a week, with always same
> diagnostics: "kernel trap 12 with interrupts disabled" and in
same
> process: "swi4: clock"
> 
>   It doesn't look as memory failure, as memtest86+ can not find any
> errors in 8 passes.
> 
>   Also, after this crash server refuse to auto-reboot, last message on
> console is "cpu_reset: Stopping other CPUs", and it hangs.
> 
>   Kernel config, booting dmesg & results of "savecore" are
attached
> (bzipped).
You have not mentioned what tasks does it perform.

Eugene Grosbein

On Wed, Jan 19, 2011 at 12:12:48AM +0300, Lev Serebryakov
wrote:
> Hello, Freebsd-stable.
> 
>   One of my servers crashes about once a week, with always same
> diagnostics: "kernel trap 12 with interrupts disabled" and in
same
> process: "swi4: clock"
> 
>   It doesn't look as memory failure, as memtest86+ can not find any
> errors in 8 passes.
> 
>   Also, after this crash server refuse to auto-reboot, last message on
> console is "cpu_reset: Stopping other CPUs", and it hangs.
> 
>   Kernel config, booting dmesg & results of "savecore" are
attached
> (bzipped).
CC'ing Jack Vogel of Intel, as this looks like it could be something the
em(4) driver might be tickling.  I do see it in the stack trace shortly
before the crash.  In the interim, can you please provide output from the
following command:

# pciconf -lbcv

And include only the entries relevant to your emX devices.

As for the "the server refuses to auto-reboot": that may be a separate
problem.  You might try toggling the hw.acpi.disable_on_reboot and
hw.acpi.handle_reboot sysctls (check what values they have on your
system first) to see if there's any improvement.

For Jack -- the core/stack trace, and dmesg are at the below URL as
attachments (and bzip2 compressed):

http://lists.freebsd.org/pipermail/freebsd-stable/2011-January/061168.html

-- 
| Jeremy Chadwick                                   jdc@parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.               PGP 4BD6C0CB |

Hello, Eugene.
You wrote 19 ?????? 2011 ?., 0:30:09:
> You have not mentioned what tasks does it perform.
  Storage of all my data with software RAID5 + torrent-box for
 25Mibt/s connection/

-- 
// Black Lion AKA Lev Serebryakov <lev@FreeBSD.org>

On 19.01.2011 03:12, Lev Serebryakov wrote:
> Hello, Freebsd-stable.
> 
> 
>   One of my servers crashes about once a week, with always same
> diagnostics: "kernel trap 12 with interrupts disabled" and in
same
> process: "swi4: clock"
> 
>   It doesn't look as memory failure, as memtest86+ can not find any
> errors in 8 passes.
> 
>   Also, after this crash server refuse to auto-reboot, last message on
> console is "cpu_reset: Stopping other CPUs", and it hangs.
> 
>   Kernel config, booting dmesg & results of "savecore" are
attached
> (bzipped).
There is known instability in em(4) driver in 8.2-RELEASE,
it may panic due to some lack of NULL pointer checks.
You should update to RELENG_8 containting fix and retest.

Eugene Grosbein

Hello, Eugene.
You wrote 19 ?????? 2011 ?., 11:44:01:
> There is known instability in em(4) driver in 8.2-RELEASE,
> it may panic due to some lack of NULL pointer checks.
> You should update to RELENG_8 containting fix and retest.
 uname -v
 FreeBSD 8.2-PRERELEASE #5: Sat Jan  8 14:38:46 MSK 2011

 It is built about hour after cvsup.

-- 
// Black Lion AKA Lev Serebryakov <lev@FreeBSD.org>