Hello! I am using very recent FreeBSD-9-STABLE snapshot: 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #0 r254986: Wed Aug 28 17:18:57 MSK 2013 I run uwsgi program (ports/www/uwsgi) on that machine. When uwsgi starts, it forks pre-configured number of worker processes. If I raise workers parameter high enough (128), I get kernel panic (100% reproducible): Fatal trap 12: page fault while in kernel mode If I compile kernel with KDB enabled, I get the following stack: pmap_demote_pde_locked() pmap_copy() vmspace_fork() fork1() sys_fork() I have only remote console for that machine, so I made 2 screenshots: 1) http://people.freebsd.org/~demon/screen1.jpg Panic screen when kernel has no KDB support compiled in 2) http://people.freebsd.org/~demon/screen2.jpg Panic screen (2nd part) with the above stack shown. I can provide any additional information if needed. Thanks!
On Wed, Aug 28, 2013 at 06:20:29PM +0400, Dmitry Sivachenko wrote:> Hello! > > I am using very recent FreeBSD-9-STABLE snapshot: > 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #0 r254986: Wed Aug 28 17:18:57 MSK 2013 > > I run uwsgi program (ports/www/uwsgi) on that machine. > > When uwsgi starts, it forks pre-configured number of worker processes. > If I raise workers parameter high enough (128), I get kernel panic (100% reproducible): > > Fatal trap 12: page fault while in kernel mode > > If I compile kernel with KDB enabled, I get the following stack: > > pmap_demote_pde_locked() > pmap_copy() > vmspace_fork() > fork1() > sys_fork() > > I have only remote console for that machine, so I made 2 screenshots: > > 1) http://people.freebsd.org/~demon/screen1.jpg > Panic screen when kernel has no KDB support compiled in > > 2) http://people.freebsd.org/~demon/screen2.jpg > Panic screen (2nd part) with the above stack shown.Look up the source line for the pmap_demote_pde_locked()+0x471 for your kernel. Dump the core from the panic. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 834 bytes Desc: not available URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20130829/41ff9362/attachment.sig>
On 29.08.2013, at 22:45, Konstantin Belousov <kostikbel at gmail.com> wrote:> On Wed, Aug 28, 2013 at 06:20:29PM +0400, Dmitry Sivachenko wrote: >> Hello! >> >> I am using very recent FreeBSD-9-STABLE snapshot: >> 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #0 r254986: Wed Aug 28 17:18:57 MSK 2013 >> >> I run uwsgi program (ports/www/uwsgi) on that machine. >> >> When uwsgi starts, it forks pre-configured number of worker processes. >> If I raise workers parameter high enough (128), I get kernel panic (100% reproducible): >> >> Fatal trap 12: page fault while in kernel mode >> >> If I compile kernel with KDB enabled, I get the following stack: >> >> pmap_demote_pde_locked() >> pmap_copy() >> vmspace_fork() >> fork1() >> sys_fork() >> >> I have only remote console for that machine, so I made 2 screenshots: >> >> 1) http://people.freebsd.org/~demon/screen1.jpg >> Panic screen when kernel has no KDB support compiled in >> >> 2) http://people.freebsd.org/~demon/screen2.jpg >> Panic screen (2nd part) with the above stack shown. > Look up the source line for the pmap_demote_pde_locked()+0x471 for your > kernel. Dump the core from the panic.Kernel dump is not generated (despite it is configured at boot), there is no "Dumping...." message on console. These screenshots shows everything I see on console. I performed some more investigations on this: I have several (14) totally identical configured machines running exactly the same software. Hardware is a bit different though. I tried to analyze motherboard differences but failed to find common things for the affected machines. Under conditions described in my initial e-mail, some of them crash (exactly the same way), some of them do not. I am confident there is no hardware problems, these machines run for months without reboot, as for now I discovered the only way to crash them. I updated one of the affected servers to 10-current and I can state it does not crash anymore with the same usage scenario.