freebsd stable - Nov 2006 - application hangs in STABLE from time to time

Hi,

I have strange problem. I'm running radius server which in turn serves 
user requests by
running application. The application connects to another host (mysql 
server) and gets data and
do some calculations and return results.
Application runs fine, but from time to time it hangs and I can see it 
using ps -ax command.
So when application hangs radius can not serve user requests any more 
waiting the application
to finish.

Before updating from FreeBSD-5.2-STABLE to RELENG_6 it was working fine 
and I didn't observe
such problem for 2 years. Since upgrading to RELENG_6 a couple of months 
ago this problem appeared.

While application was hanging, I can log into the system. I can log into 
another mysql host which is
the exactly same machine with more RAM. I checked mysql server's 
processlist by running
'mysql show processlist' and there wasn't any process running and 
locking the tables.

I'm using mysql client/server-4.0.27 from ports.

So do I have interrupt storms here and it is something related to bge?
What else should I check when application hangs again?

Here is dmesg, vmstat, ps outputs:

http://www.mnbsd.org/ftp/rad_hang.txt

thanks,

Ganbold

On Fri, Nov 24, 2006 at 10:02:23AM +0800, Ganbold wrote:
> So do I have interrupt storms here and it is something related to bge?
No, your interrupts look fine.
> What else should I check when application hangs again?
The most important thing to know is what is the application doing when
it hangs.  Unfortunately none of the information you provided shows
this.  Next time use the -o wchan argument to ps to find out what
state the process is blocked in.  You can also use kgdb to find out
where it is waiting in the kernel:

kgdb /dev/mem /boot/kernel/kernel.symbols
info threads
<find the thread corresponding to the process that is blocked>
thread <tid>
bt

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20061124/cbb2a246/attachment.pgp