asterisk users - Aug 2006 - Asterisk server crashes after two years

My Asterisk colo server has been up for almost two years.  Today it 
crashed.  When I gave the reboot command, it crashed so hard that it had 
to be power cycled.  I wasn't in attendance, but I can speculate that it 
had a kernel panic during the shutdown.

Yesterday I added a PHP agi script, and it had been user over 1000 times 
before the crash.  I don't think the Linux/Asterisk crash is coincidental.

Can someone give me things to look for?  I'm watching memory, and it has 
750MB free (out of 1GB).  When I restart Asterisk, I see 19 
processes--is this normal?  What else should I be doing to narrow down 
on this problem.

Thanks for your help.


-- 
Michael Welter
Telecom Matters Corp.
Denver, Colorado US
+1.303.414.4980
mike@TelecomMatters.net
www.TelecomMatters.net

Michael Welter wrote:
> My Asterisk colo server has been up for almost two years.  Today it 
> crashed.  When I gave the reboot command, it crashed so hard that it 
> had to be power cycled.  I wasn't in attendance, but I can speculate 
> that it had a kernel panic during the shutdown.
>
> Yesterday I added a PHP agi script, and it had been user over 1000 
> times before the crash.  I don't think the Linux/Asterisk crash is 
> coincidental.
>
> Can someone give me things to look for?  I'm watching memory, and it 
> has 750MB free (out of 1GB).  When I restart Asterisk, I see 19 
> processes--is this normal?  What else should I be doing to narrow down 
> on this problem.
>
> Thanks for your help.
>
>
Have you checked the log files?
Do you use Real-time? Is your database ok?
Have you checked the hard disk space?

2 years Asterisk sounds "strange", since I can remember there was a
bug
with the date a year ago. If you have not upgraded, than this bug is 
still in your code. Maybe you just meant no reboot for two years.

bye

Ronald

On Thu, Aug 31, 2006 at 09:40:50PM -0600, Michael Welter
wrote:
> My Asterisk colo server has been up for almost two years.  Today it 
> crashed.  When I gave the reboot command, it crashed so hard that it had 
> to be power cycled.  I wasn't in attendance, but I can speculate that
it
> had a kernel panic during the shutdown.
> 
> Yesterday I added a PHP agi script, and it had been user over 1000 times 
> before the crash.  I don't think the Linux/Asterisk crash is
coincidental.
> 
> Can someone give me things to look for?  I'm watching memory, and it
has
> 750MB free (out of 1GB).  When I restart Asterisk, I see 19 
> processes--is this normal?  
Is this kernel 2.4? If so: do they happen to have exactly the same
memory size and the same files open? If so: this is normal: threads of
the same process.
> What else should I be doing to narrow down 
> on this problem.
One way to get a (huge) trace:

strace -f -o path/to/log/file command to start asterisk

Also try starting asterisk without -p, if you normally start it with it.

-- 
Tzafrir Cohen         sip:tzafrir@local.xorcom.com
icq#16849755          iax:tzafrir@local.xorcom.com
+972-50-7952406          jabber:tzafrir@jabber.org
tzafrir.cohen@xorcom.com     http://www.xorcom.com