asterisk users - Jan 2005 - Asterisk server stopped - "0-order allocation failed " errors in the log

Hey, 

 

One of my remote Asterisk servers which has been up and running for a couple
of months now suddenly stopped. I didn't realize it because my monitoring
system only pings the machine. But my remote office complained, I checked
"iax2 show peers" and saw they weren't checked in. So I pinged the
IP of
their Asterisk server and got a response.  Tried to SSH in but it timed out.
Machine must be in some kind of locked state. So I had them reboot it, it's
now up and working fine, but the log files show:

 

.

Jan 12 15:10:00 palmtrace-nuvoip CRON[30145]: (root) CMD (test -x
/usr/sbin/run-crons && /usr/sbin/run-crons )

Jan 12 15:12:27 palmtrace-nuvoip tftpd[7279]: Serving ata000d28383ca9 to
10.9.8.49:6616

Jan 12 15:12:27 palmtrace-nuvoip tftpd[21012]: Serving atadefault.cfg to
10.9.8.49:6617

Jan 12 15:13:43 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1d2/0)

Jan 12 15:13:43 palmtrace-nuvoip VM: killing process mpg123

Jan 12 15:13:48 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:13:48 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:17 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:17 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:19 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:19 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:19 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:21 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:46 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:50 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:50 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0xf0/0)

Jan 12 15:14:50 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:50 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:50 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0xf0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0xf0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0xf0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 15:14:58 palmtrace-nuvoip __alloc_pages: 0-order allocation failed
(gfp=0x1f0/0)

Jan 12 23:29:07 palmtrace-nuvoip syslog-ng[17735]: syslog-ng version 1.6.5
starting

.

 

 

 

The only clue is the mpg123 error. Do you think it's possible all my memory
got allocated? It's running Asterisk CVS-v1-0-10/21/04-00:18:55 

 

Should I upgrade it w/ the MOH patch? Anyways, any advice or help would be
appreciated.

 

Regards,

Paul

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20050112/ad6fdb40/attachment.htm

On Wed, 2005-01-12 at 15:50 -0500, Paul Rodan wrote:
> Hey, 
> 
>  
> 
> One of my remote Asterisk servers which has been up and running for a
> couple of months now suddenly stopped. I didn?t realize it because my
> monitoring system only pings the machine. But my remote office
> complained, I checked ?iax2 show peers? and saw they weren?t checked
> in. So I pinged the IP of their Asterisk server and got a response.
>  Tried to SSH in but it timed out. Machine must be in some kind of
> locked state. So I had them reboot it, it?s now up and working fine,
> but the log files show:
> 
>  
> 
> ?
> 
> Jan 12 15:10:00 palmtrace-nuvoip CRON[30145]: (root) CMD (test
> -x /usr/sbin/run-crons && /usr/sbin/run-crons )
> 
> Jan 12 15:12:27 palmtrace-nuvoip tftpd[7279]: Serving ata000d28383ca9
> to 10.9.8.49:6616
> 
> Jan 12 15:12:27 palmtrace-nuvoip tftpd[21012]: Serving atadefault.cfg
> to 10.9.8.49:6617
> 
> Jan 12 15:13:43 palmtrace-nuvoip __alloc_pages: 0-order allocation
> failed (gfp=0x1d2/0)
> 
> Jan 12 15:13:43 palmtrace-nuvoip VM: killing process mpg123
> 
> Jan 12 15:13:48 palmtrace-nuvoip __alloc_pages: 0-order allocation
> failed (gfp=0x1f0/0)
> Jan 12 23:29:07 palmtrace-nuvoip syslog-ng[17735]: syslog-ng version
> 1.6.5 starting
> 
> The only clue is the mpg123 error. Do you think it?s possible all my
> memory got allocated? It?s running Asterisk
> CVS-v1-0-10/21/04-00:18:55 
I'm pretty sure you ran out of memory. I don't think mpg123 caused it. 
> Should I upgrade it w/ the MOH patch? Anyways, any advice or help
> would be appreciated.
Advice would be to first to ensure you don't use HTML email. 

Next, If you thought it was out of memory, it would probably be a good
idea to also mention hardware specs including RAM and swap space. 

After that, you might want to look around and see if you might have been
hit by some kind of worm. Look for any other processes that decided to
fail around the same time. I know of a few people who are experiencing
unexplained ssh probes. 
-- 
Steven Critchfield <critch@basesys.com>