Hi everyone, I am running into wall today with simultaneous call limits. I have two Asterisk machines (fast 3GHz C2D with 2GB of ram). I tried to create a lot of sip calls from one machine to the other by issuing AMI Originate commands to one machine. The machine that makes calls plays a message (demo-intruct) upon the other machine answer. The machine receives the calls just waits for 40 seconds then hangs up. Throught the manager connection, I was creating 10 calls per-second. I also have sip phone registered with the calling machine. At around 150 to 200 calls. When I call the machine that's making all the calls, most of the calls couldn't go through. For the ones that went through, most of them will drop off within seconds of the call. But here is catch. When I run 'top', the cpu is idling 97%. My question is. Is there a limit on the number of simultaneous calls Asterisk can handle? I know I have very fast systems. Shouldn't they be able to handle that many calls? What is your take? Thnx
Nicholas Blasgen
2007-Sep-21 00:56 UTC
[asterisk-users] Asterisk 1.2.24 simultaneous call limits.
Just thinking about it quickly, it's always possible it has nothing to do with Asterisk. There are many instances where I run into issues with a poorly configured servers when they have even a little bump in HTTP traffic. This was years ago though, and it was an issue to do with a web server and not Asterisk, but look into your kernel's configuration. Sometimes the kernel's settings are setup for a normal USER and not designed to handle the memory allocation a server demands. The fix for me back then was something to do with the MAXIMUM PAGE REQUESTS or SIZE maybe. Basicly the kernel couldn't keep track of all the HTTP processes. Now that I'm reading this over I doubt it's your problem because Asterisk doesn't fork. But while we're at it, tell me a bit more about your system. What operating system (and version)? The problem could also be with your method of load generation, but I wouldn't know that since I've never tried load testing a system. Lastly, I know FreeBSD started incorporating a basic DDoS protection a few years back and maybe that's also in some of these newer Linux distros. They would detect a flood and start to limit the bandwidth. These are just ideas, I don't really like any of them. Sometimes the kernel will report issues to SYSLOGD. Might want to check your error and message logs. cat /proc/meminfo On a Linux box will give you memory limits and how close you are to them. They're not exactly what I was looking for, but maybe that will help. All TCP connections require the Kernel to page the information but I can't seem to find out how to access that limit if any. On 9/20/07, Wai Wu <wkwu at calltrol.com> wrote:> > > Hi everyone, > > I am running into wall today with simultaneous call limits. I have two > Asterisk machines (fast 3GHz C2D with 2GB of ram). I tried to create a > lot of sip calls from one machine to the other by issuing AMI Originate > commands to one machine. The machine that makes calls plays a message > (demo-intruct) upon the other machine answer. The machine receives the > calls just waits for 40 seconds then hangs up. Throught the manager > connection, I was creating 10 calls per-second. I also have sip phone > registered with the calling machine. At around 150 to 200 calls. When I > call the machine that's making all the calls, most of the calls couldn't > go through. For the ones that went through, most of them will drop off > within seconds of the call. But here is catch. When I run 'top', the cpu > is idling 97%. My question is. Is there a limit on the number of > simultaneous calls Asterisk can handle? I know I have very fast systems. > Shouldn't they be able to handle that many calls? What is your take? > > Thnx > > _______________________________________________ > > Sign up now for AstriCon 2007! September 25-28th. > http://www.astricon.net/ > > --Bandwidth and Colocation Provided by http://www.api-digital.com-- > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > http://lists.digium.com/mailman/listinfo/asterisk-users >-- /Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.digium.com/pipermail/asterisk-users/attachments/20070920/b1455546/attachment.htm
Interesting. I am using PCLinuxOS(Mandrak) in console mode. Here is my memory info and you can see that I still have a lot of memory while asterisk is running [root at localhost ~]# cat /proc/meminfo MemTotal: 2076000 kB MemFree: 1855636 kB Buffers: 17224 kB Cached: 115916 kB SwapCached: 0 kB Active: 124100 kB Inactive: 73468 kB HighTotal: 1179264 kB HighFree: 992808 kB LowTotal: 896736 kB LowFree: 862828 kB SwapTotal: 1365484 kB SwapFree: 1365484 kB Dirty: 216 kB Writeback: 0 kB AnonPages: 64524 kB Mapped: 43912 kB Slab: 13168 kB PageTables: 1344 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 2403484 kB Committed_AS: 142512 kB VmallocTotal: 114680 kB VmallocUsed: 8484 kB VmallocChunk: 104492 kB [root at localhost ~]# You mentioned DDoS projection. How can I find out if my distro has it built in? ________________________________ From: asterisk-users-bounces at lists.digium.com [mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Nicholas Blasgen Sent: Thursday, September 20, 2007 8:56 PM To: Asterisk Users Mailing List - Non-Commercial Discussion Subject: Re: [asterisk-users] Asterisk 1.2.24 simultaneous call limits. Just thinking about it quickly, it's always possible it has nothing to do with Asterisk. There are many instances where I run into issues with a poorly configured servers when they have even a little bump in HTTP traffic. This was years ago though, and it was an issue to do with a web server and not Asterisk, but look into your kernel's configuration. Sometimes the kernel's settings are setup for a normal USER and not designed to handle the memory allocation a server demands. The fix for me back then was something to do with the MAXIMUM PAGE REQUESTS or SIZE maybe. Basicly the kernel couldn't keep track of all the HTTP processes. Now that I'm reading this over I doubt it's your problem because Asterisk doesn't fork. But while we're at it, tell me a bit more about your system. What operating system (and version)? The problem could also be with your method of load generation, but I wouldn't know that since I've never tried load testing a system. Lastly, I know FreeBSD started incorporating a basic DDoS protection a few years back and maybe that's also in some of these newer Linux distros. They would detect a flood and start to limit the bandwidth. These are just ideas, I don't really like any of them. Sometimes the kernel will report issues to SYSLOGD. Might want to check your error and message logs. cat /proc/meminfo On a Linux box will give you memory limits and how close you are to them. They're not exactly what I was looking for, but maybe that will help. All TCP connections require the Kernel to page the information but I can't seem to find out how to access that limit if any. On 9/20/07, Wai Wu <wkwu at calltrol.com> wrote: Hi everyone, I am running into wall today with simultaneous call limits. I have two Asterisk machines (fast 3GHz C2D with 2GB of ram). I tried to create a lot of sip calls from one machine to the other by issuing AMI Originate commands to one machine. The machine that makes calls plays a message (demo-intruct) upon the other machine answer. The machine receives the calls just waits for 40 seconds then hangs up. Throught the manager connection, I was creating 10 calls per-second. I also have sip phone registered with the calling machine. At around 150 to 200 calls. When I call the machine that's making all the calls, most of the calls couldn't go through. For the ones that went through, most of them will drop off within seconds of the call. But here is catch. When I run 'top', the cpu is idling 97%. My question is. Is there a limit on the number of simultaneous calls Asterisk can handle? I know I have very fast systems. Shouldn't they be able to handle that many calls? What is your take? Thnx _______________________________________________ Sign up now for AstriCon 2007! September 25-28th. http://www.astricon.net/ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users <http://lists.digium.com/mailman/listinfo/asterisk-users> -- /Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.digium.com/pipermail/asterisk-users/attachments/20070920/6462a629/attachment.htm
Gordon Henderson
2007-Sep-21 07:40 UTC
[asterisk-users] Asterisk 1.2.24 simultaneous call limits.
On Thu, 20 Sep 2007, Wai Wu wrote:> > Hi everyone, > > I am running into wall today with simultaneous call limits. I have two > Asterisk machines (fast 3GHz C2D with 2GB of ram). I tried to create a > lot of sip calls from one machine to the other by issuing AMI Originate > commands to one machine. The machine that makes calls plays a message > (demo-intruct) upon the other machine answer. The machine receives the > calls just waits for 40 seconds then hangs up. Throught the manager > connection, I was creating 10 calls per-second. I also have sip phone > registered with the calling machine. At around 150 to 200 calls. When I > call the machine that's making all the calls, most of the calls couldn't > go through. For the ones that went through, most of them will drop off > within seconds of the call. But here is catch. When I run 'top', the cpu > is idling 97%. My question is. Is there a limit on the number of > simultaneous calls Asterisk can handle? I know I have very fast systems. > Shouldn't they be able to handle that many calls? What is your take?200 calls using g711 needs 16Mb/sec of network bandwidth - each way. (200 * 80Kbs) This is well within the limits of a 100Mb network interface. However it also needs 50 packets per second of 160 bytes + IP overhead each way, per call, so thats 20,000 packets/second, and that might well be the bottleneck for your system, not just in the hardware issues required to shovel that many packets over the various buses, but the Linux overhead of schedulling each of the 200 threads to take/send that data in real-time. You might want to run iperf on each machine with nothing else going and see just how many UDP packets of 160 bytes you can push between the machines before packet loss starts. Gordon
The errors are gone after I set the ulimit -n to 32786. The -x is unlimited. I found that AMI is very inefficient. During the time of issue a lot of AMI command, sip calls can't go into the asterisk box. I don't know if you guys have the same experience. ________________________________ From: asterisk-users-bounces at lists.digium.com [mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of James Texter Sent: Friday, September 21, 2007 11:19 AM To: Asterisk Users Mailing List - Non-Commercial Discussion Subject: Re: [asterisk-users] Asterisk 1.2.24 simultaneous call limits. What do you have ulimit -n and ulimit -x set to? Thanks, James Texter On Fri, 2007-09-21 at 08:51 -0400, Wai Wu wrote: I am not so sure if the interrupts has any thing to do with it. I run some more test just now and I am getting these error on the console of the call receiving machine. All it does is wait for 45 seconds. I think there is more can be done on the Linux configuration, but I just don't know what. Sep 21 08:42:30 WARNING[22820]: channel.c:565 ast_channel_alloc: Channel allocation failed: Can't create alert pipe! Sep 21 08:42:30 WARNING[22820]: chan_sip.c:2797 sip_new: Unable to allocate SIP channel structure Sep 21 08:42:30 NOTICE[22820]: chan_sip.c:10843 handle_request_invite: Unable to create/find channel Sep 21 08:43:03 WARNING[22820]: channel.c:565 ast_channel_alloc: Channel allocation failed: Can't create alert pipe! Sep 21 08:43:03 WARNING[22820]: chan_sip.c:2797 sip_new: Unable to allocate SIP channel structure Sep 21 08:43:03 NOTICE[22820]: chan_sip.c:10843 handle_request_invite: Unable to create/find channel -----Original Message----- From: asterisk-users-bounces at lists.digium.com on behalf of Gordon Henderson Sent: Fri 9/21/2007 3:40 AM To: Asterisk Users Mailing List - Non-Commercial Discussion Subject: Re: [asterisk-users] Asterisk 1.2.24 simultaneous call limits. On Thu, 20 Sep 2007, Wai Wu wrote: > > Hi everyone, > > I am running into wall today with simultaneous call limits. I have two > Asterisk machines (fast 3GHz C2D with 2GB of ram). I tried to create a > lot of sip calls from one machine to the other by issuing AMI Originate > commands to one machine. The machine that makes calls plays a message > (demo-intruct) upon the other machine answer. The machine receives the > calls just waits for 40 seconds then hangs up. Throught the manager > connection, I was creating 10 calls per-second. I also have sip phone > registered with the calling machine. At around 150 to 200 calls. When I > call the machine that's making all the calls, most of the calls couldn't > go through. For the ones that went through, most of them will drop off > within seconds of the call. But here is catch. When I run 'top', the cpu > is idling 97%. My question is. Is there a limit on the number of > simultaneous calls Asterisk can handle? I know I have very fast systems. > Shouldn't they be able to handle that many calls? What is your take? 200 calls using g711 needs 16Mb/sec of network bandwidth - each way. (200 * 80Kbs) This is well within the limits of a 100Mb network interface. However it also needs 50 packets per second of 160 bytes + IP overhead each way, per call, so thats 20,000 packets/second, and that might well be the bottleneck for your system, not just in the hardware issues required to shovel that many packets over the various buses, but the Linux overhead of schedulling each of the 200 threads to take/send that data in real-time. You might want to run iperf on each machine with nothing else going and see just how many UDP packets of 160 bytes you can push between the machines before packet loss starts. Gordon _______________________________________________ Sign up now for AstriCon 2007! September 25-28th. http://www.astricon.net/ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users _______________________________________________ Sign up now for AstriCon 2007! September 25-28th. http://www.astricon.net/ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.digium.com/pipermail/asterisk-users/attachments/20070921/ab4d3952/attachment.htm