Wai Wu
2006-Feb-02 22:11 UTC
[Asterisk-Users] RE: 5, 000 concurrent calls system rolloutquestion
I don't think they are doing it with one Asterisk box. They did say
"one rack of servers". Well, that might mean up to 50 computers if
they are using blade servers.
-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com]On Behalf Of John Todd
Sent: Thursday, February 02, 2006 10:21 PM
To: asterisk-users@lists.digium.com
Subject: RE: [Asterisk-Users] RE: 5,000 concurrent calls system
rolloutquestion
[top-posting continued due to formatting sloth on my part]
So, then let me follow up with a few more comments:
1) I will make some assumptions from your note:
a) Asterisk is currently capable (unless something has "broken"
recently) of handling 2500 SIP-SIP calls with no transcoding,
including RTP sessions, if on an operating system and hardware that
is appropriately configured. This puts to rest some who have claimed
that 5000 channels is "impossible" with Asterisk regardless of
platform, at least according to Signate.
b) It is unclear if other channel drivers (IAX and Zaptel,
specifically) have had any testing with significant numbers of
channels.
c) It is unclear if anything other than pure RTP passthrough is
viable in these configurations. Maybe IVR causes collapse. ?
2) Still no claims or comments on the specific testing methods, or on
methodology. I'm left still scratching my head as to if this is
actually possible, since there is no specific claim that can be
verified. While I hope that your system can do those numbers (it
would help me greatly in the future!) I can't say that I'm confident
yet. I'll follow up in private email for further discussion.
3) Nobody else has thus far taken the bait and made any comments
about their systems. I appreciate Signate's comments; they seem to be
the only ones to publicly claim large-scale throughput using Asterisk
in a public forum. Most other people who claim thousands or even
high hundreds of connections do so offhand, without responding to
second questions when I raise my figurative eyebrows.
4) There are still no notes on other problems with scale here. I've
had systems with several hundred simultaneous SIP connections, but
"sip show channels" sure does start to take a while. What _other_
problems crop up, but don't necessarily cause a "failure"
condition?
5) I will agree that most SIP testing systems are currently too
pricey. I would love to find a well-connected network that rents out
a few of the better-known SIP testing tools to beat on Asterisk
installations in remote places for short periods of time. But this
has always been the case... test gear is a small market, and
expensive. Just look at the MSRP of new high-end HP Oscilloscopes if
you want to get a picture of price-gouging.
JT
At 11:21 AM -0800 2/2/06, William Boehlke wrote:>
>Signate has claimed 5,000 streams, or 2,500 calls, on a single Telephony
>Server 5000. The throughput has little to do with Asterisk and a lot to do
>with hardware design and operating system tuning. Our very minor code
>changes were returned to the project last year.
>
>The benchmark we used to make that initial claim was flawed, however we have
>since replicated the throughput in a different way to save our marketing
>bacon.
>
>How we actually achieve the throughput is our intellectual property but we
>have a number of customers who are scaling towards and past that traffic
>level. One of these days we hope to be able to justify the very large fee
>Hammer wants to extract from us to produce a third party verification.
>
>In production environments, of course, systems do more than switch calls. We
>think high volume system design using 32-bit systems of any kind is complex,
>and it's difficult to replicate the volumes without actual customer
traffic
>- and by then it's too late. Where do you put voicemail? Where does the
IVR
>reside?
>
>When someone needs to switch 5,000 calls with Class 5 services we would
>specify a rack of servers. The good news is that it is one rack, not three
>of them, but we need more than Asterisk alone, great though it is, to make
>everything work.
>
>
>-----Original Message-----
>From: asterisk-users-bounces@lists.digium.com
>[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of John Todd
>Sent: Wednesday, February 01, 2006 9:33 PM
>To: asterisk-users@lists.digium.com; asterisk-dev@lists.digium.com
>Subject: [Asterisk-Users] RE: 5, 000 concurrent calls system rollout
>question
>
>
>>Signate sells a single server that can get you to the call volumes you
>need.
>>
>>Paul Mahler
>><mailto:pmahler@signate.com>pmahler@signate.com
>>www.signate.com
>>
>[snip]
>
>Past conversations on this topic have generated quite a bit of
>controversy within the Asterisk development community, both publicly
>here on the list forums as well as in quite a few more quiet
>discussions with people who often do not post but have extensive
>operational experiences with Asterisk (most of whom monitor the -dev
>list and whose replies will be suited to that audience.)
>
>The subject of load on a single chassis is still the most contentious
>issue to date. The Signate numbers of >5000 calls per chassis with
>RTP are impressive, and there are others who claim more vaguely of
>1000, 2000, or more calls into a single P4 server (with or without
>media.) Others say that there are inherent limits in the Asterisk
>code which prevent more than ~500 calls from being processed with RTP
>at any one time. Opterons, FreeBSD, custom Linux loads, Solaris, and
>other operating systems or hardware have been offered as the magic
>bullets to increase call volumes. Who knows? (1) I will say that
>extraordinary claims demand extraordinary evidence, which has been
>pretty thin. I believe that most large call processing facilities
>still run on distributed systems of some type, as was described in
>the primary thread of this discussion on -users. (2)
>
>I know that there are some projects towards testing Asterisk more
>rigorously to determine these numbers. However, I would suggest that
>the community at large could benefit from a more open examination of
>high-end system claims immediately than these (better) long-term
>tests which are progressing slowly (if at all.) Let's just look at
>the "maximum" numbers. Running a big system? Selling a big
system?
>Tell us about it, in detail. What are the limits that have been hit?
>Be specific. I keep seeing hand-waving, but no programmers have come
>forward to say "It won't work because of the way X is implemented
in
>the file blah.c or libFOO."
>
>To make a bad analogy: I don't want to see the street rods; I just
>want to see the top-fuel, rocket-powered dragsters on the line. Any
>takers? It sounds like Signate has a contender, but quite a few
>people have said that it's impossible without serious modifications
>to the code. Others have claimed (publicly or privately) that they
>can match those numbers on different hardware.
>
>Here are the criteria:
> - Any O/S
> - An unmodified version of Asterisk from SVN (or CVS)
> OR patches must be available for inspection, as per the GPL
> OR you must be a Digium license-holder (patches can be secret)
> - All calls are IAX2 or SIP (both in and out)
> - No transcoding of any type is required
> - All calls are G.711, 20ms OR 30ms packet size
>
>Documentation:
> - All O/S documentation, kernel tricks, modules, hacks, patches, or
>configuration elements should be documented, but proprietary
>information need not be divulged if that is deemed "secret"
> - Testing method must be reasonably documented
> - Dialplans must be included
> - SIP.conf files must be included
> - All hardware must be fully described (part numbers required)
>
>TEST #1:
> All media must be handled by the server. This is for both legs of
>the call. The "canreinvite=no" for SIP and
"notransfer=yes" in IAX2
>must be set for all calls.
>
>TEST #2:
> Media may or may not be handled by the server. Native transfers
>should be allowed in both IAX2 and/or SIP.
>
>
>(1) I have heard various people saying that it is "impossible" for
>Asterisk to handle a large number of calls due to architectural
>issues (no, it's not just from the people that you'd
"expect" to hear
>this from.) I've not been able to validate this one way or the other
>recently. I am interested to hear what the developer community has
>as a comment on this topic. I have an Empirix Hammer system at my
>company, but honestly I just don't have the time to set it up to do
>testing due to day job time constraints...
>
>(2) There are so many ways to spread calls across an Asterisk array
>it makes my head spin, but the question STILL comes down to "how many
>calls can a single chassis handle?" Even in a farm of servers, there
>has to be a numerator in that ratio.
>
>JT
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --
Asterisk-Users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
asterisk@anime.net
2006-Feb-02 22:58 UTC
[Asterisk-Users] RE: 5, 000 concurrent calls system rolloutquestion
On Fri, 3 Feb 2006, Wai Wu wrote:> I don't think they are doing it with one Asterisk box. They did say > "one rack of servers". Well, that might mean up to 50 computers if they > are using blade servers.> At 11:21 AM -0800 2/2/06, William Boehlke wrote: >> Signate has claimed 5,000 streams, or 2,500 calls, on a single Telephony^^^^^^^^^^^^^^^^>> Server 5000.^^^^^^^^^^^ If you look at the datasheet http://www.signate.com/pdf/TelephonyServer.pdf its pretty clear its a cluster. I dont think anyone would be able to route 5,000 RTP streams on a single CPU these days, no matter how studly it is. -Dan
William Boehlke
2006-Feb-03 11:28 UTC
[Asterisk-Users] RE: 5, 000 concurrent calls system rolloutquestion
One of our Telephony Server 5000 modules will throughput between 2,000 and
2,500 SIP calls with streams if it is doing no other work. One of these days
we will again announce the details of the ongoing benchmarks that we perform
with the help of system engineers from a major computer manufacturer.
The key statement is "if it is doing no other work." If a server is
playing
IVR or hosting conferences, throughput declines in unpredictable ways
depending on the actual mix of work. So when we spec a system for a
particular call volume we use relatively conservative engineering to ensure
that the system can handle the peak load.
In real applications, we rate a box at less than half of its peak call
throughput. So for 5,000 calls, we'd probably use five servers plus an extra
one for failover.
Someone trying to do that same amount of work with PC servers might need up
to four dozen of them in a complex configuration with a central voicemail
store. The load balancing and system management problems are considerable.
-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Wai Wu
Sent: Thursday, February 02, 2006 9:11 PM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: RE: [Asterisk-Users] RE: 5,000 concurrent calls system
rolloutquestion
I don't think they are doing it with one Asterisk box. They did say
"one
rack of servers". Well, that might mean up to 50 computers if they are
using
blade servers.
-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com]On Behalf Of John Todd
Sent: Thursday, February 02, 2006 10:21 PM
To: asterisk-users@lists.digium.com
Subject: RE: [Asterisk-Users] RE: 5,000 concurrent calls system
rolloutquestion
[top-posting continued due to formatting sloth on my part]
So, then let me follow up with a few more comments:
1) I will make some assumptions from your note:
a) Asterisk is currently capable (unless something has "broken"
recently) of handling 2500 SIP-SIP calls with no transcoding,
including RTP sessions, if on an operating system and hardware that
is appropriately configured. This puts to rest some who have claimed
that 5000 channels is "impossible" with Asterisk regardless of
platform, at least according to Signate.
b) It is unclear if other channel drivers (IAX and Zaptel,
specifically) have had any testing with significant numbers of
channels.
c) It is unclear if anything other than pure RTP passthrough is
viable in these configurations. Maybe IVR causes collapse. ?
2) Still no claims or comments on the specific testing methods, or on
methodology. I'm left still scratching my head as to if this is
actually possible, since there is no specific claim that can be
verified. While I hope that your system can do those numbers (it
would help me greatly in the future!) I can't say that I'm confident
yet. I'll follow up in private email for further discussion.
3) Nobody else has thus far taken the bait and made any comments
about their systems. I appreciate Signate's comments; they seem to be
the only ones to publicly claim large-scale throughput using Asterisk
in a public forum. Most other people who claim thousands or even
high hundreds of connections do so offhand, without responding to
second questions when I raise my figurative eyebrows.
4) There are still no notes on other problems with scale here. I've
had systems with several hundred simultaneous SIP connections, but
"sip show channels" sure does start to take a while. What _other_
problems crop up, but don't necessarily cause a "failure"
condition?
5) I will agree that most SIP testing systems are currently too
pricey. I would love to find a well-connected network that rents out
a few of the better-known SIP testing tools to beat on Asterisk
installations in remote places for short periods of time. But this
has always been the case... test gear is a small market, and
expensive. Just look at the MSRP of new high-end HP Oscilloscopes if
you want to get a picture of price-gouging.
JT
At 11:21 AM -0800 2/2/06, William Boehlke wrote:>
>Signate has claimed 5,000 streams, or 2,500 calls, on a single Telephony
>Server 5000. The throughput has little to do with Asterisk and a lot to do
>with hardware design and operating system tuning. Our very minor code
>changes were returned to the project last year.
>
>The benchmark we used to make that initial claim was flawed, however we
have>since replicated the throughput in a different way to save our marketing
>bacon.
>
>How we actually achieve the throughput is our intellectual property but we
>have a number of customers who are scaling towards and past that traffic
>level. One of these days we hope to be able to justify the very large fee
>Hammer wants to extract from us to produce a third party verification.
>
>In production environments, of course, systems do more than switch calls.
We>think high volume system design using 32-bit systems of any kind is
complex,>and it's difficult to replicate the volumes without actual customer
traffic
>- and by then it's too late. Where do you put voicemail? Where does the
IVR
>reside?
>
>When someone needs to switch 5,000 calls with Class 5 services we would
>specify a rack of servers. The good news is that it is one rack, not three
>of them, but we need more than Asterisk alone, great though it is, to make
>everything work.
>
>
>-----Original Message-----
>From: asterisk-users-bounces@lists.digium.com
>[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of John Todd
>Sent: Wednesday, February 01, 2006 9:33 PM
>To: asterisk-users@lists.digium.com; asterisk-dev@lists.digium.com
>Subject: [Asterisk-Users] RE: 5, 000 concurrent calls system rollout
>question
>
>
>>Signate sells a single server that can get you to the call volumes you
>need.
>>
>>Paul Mahler
>><mailto:pmahler@signate.com>pmahler@signate.com
>>www.signate.com
>>
>[snip]
>
>Past conversations on this topic have generated quite a bit of
>controversy within the Asterisk development community, both publicly
>here on the list forums as well as in quite a few more quiet
>discussions with people who often do not post but have extensive
>operational experiences with Asterisk (most of whom monitor the -dev
>list and whose replies will be suited to that audience.)
>
>The subject of load on a single chassis is still the most contentious
>issue to date. The Signate numbers of >5000 calls per chassis with
>RTP are impressive, and there are others who claim more vaguely of
>1000, 2000, or more calls into a single P4 server (with or without
>media.) Others say that there are inherent limits in the Asterisk
>code which prevent more than ~500 calls from being processed with RTP
>at any one time. Opterons, FreeBSD, custom Linux loads, Solaris, and
>other operating systems or hardware have been offered as the magic
>bullets to increase call volumes. Who knows? (1) I will say that
>extraordinary claims demand extraordinary evidence, which has been
>pretty thin. I believe that most large call processing facilities
>still run on distributed systems of some type, as was described in
>the primary thread of this discussion on -users. (2)
>
>I know that there are some projects towards testing Asterisk more
>rigorously to determine these numbers. However, I would suggest that
>the community at large could benefit from a more open examination of
>high-end system claims immediately than these (better) long-term
>tests which are progressing slowly (if at all.) Let's just look at
>the "maximum" numbers. Running a big system? Selling a big
system?
>Tell us about it, in detail. What are the limits that have been hit?
>Be specific. I keep seeing hand-waving, but no programmers have come
>forward to say "It won't work because of the way X is implemented
in
>the file blah.c or libFOO."
>
>To make a bad analogy: I don't want to see the street rods; I just
>want to see the top-fuel, rocket-powered dragsters on the line. Any
>takers? It sounds like Signate has a contender, but quite a few
>people have said that it's impossible without serious modifications
>to the code. Others have claimed (publicly or privately) that they
>can match those numbers on different hardware.
>
>Here are the criteria:
> - Any O/S
> - An unmodified version of Asterisk from SVN (or CVS)
> OR patches must be available for inspection, as per the GPL
> OR you must be a Digium license-holder (patches can be secret)
> - All calls are IAX2 or SIP (both in and out)
> - No transcoding of any type is required
> - All calls are G.711, 20ms OR 30ms packet size
>
>Documentation:
> - All O/S documentation, kernel tricks, modules, hacks, patches, or
>configuration elements should be documented, but proprietary
>information need not be divulged if that is deemed "secret"
> - Testing method must be reasonably documented
> - Dialplans must be included
> - SIP.conf files must be included
> - All hardware must be fully described (part numbers required)
>
>TEST #1:
> All media must be handled by the server. This is for both legs of
>the call. The "canreinvite=no" for SIP and
"notransfer=yes" in IAX2
>must be set for all calls.
>
>TEST #2:
> Media may or may not be handled by the server. Native transfers
>should be allowed in both IAX2 and/or SIP.
>
>
>(1) I have heard various people saying that it is "impossible" for
>Asterisk to handle a large number of calls due to architectural
>issues (no, it's not just from the people that you'd
"expect" to hear
>this from.) I've not been able to validate this one way or the other
>recently. I am interested to hear what the developer community has
>as a comment on this topic. I have an Empirix Hammer system at my
>company, but honestly I just don't have the time to set it up to do
>testing due to day job time constraints...
>
>(2) There are so many ways to spread calls across an Asterisk array
>it makes my head spin, but the question STILL comes down to "how many
>calls can a single chassis handle?" Even in a farm of servers, there
>has to be a numerator in that ratio.
>
>JT
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --
Asterisk-Users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --
Asterisk-Users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users