asterisk users - Jan 2006 - 5,000 concurrent calls system rollout question

Hi,

we are currently considering different options for rolling out a large scale IP
PBX to handle around 3,000 + concurrent calls.

Can this be done with Asterisk? Has it been done before?

I really would like an input on this.

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20060127/ae836e6a/attachment.htm

What's you mix of calls going SIP/IAXand to PSTN?

We've done  some benchmark experiments on a 3GHz HT box with 1GB of ram,
mirrored traditional IDE disks. The box has a Digium quad-PRI, a TDM40B, a
TDM22B and a Sirrix quad-BRI board in it. This box can run 120 active calls
over 4 PRI spans. Its running
MusicOnHold into 60 of the channels, playing various GSM prompts into the
other 60. The "user" cpu usage is about 25%, the "system"
cpu about 25%
also. We can add to that 5000 registered SIP peers and 5000 registered IAX2
peers - total of about 100 registration refreshes per second. That adds
about 40% more user CPU and pretty much fills up CPU. Audio quality is still
perfectly fine, and PRI slips few and far between. Load average for the
whole mix sits between 5 and 10. About 550Kb/s out and 400Kb/s out on the
ethernet for the registration traffic.

Also on www.voip-info.org - search for dimensioning

Rob

On 1/28/06, Vic <svictor@yahoo.co.jp> wrote:
>
> Hi,
>
> we are currently considering different options for rolling out a large
> scale IP PBX to handle around 3,000 + concurrent calls.
>
> Can this be done with Asterisk? Has it been done before?
>
> I really would like an input on this.
>
> Thanks!
>
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
>
> Asterisk-Users mailing list
> To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20060128/a41546b6/attachment.htm

Vic wrote:
> Hi,
>
> we are currently considering different options for rolling out a large
> scale IP PBX to handle around 3,000 + concurrent calls.
>
> Can this be done with Asterisk? Has it been done before?
>
> I really would like an input on this.
>
> Thanks!
>
>  
>
Check out Signate Telephony Server,
http://www.signate.com/pbx.php

It can be done, are those 3000 calls sip to sip ? If so it could easily
be done, if they are not sip to sip you will need a bunch of servers.

Zoa.

Vic wrote:
> Hi,
>
> we are currently considering different options for rolling out a large
> scale IP PBX to handle around 3,000 + concurrent calls.
>
> Can this be done with Asterisk? Has it been done before?
>
> I really would like an input on this.
>
> Thanks!
>
>------------------------------------------------------------------------
>
>_______________________________________________
>--Bandwidth and Colocation provided by Easynews.com --
>
>Asterisk-Users mailing list
>To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
>  
>

The question is somewhat ludicrous, and I'm slightly surprised that
no-one has sat down and done the maths about bandwidth utilization. So I
did.

 

To handle 5000 calls coming in over a PRI, you'd need 210 or so T1s or
170 E1's.

All of those would generate 320Mega BYTES of data per second (eg,
32Gigabit/sec)

 

There is no way possible that you're going to pump that amount of data
through a PC. Don't care about codecs and dialplans, PC's just don't
have that sort of internal bandwidth from peripherals.

 

If you do, honestly, need to handle 5k calls, you'd probably have to
have a bank of Cisco 5850s doing the termination - With a max of 5 DS3
(1 DS3 = 28 T1's) into each one, you'll need 8, or probably 9 as
you'd
want to have one as a hot spare. Each of those DS3's would go into some
beefy switching fabric (note, that each one is producting 225mbit) and
you'd have some sort of asterisk box with huge internal bandwidth
handling each one. Cross connect all 9 asterisk boxes via 10Gbit
networks (note, you'll need PCI-16x 10g cards) and have a pair of
voicemail servers. I'd suggest a pair of big Sun boxes.

 

Then, of course, you have the issue of getting the calls _out_ of the
asterisk machines. You've just doubled your bandwidth requirements, so
you'll need to double up on the asterisk machines, and split the network
up further.

 

I'd take a guess that you could do it under USD$1million (just for
hardware) but I wouldn't be surprised if it was USD$10million.

 

I'm happy to sell you any of this 8-)

 

--Rob

 

 

 

 

-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Vic
Sent: Sunday, 29 January 2006 1:16 PM
To: asterisk-users@lists.digium.com
Subject: Re: [Asterisk-Users] 5,000 concurrent calls system rollout
question

 

Hi, Zoa, 

yes, these calls are from SIP to SIP. We will have more than 3000 (more
like 5000)concurrent calls come into system and we will need to handle
them. 

We will also need an IVR function as well. 

I am not up to speed on Asterisk yet, so, I am a little bit confused by
all the different ways of doing it. Someone is talking about IAX:  I
think it can only be used between Asterisk servers, right? 

In this particula rscenario we are getting calls as SIP directly from
carrier, so we will not need to do any conversion (I think). We just
route the calls to the destination, that's it. 

Any suggestions on how to proceed? Can Asterisk do it? 

I read somewhere that it takes about 30 MHz per one voice channel, so if
we want to have 5,000 calls, we will need 150,000 MHz? Thats like 50 3
GHz machines... Not going to fly with our people.  

Or do 30 MHz are only necessary for transcoding? In other words, if it
comes in as SIP and we keep it that way, can we make it a bt more
feasible number?  

  

 Zoa <zoachien@securax.org> wrote: 

	
	It can be done, are those 3000 calls sip to sip ? If so it could
easily
	be done, if they are not sip to sip you will need a bunch of
servers.
	
	Zoa.
	
	Vic wrote:
	
	> Hi,
	>
	> we are currently considering different options for rolling out
a large
	> scale IP PBX to handle around 3,000 + concurrent calls.
	>
	> Can this be done with Asterisk? Has it been done before?
	>
	> I really would like an input on this.
	>
	> Thanks!
	>
	
>-----------------------------------------------------------------------
-
	>
	>_______________________________________________
	>--Bandwidth and Colocation provided by Easynews.com --
	>
	>Asterisk-Users mailing list
	>To UNSUBSCRIBE or update options visit:
	> http://lists.digium.com/mailman/listinfo/asterisk-users
	> 
	>
	
	_______________________________________________
	--Bandwidth and Colocation provided by Easynews.com --
	
	Asterisk-Users mailing list
	To UNSUBSCRIBE or update options visit:
	http://lists.digium.com/mailman/listinfo/asterisk-users

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20060129/7094d22a/attachment.htm

> -----Original Message-----
> From: asterisk-users-bounces@lists.digium.com [mailto:asterisk-users-
> bounces@lists.digium.com] On Behalf Of Patrick
> Sent: Sunday, January 29, 2006 7:40 AM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: RE: [Asterisk-Users] 5,000 concurrent calls system rollout
> question
> 
> On Sun, 2006-01-29 at 17:24 +1000, Rob Thomas wrote:
> [snip]
> > If you do, honestly, need to handle 5k calls, you'd probably have
to
> > have a bank of Cisco 5850s doing the termination
> 
> Or have a look at the Lucent APX8100 box for some added carrier class
> humpf. Supports more than 8000 DS0's (channels) and does transcoding
in
> hardware DSP's so well suited to handle your 5000 concurrent calls and
> you don't need a stack of them like with the Cisco 5850.
> 
> Weblink:
>
http://www.lucent.com/products/subcategory/0,,CTID+2013-STID+10443-LOCL
> +1,00.html
> Datasheet:
> http://www.lucent.com/livelink/090094038000eb91_Brochure_datasheet.pdf
> 
> Like Rob I'd love to sell this to you but I doubt Lucent would even
pick
> up the phone to answer my "how to become a VAR" enquiry. Best
contact
> them directly :)
> 
> Regards,
> Patrick (no affiliation with Lucent)
> 
The original poster of this message stated in an earlier message that
the calls would be handed off to him SIP, so the media conversion is
being done buy an upstream carrier, presumably on a Lucent or Sonus.

With the growing availability of SIP origination and termination, high
density channels banks like the APX8000 are becoming items only needed
by wholesale carriers.

Of course this varies by geographical region, but to use a APX 8000 you
need at least PRI service over DS1/E1, and ideally PRI service over
DS3/E3.

The challenge I see with a 5000 INBOUND call setup originated SIP is
that the calls will need to be load balanced across many * boxes, no 1
asterisk box is going to take 5000 CONCURRENT calls (500 would impress
me).

I would suggest; 

Check to see if the SIP origination provider can give you a "round
robin" delivery of calls over 10 or so * boxes (IP addresses), or find
an external method of doing it yourself (like a smart session border
controller).

IF the calls are terminated to hardphones or softphones (as opposed to
purely IVR), make sure you can do RTP re-invites so, when appropriate,
the RTP stream is offloaded from * (but consider the impact of doing
so).

Calculate bandwidth needs carefully - 5000 * 70-75kbps  (a/ulaw plus
packet overhead) requires a GIG-E IP link from you SIP provider and some
very robust networking in between.

Terminating 5000 calls on * is relatively uncharted ground, there MAY be
some others doing it, but good luck getting them to reveal the company
jewels.

At the very least, this type of implementation would require a team of
the VERY BEST asterisk consultants - might want to call Mark himself if
you are serious.

Damon

To handle 5000 calls coming in over a PRI, you'd need 210 or so T1s or 170
E1's.All of those would generate 320Mega BYTES of data per second (eg,
32Gigabit/sec)
[Wai Wu]  He not talking about PRI here, but rather SIP to SIP

 

 

There is no way possible that you're going to pump that amount of data
through a PC. Don't care about codecs and dialplans, PC's just don't
have that sort of internal bandwidth from peripherals.

 

If all the endpints support reinvite and he is not doing any voice processing at
all, there is hardly any data going through the PC

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20060129/85ac0c74/attachment.htm

64kb/s channel will require a bandwidth of about 90kb/s using ONE voice packet
per UDP packet. The overhead is a bit low if you put TWO voice packets in ONE
UDP packet.

check this out.
http://web1.egvrn.net/tokata/VoIP%20Bandwidth%20Consumption.pdf

-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com]On Behalf Of Kristian
Larsson
Sent: Monday, January 30, 2006 10:02 AM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [Asterisk-Users] 5,000 concurrent calls system rollout
question


On Mon, Jan 30, 2006 at 12:49:15PM +0100, janvb@caselaboratories.com
wrote:
> 
> >Using G711A (ie, "worst case" bandwidth wise):
> >it's 64kbit/s not 64Kbyte/s
> >so it's 320Megabits per seconds
> > 
> >
> That will only do if you talk a lot with your mother in law! ;-) 
> 
> For the rest of the conversation (those with both speaking):
> 
> 5000 * 64k * 2 = 640M
Indeed you are correct, I'll defend myself with
stating that I presumed we were talkin "full duplex"
;)
> 
> It should in theory work with a 1Gbits Ethernet, but you would be 
> counting on ca 65% utilization. I would normally plan with < 30-40 % 
> utilization and you need 2 for redundancy anyway.
Though now you're wrong ;)
65% isn't correct. If you're counting both in and
out traffic you'll have to assume that the Gigg
card is capable of 1Gbps in each direction thus
2Gbps in total and 640M of 2000G is about 30% or
just as much as 320M is of 1G.

I don't know the average packet size of a voice
RTP packet but I guess it's quite small. Being a
network guy I've dealt quite a lot with software
routers and a normal Linux machine can forward
about 500kpps, and this is mere forwarding if you
run this via Asterisk you should probably split
that by ten.


-- 
Kristian Larsson, Net At Once AB
Email: kristian@netatonce.se
Phone: +46 470 592717
Cell: +46 704 910401
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

Asterisk-Users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

One voice packet is about 20ms of sampling depending on the codec.

-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com]On Behalf Of
janvb@caselaboratories.com
Sent: Monday, January 30, 2006 9:28 AM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [Asterisk-Users] 5,000 concurrent calls system rollout
question


>64kb/s channel will require a bandwidth of about 90kb/s using ONE voice
packet per UDP packet. The overhead is a bit low if you put TWO voice packets in
ONE UDP packet.
>  
>
And what is 'one voice packet'?

Jan


_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

Asterisk-Users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

> -----Original Message-----
> From: asterisk-users-bounces@lists.digium.com [mailto:asterisk-users-
> bounces@lists.digium.com] On Behalf Of C F
> Sent: Tuesday, January 31, 2006 4:03 PM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [Asterisk-Users] 5,000 concurrent calls system rollout
> question
> 
> I don't know how much 1+1 by you is, but lets recalculate this for a
> moment:
> First the bandwidth per channel:
> http://www.airewaves.com/aire/support/bandwidth_explain.php
> 1.5mbps (mega *BITS* not BYTES per second) to a full T1, which equals
> 1536 Kbits, each channel then takes 64kbps.
> 64*5,000=320,000kbps.
> 32,000/1,024=312.5 Mbps (round off to Mbps), no where close to a Gb.
> Every single PC made in the last 4 years I came across, can handle
> this type of bandwidth.
> BTW, this all amounts to just over 39 MBYTES per second.
312.5/8=39.0625
> 
Not that I disagree with your point, the bandwidth is not huge, but the
math is a little fuzzy;

First of all, a g.711u stream over UDP is closer 80k than 64k, the
payload is 64k + udp overhead + IP overhead.

Now consider that the call is originated as SIP (llok back a few days in
the thread), and lets assume the call goes to an external hard or
softphone, and lets also assume that there is a reason to keep the RTP
stream running through asterisk (monitoring, recording, transferring,
dtmf, ability to re-enter IVR, etc).

I make all the assumptions safely since the thread was started by
someone looking to set up a large call center and I have followed thread
out of curiosity.

So a 80k full duplex RTP stream originates on media gateway somewhere,
hits the asterisk box, is internally bridged, and is sent back out to a
phone somewhere. My math says this puts a 160kbps full duplex load in
the NIC.

Ok, now lets go for 5000 of them. 160kbps*5000=800000kbps or 800mbps -
full duplex.

Have you ever seen a NIC or switch that can run GigE full duplex at 80%
utilization and not at least start to fall apart?

To get to a comfortable load you would need 2x GigE NICs (for ~40%
utilization), of course now we are adding additional overhead for the
bonded NIC trunking protocol.

Is still contend this is not practical without multiple very high end
servers and round robin call origination from the upstream provider
delivered over something like GigE or OCx.

Maybe someone will step up and post some real-world application limits
based on experience...

I saw your previous post that signate has a solution - care to
elobaorate? (invitation for a shameless plug)
> -----Original Message-----
> From: asterisk-users-bounces@lists.digium.com [mailto:asterisk-users-
> bounces@lists.digium.com] On Behalf Of William Boehlke
> Sent: Tuesday, January 31, 2006 7:06 PM
> To: 'Asterisk Users Mailing List - Non-Commercial Discussion'
> Subject: RE: [Asterisk-Users] 5,000 concurrent calls system rollout
> question
> 
> 
> In our experience, it's not a bandwidth limitation. If you do nothing
> special, interrupt servicing for a single NIC on our high throughput
> hardware maxes at something in excess of 1,000 calls when you are
keeping
> the streams. I don't believe you can get even that far on a PC server,
but
> we haven't tried.
> 
> 
> 
> 
> 
> -----Original Message-----
> From: asterisk-users-bounces@lists.digium.com
> [mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Damon
Estep
> Sent: Tuesday, January 31, 2006 5:29 PM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: RE: [Asterisk-Users] 5,000 concurrent calls system rollout
> question
> 
> 
> 
> > -----Original Message-----
> > From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-
> > bounces@lists.digium.com] On Behalf Of C F
> > Sent: Tuesday, January 31, 2006 4:03 PM
> > To: Asterisk Users Mailing List - Non-Commercial Discussion
> > Subject: Re: [Asterisk-Users] 5,000 concurrent calls system rollout
> > question
> >
> > I don't know how much 1+1 by you is, but lets recalculate this for
a
> > moment:
> > First the bandwidth per channel:
> > http://www.airewaves.com/aire/support/bandwidth_explain.php
> > 1.5mbps (mega *BITS* not BYTES per second) to a full T1, which
equals
> > 1536 Kbits, each channel then takes 64kbps.
> > 64*5,000=320,000kbps.
> > 32,000/1,024=312.5 Mbps (round off to Mbps), no where close to a Gb.
> > Every single PC made in the last 4 years I came across, can handle
> > this type of bandwidth.
> > BTW, this all amounts to just over 39 MBYTES per second.
> 312.5/8=39.0625
> >
> 
> Not that I disagree with your point, the bandwidth is not huge, but
the
> math is a little fuzzy;
> 
> First of all, a g.711u stream over UDP is closer 80k than 64k, the
> payload is 64k + udp overhead + IP overhead.
> 
> Now consider that the call is originated as SIP (llok back a few days
in
> the thread), and lets assume the call goes to an external hard or
> softphone, and lets also assume that there is a reason to keep the RTP
> stream running through asterisk (monitoring, recording, transferring,
> dtmf, ability to re-enter IVR, etc).
> 
> I make all the assumptions safely since the thread was started by
> someone looking to set up a large call center and I have followed
thread
> out of curiosity.
> 
> So a 80k full duplex RTP stream originates on media gateway somewhere,
> hits the asterisk box, is internally bridged, and is sent back out to
a
> phone somewhere. My math says this puts a 160kbps full duplex load in
> the NIC.
> 
> Ok, now lets go for 5000 of them. 160kbps*5000=800000kbps or 800mbps -
> full duplex.
> 
> Have you ever seen a NIC or switch that can run GigE full duplex at
80%
> utilization and not at least start to fall apart?
> 
> To get to a comfortable load you would need 2x GigE NICs (for ~40%
> utilization), of course now we are adding additional overhead for the
> bonded NIC trunking protocol.
> 
> Is still contend this is not practical without multiple very high end
> servers and round robin call origination from the upstream provider
> delivered over something like GigE or OCx.
> 
> Maybe someone will step up and post some real-world application limits
> based on experience...
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
> 
> Asterisk-Users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users
> 
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
> 
> Asterisk-Users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users

> -----Original Message-----
> From: asterisk-users-bounces@lists.digium.com [mailto:asterisk-users-
> bounces@lists.digium.com] On Behalf Of C F
> Sent: Tuesday, January 31, 2006 9:50 PM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [Asterisk-Users] 5,000 concurrent calls system rollout
> question
> 
> On 1/31/06, Damon Estep <damon@suburbanbroadband.net> wrote:
> >
> >
> > > -----Original Message-----
> > > From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-
> > > bounces@lists.digium.com] On Behalf Of C F
> > > Sent: Tuesday, January 31, 2006 4:03 PM
> > > To: Asterisk Users Mailing List - Non-Commercial Discussion
> > > Subject: Re: [Asterisk-Users] 5,000 concurrent calls system
rollout
> > > question
> > >
> > > I don't know how much 1+1 by you is, but lets recalculate
this for
a
> > > moment:
> > > First the bandwidth per channel:
> > > http://www.airewaves.com/aire/support/bandwidth_explain.php
> > > 1.5mbps (mega *BITS* not BYTES per second) to a full T1, which
equals
> > > 1536 Kbits, each channel then takes 64kbps.
> > > 64*5,000=320,000kbps.
> > > 32,000/1,024=312.5 Mbps (round off to Mbps), no where close to a
Gb.
> > > Every single PC made in the last 4 years I came across, can
handle
> > > this type of bandwidth.
> > > BTW, this all amounts to just over 39 MBYTES per second.
> > 312.5/8=39.0625
> > >
> >
> > Not that I disagree with your point, the bandwidth is not huge, but
the
> > math is a little fuzzy;
> 
> No it's not, the math was meant for the T1 calculations, reread the
> post I replied to.
I guess I just assumed that that the connection to asterisk would have
to be IP since it is absolutely impossible to connect ~208 T1s directly
to a single asterisk server. You would have to use an external media
gateway. I am not aware of any 200x T1 or 8x T3 cards for asterisk :)

> 
> >
> > First of all, a g.711u stream over UDP is closer 80k than 64k, the
> > payload is 64k + udp overhead + IP overhead.
> >
> > Now consider that the call is originated as SIP (llok back a few
days in
> > the thread), and lets assume the call goes to an external hard or
> > softphone, and lets also assume that there is a reason to keep the
RTP
> > stream running through asterisk (monitoring, recording,
transferring,
> > dtmf, ability to re-enter IVR, etc).
> >
> > I make all the assumptions safely since the thread was started by
> > someone looking to set up a large call center and I have followed
thread
> > out of curiosity.
> >
> > So a 80k full duplex RTP stream originates on media gateway
somewhere,
> > hits the asterisk box, is internally bridged, and is sent back out
to a
> > phone somewhere. My math says this puts a 160kbps full duplex load
in
> > the NIC.
> >
> > Ok, now lets go for 5000 of them. 160kbps*5000=800000kbps or 800mbps
-
> > full duplex.
> >
> > Have you ever seen a NIC or switch that can run GigE full duplex at
80%
> > utilization and not at least start to fall apart?
> >
> > To get to a comfortable load you would need 2x GigE NICs (for ~40%
> > utilization), of course now we are adding additional overhead for
the
> > bonded NIC trunking protocol.
> >
> > Is still contend this is not practical without multiple very high
end
> > servers and round robin call origination from the upstream provider
> > delivered over something like GigE or OCx.
> >
> > Maybe someone will step up and post some real-world application
limits
> > based on experience...
> >
> >
> >
> >
> >
> >

On 02/01/06 09:29 Damon Estep said the following:
> Ok, now lets go for 5000 of them. 160kbps*5000=800000kbps or 800mbps -
> full duplex.
> 
> Have you ever seen a NIC or switch that can run GigE full duplex at 80%
> utilization and not at least start to fall apart?
additionally, 5000 simultaneous SIP calls at 20ms intervals will send,

5,000 * 50 * 2 = 500,000 packets per second (full duplex).

not too many boxes can handle such packet load, in spite of the relatively 
small packet sizes.

-- 
Regards,                           /\_/\   "All dogs go to heaven."
dinesh@alphaque.com                (0 0)    http://www.alphaque.com/
+==========================----oOO--(_)--OOo----==========================+
| for a in past present future; do                                        |
|   for b in clients employers associates relatives neighbours pets; do   |
|   echo "The opinions here in no way reflect the opinions of my $a
$b."  |
| done; done                                                              |
+=========================================================================+

Dinesh Nair wrote:
>
>
> On 02/01/06 09:29 Damon Estep said the following:
>
>> Ok, now lets go for 5000 of them. 160kbps*5000=800000kbps or 800mbps -
>> full duplex.
>>
>> Have you ever seen a NIC or switch that can run GigE full duplex at 80%
>> utilization and not at least start to fall apart?
>
>
> additionally, 5000 simultaneous SIP calls at 20ms intervals will send,
>
> 5,000 * 50 * 2 = 500,000 packets per second (full duplex).
>
> not too many boxes can handle such packet load, in spite of the 
> relatively small packet sizes.
>
Why not bond multiple NICs together to do a load balance output?  Would 
provide redundancy as well.

I have tested an asterisk server with over 5000 concurrent calls.

The system setup was a P4 HT 3Ghz, 4 Gb RAM, and 1 gbps Ethernet
connection on a cisco 3560 switch.

 

This works, but puts some serious stresses on the system.

Why don't u considered using g.729 codec, this will at least lower the
bandwidth consumption significantly, and, you can overcome the CPU
resource issue by just using a server grade multi CPU xeon server.

 

I would never the less still connect the system via 2 ethernet
connections, just for some redundancy, as mentioned before in this
thread.

 

Bandwidth should be about 24 kbps (half duplex) per call

 

So, 5000 * 24 is roughly 120 mbps, so a gigabit Ethernet should do just
fine.

 

Joash

 

-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Dustin
Wildes
Sent: Wednesday, February 01, 2006 8:54 AM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [Asterisk-Users] 5,000 concurrent calls system rollout
question

 

Dinesh Nair wrote:

 
> 
> 
> On 02/01/06 09:29 Damon Estep said the following:
> 
>> Ok, now lets go for 5000 of them. 160kbps*5000=800000kbps or 800mbps
-
>> full duplex.
>> 
>> Have you ever seen a NIC or switch that can run GigE full duplex at
80%
>> utilization and not at least start to fall apart?
> 
> 
> additionally, 5000 simultaneous SIP calls at 20ms intervals will send,
> 
> 5,000 * 50 * 2 = 500,000 packets per second (full duplex).
> 
> not too many boxes can handle such packet load, in spite of the 
> relatively small packet sizes.
> 
 

Why not bond multiple NICs together to do a load balance output?  Would 

provide redundancy as well.

 

_______________________________________________

--Bandwidth and Colocation provided by Easynews.com --

 

Asterisk-Users mailing list

To UNSUBSCRIBE or update options visit:

   http://lists.digium.com/mailman/listinfo/asterisk-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20060201/6c390d39/attachment.htm

>I guess I just assumed that that the connection to asterisk would have
>to be IP since it is absolutely impossible to connect ~208 T1s directly
>to a single asterisk server. You would have to use an external media
>gateway. I am not aware of any 200x T1 or 8x T3 cards for asterisk :)

Not necessarily.

Granted that you will not be able to have have that many T1's on one
system but if the load is spread across multiple "Asterisk Media
converters" you should be able to do anything and scale your system much
better.

Lets consider for example the following scenario:

------------------       ------------------
|   * Media      |       |   * Media      |
|    server      |       |    server      |
|   2x TE406P    |       |   2x TE406P    |
------------------       ------------------
               |            |
             -------------------
             |     Main *      |
             |     server      |
             |                 |
             -------------------
               |            |
------------------       ------------------
|   * Media      |       |   * Media      |
|    server      |       |    server      |
|   2x TE406P    |       |   2x TE406P    |
------------------       ------------------


This will let you serve 192 channels per media server.
Media servers will only need to convert PRI<->IP so a cheap DIY Dual
Core Xeon MP with 4MB cache would be more then enough to
process/compress 196 channels in/out of 2 TE406P's. Also media servers
do not need much RAM, hard drives and can run from flash cards.
My preference would be convert all the traffic coming out of media
servers to G.729 and IAX2 trunk it to main server. IAX2 trunking will
save you MANY interrupts and will improve your bandwidth utilisation
between Media and Main servers.

With this setup you can run Media and Main servers on private gigabit
network which would be more then enough to handle IAX2 trunked G.729
traffic from media servers. Network redundancy can easily be achieved
between Media and Main servers by adding NIC's to each and using many
known techniques (bonding, routing, VRRP, etc, etc).

The Main Asterisk server can be setup with load balancing/failover.
Media servers will need to be aware of this.

The good thing in the setup like this is that its easy to scale up when
needed, you're not exposed of loosing all of your T1's if one of media
servers fail, you can easily add more T1's in your setup.

The Main server would need a quad gigabit card (intel is a good choice)
and since it would not be hampered by Zaptel traffic and it would not
need to do any transcoding (except for odd voicemail usage, that could
be send to another server) you could use 2xDual Core Xeons. A separate
dual port (for redundancy) gigabit card would be used to serve SIP
clients.

We're working with one of the ISP's on testing and perhaps implementing
this setup for them.

This setup is considerably cheaper then $1M proposed Cisco setup and can
be made as reliable as Cisco solution is.
Please don't get me wrong, if I'd have $1M-$5M to spare would go for
Cisco.
But most of us don't have that much money and if we would, we would
never be reading any messages on asterisk-users.

Asterisk can be made as reliable and scales as good if not better then
any Cisco solution and the fraction of the cost.

Now imagine all of this with the new DS3000P in media servers!

All hail Asterisk! :)

Still,

Everybody is using T1 / E1 interfaces in servers.
I would go for purpose build voice gateways.
Vegastream or cisco GW are able to handle multiple T1/E1 connections
easily. (make sure that in a cisco GW you get enough DSP capacity)

In this scenario the asterisk server is just used to make sure everybody
gets connected to each other.

I a + 5000 call setup I would say some money is available to buy the
dedicated voice gateways.

I would go for the vegastream 400 series. Use 2 of them to prevent than
all your PSTN connections are terminated in one machine.

These kind of setups work with my customers (not 5000 concurrent calls)
but we do have connected over 3000 phones to the Asterisk server in some
locations.

You can then use a good management GUI like scopserv (be sure to mention
my name :-) :-) ) (www.scopserv.com) to manage you're asterisk servers
in a easy way (and also keep configs synchronized)

You will still be a hell of a lot cheaper then a cisco callmanager
setup, and get far more performance and features then cisco will ever be
able to offer you.

Joash


-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Boris
Bakchiev
Sent: Wednesday, February 01, 2006 10:26 AM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: RE: [Asterisk-Users] 5,000 concurrent calls system rollout
question

>I guess I just assumed that that the connection to asterisk would have
>to be IP since it is absolutely impossible to connect ~208 T1s directly
>to a single asterisk server. You would have to use an external media
>gateway. I am not aware of any 200x T1 or 8x T3 cards for asterisk :)

Not necessarily.

Granted that you will not be able to have have that many T1's on one
system but if the load is spread across multiple "Asterisk Media
converters" you should be able to do anything and scale your system much
better.

Lets consider for example the following scenario:

------------------       ------------------
|   * Media      |       |   * Media      |
|    server      |       |    server      |
|   2x TE406P    |       |   2x TE406P    |
------------------       ------------------
               |            |
             -------------------
             |     Main *      |
             |     server      |
             |                 |
             -------------------
               |            |
------------------       ------------------
|   * Media      |       |   * Media      |
|    server      |       |    server      |
|   2x TE406P    |       |   2x TE406P    |
------------------       ------------------


This will let you serve 192 channels per media server.
Media servers will only need to convert PRI<->IP so a cheap DIY Dual
Core Xeon MP with 4MB cache would be more then enough to
process/compress 196 channels in/out of 2 TE406P's. Also media servers
do not need much RAM, hard drives and can run from flash cards.
My preference would be convert all the traffic coming out of media
servers to G.729 and IAX2 trunk it to main server. IAX2 trunking will
save you MANY interrupts and will improve your bandwidth utilisation
between Media and Main servers.

With this setup you can run Media and Main servers on private gigabit
network which would be more then enough to handle IAX2 trunked G.729
traffic from media servers. Network redundancy can easily be achieved
between Media and Main servers by adding NIC's to each and using many
known techniques (bonding, routing, VRRP, etc, etc).

The Main Asterisk server can be setup with load balancing/failover.
Media servers will need to be aware of this.

The good thing in the setup like this is that its easy to scale up when
needed, you're not exposed of loosing all of your T1's if one of media
servers fail, you can easily add more T1's in your setup.

The Main server would need a quad gigabit card (intel is a good choice)
and since it would not be hampered by Zaptel traffic and it would not
need to do any transcoding (except for odd voicemail usage, that could
be send to another server) you could use 2xDual Core Xeons. A separate
dual port (for redundancy) gigabit card would be used to serve SIP
clients.

We're working with one of the ISP's on testing and perhaps implementing
this setup for them.

This setup is considerably cheaper then $1M proposed Cisco setup and can
be made as reliable as Cisco solution is.
Please don't get me wrong, if I'd have $1M-$5M to spare would go for
Cisco.
But most of us don't have that much money and if we would, we would
never be reading any messages on asterisk-users.

Asterisk can be made as reliable and scales as good if not better then
any Cisco solution and the fraction of the cost.

Now imagine all of this with the new DS3000P in media servers!

All hail Asterisk! :)

_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

Asterisk-Users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

Still,

Everybody is using T1 / E1 interfaces in servers.
I would go for purpose build voice gateways.
Vegastream or cisco GW are able to handle multiple T1/E1 connections
easily. (make sure that in a cisco GW you get enough DSP capacity)

In this scenario the asterisk server is just used to make sure everybody
gets connected to each other.

I a + 5000 call setup I would say some money is available to buy the
dedicated voice gateways.

I would go for the vegastream 400 series. Use 2 of them to prevent than
all your PSTN connections are terminated in one machine.

These kind of setups work with my customers (not 5000 concurrent calls)
but we do have connected over 3000 phones to the Asterisk server in some
locations.

You can then use a good management GUI like scopserv (be sure to mention
my name :-) :-) ) (www.scopserv.com) to manage you're asterisk servers
in a easy way (and also keep configs synchronized)

You will still be a hell of a lot cheaper then a cisco callmanager
setup, and get far more performance and features then cisco will ever be
able to offer you.

Joash


-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Boris
Bakchiev
Sent: Wednesday, February 01, 2006 10:26 AM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: RE: [Asterisk-Users] 5,000 concurrent calls system rollout
question

>I guess I just assumed that that the connection to asterisk would have
>to be IP since it is absolutely impossible to connect ~208 T1s directly
>to a single asterisk server. You would have to use an external media
>gateway. I am not aware of any 200x T1 or 8x T3 cards for asterisk :)

Not necessarily.

Granted that you will not be able to have have that many T1's on one
system but if the load is spread across multiple "Asterisk Media
converters" you should be able to do anything and scale your system much
better.

Lets consider for example the following scenario:

------------------       ------------------
|   * Media      |       |   * Media      |
|    server      |       |    server      |
|   2x TE406P    |       |   2x TE406P    |
------------------       ------------------
               |            |
             -------------------
             |     Main *      |
             |     server      |
             |                 |
             -------------------
               |            |
------------------       ------------------
|   * Media      |       |   * Media      |
|    server      |       |    server      |
|   2x TE406P    |       |   2x TE406P    |
------------------       ------------------


This will let you serve 192 channels per media server.
Media servers will only need to convert PRI<->IP so a cheap DIY Dual
Core Xeon MP with 4MB cache would be more then enough to
process/compress 196 channels in/out of 2 TE406P's. Also media servers
do not need much RAM, hard drives and can run from flash cards.
My preference would be convert all the traffic coming out of media
servers to G.729 and IAX2 trunk it to main server. IAX2 trunking will
save you MANY interrupts and will improve your bandwidth utilisation
between Media and Main servers.

With this setup you can run Media and Main servers on private gigabit
network which would be more then enough to handle IAX2 trunked G.729
traffic from media servers. Network redundancy can easily be achieved
between Media and Main servers by adding NIC's to each and using many
known techniques (bonding, routing, VRRP, etc, etc).

The Main Asterisk server can be setup with load balancing/failover.
Media servers will need to be aware of this.

The good thing in the setup like this is that its easy to scale up when
needed, you're not exposed of loosing all of your T1's if one of media
servers fail, you can easily add more T1's in your setup.

The Main server would need a quad gigabit card (intel is a good choice)
and since it would not be hampered by Zaptel traffic and it would not
need to do any transcoding (except for odd voicemail usage, that could
be send to another server) you could use 2xDual Core Xeons. A separate
dual port (for redundancy) gigabit card would be used to serve SIP
clients.

We're working with one of the ISP's on testing and perhaps implementing
this setup for them.

This setup is considerably cheaper then $1M proposed Cisco setup and can
be made as reliable as Cisco solution is.
Please don't get me wrong, if I'd have $1M-$5M to spare would go for
Cisco.
But most of us don't have that much money and if we would, we would
never be reading any messages on asterisk-users.

Asterisk can be made as reliable and scales as good if not better then
any Cisco solution and the fraction of the cost.

Now imagine all of this with the new DS3000P in media servers!

All hail Asterisk! :)

_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

Asterisk-Users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

> -----Original Message-----
> From: asterisk-users-bounces@lists.digium.com [mailto:asterisk-users-
> bounces@lists.digium.com] On Behalf Of Kristian Larsson
> Sent: Wednesday, February 01, 2006 2:49 AM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [Asterisk-Users] 5,000 concurrent calls system rollout
> question
> 
> On Tue, Jan 31, 2006 at 06:29:07PM -0700, Damon Estep wrote:
> >
> >
> > > -----Original Message-----
> > > From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-
> > > bounces@lists.digium.com] On Behalf Of C F
> > > Sent: Tuesday, January 31, 2006 4:03 PM
> > > To: Asterisk Users Mailing List - Non-Commercial Discussion
> > > Subject: Re: [Asterisk-Users] 5,000 concurrent calls system
rollout
> > > question
> > >
> > > I don't know how much 1+1 by you is, but lets recalculate
this for
a
> > > moment:
> > > First the bandwidth per channel:
> > > http://www.airewaves.com/aire/support/bandwidth_explain.php
> > > 1.5mbps (mega *BITS* not BYTES per second) to a full T1, which
equals
> > > 1536 Kbits, each channel then takes 64kbps.
> > > 64*5,000=320,000kbps.
> > > 32,000/1,024=312.5 Mbps (round off to Mbps), no where close to a
Gb.
> > > Every single PC made in the last 4 years I came across, can
handle
> > > this type of bandwidth.
> > > BTW, this all amounts to just over 39 MBYTES per second.
> > 312.5/8=39.0625
> > >
> >
> > Not that I disagree with your point, the bandwidth is not huge, but
the
> > math is a little fuzzy;
> >
> > First of all, a g.711u stream over UDP is closer 80k than 64k, the
> > payload is 64k + udp overhead + IP overhead.
> >
> > Now consider that the call is originated as SIP (llok back a few
days in
> > the thread), and lets assume the call goes to an external hard or
> > softphone, and lets also assume that there is a reason to keep the
RTP
> > stream running through asterisk (monitoring, recording,
transferring,
> > dtmf, ability to re-enter IVR, etc).
> >
> > I make all the assumptions safely since the thread was started by
> > someone looking to set up a large call center and I have followed
thread
> > out of curiosity.
> >
> > So a 80k full duplex RTP stream originates on media gateway
somewhere,
> > hits the asterisk box, is internally bridged, and is sent back out
to a
> > phone somewhere. My math says this puts a 160kbps full duplex load
in
> > the NIC.
> >
> > Ok, now lets go for 5000 of them. 160kbps*5000=800000kbps or 800mbps
-
> > full duplex.
> >
> > Have you ever seen a NIC or switch that can run GigE full duplex at
80%
> > utilization and not at least start to fall apart?
> No no no.
> 
> First you come to the conclusion that you have
> 800Mbps of traffic, but this is bi directional
> thus 400Mbps in each direction. Then you're
> comparing you're 800Mbps to 1Gbps. If you compare
> bi directional you need to count the card as
> 2Gbps.
> 
> So you are nowhere near 80% but closer to 40%.
Not true, each g.711 call generates 1x 80kps FULL DUPLEX RTP stream, if
the call originates g.711, enters *, gets bridged to another channel,
and goes back out to the phone as an additional g.711 call you are
generating 160 kbps of FULL DUPLEX traffic per call.

160kbps full duplex * 5000 calls IS 800mbps FULL DUPLEX!

Until someone steps up and says "we are currently doing this in a
PRODUCTION environment and it has been working for X months" I will
stand by my opinion that a single chassis 5000 call implementation of *
with RTP streams bridged is IMPOSSIBLE for many reasons, this being one
of them, the interrupt servicing being the next, the cpu load being 3rd,
and so on...

And if it can be done, I would still not do it - too many eggs in one
box. This should be done on 10x high end servers, 500 calls each, no
transcoding, round robin call origination - and even at that - 500 calls
per box will require a 2 to 4 CPU box with GigE NICs

Holy cow - just writing 5000 CDRs in a short time interval will bring a
box to its knees.

There have been many theoretical reasons why it can't be done posted
(including mine), and no "real world experience" posted. There are
reasons for that.
> >
> > To get to a comfortable load you would need 2x GigE NICs (for ~40%
> > utilization), of course now we are adding additional overhead for
the
> > bonded NIC trunking protocol.
> >
> > Is still contend this is not practical without multiple very high
end
> > servers and round robin call origination from the upstream provider
> > delivered over something like GigE or OCx.
> >
> > Maybe someone will step up and post some real-world application
limits
> > based on experience...
> >

> > Ok, now lets go for 5000 of them. 160kbps*5000=800000kbps or 800mbps
-
> > full duplex.
> >
> > Have you ever seen a NIC or switch that can run GigE full duplex at
80%
> > utilization and not at least start to fall apart?
> 
> additionally, 5000 simultaneous SIP calls at 20ms intervals will send,
> 
> 5,000 * 50 * 2 = 500,000 packets per second (full duplex).
> 
> not too many boxes can handle such packet load, in spite of the
relatively
> small packet sizes.
> 
> --
> Regards,                           /\_/\   "All dogs go to
heaven."
> dinesh@alphaque.com                (0 0)    http://www.alphaque.com/
> +==========================----oOO--(_)--OOo----
> ==========================+
> | for a in past present future; do
> |
> |   for b in clients employers associates relatives neighbours pets;
do
> |
> |   echo "The opinions here in no way reflect the opinions of my $a
$b."
> |
> | done; done
> |
>
+=======================================================================
Don't forget to account for the fact that the calls would be bridged, so
each call is really 2! = 10,000 RTP streams.

> >I guess I just assumed that that the connection to asterisk would
have
> >to be IP since it is absolutely impossible to connect ~208 T1s
directly
> >to a single asterisk server. You would have to use an external media
> >gateway. I am not aware of any 200x T1 or 8x T3 cards for asterisk :)
> 
> 
> Not necessarily.
Yes!
> 
> Granted that you will not be able to have have that many T1's on one
> system but if the load is spread across multiple "Asterisk Media
> converters" you should be able to do anything and scale your system
much
> better.
> 
> Lets consider for example the following scenario:
> 
> ------------------       ------------------
> |   * Media      |       |   * Media      |
> |    server      |       |    server      |
> |   2x TE406P    |       |   2x TE406P    |
> ------------------       ------------------
>                |            |
>              -------------------
>              |     Main *      |
>              |     server      |
>              |                 |
>              -------------------
>                |            |
> ------------------       ------------------
> |   * Media      |       |   * Media      |
> |    server      |       |    server      |
> |   2x TE406P    |       |   2x TE406P    |
> ------------------       ------------------
> 

The main sever is still connected via IP, correct?

Does not matter if you use * for media gateways or an APX8000 - the only
trunking options to get to the main box are IP based.
> 
> This will let you serve 192 channels per media server.
> Media servers will only need to convert PRI<->IP so a cheap DIY Dual
> Core Xeon MP with 4MB cache would be more then enough to
> process/compress 196 channels in/out of 2 TE406P's. Also media servers
> do not need much RAM, hard drives and can run from flash cards.
> My preference would be convert all the traffic coming out of media
> servers to G.729 and IAX2 trunk it to main server. IAX2 trunking will
> save you MANY interrupts and will improve your bandwidth utilisation
> between Media and Main servers.
> 
> With this setup you can run Media and Main servers on private gigabit
> network which would be more then enough to handle IAX2 trunked G.729
> traffic from media servers. Network redundancy can easily be achieved
> between Media and Main servers by adding NIC's to each and using many
> known techniques (bonding, routing, VRRP, etc, etc).
> 
> The Main Asterisk server can be setup with load balancing/failover.
> Media servers will need to be aware of this.
> 
> The good thing in the setup like this is that its easy to scale up
when
> needed, you're not exposed of loosing all of your T1's if one of
media
> servers fail, you can easily add more T1's in your setup.
> 
> The Main server would need a quad gigabit card (intel is a good
choice)
> and since it would not be hampered by Zaptel traffic and it would not
> need to do any transcoding (except for odd voicemail usage, that could
> be send to another server) you could use 2xDual Core Xeons. A separate
> dual port (for redundancy) gigabit card would be used to serve SIP
> clients.
> 
> We're working with one of the ISP's on testing and perhaps
implementing
> this setup for them.
> 
> This setup is considerably cheaper then $1M proposed Cisco setup and
can
> be made as reliable as Cisco solution is.
> Please don't get me wrong, if I'd have $1M-$5M to spare would go
for
> Cisco.
> But most of us don't have that much money and if we would, we would
> never be reading any messages on asterisk-users.
> 
> Asterisk can be made as reliable and scales as good if not better then
> any Cisco solution and the fraction of the cost.
> 
> Now imagine all of this with the new DS3000P in media servers!
> 
> All hail Asterisk! :)
>

>The main sever is still connected via IP, correct?
>Does not matter if you use * for media gateways or an APX8000 - the
only
>trunking options to get to the main box are IP based.
Are seriously going to tell me that a quad xeon/opteron would not handle
traffic from 4xGIG cards?? :)

On 02/02/06 06:13 asterisk@anime.net said the following:
> On Wed, 1 Feb 2006, Kristian Larsson wrote:
> 
>> Indeed, a FreeBSD machine doing just routing
>> lookups can handle somewhere around 600Kpps.
> 
> 
> Not to nitpick, but freebsd has routed 1M+pps using commodity hardware.
thanx, i wanted to point this out but didnt want to inadverntly start a 
linux vs freebsd flame war.

-- 
Regards,                           /\_/\   "All dogs go to heaven."
dinesh@alphaque.com                (0 0)    http://www.alphaque.com/
+==========================----oOO--(_)--OOo----==========================+
| for a in past present future; do                                        |
|   for b in clients employers associates relatives neighbours pets; do   |
|   echo "The opinions here in no way reflect the opinions of my $a
$b."  |
| done; done                                                              |
+=========================================================================+

Signate runs asterisk on a SGI box.   Nothing special, do yourself a
favor and just buy the SGI box yourself.  In fact I have 3 SGI boxes for
sale.  I'll rip off the Signate labels and sell them to you. 

 

 I worked out an asterisk load balance solution, so I don't need one all
powerful PC.  I distribute the load to many PC's... 

 

Doug


 

________________________________

From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Vic
Sent: Thursday, February 02, 2006 2:07 PM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: [Asterisk-Users] 5,000 concurrent calls system rollout question

 

Hi, 

several of your mentioned signant as a viable option. 

Has anyone ever used them? Are there any reviews for their products? 

Did they just put together a lot of Asterisks into a large scale PC? (I
am still struggling with the concept) 

Thanks, 

Vic

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20060207/22de6b67/attachment.htm