asterisk users - Jun 2003 - IAX2 trunking: codec bandwidth comparison notes and results

2003-06-28 Bandwidth Study - John Todd (jtodd @loligo.com)

Purpose:
-------------
   To obtain a better chart of actual bandwidth usage per codec as 
seen "on-the-wire" when using IAX2 trunking between two Asterisk 
telephony servers.



Discussion:
-------------
   Past threads on the asterisk-dev and asterisk-users lists have 
indicated that the optimal way to save bandwidth on multiple calls to 
the same destination is to use IAX2 (Inter-Asterisk eXchange version 
2) in "trunk" mode, which eliminates IP overhead to a large degree. 
This trunking eliminates the IP overhead found in individual VoIP IP 
streams by pipelining RTP data from multiple calls into single 
(larger) packets, thus removing the redundancy of IP overhead for 
each RTP stream and more closely allowing bandwidth scaling as a 
function of codec usage instead of a function of (codec usage + IP 
overhead.)  Of course, this mode can only be used if all the calls 
are between two specific Asterisk servers, but this is frequently the 
case with toll-avoidance situations or between two branch offices 
where there is an Asterisk server at each location.

   Various statements have been made as to which codec is the most 
efficient at moving traffic over limited bandwidth.  Often, these 
statements would quote the codec usage itself, and not include the IP 
and/or IAX2 overhead which is vital in computing actual bandwidth 
usage between two endpoints on the Internet - merely measuring codec 
theoretical usage is insufficient for real-world costing and 
provisioning purposes.

   I needed to actually quantify these numbers with 'real' on-the-wire 
examinations of traffic to build my own competence with quoting these 
figures, as they are frequently requested from me by clients.  To 
this end I had never compared the major protocols side-by-side in any 
formal manner.  Thus, on a beautiful Saturday morning when I should 
have been outside at the farmer's market or on my bike, I sat inside 
in a darkened computer room and said "blah" several hundred times at 
a movie theater recording.

   These numbers may come as no surprise to those who have configured 
them before, but I have not as yet seen a back-to-back comparison 
here, so I post it for your examination and my record-keeping.  I am 
a firm proponent of "don't believe everything you hear" which
applies
doubly-so for Internet rumors, so I had to test these codecs myself 
to prevent rude surprises in the future.  In keeping with that 
spirit, you should not believe my numbers, and test these yourself 
and post updates to the list to double-check my methods and results 
in a public forum.



Experiment notes and methods:
-------------
   - Raw figures below are bi-directional, meaning that the numbers 
include both incoming/outgoing voice traffic.  "Cooked" figures are 
included in parenthesis - raw numbers were divided by two and turned 
into kbps for your ease of reading.  Packets per second not cooked, 
and that is left as an exercise to the reader.
   - Phones were Cisco 7960 devices
   - Calls were bi-directional with audio, using the same recording at 
the PSTN side, and my use of the word "blah" repeatedly spoken into 
the 7960 at a steady rate
   - Diagram:  7960(G.711) -> *(IAX2) -> Internet -> *(IAX2) ->
T100P
-> PSTN (PRI)
   - The number for "Estimated IP Overhead" was obtained by 
subtracting (additional channel usage) from (single channel usage.) 
This is possibly inaccurate.
   - Asterisk server talking to SIP phones: Asterisk CVS-06/27/03-07:29:29
   - Asterisk server talking to PRI: Asterisk CVS-05/26/03-15:30:05
   - Method used to obtain figures "rate -b -f 'host my.iax2.host and 
not port ssh' "
   - All tests were done with IAX2 trunking turned on ("trunk=yes" in 
iax.conf for each peer)
   - All tests were verified on the remote host for use of the proper 
codec during calls
   - All tests performed over one minute
   - All measurements performed twice, and averaged
   - Calls were allowed to complete, then testing started for one 
minute, then testing stopped, and calls were hung up (in other words, 
call setup and teardown was not factored into any of these figures, 
but I suspect that will not make a difference)
   - Protocols untested: G.723.1, adpcm, mp3, slinear
   - When measuring bandwidth, "kilobit" is 1000 bits per second (not 
1024) and a megabit is 1,000,000 bits per second




Testing results:
-------------
G.711 (ulaw)
  one call:   164333.75 bps/94.26 pps   ( 82.1 kbps)
  two calls:  296171.60 bps/101.46 pps  (148.0 kbps)

  Thus:
   For every additional call:      131837 bps (65.9 kbps)
   Est. IP/IAX2 overhead (1 call):  32495 bps (16.0 kbps)
   Raw number of calls per megabit: 15

-------------
ILBC:
  one call:   56134.91 bps/67.45 pps    (28.0 kbps)
  two calls:  98679.11 bps/102.41 pps   (49.3 kbps)

  Thus:
   For every additional call:       42544 bps (21.2 kbps)
   Est. IP/IAX2 overhead (1 call):  13590 bps ( 6.7 kbps)
   Raw number of calls per megabit: 47

-------------
G.729
  one call:   60124.33 bps/101.26 pps   (30.0 kbps)
  two calls:  79496.23 bps/102.85 pps   (39.7 kbps)

  Thus:
   For every additional call:       19372 bps ( 9.6 kbps)
   Est. IP/IAX2 overhead (1 call):  40752 bps (20.3 kbps)
   Raw number of calls per megabit: 103

-------------
GSM
  one call:   70958.16 bps/102.13 pps   (35.4 kbps)
  two calls: 100455.23 bps/102.63 pps   (50.2 kbps)

  Thus:
   For every additional call:       29497 bps (14.7 kbps)
    Est. IP/IAX2 overhead (1 call): 41461 bps (20.7 kbps)
   Raw number of calls per megabit: 68

-------------
LPC10
  one call:   43855.44 bps/89.94  pps   (21.9 kbps)
  two calls:  56059.18 bps/100.81 pps   (28.0 kbps)

  Thus:
   For every additional call:       12203 bps ( 6.1 kbps)
   Est. IP/IAX2 overhead (1 call):  31561 bps (15.8 kbps)
   Raw number of calls per megabit: 164

-------------
SPEEX
  one call:   74817.18 bps/101.06 pps   (37.4 kbps)
  two calls: 109692.68 bps/102.18 pps   (54.8 kbps)

  Thus:
   For every additional call:       34875 bps (17.4 kbps)
   Est. IP/IAX2 overhead (1 call):  39941 bps (19.9 kbps)
   Raw number of calls per megabit: 57



-------------
Conclusions:
   lpc10 is the clear winner for thrifty use of bandwidth.  However, 
voice quality is on the robotic-sounding side, and may not be 
acceptable for users who are expecting something near toll-quality. 
Calls are  perfectly understandable, but nuances are lost with this 
codec.  This was the only codec that had significant quality issues 
that were worth mentioning in this study; other codecs have different 
sound quality ratings, but their differences are minor enough that 
they are not relevant to the scope of this examination.  That being 
said about call quality, there is certainly a tradeoff to getting 
~164 calls in a megabit, which is extremely impressive.

   G.729 is the next best from a bandwidth standpoint, and expected 
additional channel usage is very close to the quoted bandwidth of 
G.729 codec usage - ~9.6kbps in each direction.  The next most 
efficient codec would be GSM at 14.7kbps, and third would be Speex at 
17.4kbps.

   Somewhat surprising were the IP/IAX2 overhead figures for ILBC - 
they are almost one-third that of some other protocols.  I tested 
that particular number three times to make sure I did not make any 
errors, and all tests were within 3% of each other, so that is an 
unexpected but consistent result.  Of course, my method for obtaining 
these estimated IP/IAX2 overhead figures may be inappropriate - I 
welcome comments on the method used to estimate those numbers.


-------------
Caveats:
   I only had two phones with which to test, and only a limited amount 
of time.  More accurate testing would be achieved by putting many 
more lines into a trunk and watching the bits-per-second growth of 
the IAX2 trunk - two calls is probably not sufficient to examine true 
bandwidth usage in anything other than a general way.  Most 
interesting would be to watch the growth of LPC10 channel usage, as 
they are supposed to be only 2.4kbps, which could possibly result in 
many more channels per megabit if the overhead figures stay roughly 
the same with more channels added.  The 6.1 number for LPC10 seems a 
bit high, given that the quoted usage is 2.4kbps.



-------------
References:
http://www.asterisk.org/ - Asterisk open-source VoIP/CTI platform
http://s-tech.elsat.net.pl/bmtools/ - for the "rate" bandwidth
measurement tool
http://www.erlang.com/bandwidth.html - basic common IP rates for some codecs



---------------
Shameless plug:
   I am a consultant, and I'm happy to do this kind of work to further 
develop your products or deployments of VoIP/Asterisk systems.  Mail 
me (jtodd @loligo.com) for details.


JT

>    - The number for "Estimated IP Overhead" was obtained by
> subtracting (additional channel usage) from (single channel usage.)
> This is possibly inaccurate.
It should generally be pretty accurate.  You might try running 3 calls
just to confirm.
> ILBC:
>   one call:   56134.91 bps/67.45 pps    (28.0 kbps)
>   two calls:  98679.11 bps/102.41 pps   (49.3 kbps)
>
>   Thus:
>    For every additional call:       42544 bps (21.2 kbps)
>    Est. IP/IAX2 overhead (1 call):  13590 bps ( 6.7 kbps)
>    Raw number of calls per megabit: 47
Remember ILBC uses a different frame length (thus the lower pps count) and
because it's not going to line up exactly with the G.729 or ulaw frames,
there will be even fewer.  However, I still don't see only 6.7kbps.  That
seems just a bit too low.
> GSM
>   one call:   70958.16 bps/102.13 pps   (35.4 kbps)
>   two calls: 100455.23 bps/102.63 pps   (50.2 kbps)
>
>   Thus:
>    For every additional call:       29497 bps (14.7 kbps)
>     Est. IP/IAX2 overhead (1 call): 41461 bps (20.7 kbps)
>    Raw number of calls per megabit: 68
IP overhead is purely a factor of the number of packets (PPS).  GSM, ulaw,
Speex, and G.729 should all have identical overheads, in principle.
> -------------
> LPC10
>   one call:   43855.44 bps/89.94  pps   (21.9 kbps)
>   two calls:  56059.18 bps/100.81 pps   (28.0 kbps)
>
>   Thus:
>    For every additional call:       12203 bps ( 6.1 kbps)
>    Est. IP/IAX2 overhead (1 call):  31561 bps (15.8 kbps)
>    Raw number of calls per megabit: 164
I would predict LPC10 to be around 4.8, not 6.1....  I wonder why the
discrepency.  Perhaps the IAX overhead is coming into play more here...
> SPEEX
>   one call:   74817.18 bps/101.06 pps   (37.4 kbps)
>   two calls: 109692.68 bps/102.18 pps   (54.8 kbps)
>
>   Thus:
>    For every additional call:       34875 bps (17.4 kbps)
>    Est. IP/IAX2 overhead (1 call):  39941 bps (19.9 kbps)
>    Raw number of calls per megabit: 57
Changing Asterisk's selection of options on Speex could improve this.
>    I am a consultant, and I'm happy to do this kind of work to further
> develop your products or deployments of VoIP/Asterisk systems.  Mail
> me (jtodd @loligo.com) for details.
Your work speaks for itself, but for what it's worth, I also give you my
own thumbs up for your dedicated understanding of the mechanics,
debugging, and testing of Asterisk.

Mark