asterisk users - Dec 2008 - Using Asterisk to measure call quality: Introducing Recqual

Hey everyone,

   A while back I worked on a project to measure call quality.  I've
finally gotten around to releasing it and I'm calling it recqual (Real
Call Quality).  There isn't much to it and it should be considered
alpha quality.  I'm hoping some of the bright minds on the list can
help me out with it.  I'll include the intro text from the README in
the tarball:

----
Recqual is collection of scripts using Asterisk and other
Linux utilities to measure call quality on an automated
basis.

-How it Works-
Recqual was designed to detect audio quality problems in a call
path that may not be visible from the technology being used
locally.  Whether it's SIP, ZAP, or IAX the fact is there are
many potential sources of call quality problems in just about
any call being made.  Often times a SIP "provider" may resell
services, for example.  While the delivery of IP packets to/from
this provider may look excellent there may be other problems
upstream that an analysis of the IP packets, path, etc may not
be able to detect.

In scenarios such as this the only way to identify call quality
problems is to analyze the audio itself.  Regardless of method
or transport being used, the goal of any telephony system is to
deliver reliable, consistent call quality.

Recqual is designed to allow you to place a large number of
automated calls (using Asterisk) using different call scenario
files.

The key here is consistency.  When Asterisk places the outbound
call (and answers the inbound call) it will generate a set of
tones while recording the return audio path.  Once the run has
finished Ecasound will run with various filters and noise gates
to detect certain amounts of distortion, signal loss, etc.

Calls either pass or fail based on how much variation there is
in the audio once it has been returned.  Of course you can pass
audio through any combination of networks - including the PSTN.

Almost any "call quality" problem(s) can be detected with this
method.  Whether it's one way calls, echo, dropped packets,
distortion, etc ecasound should be able to isolate the problem
calls.  If not you can just tweak the script ;).

Only calls that fail are saved.  These files can be imported
into your favorite audio processing utility and/or run through
Ecasound again if you'd like to tweak the process script to
detect them automatically.

Recqual has been designed (and optimized) to work with SIP
channels.  For example, it has the ability to correlate problem
calls with specific RTP endpoint IP addresses.  However, due to
the protocol independent nature of Asterisk you can use just
about any channel type with a few simple changes.
---

  So there you have it.  I've used this with a great deal of success
but I think there is still a lot to be done.  More on my blog here:

http://blog.krisk.org

  Thoughts?

-- 
Kristian Kielhofner
http://blog.krisk.org
http://www.submityoursip.com
http://www.astlinux.org
http://www.star2star.com

On Mon, Dec 22, 2008 at 10:37:01AM -0500, Kristian Kielhofner
wrote:
> Hey everyone,
> 
>    A while back I worked on a project to measure call quality.  I've
> finally gotten around to releasing it and I'm calling it recqual (Real
> Call Quality).  There isn't much to it and it should be considered
> alpha quality.  I'm hoping some of the bright minds on the list can
> help me out with it.  I'll include the intro text from the README in
> the tarball:
You seem to have ommited the relevant links:

http://blog.krisk.org/2008/12/introducing-recqual.html
http://admin.star2star.com/recqual/

Looks interesting...

-- 
               Tzafrir Cohen
icq#16849755              jabber:tzafrir.cohen at xorcom.com
+972-50-7952406           mailto:tzafrir.cohen at xorcom.com
http://www.xorcom.com  iax:guest at local.xorcom.com/tzafrir

On Mon, Dec 22, 2008 at 5:37 PM, Kristian Kielhofner
<kristian.kielhofner at gmail.com> wrote:
> Hey everyone,
>
>   A while back I worked on a project to measure call quality.  I've
> finally gotten around to releasing it and I'm calling it recqual (Real
> Call Quality).  There isn't much to it and it should be considered
> alpha quality.  I'm hoping some of the bright minds on the list can
> help me out with it.  I'll include the intro text from the README in
> the tarball:
>
> ----
> Recqual is collection of scripts using Asterisk and other
> Linux utilities to measure call quality on an automated
> basis.
>
> -How it Works-
> Recqual was designed to detect audio quality problems in a call
> path that may not be visible from the technology being used
> locally.  Whether it's SIP, ZAP, or IAX the fact is there are
> many potential sources of call quality problems in just about
> any call being made.  Often times a SIP "provider" may resell
> services, for example.  While the delivery of IP packets to/from
> this provider may look excellent there may be other problems
> upstream that an analysis of the IP packets, path, etc may not
> be able to detect.
>
> In scenarios such as this the only way to identify call quality
> problems is to analyze the audio itself.  Regardless of method
> or transport being used, the goal of any telephony system is to
> deliver reliable, consistent call quality.
>
> Recqual is designed to allow you to place a large number of
> automated calls (using Asterisk) using different call scenario
> files.
>
> The key here is consistency.  When Asterisk places the outbound
> call (and answers the inbound call) it will generate a set of
> tones while recording the return audio path.  Once the run has
> finished Ecasound will run with various filters and noise gates
> to detect certain amounts of distortion, signal loss, etc.
>
> Calls either pass or fail based on how much variation there is
> in the audio once it has been returned.  Of course you can pass
> audio through any combination of networks - including the PSTN.
>
> Almost any "call quality" problem(s) can be detected with this
> method.  Whether it's one way calls, echo, dropped packets,
> distortion, etc ecasound should be able to isolate the problem
> calls.  If not you can just tweak the script ;).
>
> Only calls that fail are saved.  These files can be imported
> into your favorite audio processing utility and/or run through
> Ecasound again if you'd like to tweak the process script to
> detect them automatically.
>
> Recqual has been designed (and optimized) to work with SIP
> channels.  For example, it has the ability to correlate problem
> calls with specific RTP endpoint IP addresses.  However, due to
> the protocol independent nature of Asterisk you can use just
> about any channel type with a few simple changes.
> ---
>
>  So there you have it.  I've used this with a great deal of success
> but I think there is still a lot to be done.  More on my blog here:
>
> http://blog.krisk.org
>
>  Thoughts?
>
Hi,

This is good idea, and i will probably try it out someday next year
(too busy completing my business requirements :)

I took a look at asterisk patch, and it seems quite simple. I just
don't see the point of removing "if(debug)". You could easily get
this
additional logging into Asterisk trunk (if preserving RTP info in
debug level), and starting asterisk with "debug 1". So, then it would
be easier to install recqual. Also, being able to run on unmodified
version of Asterisk, it would be good to allow keeping current
dialplan and just route test calls trough it. So, people would be able
to keep track of their billing, etc for those test calls.

Also, thanks for showing us magics of ecasound. I have similar project
(pbx-test-framework) that allows IVR/Queue/etc testing in automated
mode. Recording everything and checking voice quuailty would be great
addition :)

Regards,
Atis

-- 
Atis Lezdins,
VoIP Project Manager / Developer,
IQ Labs Inc,
atis at iq-labs.net
Skype: atis.lezdins
Cell Phone: +371 28806004
Cell Phone: +1 800 7300689
Work phone: +1 800 7502835