Hi all, As I am starting to have stable live releases of a dialplan and development work going on in parallel I need to have some sort of regression test in place to ensure that no key functions of the current dialplan are broken by a new version. Does anyone have pointers to the best way to run an automated test on the dialplan, what I am really hoping for is something that looks and works a bit like an nUnit type automated test but I'll take almost any automated testing approach over the alternative - which is manual re-testing every time we change a dialplan. Thanks, Nic
Y'know, I was thinking about a similar idea recently, primarily because I do a lot of work with dialplan based apps. It would be great if there was a way to set up a _complete_ call (meaning it would include what digits to enter when, etc) in a test and then run it against the dialplan being worked on. One thing that I have found is really helpful is coding your dialplans in AEL2 and running the aelparse program with the -d option, which uses the CWD as the base to read from. The parser tells you when something's not right and tells you where it found the error. However this isn't helpful for things that only take place during a call like database lookups and variable values. I'm interested to see the answers that come from your post, thanks for putting it out there! Sherwood McGowan Consultant -----Original Message----- From: asterisk-users-bounces@lists.digium.com [mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of Nic Hughes Sent: Sunday, July 16, 2006 6:31 PM To: asterisk-users@lists.digium.com Subject: [asterisk-users] Regression testing dialplan changes Hi all, As I am starting to have stable live releases of a dialplan and development work going on in parallel I need to have some sort of regression test in place to ensure that no key functions of the current dialplan are broken by a new version. Does anyone have pointers to the best way to run an automated test on the dialplan, what I am really hoping for is something that looks and works a bit like an nUnit type automated test but I'll take almost any automated testing approach over the alternative - which is manual re-testing every time we change a dialplan. Thanks, Nic _______________________________________________ --Bandwidth and Colocation provided by Easynews.com -- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
On Sun, Jul 16, 2006 at 11:31:24PM +0100, Nic Hughes wrote:> Hi all, > > As I am starting to have stable live releases of a dialplan and > development work going on in parallel I need to have some sort of > regression test in place to ensure that no key functions of the current > dialplan are broken by a new version. Does anyone have pointers to the > best way to run an automated test on the dialplan, what I am really > hoping for is something that looks and works a bit like an nUnit type > automated test but I'll take almost any automated testing approach over > the alternative - which is manual re-testing every time we change a > dialplan.Do you expect those tests to run on a running Asteris system or a copy of the configuration file? What do you want to test, exactly? 'show dialplan' can show the parsed configuration (also after AEL include files expansion). This can help validateproper configuration generation. -- Tzafrir Cohen sip:tzafrir@local.xorcom.com icq#16849755 iax:tzafrir@local.xorcom.com +972-50-7952406 jabber:tzafrir@jabber.org tzafrir.cohen@xorcom.com http://www.xorcom.com
From: Tzafrir Cohen <tzafrir.cohen@xorcom.com>> > Do you expect those tests to run on a running Asteris system or a copy > of the configuration file? >What I am after is the current best practice for running regression tests against a running Asterisk system with config files, AGI scripts etc all in place.> What do you want to test, exactly? >I see two main risk areas that I want to deal with right now: 1. Changes to my dialplan causing existing functions of the dialplan to fail 2. Changes to an underlying component (mostly AGI and then database) causing an unexpected error in call handling via the dialplan. To put this into perspective I currently have quite a bit of my more complex logic in AGI scripts - as these are written in PHP I have PHPUnit scripts which can be run very quickly to validate that nothing has ceased working that was working last time I ran the test set. I have a similar test suite for my web-based management interface to the application. The missing link is the dialplan itself - I currently have no test suite which I can quickly run against the dialplan to confirm that it is still working as expected. I would like to shift away from using AGI quite so much for performance reasons but if that means I cannot put my hand on my heart and say "yes it is working, I have re-run every test" after every version upgrade or bug-fix then I am not going to do it. -- Nic