asterisk users - Nov 2006 - What's up with the Manager Interface?!?!

The Asterisk Manager Interface is driving me nuts.
Whoever wrote it should be drawn and quartered.

Sometimes the data comes back separated by \r\n, and sometimes it's
separated by \n.
The whole thing is completely inconsistent, and trying to write any kind of API
for it is -GHASTLY-

Doug.

> Sometimes the data comes back separated by \r\n, and sometimes it's
> separated by \n.
> The whole thing is completely inconsistent, and trying to write any
kind
> of API for it is -GHASTLY-
Doug,

What language(s) are you using?  Just curious.  I've been tinkering with
Perl, POE, and POE::Component::Client::Asterisk::Manager.  These have
abstracted away the lowest level of programming. 

I know you've done Python in the past - I hear that there's a module for
AMI called py-Asterisk.  Have you seen or tried that?  Ditto with Ruby -
a module called RAMI.  Both are on sourceforge.

Also, could you hum a few bars about what you're trying to accomplish
with your API?  I'm curious about the big picture.

Thanks!

-MC

Douglas Garstang wrote:
> The Asterisk Manager Interface is driving me nuts.
> Whoever wrote it should be drawn and quartered.
>
> Sometimes the data comes back separated by \r\n, and sometimes it's
separated by \n.
> The whole thing is completely inconsistent, and trying to write any kind of
API for it is -GHASTLY-
>
> Doug.
> _______________________________________________
>   
just wait till you get a 'hiccup' that causes a line to get cut off, 
drop a char, and continue on next line. <G>
(examples below)

this is an older manager.c
there have been a lot of mods to the manager interface in the 1.4 tree, 
but there is no way i would put that into a production envir.

-------------
Event: OriginateFailure
Privilege: call,all
Channel: Zap/g1/xxxxxxxxxx
Context: gdincoming
Exten:
Reason: 3
Uniqueid: <null>
CallerID: xxxxxxxxxx
CallerIDName: ~308C
D13-47426-true~

-------------
Event: OriginateFailure
Privilege: call,all
Channel: Zap/g1/xxxxxxxxxx
Context: gdincoming
Exten:
Reason: 5
(rest was gone)

-------------
Event: OriginateFailure
Privilege: call,all
Channel: Zap/g1/xxxxxxxxxx
Context: gdincoming
Exten:
Reason: 0
Uniqueid: <null>
CallerID: xxxxxxxxxx
Ca
lerIDName: ~308CLD14-40566-true~

-------------
Event: OriginateSuccess
Privilege: call,all
Channel: Zap/g1/xxxxxxxxxx
Context: gdincoming
Exten:
Reaso
: 4
Uniqueid: 1163128185.2006
CallerID: xxxxxxxxxx
CallerIDName: ~308CLD13-50454-true~

> -----Original Message-----
> From: Michael Collins [mailto:mcollins@fcnetwork.com]
> Sent: Wednesday, November 29, 2006 11:20 AM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: RE: [asterisk-users] What's up with the Manager Interface?!?!
> 
> 
> > Sometimes the data comes back separated by \r\n, and sometimes
it's
> > separated by \n.
> > The whole thing is completely inconsistent, and trying to write any
> kind
> > of API for it is -GHASTLY-
> 
> Doug,
> 
> What language(s) are you using?  Just curious.  I've been 
> tinkering with
> Perl, POE, and POE::Component::Client::Asterisk::Manager.  These have
> abstracted away the lowest level of programming. 
> 
> I know you've done Python in the past - I hear that there's a 
> module for
> AMI called py-Asterisk.  Have you seen or tried that?  Ditto 
> with Ruby -
> a module called RAMI.  Both are on sourceforge.
> 
> Also, could you hum a few bars about what you're trying to accomplish
> with your API?  I'm curious about the big picture.
Michael, I'm using python.

Here's a good example. I'm trying to get SIP blf. I managed to split my
result into a list of lines by splitting on ANY of \r\n, \n or \r.  I was going
use the column headings from the third line as my keys for my dictionary/hash,
rather than hard coding them. Notice anything? The 'Call ID' column has
a space right in the middle which means I can't simply split this up by
white-space.

        Response: Follows
        Privilege: Command
        Peer             User        Call ID      Extension        Last state   
Type
        xxx.187.128.105  2944090     f7ee98da-6d  2944006          InUse        
xpidf+xml
        xxx.187.128.105  2944090     111e388b-6b  2944077          Idle         
xpidf+xml

I think I looked at the python module and was underwhelmed by it.

Doug.

On 11/29/06, Douglas Garstang <dgarstang@oneeighty.com>
wrote:
> The Asterisk Manager Interface is driving me nuts.
> Whoever wrote it should be drawn and quartered.
You would need a ton of rope and a few hundred horses for that :)

The Manager API code is distributed across dozens of source files in
the Asterisk code base and if you include every possible input and
output of it you are talking about code contributions from over a
hundred people(including me).

The problem is that the Manager API is not centrally coded and has
evolved over the last 5 years to accommodate a lot of functionality
that it was never designed to do. In my experience, the Manager API is
more stable in 1.2 than it was in 1.0, and it was more stable in 1.0
than it was in the pre 1.0 CVS codebase.

Having developed a Manager Queue system 3 years ago, I fought a lot of
the Manage API battles and eventually got to a fairly robust and
non-blocking system that deals a lot of the flaws of the Manager API,
but it was by no means a full-featured solution that worked with every
kind of Manager input or output.

My recommendation to people starting out with Manager API programming
is to either specialize it as much as you can to the tasks you need to
do with it, or use something like the AstManProxy to handle the
connection and interface mess.

MATT---

> -----Original Message-----
> From: Matt Florell [mailto:astmattf@gmail.com]
> Sent: Wednesday, November 29, 2006 12:52 PM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [asterisk-users] What's up with the Manager Interface?!?!
> 
> 
> On 11/29/06, Douglas Garstang <dgarstang@oneeighty.com> wrote:
> > The Asterisk Manager Interface is driving me nuts.
> > Whoever wrote it should be drawn and quartered.
> 
> You would need a ton of rope and a few hundred horses for that :)
> 
> The Manager API code is distributed across dozens of source files in
> the Asterisk code base and if you include every possible input and
> output of it you are talking about code contributions from over a
> hundred people(including me).
> 
> The problem is that the Manager API is not centrally coded and has
> evolved over the last 5 years to accommodate a lot of functionality
> that it was never designed to do. In my experience, the Manager API is
> more stable in 1.2 than it was in 1.0, and it was more stable in 1.0
> than it was in the pre 1.0 CVS codebase.
> 
> Having developed a Manager Queue system 3 years ago, I fought a lot of
> the Manage API battles and eventually got to a fairly robust and
> non-blocking system that deals a lot of the flaws of the Manager API,
> but it was by no means a full-featured solution that worked with every
> kind of Manager input or output.
Agreed on all that, except I found AstManProxy to have just as many issues in
the way it rendered XML output.

Doug.

> -----Original Message-----
> From: Douglas Garstang 
> Sent: Wednesday, November 29, 2006 12:26 PM
> To: 'Asterisk Users Mailing List - Non-Commercial Discussion'
> Subject: RE: [asterisk-users] What's up with the Manager Interface?!?!
> 
> 
> > -----Original Message-----
> > From: Michael Collins [mailto:mcollins@fcnetwork.com]
> > Sent: Wednesday, November 29, 2006 11:20 AM
> > To: Asterisk Users Mailing List - Non-Commercial Discussion
> > Subject: RE: [asterisk-users] What's up with the Manager 
> Interface?!?!
> > 
> > 
> > > Sometimes the data comes back separated by \r\n, and 
> sometimes it's
> > > separated by \n.
> > > The whole thing is completely inconsistent, and trying to 
> write any
> > kind
> > > of API for it is -GHASTLY-
> > 
> > Doug,
> > 
> > What language(s) are you using?  Just curious.  I've been 
> > tinkering with
> > Perl, POE, and POE::Component::Client::Asterisk::Manager.  
> These have
> > abstracted away the lowest level of programming. 
> > 
> > I know you've done Python in the past - I hear that there's a 
> > module for
> > AMI called py-Asterisk.  Have you seen or tried that?  Ditto 
> > with Ruby -
> > a module called RAMI.  Both are on sourceforge.
> > 
> > Also, could you hum a few bars about what you're trying to 
> accomplish
> > with your API?  I'm curious about the big picture.
> 
> Michael, I'm using python.
> 
> Here's a good example. I'm trying to get SIP blf. I managed 
> to split my result into a list of lines by splitting on ANY 
> of \r\n, \n or \r.  I was going use the column headings from 
> the third line as my keys for my dictionary/hash, rather than 
> hard coding them. Notice anything? The 'Call ID' column has a 
> space right in the middle which means I can't simply split 
> this up by white-space. 
> 
>         Response: Follows
>         Privilege: Command
>         Peer             User        Call ID      Extension   
>      Last state     Type           
>         xxx.187.128.105  2944090     f7ee98da-6d  2944006     
>      InUse          xpidf+xml      
>         xxx.187.128.105  2944090     111e388b-6b  2944077     
>      Idle           xpidf+xml  
> 
> I think I looked at the python module and was underwhelmed by it.
Grrrr. Here's another example...

Action: Command
Command: sip show peer 2944093

Response: Follows
Privilege: Command


  * Name       : 2944093
  Secret       : <Set>
  MD5Secret    : <Not set>
  Context      : 180o_CallStart
  Subscr.Cont. : 180o_WatchBLF

Why the HELL is there an asterisk before 'Name'? Now I have to strip the
bloody thing out!
And why is there TWO empty lines before it?
Good grief!

Doug.

> -----Original Message-----
> From: Steve Edwards [mailto:asterisk.org@sedwards.com]
> Sent: Wednesday, November 29, 2006 2:55 PM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: RE: [asterisk-users] What's up with the Manager Interface?!?!
> 
> 
> On Wed, 29 Nov 2006, Douglas Garstang wrote:
> 
> > Grrrr. Here's another example...
> >
> > Action: Command
> > Command: sip show peer 2944093
> >
> > Response: Follows
> > Privilege: Command
> >
> >
> >  * Name       : 2944093
> >  Secret       : <Set>
> >  MD5Secret    : <Not set>
> >  Context      : 180o_CallStart
> >  Subscr.Cont. : 180o_WatchBLF
> >
> > Why the HELL is there an asterisk before 'Name'? Now I have 
> to strip the bloody thing out!
> > And why is there TWO empty lines before it?
> > Good grief!
> >
> > Doug.
> 
> Would it be a better use of your time to "fix" the offending
modules
> rather than kludge your code to handle the inconsistencies?
> 
> Is AMI spec'd or would that be the first step?
Steve,

No... I'm not a C programmer. A standard interface would be a first step. :)

Doug.

> -----Original Message-----
> From: James Texter [mailto:james.texter@cox.net]
> Sent: Wednesday, November 29, 2006 3:03 PM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [asterisk-users] What's up with the Manager Interface?!?!
> 
> 
> Doug,
>     Your issue isn't with the manager.  It's with the CLI 
> output you are
> trying to hijack via manager :D  If you run "sip show peer 
> 2944093" in the
> CLI, you'll see a blank line, followed by a line that is "*
Name".  It
> appears what you really want is a manager Action to show a 
> sip peer, in
> which case I would recommend adding a new manager command 
> that returns a
> string which is much more machine readable.  Remember, CLI output is
> designed to be human readable.
James.

Ok... that sounds like an objective distinction. Maybe it's just the output
that I get as a result of:

Action: Command
Command: foo

that's causing problems.

eg:
Action: Command
Command: sip show subscriptions

I don't know why every CLI command doesn't have a corresponding action. 
I won't be adding any new manager commands, as I am not a C programmer.

Doug.

> -----Original Message-----
> From: James Texter [mailto:james.texter@cox.net]
> Sent: Wednesday, November 29, 2006 3:03 PM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [asterisk-users] What's up with the Manager Interface?!?!
> 
> 
> Doug,
>     Your issue isn't with the manager.  It's with the CLI 
> output you are
> trying to hijack via manager :D  If you run "sip show peer 
> 2944093" in the
> CLI, you'll see a blank line, followed by a line that is "*
Name".  It
> appears what you really want is a manager Action to show a 
> sip peer, in
> which case I would recommend adding a new manager command 
> that returns a
> string which is much more machine readable.  Remember, CLI output is
> designed to be human readable.
> 
> Just my $0.02.
James, even the Action: commands are inconsistent.
Arrgh!!!

The 'sippeers' action completes with:

Event: PeerlistComplete
ListItems: 79

while the 'queues' action completes with NOTHING. Nothing is printed
that indicates it's output has completed.
These are _both_ action commands and yet they yield inconsistent results!

Action: queues

180o_QCCare  has 0 calls (max unlimited) in 'rrmemory' strategy (27s
holdtime), W:0, C:39, A:8, SL:0.0% within 0s
   Members: 
      Agent/80014053 (Unavailable) has taken no calls yet
      Agent/80014009 (Unavailable) has taken no calls yet
      Agent/80014054 (Unavailable) has taken no calls yet
      Agent/80014050 (paused) (Unavailable) has taken 1 calls (last was 99158
secs ago)
      Agent/80014033 (Unavailable) has taken no calls yet
      Agent/80014051 (Not in use) has taken 17 calls (last was 5842 secs ago)
      Agent/80014052 (Unavailable) has taken no calls yet
      Agent/80014057 (Busy) has taken 20 calls (last was 769 secs ago)
      Agent/80014055 (paused) (Not in use) has taken 1 calls (last was 100494
secs ago)
   No Callers

180o_QTech   has 0 calls (max unlimited) in 'rrmemory' strategy (11s
holdtime), W:0, C:106, A:23, SL:0.0% within 0s
   Members: 
      Agent/80014153 (Unavailable) has taken no calls yet
      Agent/80014154 (Unavailable) has taken no calls yet
      Agent/80014109 (Unavailable) has taken 17 calls (last was 87502 secs ago)
      Agent/80014150 (Unavailable) has taken no calls yet
      Agent/80014133 (Not in use) has taken 54 calls (last was 172 secs ago)
      Agent/80014151 (Unavailable) has taken no calls yet
      Agent/80014152 (paused) (Not in use) has taken 35 calls (last was 6479
secs ago)
      Agent/80014157 (Unavailable) has taken no calls yet
      Agent/80014155 (Unavailable) has taken no calls yet
   No Callers

See! There's no Event that indicates the output has finished!

That data is no easier to parse than the output generated from:

Action: Command
Command: sip show peer <peer>

Note the colons used both as a field delimiter AND stuck in the regcontact.

CodecOrder: ulaw,g729,gsm
Status: UNKNOWN
SIP-Useragent: PolycomSoundPointIP-SPIP_601-UA/1.6.7.0098
Reg-Contact : sip:2944093@216.187.128.77
ChanVariable:
 parkstart,10

Doug.
> -----Original Message-----
> From: Richard Lyman [mailto:pchammer@dynx.net]
> Sent: Wednesday, November 29, 2006 3:30 PM
> To: Asterisk Users Mailing List - Non-Commercial Discussion
> Subject: Re: [asterisk-users] What's up with the Manager Interface?!?!
> 
> 
> James Texter wrote:
> > Doug,
> >     Your issue isn't with the manager.  It's with the CLI 
> output you are
> > trying to hijack via manager :D  If you run "sip show peer 
> 2944093" in the
> > CLI, you'll see a blank line, followed by a line that is "* 
> Name".  It
> > appears what you really want is a manager Action to show a 
> sip peer, in
> > which case I would recommend adding a new manager command 
> that returns a
> > string which is much more machine readable.  Remember, CLI output is
> > designed to be human readable.
> >
> > Just my $0.02.
> >
> >   
> action: sippeers
> 
> or
> 
> action: sipshowpeer
> peer: name
> 
> maybe you should do the below to refresh  your memory
> 
> action: command
> command: show manager commands
> 
> 
> 
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
> 
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users
>

Doug, Everyone:

I'll make you an offer you (hopefully) can't refuse:

I've been fixing manager bugs here and there, and am willing to take on
any manager issues out there, for 1.4, and trunk, especially, so as to
have things nice and solid for 1.4 before it gets out of beta.

So, give me some details. I will file the bug, if you don't. I will
reproduce(if I can), and debug,  and fix 'em. Just tell me (as
explicitly as possible, please!) what the problems are-- especially you,
Doug-- where are those inconsistencies, exactly? Richard-- I'll lab up
1.4 and see if I can get the hiccups you mention.

murf

----------------------------------------


Douglas Garstang wrote:
> The Asterisk Manager Interface is driving me nuts.
> Whoever wrote it should be drawn and quartered.
>
> Sometimes the data comes back separated by \r\n, and sometimes it's
separated by \n.
> The whole thing is completely inconsistent, and trying to write any kind of
API for it is -GHASTLY-
>
And,

Richard Lyman  wrote:
just wait till you get a 'hiccup' that causes a line to get cut off, 
drop a char, and continue on next line. <G>
(examples below)

this is an older manager.c
there have been a lot of mods to the manager interface in the 1.4 tree, 
but there is no way i would put that into a production envir.

-------------
Event: OriginateFailure
Privilege: call,all
Channel: Zap/g1/xxxxxxxxxx
Context: gdincoming
Exten:
Reason: 3
Uniqueid: <null>
CallerID: xxxxxxxxxx
CallerIDName: ~308C
D13-47426-true~

-------------
Event: OriginateFailure
Privilege: call,all
Channel: Zap/g1/xxxxxxxxxx
Context: gdincoming
Exten:
Reason: 5
(rest was gone)

-------------
Event: OriginateFailure
Privilege: call,all
Channel: Zap/g1/xxxxxxxxxx
Context: gdincoming
Exten:
Reason: 0
Uniqueid: <null>
CallerID: xxxxxxxxxx
Ca
lerIDName: ~308CLD14-40566-true~

-------------
Event: OriginateSuccess
Privilege: call,all
Channel: Zap/g1/xxxxxxxxxx
Context: gdincoming
Exten:
Reaso
: 4
Uniqueid: 1163128185.2006
CallerID: xxxxxxxxxx
CallerIDName: ~308CLD13-50454-true~
-- 
Steve Murphy
Software Developer
Digium
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3227 bytes
Desc: not available
Url :
http://lists.digium.com/pipermail/asterisk-users/attachments/20061207/6f21c1d6/smime.bin

Hi Steve.

Thanks, but unfortunately, I can't be involved in that. We are running
Asterisk in a production environment and we're using 1.2, not 1.4. I
don't have the resources to work with 1.4. Last time I filed a bug against
1.2 I got told off.

Here's an example of that cruddy output. 

hestia*CLI> dundi show peer 00:0e:0c:a1:92:4d
Peer:    00:0e:0c:a1:92:4d
Model:   Symmetric
Host:    xxx.187.142.203
Dynamic: no
KeyPend: no
Reg:     No
In Key:  dundikey
Out Key: dundikey
Include logic:
-- include all
Query logic:
-- permit all
hestia*CLI> 

The delimiter should not be the colon, as the data may also contain a colon (in
this case the MAC address). That makes it really difficult to split the data
into fields. Also, the apparent key:value rule gets broken when you get down to
the Include Logic line. The '--include all' should be on the same line!

Just about every single Asterisk command has screwed up output like this. Fixing
all this would be a LOT of work.

Doug.
> -----Original Message-----
> From: Steve Murphy [mailto:murf@digium.com]
> Sent: Thursday, December 07, 2006 4:26 PM
> To: asterisk-users@lists.digium.com
> Subject: [asterisk-users] Re: What's up with the Manager Interface?!?!
> 
> 
> Doug, Everyone:
> 
> I'll make you an offer you (hopefully) can't refuse:
> 
> I've been fixing manager bugs here and there, and am willing 
> to take on
> any manager issues out there, for 1.4, and trunk, especially, so as to
> have things nice and solid for 1.4 before it gets out of beta.
> 
> So, give me some details. I will file the bug, if you don't. I will
> reproduce(if I can), and debug,  and fix 'em. Just tell me (as
> explicitly as possible, please!) what the problems are-- 
> especially you,
> Doug-- where are those inconsistencies, exactly? Richard-- I'll lab up
> 1.4 and see if I can get the hiccups you mention.
> 
> murf
> 
> ----------------------------------------
> 
> 
> Douglas Garstang wrote:
> > The Asterisk Manager Interface is driving me nuts.
> > Whoever wrote it should be drawn and quartered.
> >
> > Sometimes the data comes back separated by \r\n, and 
> sometimes it's separated by \n.
> > The whole thing is completely inconsistent, and trying to 
> write any kind of API for it is -GHASTLY-
> >
> 
> And,
> 
> Richard Lyman  wrote:
> just wait till you get a 'hiccup' that causes a line to get cut
off,
> drop a char, and continue on next line. <G>
> (examples below)
> 
> this is an older manager.c
> there have been a lot of mods to the manager interface in the 
> 1.4 tree, 
> but there is no way i would put that into a production envir.
> 
> -------------
> Event: OriginateFailure
> Privilege: call,all
> Channel: Zap/g1/xxxxxxxxxx
> Context: gdincoming
> Exten:
> Reason: 3
> Uniqueid: <null>
> CallerID: xxxxxxxxxx
> CallerIDName: ~308C
> D13-47426-true~
> 
> -------------
> Event: OriginateFailure
> Privilege: call,all
> Channel: Zap/g1/xxxxxxxxxx
> Context: gdincoming
> Exten:
> Reason: 5
> (rest was gone)
> 
> -------------
> Event: OriginateFailure
> Privilege: call,all
> Channel: Zap/g1/xxxxxxxxxx
> Context: gdincoming
> Exten:
> Reason: 0
> Uniqueid: <null>
> CallerID: xxxxxxxxxx
> Ca
> lerIDName: ~308CLD14-40566-true~
> 
> -------------
> Event: OriginateSuccess
> Privilege: call,all
> Channel: Zap/g1/xxxxxxxxxx
> Context: gdincoming
> Exten:
> Reaso
> : 4
> Uniqueid: 1163128185.2006
> CallerID: xxxxxxxxxx
> CallerIDName: ~308CLD13-50454-true~
> -- 
> Steve Murphy
> Software Developer
> Digium
>