asterisk users - Feb 2018 - Which CDR processing for high load ?

Hello,

I'm load testing a new Asterisk 13 system (Debian  Stretch, packaged
asterisk).
One system writes CDR though an ODBC connection to a local Postgres
database over the LAN.


When sending 50 new calls per second with SIPp, I'm seeing one system
outputs :
taskprocessor.c: The 'subm:cdr_engine-00000003' task processor queue
reached 5000 scheduled tasks again.

This [1] thread mentions such limit.

I was thinking of simply writing CDR entries to a local file before
centralizing them into my database.

What would you suggest on this topic ?

Best regards

[1] http://lists.digium.com/pipermail/asterisk-dev/2016-June/075607.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20180222/1da17be6/attachment.html>

Have you looked at the proc limits for the Asterisk PID?


On Thu, Feb 22, 2018 at 6:23 AM, Olivier <oza.4h07 at gmail.com> wrote:
> Hello,
>
> I'm load testing a new Asterisk 13 system (Debian  Stretch, packaged
> asterisk).
> One system writes CDR though an ODBC connection to a local Postgres
> database over the LAN.
>
>
> When sending 50 new calls per second with SIPp, I'm seeing one system
> outputs :
> taskprocessor.c: The 'subm:cdr_engine-00000003' task processor
queue
> reached 5000 scheduled tasks again.
>
> This [1] thread mentions such limit.
>
> I was thinking of simply writing CDR entries to a local file before
> centralizing them into my database.
>
> What would you suggest on this topic ?
>
> Best regards
>
> [1] http://lists.digium.com/pipermail/asterisk-dev/2016-June/075607.html
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> Check out the new Asterisk community forum at: https://community.asterisk.
> org/
>
> New to Asterisk? Start here:
>       https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20180222/788162fb/attachment.html>

On Thu, Feb 22, 2018 at 5:23 AM, Olivier <oza.4h07 at gmail.com> wrote:
> Hello,
>
> I'm load testing a new Asterisk 13 system (Debian  Stretch, packaged
> asterisk).
> One system writes CDR though an ODBC connection to a local Postgres
> database over the LAN.
>
>
> When sending 50 new calls per second with SIPp, I'm seeing one system
> outputs :
> taskprocessor.c: The 'subm:cdr_engine-00000003' task processor
queue
> reached 5000 scheduled tasks again.
>
> This [1] thread mentions such limit.
>
> I was thinking of simply writing CDR entries to a local file before
> centralizing them into my database.
>
> What would you suggest on this topic ?
>
You should get better CDR performance enabling batch mode which is disabled
by default for legacy reasons.

The CDR code processes all call events in one thread to create CDRs.  This
thread is what processes the
'subm:cdr_engine-00000003' task processor queue you mention above.  If
you
don't have batch mode enabled
then that same thread also has to immediately write the CDRs to each back
end configured.  If you do have
batch mode enabled then the CDRs are passed to another thread to write to
the back ends.

You need to be using at least v13.19.1 or v15.2.1 to also have some CDR
performance enhancements to
help CDR processing of call events.

For information about ODBC connection pooling performance problems see [2].

Richard

[2] http://blogs.asterisk.org/2016/06/15/asterisk-odbc-connections/

>
>
> Best regards
>
> [1] http://lists.digium.com/pipermail/asterisk-dev/2016-June/075607.html
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> Check out the new Asterisk community forum at: https://community.asterisk.
> org/
>
> New to Asterisk? Start here:
>       https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20180222/e68e55f0/attachment.html>

2018-02-22 17:12 GMT+01:00 Dovid Bender <dovid at telecurve.com>:
> Have you looked at the proc limits for the Asterisk PID?
>
>
>
No I haven't.
What shall I look for, exactly ?

The only thing I configured for performance is asterisk.conf 's maxfiles
parameter.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20180222/237c0a58/attachment.html>

1. Would say CDR_ODBC has a greater chance than CDR_CUSTOM (if I may call
them both as such) to become a bottleneck under pressure ?

2. I didn't know about batch mode existence. Thanks for presenting it.
In such batch mode, where are stored CDRs before being written to files or
database ? Does it also apply to data later found in cdr-scv/Master.csv
file ?

2018-02-22 18:14 GMT+01:00 Richard Mudgett <rmudgett at digium.com>:
>
>
> On Thu, Feb 22, 2018 at 5:23 AM, Olivier <oza.4h07 at gmail.com>
wrote:
>
>> Hello,
>>
>> I'm load testing a new Asterisk 13 system (Debian  Stretch,
packaged
>> asterisk).
>> One system writes CDR though an ODBC connection to a local Postgres
>> database over the LAN.
>>
>>
>> When sending 50 new calls per second with SIPp, I'm seeing one
system
>> outputs :
>> taskprocessor.c: The 'subm:cdr_engine-00000003' task processor
queue
>> reached 5000 scheduled tasks again.
>>
>> This [1] thread mentions such limit.
>>
>> I was thinking of simply writing CDR entries to a local file before
>> centralizing them into my database.
>>
>> What would you suggest on this topic ?
>>
>
> You should get better CDR performance enabling batch mode which is
> disabled by default for legacy reasons.
>
> The CDR code processes all call events in one thread to create CDRs.  This
> thread is what processes the
> 'subm:cdr_engine-00000003' task processor queue you mention above. 
If you
> don't have batch mode enabled
> then that same thread also has to immediately write the CDRs to each back
> end configured.  If you do have
> batch mode enabled then the CDRs are passed to another thread to write to
> the back ends.
>
> You need to be using at least v13.19.1 or v15.2.1 to also have some CDR
> performance enhancements to
> help CDR processing of call events.
>
> For information about ODBC connection pooling performance problems see [2].
>
> Richard
>
> [2] http://blogs.asterisk.org/2016/06/15/asterisk-odbc-connections/
>
>
>>
>>
>> Best regards
>>
>> [1]
http://lists.digium.com/pipermail/asterisk-dev/2016-June/075607.html
>>
>> --
>> _____________________________________________________________________
>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>
>> Check out the new Asterisk community forum at:
>> https://community.asterisk.org/
>>
>> New to Asterisk? Start here:
>>       https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>>
>> asterisk-users mailing list
>> To UNSUBSCRIBE or update options visit:
>>    http://lists.digium.com/mailman/listinfo/asterisk-users
>>
>
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> Check out the new Asterisk community forum at: https://community.asterisk.
> org/
>
> New to Asterisk? Start here:
>       https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20180222/6b5e5523/attachment.html>

what kind of ippbx does 50cps ? I think you must be looking for something
like opensips.

On Feb 22, 2018 9:13 PM, "Olivier" <oza.4h07 at gmail.com>
wrote:



2018-02-22 17:12 GMT+01:00 Dovid Bender <dovid at telecurve.com>:
> Have you looked at the proc limits for the Asterisk PID?
>
>
>
No I haven't.
What shall I look for, exactly ?

The only thing I configured for performance is asterisk.conf 's maxfiles
parameter.

--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.
org/

New to Asterisk? Start here:
      https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20180222/63b44c7a/attachment.html>