opus - Feb 2018 - [EXTERNAL] Re: Developing OPUS on TI CC3220

Thanks Jean-Marc,



I was able to get both encode and decode working the CC3220 device! But for
bi-directional communication, I need decode and encode to occur in less time
than the frame size I’m sending (20 ms).



Currently decode takes 16~22 ms and encode is ~13 ms. What is the best way to
try to reduce this time? Also, unsure why encode is taking less time than
decode...



I've also noticed if I change my encoder to silk only, I get a Heap
overflow... I have dynamically allocated a heap for the codec, got the size
using the api and used encode init so that the memory is allocated to my created
heap. I have 30k currently in heap, since the return of get  size was ~25 kB.
This seems to happen when I change the application as well. :/



Currently here is how I am setting up my encoder:





    int size = opus_encoder_get_size(1);

    sOpusEnc = Memory_alloc(OpusHeap,size,NULL,NULL);

//

//    sOpusEnc = opus_encoder_create(16000,

//                                   1,

//                                   OPUS_APPLICATION_VOIP, &i32error);

    i32error = opus_encoder_init(sOpusEnc, 16000, 1,
OPUS_APPLICATION_RESTRICTED_LOWDELAY);





    opus_encoder_ctl(sOpusEnc, OPUS_SET_BITRATE(16000));

    opus_encoder_ctl(sOpusEnc, OPUS_SET_VBR(1));

    opus_encoder_ctl(sOpusEnc, OPUS_SET_VBR_CONSTRAINT(0));

    opus_encoder_ctl(sOpusEnc, OPUS_SET_COMPLEXITY(0));

    opus_encoder_ctl(sOpusEnc, OPUS_SET_INBAND_FEC(0));

    opus_encoder_ctl(sOpusEnc, OPUS_SET_FORCE_CHANNELS(1));//Mono

    opus_encoder_ctl(sOpusEnc, OPUS_SET_PACKET_LOSS_PERC(0));

    opus_encoder_ctl(sOpusEnc, OPUS_SET_LSB_DEPTH(16)

                     );

    opus_encoder_ctl(sOpusEnc,

                     OPUS_SET_EXPERT_FRAME_DURATION(OPUS_FRAMESIZE_20_MS));

    opus_encoder_ctl(sOpusEnc, OPUS_SET_FORCE_MODE(MODE_CELT_ONLY));









Thanks again for your help!



VR



-----Original Message-----
From: Jean-Marc Valin [mailto:jmvalin at jmvalin.ca]
Sent: Tuesday, February 20, 2018 6:23 PM
To: Rodriguez, Vince; opus at xiph.org
Subject: Re: [EXTERNAL] Re: [opus] Developing OPUS on TI CC3220



On 02/20/2018 06:48 PM, Rodriguez, Vince wrote:
> I am trying to get the library to build without using the pseudostack
> define, and use either VAR_ARRAYS or ALLOC, but it seems the global
> stack is not defined.


Not sure what you mean here. If your compiler supports C99 VLAs, then by all
means just define VAR_ARRAYS and you're good to go. If not, the second best
option is alloca(), which you can use by defining USE_ALLOCA. If your compiler
supports alloca() and USE_ALLOCA doesn't work, then you may need to make
minor tweaks to celt/stack_alloc.h. Do not use NONTHREADSAFE_PSEUDOSTACK unless
all other options fail.



Cheers,



               Jean-Marc


> -----Original Message----- From: Jean-Marc Valin
> [mailto:jmvalin at jmvalin.ca] Sent: Tuesday, February 20, 2018 5:40 PM
> To: Rodriguez, Vince; opus at xiph.org<mailto:opus at xiph.org>
Subject: [EXTERNAL] Re: [opus]
> Developing OPUS on TI CC3220
>
> Hi Vince,
>
> Normally, Opus should work fine on the CC3220, though real-time
> encoding may be a bit tight depending on what configuration you
> choose. You'll probably want to configure with --enable-fixed-point
> and --disable-float-api (or define FIXED_POINT and DISABLE_FLOAT_API
> if you don't use autoconf).
>
> The encoder has a bunch of run-time settings that you may want to set,
> depending on your application. These include bitrate, complexity
> (probably a low setting for that that chip), and bitrate management
> (CBR/VBR/CVBR).
>
> In the end, aside from the complexity requirements, you should need to
> do anything special to get Opus running on your chip.
>
> Cheers,
>
> Jean-Marc
>
> On 02/20/2018 05:46 PM, Rodriguez, Vince wrote:
>> I'm looking to try to port a Opus Decoder and Encoder onto the TI
>> CC3220SF (http://www.ti.com/product/CC3220) device. Currently, I have
>> successfully been able to get the decode working for a .ogg file
>> saved locally to the serial flash. My end goal is bidirectional audio
>> using OPUS between two devices. While looking into the documentation
>> for the Encoder, I was not sure what the best step forward would be
>> in trying to enable this streaming application.
>>
>>
>>
>> Can someone give me a little guidance on how to properly setup the
>> encoder for streaming applications, and if I need to do anything
>> special to ensure the encoder works correctly on an ARM M4 (not
>> Floating point) MCU?
>>
>>
>>
>> Thanks!
>>
>>
>>
>> *Vince Rodriguez*
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________ opus mailing list
>> opus at xiph.org<mailto:opus at xiph.org>
http://lists.xiph.org/mailman/listinfo/opus
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xiph.org/pipermail/opus/attachments/20180223/edc7ebbd/attachment.html>

On 02/23/2018 10:47 AM, Rodriguez, Vince wrote:
> Currently decode takes 16~22 ms and encode is ~13 ms. What is the best
> way to try to reduce this time? Also, unsure why encode is taking less
> time than decode...
Tried running a profiler?
> I've also noticed if I change my encoder to silk only, I get a Heap
> overflow... I have dynamically allocated a heap for the codec, got the
> size using the api and used encode init so that the memory is allocated
> to my created heap. I have 30k currently in heap, since the return of
> get  size was ~25 kB. This seems to happen when I change the application
> as well. :/
SILK and CELT use exactly as much heap since the only heap Opus needs is
the encoder state which holds both the SILK and the CELT state. What
stack allocator did you end up choosing? If you used the pseudo-stack,
then that allocates a lot of heap memory. If not, then maybe you have
some sort of collision between the stack and heap?
>     i32error = opus_encoder_init(sOpusEnc, 16000, 1,
> OPUS_APPLICATION_RESTRICTED_LOWDELAY);
OPUS_APPLICATION_RESTRICTED_LOWDELAY will force CELT-only mode here,
regardless of other options.
>    opus_encoder_ctl(sOpusEnc, OPUS_SET_FORCE_CHANNELS(1));//Mono
No need to force mono since you're only encoding one channel (this does
nothing).

	Jean-Marc

On 02/23/2018 07:47 AM, Rodriguez, Vince wrote:
> Currently decode takes 16~22 ms and encode is ~13 ms. What is the best
> way to try to reduce this time? Also, unsure why encode is taking less
> time than decode...
Last month a user posted some benchmarks on a STM32F4, which is a the
same core as your cc3220 but at 168MHz rather than 80Mhz. Here's the
results:

http://lists.xiph.org/pipermail/opus/attachments/20180114/a0340dfa/attachment-0012.png

Based on that it seems that your decode should take about ~10ms, so
it'll be a bit of a squeeze to get full duplex in. But your numbers are
significantly worse. Are you running out of XIP flash? Do you have all
the flash accelerators and caches enabled?

Jean-Marc,

Thanks for the quick reply.

Using your comments, I was able to get my implementation to be much closer to
working. Decode taking ~8 msecs and encode is ~13 ms.

I noticed with decode, if I use a file saved on the device, I can decode it in 4
ms. This is the timing I need to make the bi-directional work.

Looking into how my encoder is setup. It seems I can isolate my overflow
condition to the Silk implementation. On the first call to Encode with silk, the
codec does a malloc, and I get a null pointer returned.

I guess I'm a little hazy on the memory requirements. We have a heap, that
holds the state of the codec (~25 kB), but do we also need to malloc other
memory to encode? Or does the codec use the heap allocated to do all encoding?

Thanks!

VR

-----Original Message-----
From: Jean-Marc Valin [mailto:jmvalin at jmvalin.ca] 
Sent: Friday, February 23, 2018 11:12 AM
To: Rodriguez, Vince; opus at xiph.org
Subject: Re: [EXTERNAL] Re: [opus] Developing OPUS on TI CC3220

On 02/23/2018 10:47 AM, Rodriguez, Vince wrote:
> Currently decode takes 16~22 ms and encode is ~13 ms. What is the best 
> way to try to reduce this time? Also, unsure why encode is taking less 
> time than decode...
Tried running a profiler?
> I've also noticed if I change my encoder to silk only, I get a Heap 
> overflow... I have dynamically allocated a heap for the codec, got the 
> size using the api and used encode init so that the memory is 
> allocated to my created heap. I have 30k currently in heap, since the 
> return of get  size was ~25 kB. This seems to happen when I change the 
> application as well. :/
SILK and CELT use exactly as much heap since the only heap Opus needs is the
encoder state which holds both the SILK and the CELT state. What stack allocator
did you end up choosing? If you used the pseudo-stack, then that allocates a lot
of heap memory. If not, then maybe you have some sort of collision between the
stack and heap?
>     i32error = opus_encoder_init(sOpusEnc, 16000, 1, 
> OPUS_APPLICATION_RESTRICTED_LOWDELAY);
OPUS_APPLICATION_RESTRICTED_LOWDELAY will force CELT-only mode here, regardless
of other options.
>    opus_encoder_ctl(sOpusEnc, OPUS_SET_FORCE_CHANNELS(1));//Mono
No need to force mono since you're only encoding one channel (this does
nothing).

	Jean-Marc