Hi all !
If someone could give me a hint on how to proceed with the following i'd be
very happy:
I have a test setup on an nrf52832 (Cortex M4) in which I receive audio
from a PDM microphone (64 sample frame) and pass it directly to an I2S
device i.e. from ISR to ISR. With uncompressed audio this works just fine.
Now I try to insert OPUS1.3 in the path but cannot make it work. The audio
passes through but ends up “down pitched” and heavily distorted.
Due to the 32MHz base clock (+ prescalers) available it is not possible to
get the desired 16kHz sample rate. I am therefore forced to work with
15625Hz.
As I understand it the opus encoder does not mind a strange sample rate as
it will re-sample it to 48kHz (Correct?). The decoder, however, will always
output 48kHz, 24kHz, 16kHz etc (Correct?) which is incompatible with my
15625Hz rate. (However; If my assumptions are correct there could be a
possibility to make my I2S device master and, thus, be able to pull the
data at 16kHz)
Consequently I have been looking at “opus custom” as a solution but even
with a (seemingly) low complexity it does not yield the desired result.
*Setup code:*
mode = opus_custom_mode_create(15625, 64, &err);
size = opus_custom_encoder_get_size(mode, 1);
enc = malloc(size);
err = opus_custom_encoder_init(enc, mode, 1);
err = opus_custom_encoder_ctl(enc, OPUS_SET_BITRATE(16000));
err = opus_custom_encoder_ctl(enc, OPUS_SET_COMPLEXITY(0));
size = opus_custom_decoder_get_size(mode, 1);
dec = malloc(size);
err = opus_custom_decoder_init(dec, mode, 1);
*ISR code (called once every 4.096ms):*
nbBytes = opus_custom_encode(
enc, (opus_int16 *)lMicBuf, 64, codedMicBuf, 64);
frame_size = opus_custom_decode(
dec, codedMicBuf, 64, decodedMicBuf, 64);
*Build:*
C:\Program Files\SEGGER\SEGGER Embedded Studio for ARM
3.50\gcc\arm-none-eabi\bin\cc1" -fmessage-length=0
-fno-diagnostics-show-caret -mcpu=cortex-m4 -mlittle-endian
-mfloat-abi=hard -mfpu=fpv4-sp-d16 -mthumb -mtp=soft -nostdinc "-isystem
-D__SIZEOF_WCHAR_T=4 -D__ARM_ARCH_7EM__ -D__SES_ARM
-D__ARM_ARCH_FPV4_SP_D16__ -D__SES_VERSION=35000 -DARM_MATH_CM4
-D__FPU_PRESENT -DDEBUG "-D DEBUG_NRF" -DCONFIG_GPIO_AS_PINRESET
-DINITIALIZE_USER_SECTIONS -DNO_VTOR_CONFIG -DNRF52 -DNRF52832_XXAA
-DCONFIG_NFCT_PINS_AS_GPIOS -DOPUS_BUILD -DVAR_ARRAYS -DFIXED_POINT
-DDISABLE_FLOAT_API -DCUSTOM_MODES "-DARM_MATH_CM4 "
"-DOPUS_ARM_INLINE_EDSP " -DDOPUS_ARM_INLINE_ASM
-DOPUS_ARM_INLINE_MEDIA
-MD
Any ideas?
Regards
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xiph.org/pipermail/opus/attachments/20190217/93dd8087/attachment.html>
Don't use OPUS_SET_COMPLEXITY(0) unless you absolutely have to. That means "give me all the distortions" so I wouldn't be surprised if that's where those are coming from. If your encoding path can't handle full 10 complexity, dial it down to 9, 8, etc until it runs ok. opus_custom_decode doesn't downsample from 48kHz, so your playback code must still be treating it as if it's 15kHz, thus the pitch down. If you can swing it, play as 48kHz, otherwise you have to insert another CPU-hogging resampler. If you do need it and don't have a resampler handy, Opus has silk_resampler_init & silk_resampler available for you. Emily Bowman On Sun, Feb 17, 2019 at 12:47 PM Peter Svensson <petersvenss85 at gmail.com> wrote:> Hi all ! > > > > If someone could give me a hint on how to proceed with the following i'd > be very happy: > > > > I have a test setup on an nrf52832 (Cortex M4) in which I receive audio > from a PDM microphone (64 sample frame) and pass it directly to an I2S > device i.e. from ISR to ISR. With uncompressed audio this works just fine. > > > > Now I try to insert OPUS1.3 in the path but cannot make it work. The audio > passes through but ends up “down pitched” and heavily distorted. > > > > Due to the 32MHz base clock (+ prescalers) available it is not possible to > get the desired 16kHz sample rate. I am therefore forced to work with > 15625Hz. > > > > As I understand it the opus encoder does not mind a strange sample rate as > it will re-sample it to 48kHz (Correct?). The decoder, however, will always > output 48kHz, 24kHz, 16kHz etc (Correct?) which is incompatible with my > 15625Hz rate. (However; If my assumptions are correct there could be a > possibility to make my I2S device master and, thus, be able to pull the > data at 16kHz) > > > > Consequently I have been looking at “opus custom” as a solution but even > with a (seemingly) low complexity it does not yield the desired result. > > > > *Setup code:* > > > > mode = opus_custom_mode_create(15625, 64, &err); > > > > size = opus_custom_encoder_get_size(mode, 1); > > enc = malloc(size); > > err = opus_custom_encoder_init(enc, mode, 1); > > err = opus_custom_encoder_ctl(enc, OPUS_SET_BITRATE(16000)); > > err = opus_custom_encoder_ctl(enc, OPUS_SET_COMPLEXITY(0)); > > > > size = opus_custom_decoder_get_size(mode, 1); > > dec = malloc(size); > > err = opus_custom_decoder_init(dec, mode, 1); > > > > *ISR code (called once every 4.096ms):* > > > > nbBytes = opus_custom_encode( > > enc, (opus_int16 *)lMicBuf, 64, codedMicBuf, 64); > > > > frame_size = opus_custom_decode( > > dec, codedMicBuf, 64, decodedMicBuf, 64); > > > > > > *Build:* > > > > C:\Program Files\SEGGER\SEGGER Embedded Studio for ARM > 3.50\gcc\arm-none-eabi\bin\cc1" -fmessage-length=0 > -fno-diagnostics-show-caret -mcpu=cortex-m4 -mlittle-endian > -mfloat-abi=hard -mfpu=fpv4-sp-d16 -mthumb -mtp=soft -nostdinc "-isystem > > > > -D__SIZEOF_WCHAR_T=4 -D__ARM_ARCH_7EM__ -D__SES_ARM > -D__ARM_ARCH_FPV4_SP_D16__ -D__SES_VERSION=35000 -DARM_MATH_CM4 > -D__FPU_PRESENT -DDEBUG "-D DEBUG_NRF" -DCONFIG_GPIO_AS_PINRESET > -DINITIALIZE_USER_SECTIONS -DNO_VTOR_CONFIG -DNRF52 -DNRF52832_XXAA > -DCONFIG_NFCT_PINS_AS_GPIOS -DOPUS_BUILD -DVAR_ARRAYS -DFIXED_POINT > -DDISABLE_FLOAT_API -DCUSTOM_MODES "-DARM_MATH_CM4 " > "-DOPUS_ARM_INLINE_EDSP " -DDOPUS_ARM_INLINE_ASM -DOPUS_ARM_INLINE_MEDIA > -MD > > > > Any ideas? > > > > Regards > > Peter > _______________________________________________ > opus mailing list > opus at xiph.org > http://lists.xiph.org/mailman/listinfo/opus >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/opus/attachments/20190219/e1e35ad5/attachment.html>
---------- Forwarded message --------- From: Peter Svensson <petersvenss85 at gmail.com> Date: tis 19 feb. 2019 kl 20:43 Subject: Re: [opus] Custom mode To: Emily Bowman Hi Emily ! Thank you for responding. I think my problem is not (yet) with OPUS itself. Encoding at complexity 0 takes 1.6ms ( 4.342ms at complexity 10 !) and decoding takes 1.9ms. 3.5ms, out of my 4.096ms budget, is simply too much for the uncompressed audio stream to stay sound (no pun intended). Alas the distortion and aliasing experienced. If either encoder, or decoder, is removed everything sounds fine again. Now, encoding and decoding in the same device does, obviously, not make any sense. The whole idea is to pass the compressed audio via a radio link to another device. Since I have this link up and running I will try to split encoding/decoding onto two devices instead. This way I will hopefully have a chance to try the actual encoding/decoding quality. It just seamed a lot easier to experiment with loop-back on a single device... Question is if this is a dead end. Some Googling suggests that others have succeded in using Opus on a Cortex M4 device. It might, however, not be for realtime streaming. I hope to be able to transfer voice with a "decent" quality and need a minimum of 50% data compression. Can it be be done without a major optimization effort ? Given my current OPUS setting and compiler optimization flags; is there a way to ease the CPU requirements even further ? Is there any document explaining the purpose/implication of each define, e.g. OPUS_ARM_INLINE_MEDIA ??? Fixed point vs. float ? Many questions seek answers ... Thanks Peter Den tis 19 feb. 2019 kl 11:31 skrev Emily Bowman <silverbacknet at gmail.com>:> Don't use OPUS_SET_COMPLEXITY(0) unless you absolutely have to. That means > "give me all the distortions" so I wouldn't be surprised if that's where > those are coming from. If your encoding path can't handle full 10 > complexity, dial it down to 9, 8, etc until it runs ok. > > opus_custom_decode doesn't downsample from 48kHz, so your playback code > must still be treating it as if it's 15kHz, thus the pitch down. If you can > swing it, play as 48kHz, otherwise you have to insert another CPU-hogging > resampler. If you do need it and don't have a resampler handy, Opus has > silk_resampler_init & silk_resampler available for you. > > Emily Bowman > > > > On Sun, Feb 17, 2019 at 12:47 PM Peter Svensson <petersvenss85 at gmail.com> > wrote: > >> Hi all ! >> >> >> >> If someone could give me a hint on how to proceed with the following i'd >> be very happy: >> >> >> >> I have a test setup on an nrf52832 (Cortex M4) in which I receive audio >> from a PDM microphone (64 sample frame) and pass it directly to an I2S >> device i.e. from ISR to ISR. With uncompressed audio this works just fine. >> >> >> >> Now I try to insert OPUS1.3 in the path but cannot make it work. The >> audio passes through but ends up “down pitched” and heavily distorted. >> >> >> >> Due to the 32MHz base clock (+ prescalers) available it is not possible >> to get the desired 16kHz sample rate. I am therefore forced to work with >> 15625Hz. >> >> >> >> As I understand it the opus encoder does not mind a strange sample rate >> as it will re-sample it to 48kHz (Correct?). The decoder, however, will >> always output 48kHz, 24kHz, 16kHz etc (Correct?) which is incompatible with >> my 15625Hz rate. (However; If my assumptions are correct there could be a >> possibility to make my I2S device master and, thus, be able to pull the >> data at 16kHz) >> >> >> >> Consequently I have been looking at “opus custom” as a solution but even >> with a (seemingly) low complexity it does not yield the desired result. >> >> >> >> *Setup code:* >> >> >> >> mode = opus_custom_mode_create(15625, 64, &err); >> >> >> >> size = opus_custom_encoder_get_size(mode, 1); >> >> enc = malloc(size); >> >> err = opus_custom_encoder_init(enc, mode, 1); >> >> err = opus_custom_encoder_ctl(enc, OPUS_SET_BITRATE(16000)); >> >> err = opus_custom_encoder_ctl(enc, OPUS_SET_COMPLEXITY(0)); >> >> >> >> size = opus_custom_decoder_get_size(mode, 1); >> >> dec = malloc(size); >> >> err = opus_custom_decoder_init(dec, mode, 1); >> >> >> >> *ISR code (called once every 4.096ms):* >> >> >> >> nbBytes = opus_custom_encode( >> >> enc, (opus_int16 *)lMicBuf, 64, codedMicBuf, 64); >> >> >> >> frame_size = opus_custom_decode( >> >> dec, codedMicBuf, 64, decodedMicBuf, 64); >> >> >> >> >> >> *Build:* >> >> >> >> C:\Program Files\SEGGER\SEGGER Embedded Studio for ARM >> 3.50\gcc\arm-none-eabi\bin\cc1" -fmessage-length=0 >> -fno-diagnostics-show-caret -mcpu=cortex-m4 -mlittle-endian >> -mfloat-abi=hard -mfpu=fpv4-sp-d16 -mthumb -mtp=soft -nostdinc "-isystem >> >> >> >> -D__SIZEOF_WCHAR_T=4 -D__ARM_ARCH_7EM__ -D__SES_ARM >> -D__ARM_ARCH_FPV4_SP_D16__ -D__SES_VERSION=35000 -DARM_MATH_CM4 >> -D__FPU_PRESENT -DDEBUG "-D DEBUG_NRF" -DCONFIG_GPIO_AS_PINRESET >> -DINITIALIZE_USER_SECTIONS -DNO_VTOR_CONFIG -DNRF52 -DNRF52832_XXAA >> -DCONFIG_NFCT_PINS_AS_GPIOS -DOPUS_BUILD -DVAR_ARRAYS -DFIXED_POINT >> -DDISABLE_FLOAT_API -DCUSTOM_MODES "-DARM_MATH_CM4 " >> "-DOPUS_ARM_INLINE_EDSP " -DDOPUS_ARM_INLINE_ASM -DOPUS_ARM_INLINE_MEDIA >> -MD >> >> >> >> Any ideas? >> >> >> >> Regards >> >> Peter >> _______________________________________________ >> opus mailing list >> opus at xiph.org >> http://lists.xiph.org/mailman/listinfo/opus >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/opus/attachments/20190220/5283a83e/attachment.html>