opus - Jul 2013 - Encoder state management - 'chunked' Opus?

Hi,

I'm playing around using Opus in a 'chunked' streaming context,
where
chunks of media are served in separate HTTP responses.

I am trying to hunt down the source of some clicks-and-pops during 
playback, and while it is very likely that these glitches are due to the 
low quality of my code, I wanted to ask if the admonition in the API 
docs[1] that "encoder state *must* *not* be re-initialized for each 
frame" might be flagging anything more than simple inefficiency?

My code does not recreate encoder state per-frame, but it does create 
new state at the start of each 'chunk' (where a chunk is 1 second of 
consecutive Opus frames, right now).  Is this an unsupported use of the 
codec (and therefore a potential source of my issue), or should this 
have little impact on the output other than reduced efficiency?


ta,
dave


[1] 
http://www.opus-codec.org/docs/html_api-1.0.2/group__opus__encoder.html#details


-- 
http://david.holroyd.me.uk/

David Holroyd wrote:
> My code does not recreate encoder state per-frame, but it does create
> new state at the start of each 'chunk' (where a chunk is 1 second
of
> consecutive Opus frames, right now).  Is this an unsupported use of the
> codec (and therefore a potential source of my issue), or should this
> have little impact on the output other than reduced efficiency?
This does sound like the source of your issue. If you don't reinitialize 
the decoder at the start of each chunk, then you'll decode extra padding 
samples at the start of each one (and lose the corresponding samples 
from the end of the previous chunk).

If you're putting these frames in, e.g., Ogg, then you can start a new 
chain for each chunk, which will indicate to the decoder that it needs 
to reset between them. Make sure you're properly setting the preskip 
value in the header to get it to skip the padding samples at the start, 
and encoding at least that many samples extra to flush the buffered 
samples at the end.

That will ensure you're at least encoding and decoding the right samples 
(at some efficiency cost). However, because of the lossy nature of the 
compression, there may still be small artifacts when switching from one 
chunk to another. See 
<http://www.ietf.org/mail-archive/web/codec/current/msg02972.html> for 
some ideas that would eliminate this in the encoder. Unfortunately, 
these are not yet implemented, and require tracking state between chunks 
(which, if you could do that easily, presumably you wouldn't be 
resetting the encoder state for each one).

N.B., not all players handle chained streams very well (e.g., Firefox 
treats them as unseekable).

On 22/07/13 15:55, Timothy B. Terriberry wrote:
> David Holroyd wrote:
>> My code does not recreate encoder state per-frame, but it does create
>> new state at the start of each 'chunk'
> This does sound like the source of your issue. If you don't
reinitialize
> the decoder at the start of each chunk, then you'll decode extra
padding
> samples at the start of each one (and lose the corresponding samples
> from the end of the previous chunk).
Right.  I will try resetting both encoder and decoder.
>> If you're putting these frames in, e.g., Ogg, then you can start a
new
>> chain for each chunk, which will indicate to the decoder that it needs
>> to reset between them. Make sure you're properly setting the
preskip
>> value in the header to get it to skip the padding samples at the start,
>> and encoding at least that many samples extra to flush the buffered
>> samples at the end.
I didn't bother with a proper container, thinking I could get an 
implementation up and running faster by just shipping opus frames with 
length headers.  This is not how to do things in a 'real' implementation
of course :-D

Would padding need to be handled 'by hand' in my case, or is resetting 
the decoder enough?  Clearly I would take a proper look at the opus_demo 
code.

If I've missed out a lot of considerations that ogg already handles, 
then I might need to just hurry up and start generating lots of 1 second 
.ogg files!

>> [..link to ideas..]. Unfortunately,
>> these are not yet implemented, and require tracking state between
chunks
>> (which, if you could do that easily, presumably you wouldn't be
>> resetting the encoder state for each one).
Your assumption is correct.  No encoder state (outside of a given 
'chunk' of work) is one of my design goals.
>> N.B., not all players handle chained streams very well (e.g., Firefox
>> treats them as unseekable).
I had assumed that Firefox would not provide me with a way of retrieving 
a chunked Opus stream anyway.  Maybe this becomes a possibility once 
Media Source extension support lands?  In the meantime, I stream using 
XMLHttpRequest, decode using a javascript libopus port[1] and use the 
Web Audio API in Chrome for playback.

Of course, if something out there already does chunked opus so that I 
don't need to hand craft it, I would appreciate pointers!


Many thanks for all your help,
dave


[1] https://github.com/kazuki/opus.js-sample/

-- 
http://david.holroyd.me.uk/