opus - Aug 2013 - preskip and seeking suing Opus

Hi,

I've been studying the Opus code and documentation for a while and have
seen it mentioned several times that Opus uses pre-skip to allow the codec
to converge.  What convergence are they referring to?   Rate control?
Energy envelope prediction after seeking?
Is there a discussion of this somewhere?

Also, I'm also interested in the seeking behaviour of the decoder.
If, say, I want to skip ahead in an Opus bitstream, can I start decoding at
any intra frame without artifacts?

Thanks!
-Bob Estes
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/opus/attachments/20130814/e933c2ee/attachment.htm

On 13-08-14 10:09 PM, Bob Estes wrote:
> I've been studying the Opus code and documentation for a while and have
> seen it mentioned several times that Opus uses pre-skip to allow the
> codec to converge.  What convergence are they referring to?   Rate
> control?  Energy envelope prediction after seeking?
Not rate control, but there are a number of predictors running in the
decoder which need some amount of data to converge with their expected
values when the encoder and decoder would be out of sync for whatever
reason.

Pre-skip can be used by a muxer to describe how much data must be run
through the decoder before valid output is obtained at the start of a
file. This could be e.g. 80 ms if the muxer is creating a file from the
middle of another one (lossless editing, webrtc stream recording).

More commonly it's used to account for resampler and algorithmic delay
so the output can be phase-aligned with the input.
> Also, I'm also interested in the seeking behaviour of the decoder.
> If, say, I want to skip ahead in an Opus bitstream, can I start decoding
> at any intra frame without artifacts?
You must start feeding data to the decoder about 80 ms before the
intended seek point, and discard the output. This is for convergence as
described above, but for seeking you can use the general 80 ms rule as
described in the Ogg Opus draft. This is referred to as pre-roll, and
doesn't need to be specially specified by the muxer like pre-skip since
it's constant and applies to any seek point, not just the start of playback.

Does that help?

 -r

Bob Estes wrote:
> If, say, I want to skip ahead in an Opus bitstream, can I start decoding
> at any intra frame without artifacts?
Also, to be clear, there is no such thing as an intra frame in Opus. 
Even CELT mode with intra-coded band energy and no pitch filter has a 
pre-emphasis stage that uses state from the last frame.

Yes, that's a start.

Ultimately, though, I'm hoping to reduce the 80ms requirement, and am
trying to get a handle on what state in the decoder must converge and what
complications I might be up against.  I'm also only considering CELT-based
encodings if that simplifies things.

I know that Opus can do inter-frame energy envelope prediction, but that
dependency can be eliminated by using intra-frames.

There's also a dependency introduced by the MDCT overlap of CELT.  Without
the previous frame, this could be a major term in the convergence
requirement.

Other than that, I'm not sure what other dependencies I would need to
consider.

Is their a paper or some studies or discussions available about where the
80 ms comes from?
I'm a fairly new member to this group, but did scan the mailing list
archives for similar topics ...

Thanks again,
-Bob Estes



On Thu, Aug 15, 2013 at 5:30 AM, Ralph Giles <giles at thaumas.net> wrote:
> On 13-08-14 10:09 PM, Bob Estes wrote:
>
> > I've been studying the Opus code and documentation for a while and
have
> > seen it mentioned several times that Opus uses pre-skip to allow the
> > codec to converge.  What convergence are they referring to?   Rate
> > control?  Energy envelope prediction after seeking?
>
> Not rate control, but there are a number of predictors running in the
> decoder which need some amount of data to converge with their expected
> values when the encoder and decoder would be out of sync for whatever
> reason.
>
> Pre-skip can be used by a muxer to describe how much data must be run
> through the decoder before valid output is obtained at the start of a
> file. This could be e.g. 80 ms if the muxer is creating a file from the
> middle of another one (lossless editing, webrtc stream recording).
>
> More commonly it's used to account for resampler and algorithmic delay
> so the output can be phase-aligned with the input.
>
> > Also, I'm also interested in the seeking behaviour of the decoder.
> > If, say, I want to skip ahead in an Opus bitstream, can I start
decoding
> > at any intra frame without artifacts?
>
> You must start feeding data to the decoder about 80 ms before the
> intended seek point, and discard the output. This is for convergence as
> described above, but for seeking you can use the general 80 ms rule as
> described in the Ogg Opus draft. This is referred to as pre-roll, and
> doesn't need to be specially specified by the muxer like pre-skip since
> it's constant and applies to any seek point, not just the start of
> playback.
>
> Does that help?
>
>  -r
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/opus/attachments/20130815/651f98aa/attachment.htm