theora dev - Feb 2007 - AW: How to do Theora playback efficiently ?

Hi Ralph,

thanks for your posting.

Yes, the standard example player in the theora distribution could also do it
when theora would not need so much time. I removed the frame dropping from
the example because it's based on some audio stuff under Linux that
isn't
available on Windows. Currently I've no frame dropping handling build in
because the first goal is a good raw performance of the whole loop.

If there is no frame dropping mechanism, I would expect that the audio/video
playback goes out of sync... but this is not the case. Instead the video
playback slows down and the audio playback has gaps (even with precached
decoded audiobuffers). Not only the Theora decoding and the YUV2RGB stuff
takes time and those seem to be not the main reason why audio gets stalled
(the vorbis decoder is in the same loop and it is called very frequently,
also when heavy work load on theora). The problem occures when
ogg_stream_pagein() is called. In case of the theora stream it has to be
called about 20-30 times successively... and this takes up to 30ms (for one
frame)!! During this time BOTH decoders are 20-30 times called but because
of no data stalled. 

So I don't understand why these ogg_stream_pagein() calls take so much
time... sometimes more than theora decoding.

Now I try to use a second thread which task is to store as many pages as
needed, to have at least one audio page ready. I think this is a good
attempt but currently it is not working right... because ogg_sync_pageout()
can only reserve memory for 2 pages... nothing about those things are in the
ogg documentation, that's very frustrating.
Storing packets makes so sense in my opinion... because if there are no new
vorbis pages (only all those theora pages) you don't get any new packets and
so run out of data.

Thanks for the hint related to the liboggplay API.
But did you read it's source code? Please tell me, why there you can't
find
any of these functions or states we are talking about. There is no
ogg_sync_state, no ogg_stream_state, no pagein, no pageout, no packeout,
nothing at all besides vorbis/theora decode calls. So why the developer is
working with his own functions only? 
I think it's because most of the ogg sync/stream handling functions are crap
or at least not usable in efficient real world applications... and their
documentation is even worse. 
This is what I mean when I say, there's a lot of hidden stuff where it's
not
clear what actually happens.
>> I think, an ideal solution are two "pointers" in seperate
threads reading
>> independently the physical source stream and grab their pages (and
>> prepare/decode them) but ignore the other ones and don't need to
wait. So
I
>> need two ogg_sync_states or something like that? 
> Yes, you'd need two ogg_sync_states that way, and you'd just drop
any
> pages you weren't interested in inside a given thread.
I hope this is not the only way to work with the given ogg_stream/ogg_sync
functions.

Regards,
Robin.

Hi Robin,

I'm the liboggplay developer :)

The only reason I don't use any of libogg's function calls is that
liboggplay is based around a very nice little library called liboggz:

http://www.annodex.net/software/liboggz/index.html for documentation
http://svn.annodex.net/liboggz/trunk/ for code

Liboggz abstracts away a lot of the low-level manipulation you need to
do with libogg, but it is based entirely on top of libogg.

btw, liboggplay will definitely be running on Windows Real Soon Now -
in fact one of our developers at work has just started to look at
porting it across (and it shouldn't be a big job).

Cheers,
    -Shane Stephens

On 2/16/07, Robin Siegemund <r.siegemund@digitalpublishing.de>
wrote:
> Hi Ralph,
>
> thanks for your posting.
>
> Yes, the standard example player in the theora distribution could also do
it
> when theora would not need so much time. I removed the frame dropping from
> the example because it's based on some audio stuff under Linux that
isn't
> available on Windows. Currently I've no frame dropping handling build
in
> because the first goal is a good raw performance of the whole loop.
>
> If there is no frame dropping mechanism, I would expect that the
audio/video
> playback goes out of sync... but this is not the case. Instead the video
> playback slows down and the audio playback has gaps (even with precached
> decoded audiobuffers). Not only the Theora decoding and the YUV2RGB stuff
> takes time and those seem to be not the main reason why audio gets stalled
> (the vorbis decoder is in the same loop and it is called very frequently,
> also when heavy work load on theora). The problem occures when
> ogg_stream_pagein() is called. In case of the theora stream it has to be
> called about 20-30 times successively... and this takes up to 30ms (for one
> frame)!! During this time BOTH decoders are 20-30 times called but because
> of no data stalled.
>
> So I don't understand why these ogg_stream_pagein() calls take so much
> time... sometimes more than theora decoding.
>
> Now I try to use a second thread which task is to store as many pages as
> needed, to have at least one audio page ready. I think this is a good
> attempt but currently it is not working right... because ogg_sync_pageout()
> can only reserve memory for 2 pages... nothing about those things are in
the
> ogg documentation, that's very frustrating.
> Storing packets makes so sense in my opinion... because if there are no new
> vorbis pages (only all those theora pages) you don't get any new
packets and
> so run out of data.
>
> Thanks for the hint related to the liboggplay API.
> But did you read it's source code? Please tell me, why there you
can't find
> any of these functions or states we are talking about. There is no
> ogg_sync_state, no ogg_stream_state, no pagein, no pageout, no packeout,
> nothing at all besides vorbis/theora decode calls. So why the developer is
> working with his own functions only?
> I think it's because most of the ogg sync/stream handling functions are
crap
> or at least not usable in efficient real world applications... and their
> documentation is even worse.
> This is what I mean when I say, there's a lot of hidden stuff where
it's not
> clear what actually happens.
>
> >> I think, an ideal solution are two "pointers" in
seperate threads reading
> >> independently the physical source stream and grab their pages (and
> >> prepare/decode them) but ignore the other ones and don't need
to wait. So
> I
> >> need two ogg_sync_states or something like that?
>
> > Yes, you'd need two ogg_sync_states that way, and you'd just
drop any
> > pages you weren't interested in inside a given thread.
>
> I hope this is not the only way to work with the given ogg_stream/ogg_sync
> functions.
>
> Regards,
> Robin.
> _______________________________________________
> theora-dev mailing list
> theora-dev@xiph.org
> http://lists.xiph.org/mailman/listinfo/theora-dev
>