theora dev - Feb 2004 - Ogg mux design

Monty,

Thanks for writing up your thoughts on the mux design 
(ogg/doc/ogg-multiplex.html in cvs) Now that there's something to argue 
with, I'd like to comment.

To recap, the documented proposal is that we make two categories of 
streams within the OggFile multiplexing library. Pages are sorted 
chronologically by the timestamp equivalent of their granulepos fields. 
Normal data like audio and video are thought of as 'continuous' and use 
the current spec of setting the page granulepos to correspond to the 
timestamp of the last data element decodable in that page. Other, sparse 
data like captions are thought of as 'discontinous' and use a different 
convention: the granulpos corresponds to the timestamp of the *first* 
data element decodable in that page. These rules arrange the stream so 
that the discontinous pages "fall out" of the stream in time to be 
useful without necessiting excessive buffering of higher-bitrate data 
as would be the case if they were also sorted by end time.

What Jack and I came up with originally was slightly different. All 
pages are marked by end time and there is no distinction between stream 
types. Pages are still sorted chronologically. The difference is that 
*all* pages are sorted by their start time, which is to say by the 
granulepos of the previous page in that logical bitstream.

I believe this handles the requirements with about equal effectiveness, 
and I like it better conceptually, else I wouldn't bring it up. By 
treating all the streams the same we have less to think about (though 
seeking gets even more complicated). The mux logic in OggFile is 
probably equivilent code size, but there's one less flag to pass across 
the interface and it looks simpler from the point of view of the codec 
glue, which is the part more people have to deal with. Also, we don't 
have to change any part of Ogg that's already been published.

Discontinuous streams are sorted exactly the same; what's treated 
differently are the continuous streams. In the case of an a/v stream, 
the larger video packets will come before the smaller but more numerous 
audio packets that need to be played in the same timespan. You have to 
buffer both during that frame anyway; which wants a longer latency is 
really something only the playback app knows, so this doesn't make much 
difference in terms of buffering.

Am I missing anything?

Finally, I'm not convinced the continuous/discontinuous classification 
is all that helpful in the end. The real issue is that we're 
multiplexing time-linear streams with wildly-different bitrates. That's 
the real difference between a caption and a video bitstream, and the 
issue we're trying to solve is not having to buffer the high-bitrate 
stream while waiting for a page from the low-bitrate one. It's really a 
matter of degree. The issue's obvious with captions because the bitrate 
really can be tiny on the multimedia scale, but captions vs 'web video' 
bitrate ratio isn't all that different from the audio vs video tracks in 
uncompressed hd.

 -r
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'theora-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

On Mon, Feb 16, 2004 at 04:25:43PM -0800, Ralph Giles wrote:
> What Jack and I came up with originally was slightly different. All 
> pages are marked by end time and there is no distinction between stream 
> types. Pages are still sorted chronologically. The difference is that 
> *all* pages are sorted by their start time, which is to say by the 
> granulepos of the previous page in that logical bitstream.
Yes, I remember this.  Unfortunately it makes some repagination
operations impossible, as you can't as easily reconstruct granule
positions going backwards.  You have a granpos for every packet in
encode; you may not be able to regenerate that later.
> I believe this handles the requirements with about equal effectiveness, 
> and I like it better conceptually, else I wouldn't bring it up.
Conceptually, working by starting time is cleaner all around.
However, it's not entirely possible when granpos marks end time of a
page.
> Discontinuous streams are sorted exactly the same; what's treated 
> differently are the continuous streams. In the case of an a/v stream, 
> the larger video packets will come before the smaller but more numerous 
> audio packets that need to be played in the same timespan. You have to 
> buffer both during that frame anyway; which wants a longer latency is 
> really something only the playback app knows, so this doesn't make much
> difference in terms of buffering.
the continuous streams aren't actually sorted differently if you come
to think of granpos as defining the time between two pages, not the
time of a page (or packet).  The only difference is where the granpos
is stored.
> Finally, I'm not convinced the continuous/discontinuous classification 
> is all that helpful in the end. 
It is; you get buffer regulation for free if you just lean on the
continuous streams and let it fall out.  The real difference between
continuous an discontinuous really matters only to buffer regulation.  

It's not a 'big vs. small' issue really.

Monty
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'theora-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.