Vorbis dev - Oct 2002 - Carrying non-audio data in an Ogg/Vorbis I stream

Hello,

I would like to "piggy-back" other data in an Ogg/Vorbis I physical
stream
(i.e. file) in a manner that does not conflict with specifications and
existing decoders/players/editors etc.

As far as I can see, I have some possible choices:

1 - Chaining a logical bistream at the end. If this is the way to go, two
questions arise to begin with:

q1a - How to identify this stream in a future-proof way? Just invent a new
header, beginning with 'm' 'y' 'h' 'e'
'a' 'd' 'e' 'r' ?

q1b - I have not found any specifications on how a decoder should handle
this - obviously I want a silent ignore, not an error message or even worse,
a failure to decode. Is this defined somewhere?

2 - Inserting non-audio packets in the actual stream. This situation is
defined to be decoded as 'a non-audio packet when audio is expected
indicates stream corruption or a non-compliant stream. The decoder must
ignore the packet and not attempt decoding it to audio.'. The second part of
this is ok for my purpose, but I don't like the sound of the first part
'indicates stream corruption' etc. An overly ambitiuos decoder might
flag
this as an error for example.

3 - Another choice, which for various reasons feel less satisfying, is to
encode it as comments. The problem with this is first of all that it may be
attepmpted to be shown by decoders/players/editors, and second of all that
the data in question may be *large* (i.e. significantly larger than the
actual audio) - and it is unfortunate to have it at the front...

4 - Can I possibly just concatenate a completely separate Ogg stream at the
end of the first one? This seems to violate the constraints of an Ogg/Vorbis
I stream though.

In general, is there really no defined way to introduce data into an
Ogg/Vorbis I stream that is well-defined to be silently ignored by current
decoders, without this implying an error, non-compliance or whatever?

Ideas, comments - or just pointers to the appropriate specifications are
appreciated.

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

Hello Svante,

Monday, October 28, 2002, 4:46:28 PM, you wrote:
SS> I would like to "piggy-back" other data in an Ogg/Vorbis I
physical stream
SS> (i.e. file) in a manner that does not conflict with specifications and
SS> existing decoders/players/editors etc.
If you read the OGG framing:
http://www.xiph.org/ogg/vorbis/doc/oggstream.html
http://www.xiph.org/ogg/vorbis/doc/framing.html
You'll learn that Ogg is the octet stream framing spec and vorbis an
audio compression codec. Basically, in Ogg you put a series of packets
that contain any stream. And you can multiplex this streams. The only
constraint is that the ogg headers of each stream must be consecutive
and no new stream can begin until all streams have ended. So when you
concatenate any number of valid ogg files you get another valid ogg
file (which is breakable). So what you want is to define a different
ogg stream, a non vorbis stream, and multiplexit. If it's very little
data you could get awway with one or two pages, which may fit in just
two ogg pages.
Or you could actually make an ogg framed (whatever your data model is
named) and concatenate it _before_ the ogg vorbis. I'm not sure how
players would react to this, but may be usefull as a validating test.
The Theora project is working specifically on this infrastructure.You
should join their list.

SS> As far as I can see, I have some possible choices:

SS> 1 - Chaining a logical bistream at the end. If this is the way to
SS> go, two questions arise to begin with:
You could but is not the best alternative.
SS> q1a - How to identify this stream in a future-proof way? Just
SS> invent a new header, beginning with 'm' 'y' 'h'
'e' 'a' 'd' 'e'
SS> 'r' ?
Using a different ogg header. You should actually contact the Theo
Project.
SS> q1b - I have not found any specifications on how a decoder should
SS> handle this - obviously I want a silent ignore, not an error
SS> message or even worse, a failure to decode. Is this defined
SS> somewhere?
If it has a non vorbis header, it should be ignored by a vorbis player
(by specification).
SS> 2 - Inserting non-audio packets in the actual stream. This
SS> situation is defined to be decoded as 'a non-audio packet when
SS> audio is expected indicates stream corruption or a non-compliant
SS> stream. The decoder must ignore the packet and not attempt
SS> decoding it to audio.'. The second part of this is ok for my
SS> purpose, but I don't like the sound of the first part 'indicates
SS> stream corruption' etc. An overly ambitiuos decoder might flag
SS> this as an error for example.
No, niet, nicht, la, nunca.
SS> 3 - Another choice, which for various reasons feel less
SS> satisfying, is to encode it as comments. The problem with this is
SS> first of all that it may be attepmpted to be shown by
SS> decoders/players/editors, and second of all that the data in
SS> question may be *large* (i.e. significantly larger than the actual
SS> audio) - and it is unfortunate to have it at the front...
It's not the intended purpose of that field.
SS> 4 - Can I possibly just concatenate a completely separate Ogg
SS> stream at the end of the first one? This seems to violate the
SS> constraints of an Ogg/Vorbis I stream though.
No. Don't confuse vorbis and ogg. It may be a kind of solution, but
then if I concatenate variousvorbis files (like the tracks of a CD) it
may breack some stupid players. And if some of those files have your
data and some don't, you get a not very clear situation.

Regards,
Alejandro Belluscio

<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

> Hello,
>
> I would like to "piggy-back" other data in an Ogg/Vorbis I
physical stream
> (i.e. file) in a manner that does not conflict with specifications and
> existing decoders/players/editors etc.
>
> As far as I can see, I have some possible choices:
>
> 1 - Chaining a logical bistream at the end. If this is the way to go, two
> questions arise to begin with:
Probably not the way to go.
> q1a - How to identify this stream in a future-proof way? Just invent a new
> header, beginning with 'm' 'y' 'h' 'e'
'a' 'd' 'e' 'r' ?
'myheader' is a little bit too generic. You probably want to find a more
descriptive name.
> 2 - Inserting non-audio packets in the actual stream. This situation is
> defined to be decoded as 'a non-audio packet when audio is expected
> indicates stream corruption or a non-compliant stream. The decoder must
> ignore the packet and not attempt decoding it to audio.'. The second
part
of
> this is ok for my purpose, but I don't like the sound of the first part
> 'indicates stream corruption' etc. An overly ambitiuos decoder
might flag
> this as an error for example.
Correct. This is a bad idea.
> 3 - Another choice, which for various reasons feel less satisfying, is to
> encode it as comments. The problem with this is first of all that it may
be
> attepmpted to be shown by decoders/players/editors, and second of all that
> the data in question may be *large* (i.e. significantly larger than the
> actual audio) - and it is unfortunate to have it at the front...
<p>Correct again. This is also a bad idea.
> 4 - Can I possibly just concatenate a completely separate Ogg stream at
the
> end of the first one? This seems to violate the constraints of an
Ogg/Vorbis
> I stream though.
Sure. Multiple Ogg files can be chained by 'cat' (or the weird 'copy
/b + '
syntax on DOS/Windows). This is perfectly legal (and gapless).
> In general, is there really no defined way to introduce data into an
> Ogg/Vorbis I stream that is well-defined to be silently ignored by current
> decoders, without this implying an error, non-compliance or whatever?
>
> Ideas, comments - or just pointers to the appropriate specifications are
> appreciated.
You probably want to interleave your stream with audio, right? Take a look
at the ogmtools, oggmux, oggmerge and similar. They mostly seem to be
designed for a single purpose right now (audio+video+subtitles, or
vorbis+midi), but it shouldn't be too dificult to modify one of the above to
recognise your new custom stream.

Peter Harris

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

Mandag, 28 oktober 2002, skrev du:
>>> 1 - Chaining a logical bistream at the end. If this is the way 
>>> to go, two questions arise to begin with:
>>
>>Probably not the way to go.
>
>Ok...
>
>>> 4 - Can I possibly just concatenate a completely separate Ogg 
>>> at the end of the first one? This seems to violate the 
>>> constraints of an Ogg/Vorbis I stream though.
>>
>>Sure. Multiple Ogg files can be chained by 'cat'.
>>This is perfectly legal (and gapless).
But isn't concatenating separate ogg files exactly the same as chaining 
logical bit streams?

A problem I've encountered when splitting ogg files is, as you already 
mentioned, how to determine the content type of the logical stream.
An Ogg file contains no information about what exactly its logical 
streams are carrying, and a demultiplexer has to "peek" into the 
streams and run a sort of try-and-error loop to determine the stream's 
content type.

I am not familiar with other media APIs, but the Java Media Framework 
offers a plugin interface, where demultiplexing of streams and decoding 
of content is completely separated. This means that having an Ogg 
demultiplexer and an MPEG decoder would theoretically make the framework 
capable of playing back MPEG data carried by an Ogg stream, without 
the demultiplexer and decoder plugins being aware of eachother. Due 
to the lack og content description in the Ogg format, and the requirement 
in the JMF API that the demultiplexer must know the content type 
of it's resulting streams, it is not possible to implement a reasonable 
Ogg demultiplexer.

Tor

<p><p>==================================================================EASY
and FREE access to your email anywhere: http://Mailreader.com/
==================================================================
<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

Mandag, 28 oktober 2002, skrev du:
>I don't understand how per-packet metadata makes any difference here. 
>The demux happens at the Ogg level, and yes it has too peek at the 
>headers to see which codec to hand each stream to. How is that 
>different?
The JMF is a "pluggable" API, where demultiplexing and decoding is 
completely separated, and the different plugins are unaware of eachother.
A demultiplexer plugin must take an input stream and split it into 
any number of output streams, and it also has to determine the content 
type of each output stream, and if there are more output streams,
which are to be synchronized, each output stream has to know its 
current media time.

With the Ogg file format, this causes at least the following problems:

- Since the demux has to know the content type of each resulting 
output stream, the Ogg demux has to contain logic, to "peek" into 
the logical streams and try to determine the content type, although 
these steps are completely separated in the API, and the demux should 
not really be concerned about the actual content.

- The demux has to know quite a lot about the content format to be 
able to tell the current media time. In Ogg/Vorbis, the time for 
an audio packet is calculated using the granule position in the ogg 
framing, the sample rate and the number of PCM samples in each audio 
packet. Without parsing the Vorbis stream, the demux is not able 
to determine the sample rate and the number of samples in each packet.

<p><p>Tor

<p><p>==================================================================EASY
and FREE access to your email anywhere: http://Mailreader.com/
==================================================================
<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

Tirsdag, 29 oktober 2002, skrev du:
>I don't see why a demux plug-in must know what media is each stream.
>But I don't know jack, so never mind. 
The Java Media Framework uses the content type returned by the demux 
plugin to find a suitable decoder for the actual content stream. 
A data source is not necessarily seekable, so it is not possible 
for the framework to try all available decoders, to see if any will 
"fit".

Tor

<p><p>==================================================================EASY
and FREE access to your email anywhere: http://Mailreader.com/
==================================================================
<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

Tirsdag, 29 oktober 2002, skrev du:
>I don't quite understand. If the demux must return the content type,
>it must know about all data type, or the file format must include an
>explicit "type" field in the header of each stream. But then it
either
>must know all the mappings from "type" to "codec" or the
codec must
>know all it's magic numbers in different file formats. What am I
>missing?
Exaclty. It would have made sense in Ogg, if each logical stream 
would have some sort of header containing information about the content 
type. The WAV and AVI formats contain such information.

The JMF has a sort of registry, where all demux and codec plugins 
are registered. When playing back an Ogg file, the framework knows 
from the ending .ogg that the file has a content type of application/x-
ogg and searches for a demux plugin supporting this format. An Ogg 
demux plugin would examine the file and tell the framework something 
like "I've found one logical stream of type audio/x-vorbis". The 
framework will then look for a codec plugin supporting that input 
format, and if necessary even chain more codecs to produce an output 
format, which is supported by the last drain.

Tor

<p><p>==================================================================EASY
and FREE access to your email anywhere: http://Mailreader.com/
==================================================================
<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.