[vorbis-dev] Ogg MIDI proposal

Ralph Giles
Sat, 25 Aug 2001 19:34:26 -0000

On Saturday, August 25, 2001, at 05:01 , Kenneth Arnold wrote:

> How about allowing multiple Format 0 streams (thus representing Format
> 1 and even (partially?) Format 2) in an Ogg stream, so long as the
> pages are small enough and spaced properly so everything is
> simultaneous (if the application requires it)?

Jack said the format 1 -> format 0 conversion could be done without loss of
generality, so I
generally agreed the '0 only' spec made sense. I'm not entirely
clear on whether this imposes a 16
'device' limitation for Ogg MIDI, or not, and how onerous that actually
is.

In theory yes, you should be able to group multiple midi streams. But they might
be alternate
tracks instead of tracks to be mixed in. Same would go for multiple vorbis
streams, and so on,
which brings us again to the metadata can of worms. See below. :)

>> Changes to Vorbis
>> -----------------
>> libvorbisfile will need to be modified to recognize substreams and
ignore
>> all but the first valid Vorbis stream.
>
> I think it's about time to get the full Ogg stream interleave format
> together. I have not seen the MNG work (Ralph, could you commit it to
> CVS?), but I suspect this has already addressed some of the issues

Not really. Most of this is still at the design stage.

A lot of what you mention is important work from the tools point of view; mostly
we've been
thinking at the spec level, which has something to do with why we missed your
concerns. At the ogg
and vorbis levels, we already have everything we need.

Figuring out how to properly interleave all these bitstream types is also *hard*
in a way that had
me assuming much of it belonged at the application level. And there will be so
many different
types. I do expect oggmerge will eventually have either plugins or a nice
interface for compiling
in support for various formats, but I haven't seen how we can have a unified
library-level
architecture that handles everything. Can you explain your idea?

> [...]
> * table of contents header
>   - to provide information about what codecs are used in the stream,
>     for player selection or desktop environments -- more of a
>    future-proofing than actually needed now

Right, I maintain we don't need this yet. The 'principle use first'
doctrine (why the vorbis
header must come first in an OggMIDI file) takes care of simple player selection
and mimetype
issues. Anything sophisticated enough to understand that this file has both
vorbis and midi data
can search the headers just like the players do. ogginfo could also be invoked
to implement this
for scripts.

My primary concern is that we don't have a complete requirements list for
this stream-description
metadata, and I don't want this 'future proofing' to turn into
baggage. Defining everything
implicitly, as in this proposal, avoids the issue in a way that is elegantly
compatible with
simpler implementations.

Requirements gathering could certainly be happening faster. The two main things
I wanted to see
happen is a video player implementation that supports all the overlay and
alternate track features
we've discussed (dvd done right) and some kind of support for structured
text data (my infamous
xml proposal). Audio, Video, and Text are the three pillars of our grand unified
theory of
multimedia, and I'd want us to have a experience and a plan for each of them
before we write this
stream description format.

Now, anyone could slap an ogg packet format around the experimental tarkin or
w3d codecs and start
playing with the video issues; that's ready anytime. Metadata I think needs
to be connected to an
external database, so more thinking should happen there. Or we could settle on
'musicbrainz is
good enough' for now.

> One final concern: will hardware players using libvorbis 1.0 or even
> rc1 be able to play back these future Ogg streams, or at least
> intelligently skip past what they don't know about and just play the
> audio?

Yes, this is a real problem. Current libvorbisfile WILL choke on anything but a
degenerate vorbis
stream and that's a bug! It will cause much grief if we don't fix it and
we must make clear that
handling multiplexed streams is part of the ogg vorbis spec. Jack mentioned this
in his post, and
that should be all the future proofing we need.

IMHO,
 -r

BTW, sorry about not mentioning the Ogg MNG code earlier. My normal mailhost is
down and the
announcement got queued.