----- Original Message ----- From: "Ian Malone" <ibmalone@gmail.com> To: <ogg-dev@xiph.org> Sent: Friday, March 10, 2006 8:42 AM Subject: Re: [ogg-dev] oggfile, skeleton and vorbis tools> Re-reading Monty's email: when you talk about vorbis-only vorbisfile and > the concurrent vorbis stream case do you mean vorbisfile will need to be > able to choose? I'd been thinking (as above) about cases where there is > a Vorbis stream in with other stuff, the first two examples are > realities. The obvious applications for multiple concurrent Vorbis > streams are the same as for DVD: multiple languages, commentaries, > scores (possibly needing mixing). Deciding what to play needs > application involvement; how much should vorbisfile be able to do, and > how much should wait until oggfile (and how much should be left to media > players that understand all the complexities involved)? >Re: multiple audio tracks, there are quite a few ogm files out there with multiple vorbis streams, typically the default language is first. So if you made vorbisfile drop unknown streams, these are potential files people might try to play with it. The other case, i have a few such files for testing, is two audio streams designed to play concurrently. For example a dj type scenario... intentionally playing two files over the top of each other. An even more useful case is for example an online stream, where the music is in one stream, and the dj/announcer can voiceover in the other stream... i've got a few test files like this also. Vorbis+Vorbis and Speex+Vorbis examples. I think somewhere i've got a file that plays four songs simultaneously, even though this file is not particularly useful, it was just a good test to see how my directshow filters would handle it. Another scenario where this would be useful might be having different instruments in each track. Obviously neither vorbis file nor a player can tell whether the file author intended for the streams to play at the same time or be alternates... or even more complicated would be groups that are alternates, ie either play these two together, other wise play these two together... or always play this one with the music, and choose one from voiceover streams to play concurrently with the music. VLC for example will let you choose an audio stream, however there is no option for playing them at the same time. This is something we might want to extend skeleton to support, i know we've talked about this kind of thing before. So that skeleton can provide some kind of mapping as to how the various streams relate to each other. Cheers, Zen.
On Fri, Mar 10, 2006 at 08:56:20AM +0800, illiminable wrote:> Obviously neither vorbis file nor a player can tell whether the file author > intended for the streams to play at the same time or be alternates... or > even more complicated would be groups that are alternates, ie either play > these two together, other wise play these two together... or always play > this one with the music, and choose one from voiceover streams to play > concurrently with the music.Note that this is exactly the problem with have with distinguishing multiple video streams and overlays, so this is something I would like to see skeleton solve. -r
illiminable wrote:> > ----- Original Message ----- From: "Ian Malone" <ibmalone@gmail.com> > To: <ogg-dev@xiph.org> > Sent: Friday, March 10, 2006 8:42 AM > Subject: Re: [ogg-dev] oggfile, skeleton and vorbis tools > > >> Re-reading Monty's email: when you talk about vorbis-only vorbisfile and >> the concurrent vorbis stream case do you mean vorbisfile will need to be >> able to choose? I'd been thinking (as above) about cases where there is >> a Vorbis stream in with other stuff, the first two examples are >> realities. The obvious applications for multiple concurrent Vorbis >> streams are the same as for DVD: multiple languages, commentaries, >> scores (possibly needing mixing). Deciding what to play needs >> application involvement; how much should vorbisfile be able to do, and >> how much should wait until oggfile (and how much should be left to media >> players that understand all the complexities involved)? >> > > Re: multiple audio tracks, there are quite a few ogm files out there > with multiple vorbis streams, typically the default language is first. > So if you made vorbisfile drop unknown streams, these are potential > files people might try to play with it. >I didn't realise there were files in the wild like this, that's interesting. So the question is, what should vorbisfile be expected to do on encountering one? The motivation is that players based on vorbisfile will continue happily playing Oggs with annotation streams, so you can start confidently rolling out things like skeleton in the default tools without breaking players. Once you have multiple concurrent audio streams you've reached a much more complex situation (my understanding: we've got an audio player looking through content; if it finds one and only one audio stream we know what to do, regardless of what else is there, if it finds more than one we are here...).> The other case, i have a few such files for testing, is two audio > streams designed to play concurrently. For example a dj type scenario... > intentionally playing two files over the top of each other. An even more > useful case is for example an online stream, where the music is in one > stream, and the dj/announcer can voiceover in the other stream... i've > got a few test files like this also. Vorbis+Vorbis and Speex+Vorbis > examples. I think somewhere i've got a file that plays four songs > simultaneously, even though this file is not particularly useful, it was > just a good test to see how my directshow filters would handle it. > Another scenario where this would be useful might be having different > instruments in each track. >Actually, this is kind of what I meant by 'possibly needing mixing', unlike a DVD which stores the soundtracks seperately you could have the score as one stream, the dialogue (in multiple languages) in another stream (similar to your dj voiceover example). Put a third commentary stream in and now you might want to dynamically change the mixing. You'd need CMML to do that as well as skeleton (I'm guessing you could do this in CMML somehow). (I'm not so sure about the seperate tracks per instrument thing though, if you're mastering wouldn't you want a lossless format? Maybe the Flamming Lips might use it eventually.)> Obviously neither vorbis file nor a player can tell whether the file > author intended for the streams to play at the same time or be > alternates... or even more complicated would be groups that are > alternates, ie either play these two together, other wise play these two > together... or always play this one with the music, and choose one from > voiceover streams to play concurrently with the music. > > VLC for example will let you choose an audio stream, however there is no > option for playing them at the same time. > > This is something we might want to extend skeleton to support, i know > we've talked about this kind of thing before. So that skeleton can > provide some kind of mapping as to how the various streams relate to > each other. >You need player support to be able to collect the options and present them to the user. Without skeleton and something like a fully capable oggfile you can't do it, as you've pointed out you can mix Speex and Vorbis. (I recently, accidently, had a file which contained mp3 and Vorbis. Obviously no Xiph tool will ever handle or produce that.) So, with the aim that it's intended for simple audio only players (probably ones already using it, or operating with limited resources), what should vorbisfile be capable of? -- imalone
----- Original Message ----- From: "Ian Malone" <ibmalone@gmail.com> To: "ogg-dev" <ogg-dev@xiph.org> Sent: Friday, March 10, 2006 5:24 PM Subject: Re: [ogg-dev] oggfile, skeleton and vorbis tools>> This is something we might want to extend skeleton to support, i know >> we've talked about this kind of thing before. So that skeleton can >> provide some kind of mapping as to how the various streams relate to each >> other. >> > > You need player support to be able to collect the options and present > them to the user. Without skeleton and something like a fully capable > oggfile you can't do it, as you've pointed out you can mix Speex and > Vorbis. (I recently, accidently, had a file which contained mp3 and > Vorbis. Obviously no Xiph tool will ever handle or produce that.) So, > with the aim that it's intended for simple audio only players (probably > ones already using it, or operating with limited resources), what > should vorbisfile be capable of?Certainly player support is needed, and there needs to be someway to get information about track correlation, probably skeleton. From my projects perspective, oggfile is not really required. Between directshow's framework and my filters, i've already implemented all the capability oggfile might offer, and i'd say the same goes for gstreamer and vlc. I believe my directshow filters would be fine with ogm/mp3+vorbis and i'm fairly sure vlc would be fine with it. My filters default operation is to mix all streams and play simultaneously, though other directshow filters (ie morgan stream switcher) can provide stream switching (1 of n), though i will eventually build that in. VLC on the other hand defaults to play one and let you choose 1 of n. Also, as of 0.70 the demuxer and codecs are completely decoupled. In other words, it's possible for a third party to add a new stream, implement the required interfaces and distribute a directshow decode filter, and it will just work without any changes needed from my end. I'd say at this stage, it's probably fine for vorbisfile to dump anything non-vorbis, and play only the first vorbis stream it sees, ignoring the rest. Making it much more complicated than that you might as well implement a fully functional oggfile. Zen.