Vorbis dev - Nov 2011 - Multi-channel labels in Vorbis comments

Hi devs :-)

At the Mozilla Festival in London recently, I took part in a Hack the DJ
workshop, looking at ways to take digital DJ'ing to the next level:
http://www.bbc.co.uk/blogs/researchanddevelopment/2011/11/setting-a-dj-challenge-at-the.shtml

One of the ideas proposed was stem mixing, using multichannel files in
DJ applications. A proprietary implementation of this idea is Fireplayer
(http://itunes.apple.com/gb/app/fireplayer-remixer/id367791455?mt=8) but
this app is built with the intention that users will buy remixable
versions of (a very limited number of) well-known songs from an in-app
store.

We would like to work towards a new open standard for stem mixing,
something that is compatible with sharing our mixes on the open web -
legally, of course - but could also be used by record labels that sell
tracks to DJs.

For example, eight channel Ogg Vorbis files where the first two tracks
are a stereo mix of the drums, third and fourth stereo bass, fifth and
sixth stereo vocals, and seventh and eighth tracks everything else. This
means that you can mute or solo individual stems in the mix, giving you
the versatility of four-deck or eight-deck mixing but without the
problems of keeping many decks in sync.

It seems that Ogg Vorbis already has all the features we need in a stem
format for DJs, but there's the question of how we identify individual
channels within the file. There are conventions for 5.1 and 7.1 surround
mixes, of course, but we're not aware of any published convention for
stems. So it would be useful to have a text label for each channel saved
in a UTF-8 Vorbis comment which could be displayed in applications. I
was thinking of something like:

CHANNEL1=Drums L
CHANNEL2=Drums R
CHANNEL3=Bass L
CHANNEL4=Bass R
CHANNEL5=Vocals L
CHANNEL6=Vocals R
CHANNEL7=Synth L
CHANNEL8=Synth R

The labels could be edited for different languages, while preserving the
audio content - for example if you had a Soundcloud-type site for DJs in
Russia, you might would want to label the channels in Russian.

Is there any chance that a standard for labelling channels might make it
into the official Vorbis comment spec? Or would it be rejected as
'structured data'?

Cheers!

Daniel

On 16 November 2011 18:54, Daniel James <daniel at 64studio.com>
wrote:
> Hi devs :-)
>
> At the Mozilla Festival in London recently, I took part in a Hack the DJ
> workshop, looking at ways to take digital DJ'ing to the next level:
>
http://www.bbc.co.uk/blogs/researchanddevelopment/2011/11/setting-a-dj-challenge-at-the.shtml
>
> One of the ideas proposed was stem mixing, using multichannel files in
> DJ applications. A proprietary implementation of this idea is Fireplayer
> (http://itunes.apple.com/gb/app/fireplayer-remixer/id367791455?mt=8) but
> this app is built with the intention that users will buy remixable
> versions of (a very limited number of) well-known songs from an in-app
> store.
>
> We would like to work towards a new open standard for stem mixing,
> something that is compatible with sharing our mixes on the open web -
> legally, of course - but could also be used by record labels that sell
> tracks to DJs.
>
> For example, eight channel Ogg Vorbis files where the first two tracks
> are a stereo mix of the drums, third and fourth stereo bass, fifth and
> sixth stereo vocals, and seventh and eighth tracks everything else. This
> means that you can mute or solo individual stems in the mix, giving you
> the versatility of four-deck or eight-deck mixing but without the
> problems of keeping many decks in sync.
>
> It seems that Ogg Vorbis already has all the features we need in a stem
> format for DJs, but there's the question of how we identify individual
> channels within the file. There are conventions for 5.1 and 7.1 surround
> mixes, of course, but we're not aware of any published convention for
> stems. So it would be useful to have a text label for each channel saved
> in a UTF-8 Vorbis comment which could be displayed in applications. I
> was thinking of something like:
>
> CHANNEL1=Drums L
> CHANNEL2=Drums R
> CHANNEL3=Bass L
> CHANNEL4=Bass R
> CHANNEL5=Vocals L
> CHANNEL6=Vocals R
> CHANNEL7=Synth L
> CHANNEL8=Synth R
>
> The labels could be edited for different languages, while preserving the
> audio content - for example if you had a Soundcloud-type site for DJs in
> Russia, you might would want to label the channels in Russian.
>
> Is there any chance that a standard for labelling channels might make it
> into the official Vorbis comment spec? Or would it be rejected as
> 'structured data'?
Hi Daniel,

Sounds like a fun application :)

Rather than a single multi-channel Vorbis file, you could also use a
file with multiple Ogg tracks, each containing a stereo Vorbis stream.
Then you would only need to label each stream once without numbering
or re-assigning stereo channels:

PART="Bass"
PART_RU="???"

and existing panning, stereo compression etc. would continue to work
for each stream (or indeed, each stream could be surround if that's
your thing). Also it'd be straightforward to losslessly rip out any of
these streams into an independent file, and to merge streams without
re-encoding.

cheers,

Conrad.

On 17 November 2011 18:55, ogg.k.ogg.k at googlemail.com
<ogg.k.ogg.k at googlemail.com> wrote:
>> We did look at that, possibly using tar to wrap up four stereo Vorbis
>> files into a single package. That should reduce overall filesize,
>
> What Conrad meant is that you can include more than one single Vorbis
> stream in a single Ogg stream (most often seen whem you have a video
> with English audio, German audio, etc).
> Each such stream may be mono, stereo, or more, and you can have any
> reasonable number of Vorbis streams in there.
> So you end up with a single file, which you can easily preview in,
> say, VLC, by choosing the track you want from the audio menu. No need
> for an extra step such as tar.
> The advantage is that remuxing tools already know of such a step (eg,
> the oggz tools).
Yup. Also the stems would already be time-synchronized, ie. the whole
file would be streamable.

Conrad.

Hi Conrad,
> Yup. Also the stems would already be time-synchronized, ie. the whole
> file would be streamable.
It seems like your tool oggz-merge would do the job:

http://linux.die.net/man/1/oggz-merge

but having tried muxing a couple of .ogg files, neither vlc, audacity or
sweep could open the interleaved file and play back both stereo streams.
For instance, vlc reported:

[0x7f2bb0002038] main input error: ES_OUT_RESET_PCR called
[0x18f4338] ogg demux error: Broken Ogg stream (serialno) mismatch

Do you know of any playback app that supports the audio output of
oggz-merge?

Thanks!

Daniel

> but having tried muxing a couple of .ogg files, neither vlc, audacity or
> sweep could open the interleaved file and play back both stereo streams.
> For instance, vlc reported:
>
> [0x7f2bb0002038] main input error: ES_OUT_RESET_PCR called
> [0x18f4338] ogg demux error: Broken Ogg stream (serialno) mismatch
This sounds a lot like you tried to mux together streams that had identical
serialnos. oggz merge ought to moan about trying to do this, maybe it does
not. serialnos should be unique within a physical stream (eg, a file), so each
component stream should have a different one. Usually, an encoder uses a
random number, which makes collisions unlikely.
> Do you know of any playback app that supports the audio output of
> oggz-merge?
Most should. Output from oggz merge should not be much different from
the output of another program which writes already muxed audio.

Hi ogg.k,
>> Do you know of any playback app that supports the audio output of
>> oggz-merge?
> 
> Most should. Output from oggz merge should not be much different from
> the output of another program which writes already muxed audio.
I did wonder if having separate stereo streams inside an Ogg container
would make it easier to process them independently. But then we would
have the potential for the streams to get out of sync (for example if we
loop one stream, or if we use an effect which adds latency to only one
stream).

Cheers!

Daniel