Ivo Emanuel Gonçalves
2007-Jan-15 11:24 UTC
[Flac-dev] Proposal: An extension to rules all others
Proposal: An extension to rules all others Copyright Notice None. Public Domain. Abstract Extension flamewars surrounding media formats inside Ogg containers are a pain in the behind, cause problems in the adoption of said formats, and confuse users. This modest proposal directed at the maintainers of Vorbis, Theora, Speex and FLAC will suggest a solution to solve these issues (and others). 1. Introduction According to Lore, there in ancient days was Ogg, a container for all kinds of media formats, but for years to come, Ogg had only one companion, a music box known as Vorbis. Together, the two tried to battle the evil kingdom of MP3, and though not successful, songs were composed of Ogg and Vorbis feats. However, misunderstanding arouse, and the two became associated as one entity: the OGG. Now, fast-forward to the present. The "OGG" misconception states that any file with the .ogg extension is "OGG", which is some kind of nerds-only format for music, or an MP3-wannabe. Now, we of Xiph know this is bullshit, but this kind of mentality has spread so widely that it's hurting the Ogg project as a whole, and its family of formats. 2. The Problems Caused by the OGG disease When one speaks with the average computer user, the one that may occasionally check Slashdot (or God forbid, Digg), s/he will tell you in the world of Open Source (or FOSS) there are two formats for audio: OGG and FLAC. More or less, s/he has a basic idea of what OGG is: it's some kind of audio format like MP3, whose name sounds like caveman speech, and nobody uses. S/he probably likes FLAC, though. The FLAC fad is yet to come, but its day seems to draw near. There isn't one problem here; there's several. The person described above may be a blogger, but he has no idea what Speex is. The person above may have an iPod, and s/he won't care much for any audio format other than MP3. S/he won't bother to look for a hardware player with support for other audio formats, and more importantly, s/he won't even bother consider nagging Apple about adding support for other formats. If this person has ever gone to Wikipedia, s/he may have come across media files with an .ogg extension, but s/he has no idea why it's labeled as video. "Isn't OGG supposed to be that nerds-only music format? What's this all about?" This person will try to make the video work, following the useful instructions on Wikipedia Media Help page to install illi's codecs, and so on. The problem is when s/he tries to play it on any MS Windows player that isn't VLC or RealPlayer. S/he'll be royaly screwed. What with the majority of the few programs that recognize the .ogg extension assuming it's Vorbis. At this point, the person will simply give up. The video wasn't that important to being with. However, the person may think, "Hey, other people will have the same problem too", so s/he will go ahead and complain to Wikipedia about it. Now make that several people a week, and not only in Wikipedia, but anywhere else hosting Theora videos. And remember that it's only the peopler that care. Like 1% of the Internet population, maybe less. Nobody else cares. I think there's a lack of consistency and correctness in how the different developers in Xiph are doing things. We need some unity if we are ever to take on the world. Not that I'm saying I'm the Messiah, and I have the answer for everything, or that I know of all the problems there are (or not). But on this issue, however, I hope to have your full attention, gentlemen. 3. One extension to rule them all This title is misleading, because it's not possible to have one single extension to fix the problems described above. Especially, if one considers this proposal aims for backward AND forward compatibility. The proposal isn't perfect, but in my opinion it's a fine step towards solving some of the problems plaging the projects surrounding Ogg. I propose that each project (Vorbis, Speex, etc.) states in its specification two file extensions, and those two extensions are to be the only ones allowed for that given format. Implementators are supposed to support both extensions. Content creators are supposed to choose one of them according to whatever criteria they care about. Are there arguments against this? Of course there are. One might say that .avi is always .avi even though it might be MPEG 1, XviD, DivX, or anything else. Yet, you forget that .avi is ALWAYS for video, so most software won't complain when they see an .avi file (except for WMP that complains for everything). The worst that may happen is for the software to ask the user for the appropriate codecs. This doesn't happen with .ogg, though, because the Ogg container might be used for so many different purposes that programs don't have an uniform way of dealing with .ogg. Matroska, for instance, specifies .mkv for video/video+audio, and .mka for audio. This may not be the greatest approach ever, but it works in real life, and that's what we should aim for. And yet, there's further opposition? Some may say "extensions don't matter", but they forget this is a computer world ruled by MS Windows, where the average user is not smart enough to know what's wrong when the video or audio file s/he downloaded won't work. Extensions are needed. Anyone ever saw the UNIX "file" command trying to guess what type a file is? Extensions are needed. Now, assuming you guys dig what I say, there's another problem. Choose a new extension for your project, say .vorbis for Vorbis and we all know what happens. Existing software will stop playing older Vorbis streams using the .ogg extension. The solution as I mentioned above is to allow two extensions, no more, nor less. Example: .ogg as default, .vorbis as secondary. OK, but that's for Vorbis. Why should FLAC or Speex care? Because of consistency. If we want to move Ogg away from the misunderstanding that it's Vorbis only (the "OGG"), we have to allow .ogg to be used on FLAC and Speex files. And because of correctness. If we want unity between our projects, we have to allow .ogg to be used on FLAC and Speex files. So, this is what I suggest: Vorbis: .ogg & .vorbis Theora: .ogg & .theora (or .video as I've seen suggested several times) FLAC: .ogg & .flac (yes, I'm fully aware that FLAC can exist without Ogg, but it's about unity) Speex: .ogg & .spx OggPCM: .ogg & .oggpcm OggUVS: .ogg & .ogguvs For mixed content of other kinds, like Writ, or crazy combinations of the above, or external codecs (MPEG 4, whatever), one has accept the default .ogg only, because it's not possible to state an extension for every possible use of Ogg. The idea is to allow dual extension on the most important projects under Xiph. Notice that most of the suggested extensions are outside 8.3 limitations. It's 2007; we do not need to worry about this anymore. That list is a suggestion, and this you are reading is a proposal. It's up for each maintainer to decide if it's viable, or not, but I ask you to consider it. This proposal isn't only philosophical. It's about making life easier for end-users and to tell software that if it supports Speex, then it needs to look in the current directory for both .ogg and .spx files as source for Speex streams. Extensions are meaningless, and .avi sure as hell isn't just "Video for Windows", so let's make it clear what extensions are allowed for the different projects under Xiph, even if it's crazy talk. 4. More Solutions Ralph Giles has been studying the possibility of adding a "disposition-type" field on the MIME application/ogg. This is an approach taken by the RSS project, and I believe it's another step in the right direction. But it's a completely different debate, though I'm mentioning it, because this and the solution described in this proposal should both be used. Operating Systems don't care about MIME types, but care about extensions. On the other hand, HTTP servers don't care as much about extensions, but care a lot about MIME types. The RSS project is considering using disposition-type with Dublin Core attributes like "moving-pictures", "sound" and "mixed". This syntax may not be most appropriate for Ogg streams, but the idea behind it is good. Let's take it in consideration. Example: application/ogg; disposition-type=sound One more solution to the problem of consistency may be the use of the name "Ogg family" when referring to the media formats that use Ogg containers. I think it will help fortify the ties between the different projects under Xiph, and make easier my job, which is to promote those projects. Ogg family. 5. In Conclusion Hi, my name is Ivo Emanuel Gon?alves. I'm new to Xiph.Org, and mostly unknown. My project is called SpreadOgg. It's an attempt at doing whatever it's needed to promote the media formats under Xiph.Org. It was hard for me to write this proposal. Very hard. Because it forces change, and change is never welcomed. Because it's bold, and boldness tends to attract enmity. Because it's probably going to be ignored. And because it may add more complexity to an issue that is already complex. But in the end, I thought this issue was important, and here we are. From what I've heard, whole flame wars regarding extensions have happened before, and they led to nothing. Why not end this issue once and for all, and go on with more important matters? This message is public and is made available in ogg-dev, vorbis-dev, flac-dev, speex-dev, and theora-dev mailing lists. Although it's directed at the maintainers of these projects, casual readers of the mailing lists may discuss this proposal if they wish, but please make sure the contribution is significant and/or useful. More ranting is pretty much unnecessary. Que seja o que Deus quiser.
Ian Malone
2007-Jan-15 21:12 UTC
[Speex-dev] Re: [Vorbis-dev] Proposal: An extension to rules all others
(Ah... followups to ogg-dev...) (I've got a couple of tiny comments but they're not important, they do however tie into my main point at the bottom.) Ivo Emanuel Gon?alves wrote:> Proposal: An extension to rules all others > > Copyright Notice > None. Public Domain. > > There isn't one problem here; there's several. The person described > above may be a blogger, but he has no idea what Speex is. The personRather kind to ascribe technological competence simply because the person is a blogger...> above may have an iPod, and s/he won't care much for any audio format > other than MP3. S/he won't bother to look for a hardware player with > support for other audio formats, and more importantly, s/he won't even > bother consider nagging Apple about adding support for other formats.There is a point of view that Apple is quite bent on preventing uptake of other formats. <snip explanation of the problems caused by one extension combined with poor support for shared extensions>> I propose that each project (Vorbis, Speex, etc.) states in its > specification two file extensions, and those two extensions are to be > the only ones allowed for that given format. Implementators are > supposed to support both extensions. Content creators are supposed to > choose one of them according to whatever criteria they care about. >Part of the problem you describe is in implementation, so I suspect that you can't really rely on implementors.> Are there arguments against this? Of course there are. > > One might say that .avi is always .avi even though it might be MPEG 1, > XviD, DivX, or anything else. Yet, you forget that .avi is ALWAYS for > video, so most software won't complain when they see an .avi file > (except for WMP that complains for everything). The worst that may > happen is for the software to ask the user for the appropriate codecs. > This doesn't happen with .ogg, though, because the Ogg container > might be used for so many different purposes that programs don't have > an uniform way of dealing with .ogg. > > Matroska, for instance, specifies .mkv for video/video+audio, and .mka > for audio. This may not be the greatest approach ever, but it works > in real life, and that's what we should aim for. >See <http://xiph.org/minutes/2006/10/200610_meeting.txt>, there doesn't really seem to be any resistance to doing something, the question is really what. <Okay, next bit is long, leaving it in for context>> And yet, there's further opposition? Some may say "extensions don't > matter", but they forget this is a computer world ruled by MS Windows, > where the average user is not smart enough to know what's wrong when > the video or audio file s/he downloaded won't work. Extensions are > needed. Anyone ever saw the UNIX "file" command trying to guess what > type a file is? Extensions are needed. > > Now, assuming you guys dig what I say, there's another problem. > Choose a new extension for your project, say .vorbis for Vorbis and we > all know what happens. Existing software will stop playing older > Vorbis streams using the .ogg extension. The solution as I mentioned > above is to allow two extensions, no more, nor less. > > Example: .ogg as default, .vorbis as secondary. > > OK, but that's for Vorbis. Why should FLAC or Speex care? Because of > consistency. If we want to move Ogg away from the misunderstanding > that it's Vorbis only (the "OGG"), we have to allow .ogg to be used on > FLAC and Speex files. And because of correctness. If we want unity > between our projects, we have to allow .ogg to be used on FLAC and > Speex files. > > So, this is what I suggest: > Vorbis: .ogg & .vorbis > Theora: .ogg & .theora (or .video as I've seen suggested several times) > FLAC: .ogg & .flac (yes, I'm fully aware that FLAC can exist without > Ogg, but it's about unity) > Speex: .ogg & .spx > OggPCM: .ogg & .oggpcm > OggUVS: .ogg & .ogguvs > > For mixed content of other kinds, like Writ, or crazy combinations of > the above, or external codecs (MPEG 4, whatever), one has accept the > default .ogg only, because it's not possible to state an extension for > every possible use of Ogg. The idea is to allow dual extension on the > most important projects under Xiph. > > Notice that most of the suggested extensions are outside 8.3 > limitations. It's 2007; we do not need to worry about this anymore. > > That list is a suggestion, and this you are reading is a proposal. > It's up for each maintainer to decide if it's viable, or not, but I > ask you to consider it. > > This proposal isn't only philosophical. It's about making life easier > for end-users and to tell software that if it supports Speex, then it > needs to look in the current directory for both .ogg and .spx files as > source for Speex streams. > > Extensions are meaningless, and .avi sure as hell isn't just "Video > for Windows", so let's make it clear what extensions are allowed for > the different projects under Xiph, even if it's crazy talk. >First, there is apparently a new version of MS DOS coming out shortly, does anyone know if it is any more sensible than it's predecessors? It may be worth finding out. Do Macs care? I don't believe Linux does, so it's Windows that's the problem (and it is a big problem, market share and all that), but Windows is Windows Media Player, so if that has caught up with the rest of the world there's less need to worry. Secondly, it's become increasingly clear that end users are less computer literate (yes more people can type, but a decreasing proportion understand how their computers work, maybe I should say the quality of copmuter literacy is decreasing). They don't care about technologies (or people wouldn't use iPods), what they do care about is branding. I immediately thought about being flippant and suggesting .mp3 for Ogg containing audio only and .avi for Ogg containing .avi, making it the WMP's problem. But, that would make Xiph the bad guys... However there's something to this idea; it recognises the recognition that .avi and .mp3 posses (having recently had to load someone's iPod for them I suspect there may be quite a few people using iPods thinking that mp3 is aac). So what about: .music (Vorbis) .video (Theora + audio) .voice (Speex) .music-perfect + .voice-perfect (FLAC & PCM) .video-perfect (Lossless video codecs + audio)? I really am serious. They're unused (AFAIK), they tell you about content. If people start trying to call other things .music (or similar), then they'll run into any extension problems, and if MS suddenly find people whose mp3, wma etc. isn't working complaining they are more likely to try and do something about that than they have been to act on Ogg. The downside is losing the Ogg/codec branding, though I think this is more important for content publishers than consumers. -- imalone
Ivo Emanuel Gonçalves
2007-Jan-16 07:29 UTC
[ogg-dev] Re: [Vorbis-dev] Proposal: An extension to rules all others
On 1/15/07, Ian Malone <ibmalone@gmail.com> wrote:> There is a point of view that Apple is quite bent on preventing uptake > of other formats.I've been thinking about that. Considering Apple is an American company, can't they be liable for an anti-trust case against other formats? They do have an interest in keeping little competition in AAC'a territory.> First, there is apparently a new version of MS DOS coming out shortly, > does anyone know if it is any more sensible than it's predecessors?I've not heard of anything in that regard. Actually, I'm not sure it would make sense for Microsoft to develop a new version of DOS. Although, I did hear something about a new completely different Command Line in Windows Vista, but I doubt that would be affected by 8.3 limits.> Secondly, it's become increasingly clear that end users are less > computer literate (yes more people can type, but a decreasing > proportion understand how their computers work, maybe I should say > the quality of computer literacy is decreasing). They don't care > about technologies (or people wouldn't use iPods), what they do > care about is branding. I immediately thought about being flippant > and suggesting .mp3 for Ogg containing audio only and .avi for > Ogg containing .avi, making it the WMP's problem. But, that would > make Xiph the bad guys... > > However there's something to this idea; it recognises the recognition > that .avi and .mp3 posses (having recently had to load someone's iPod > for them I suspect there may be quite a few people using iPods > thinking that mp3 is aac). So what about: > .music (Vorbis) > .video (Theora + audio) > .voice (Speex) > .music-perfect + .voice-perfect (FLAC & PCM) > .video-perfect (Lossless video codecs + audio)? > > I really am serious. They're unused (AFAIK), they tell you about > content. If people start trying to call other things .music (or > similar), then they'll run into any extension problems, and if > MS suddenly find people whose mp3, wma etc. isn't working > complaining they are more likely to try and do something about that > than they have been to act on Ogg. > > The downside is losing the Ogg/codec branding, though I think this > is more important for content publishers than consumers.Yes! Your extensions list is actually more sensible than mine. This is aggresive marketing! And it can work. As long as software developers are warned to support both new and legacy extensions, and content developers encouraged to use the new extensions, it might just work. Pretty much, it's still a dual extension proposal, but with foresight of promotion and locking illiterate users to think .music is actually music, and .video is actually video, and nothing else. And I'm sure audiophiles will be happy with .music-perfect. What do others think? Monty, Josh, Ralph, Jean-Marc, Mike? Do any of you agree on this proposal? The intent to get rid of extension flamewars due to exclusive use of.ogg is still here. Other priorities at work here are promotion and clear confusion from the part of users. -Ivo P.S: There's still the disposition-type proposal to discuss. Should we schedule a Monthly Meeting to discuss this and the above?
Sebastian Olter
2007-Jan-20 22:33 UTC
[Vorbis-dev] Proposal: An extension to rules all others
Why change something that worked well for several years? People, including me, know that something.ogg is a song "Something" encoded in Vorbis, and they like vorbis due to its simplicity; indeed, there is no simplest way to create high quality music encodings than do it with OGG. Regardless of many good ideas discussed here, I think the best way to name Theora videos is to give them an .ogv extension - it will maintain current simplicity and allow us to distinguish between audio and video files. The only (well known) file explorer which looks into media files to make a preview is Windows Explorer. We need a different extension for videos in order to find them elsewhere; think about shell scripts. Consider a Linux directory tree with many Vorbis and Theora files, all with .ogg extension. When we want to see videos (we cannot see music itself), such a command in bash might be applicable: for file in $(ls -R | grep .ogg); do [ $(oggtest "$file") == "video" ] && ln "$file" .. done where oggtest is a hypothetical script to distinguish between audio and video oggs, which takes, say, 0.3s to do what is supposed to; it could return ''video'' for videos and ''audio'' for music. Wouldn''t it be easier to just check for extension?