theora - Apr 2007 - Should Join Discussion - Fwd: [whatwg] Give guidance about RFC 4281 codecs parameter

Hello,

You guys might want to join this discussion.  (And maybe some others too.)

On the WhatWG mailing list, there's been a fury of e-mails about
adding a <video> element to HTML5.  They're talking about codecs and
containers now.  It might be helpful to have your representatives
there.  And help shape things.


See ya

---------- Forwarded message ----------
From: Dave Singer <singer@apple.com>
Date: Apr 9, 2007 10:15 AM
Subject: Re: [whatwg] Give guidance about RFC 4281 codecs parameter
To: Henri Sivonen <hsivonen@iki.fi>, WHAT working group
<whatwg@lists.whatwg.org>
Cc: Randall Gellens <randy@qualcomm.com>, "Per Fr?jdh (KI/EAB)"
<per.frojdh@ericsson.com>



WARNING:  I have CC'd the co-authors of the RFC, as I think they might
like to see the discussion, comment on my answers, and possibly
correct me.  I also have a question whether there is a typo in the
RFC...


* * * * *




Henry


these are all great questions.  Let me see how many I can answer.


Overall, the RFC was struggling with the issue that there is no
'uniform' naming of codecs;  the namespace for codecs is dependent on
the container format, so products that do container conversion have to
have tables of code matches.  ugh.  That's why the RFC is as it is.


The RFC suggests that updated information would be done with RFCs,
which is a little heavy.  The RFC as written formally applies to 3GPP
files and 3GPP2 files, but the definitions are applicable for all
ISO-family files.


As you'll see below, 3GPP has also defined it for avc1 in MP4-family
containers, but no spec. or registration authority provides a pointer.
 We might want to ask IANA whether we could add something to the MIME
registry.




At 11:37  +0300 8/04/07, Henri Sivonen wrote:

  * Theora video and Vorbis audio in Ogg container. (application/ogg; .ogg)
  * Dirac video and Vorbis audio in Ogg container. (application/ogg; .ogg)
  * Theora video and Vorbis audio in Matroska container.
(video/x-matroska; .mkv)
  * Dirac video and Vorbis audio in Matroska container.
(video/x-matroska; .mkv)


My understanding is that the Ogg container is 'specific' to these
codecs, and therefore the codecs parameter is not needed.  But I am
not an Ogg or Matroska expert;  perhaps they could chime in?


We did have the discussion over profiles of these codecs;  I
understand profiles are not used, but I am still unclear as to which
of the following is true:
a) features are never added to the bitstreams, so any release of the
decoder will decode bitstreams made by any release of the encoder
(modulo bugs);
b) the decoder release needed is signalled somehow in the bitstream
('need at least the April 2005 release or later to decode this file')
c) neither of the above are true, it's left to the users to stay up to
date, and if they don't, then, well, that's their problem.


If there are profiles, release versions etc. signalled, then a
parameter might be appropriate, and if the container formats are
general, a codecs parameter might be appropriate.


 * H.264 Baseline profile video and Low-Complexity AAC audio in MP4
container. (video/mp4; .mp4)


The AAC is covered by the RFC;  the example is there - mp4a.40.2.


H.264 was recently covered by 3GPP.  See 26.244 V7.1.0 section A.2.2,
available from www.3gpp.org.


When the first element of a value is 'avc1', indicating H.264 (AVC)
video [29], the second element is the hexadecimal representation of
the following three bytes in the sequence parameter set NAL unit
specified in [29]: 1) profile_idc, 2) a byte composed of the values of
constraint_set0_flag, constraint_set1_flag, constraint_set2_flag,
constraint_set3_flag, and reserved_zero_4bits in bit-significance
order, starting from the most significant bit, and 3) level_idc. Note
that reserved_zero_4bits is required to be equal to 0 in [29], but
other values for it may be specified in the future by ITU-T or
ISO/IEC.


You don't give me a level number, so I put xx for those hex digits below.


Assuming we're simple baseline, and also main and extended compatible
avc1.42E0xx




 * H.264 Extended profile video and Low-Complexity AAC audio in MP4
container. (video/mp4; .mp4

 Extended profile has the value (decimal) 88, and typically Extended
profile streams would be marked as Baseline compatible, at least.
Here is an example for this AVC


avc1.58A0xx



 * H.264 Main profile video and Low-Complexity AAC audio in MP4
container. (video/mp4; .mp4)

 Main profile is decimal 77, so something like
avc1.4D40xx

 * H.264 High profile video and Low-Complexity AAC audio in MP4
container. (video/mp4; .mp4)


There are a number of high profiles, confusingly, though there is one
called 'high', with value decimal 100.
avc1.6400xx


if it's not compatible with main, baseline, or extended profiles.


 * MPEG-4 Simple Profile profile video and Low-Complexity AAC audio in
MP4 container. (video/mp4; .mp4)


Covered by the RFC:


For example, MPEG-4 Visual Simple Profile Level 0 has the value 9,
    so a complete string for MPEG-4 Visual Simple Profile Level 0 would
    be "mp4v.20.9".


Though when checking the next answer, I read in the spec. that we may
have a typo here, it might be 8.  Ooops, if so.


Simple Profile/Level 0  00001000
 Reserved                        00001001 - 00001111



 * MPEG-4 Advanced Simple Profile profile video and Low-Complexity AAC
audio in MP4 container. (video/mp4; .mp4)

 Advanced Simple  Profile/Level 0        11110000


so, mp4v.20.240

 * MPEG-4 Simple Profile profile video and AMR audio in 3GPP
container. (video/3gpp; .3gp)


Video we've covered.  AMR is in 26.244,


samr




 * WMV9 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)
  * WMV8 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)


These would be up to Microsoft to define, I assume.  I am not aware of
a definition.


 * VC-1 video and WMA 10 Pro audio in ASF container. (video/x-ms-wmv; .wmv)


VC-1 is standardized by SMPTE, but again this container format is
Microsoft's.


 * Real Video 10 video and High-Efficiency AAC audio in Real Media
container. (application/vnd.rn-realmedia; rm)

 We'd have to ask Real, but I don't think it is defined.

 * XviD video and MP3 audio in AVI container. (video/x-msvideo; .avi)
  * Motion-JPEG video and uncompressed PCM audio in AVI container.
(video/x-msvideo; .avi)


AVI I *think* is owned by Microsoft, and I *think* they deprecate it;
it's up to the owner to define.  Again, I am not aware of a
definition, but the 4CC style from MP4 might be appropriate.


 * MPEG-1 video and MPEG-1 Audio Layer II audio in MPEG-1 program
stream (video/mpeg; .mpg)


MPEG has not defined the codecs parameter for these 'file' (stream)
formats, as far as I know.



 (That's a lot of cases and, yet, none are contrived.)

I tried to figure this out on my own with RFC 4281 and concluded that
this is not something that authors or even browser implementors are
likely to get right without an expert-created lookup table. I see that
at least of the RFC authors reads this mailing list. :-)

 --


David Singer
 Apple Computer/QuickTime




-- 
    Charles Iliya Krempeaux, B.Sc.

    charles @ reptile.ca
    supercanadian @ gmail.com

    developer weblog: http://ChangeLog.ca/

Hello,

Sorry... forgot to include this in the last e-mail.  You can join the
mailing list here...

http://lists.whatwg.org/listinfo.cgi/whatwg-whatwg.org


See ya

On 4/9/07, Charles Iliya Krempeaux <supercanadian@gmail.com>
wrote:
> Hello,
>
> You guys might want to join this discussion.  (And maybe some others too.)
>
> On the WhatWG mailing list, there's been a fury of e-mails about
> adding a <video> element to HTML5.  They're talking about codecs
and
> containers now.  It might be helpful to have your representatives
> there.  And help shape things.
>
>
> See ya
>
> ---------- Forwarded message ----------
> From: Dave Singer <singer@apple.com>
> Date: Apr 9, 2007 10:15 AM
> Subject: Re: [whatwg] Give guidance about RFC 4281 codecs parameter
> To: Henri Sivonen <hsivonen@iki.fi>, WHAT working group
> <whatwg@lists.whatwg.org>
> Cc: Randall Gellens <randy@qualcomm.com>, "Per Fr?jdh
(KI/EAB)"
> <per.frojdh@ericsson.com>
>
>
>
> WARNING:  I have CC'd the co-authors of the RFC, as I think they might
> like to see the discussion, comment on my answers, and possibly
> correct me.  I also have a question whether there is a typo in the
> RFC...
>
>
> * * * * *
>
>
>
>
> Henry
>
>
> these are all great questions.  Let me see how many I can answer.
>
>
> Overall, the RFC was struggling with the issue that there is no
> 'uniform' naming of codecs;  the namespace for codecs is dependent
on
> the container format, so products that do container conversion have to
> have tables of code matches.  ugh.  That's why the RFC is as it is.
>
>
> The RFC suggests that updated information would be done with RFCs,
> which is a little heavy.  The RFC as written formally applies to 3GPP
> files and 3GPP2 files, but the definitions are applicable for all
> ISO-family files.
>
>
> As you'll see below, 3GPP has also defined it for avc1 in MP4-family
> containers, but no spec. or registration authority provides a pointer.
>  We might want to ask IANA whether we could add something to the MIME
> registry.
>
>
>
>
> At 11:37  +0300 8/04/07, Henri Sivonen wrote:
>
>   * Theora video and Vorbis audio in Ogg container. (application/ogg; .ogg)
>   * Dirac video and Vorbis audio in Ogg container. (application/ogg; .ogg)
>   * Theora video and Vorbis audio in Matroska container.
> (video/x-matroska; .mkv)
>   * Dirac video and Vorbis audio in Matroska container.
> (video/x-matroska; .mkv)
>
>
> My understanding is that the Ogg container is 'specific' to these
> codecs, and therefore the codecs parameter is not needed.  But I am
> not an Ogg or Matroska expert;  perhaps they could chime in?
>
>
> We did have the discussion over profiles of these codecs;  I
> understand profiles are not used, but I am still unclear as to which
> of the following is true:
> a) features are never added to the bitstreams, so any release of the
> decoder will decode bitstreams made by any release of the encoder
> (modulo bugs);
> b) the decoder release needed is signalled somehow in the bitstream
> ('need at least the April 2005 release or later to decode this
file')
> c) neither of the above are true, it's left to the users to stay up to
> date, and if they don't, then, well, that's their problem.
>
>
> If there are profiles, release versions etc. signalled, then a
> parameter might be appropriate, and if the container formats are
> general, a codecs parameter might be appropriate.
>
>
>  * H.264 Baseline profile video and Low-Complexity AAC audio in MP4
> container. (video/mp4; .mp4)
>
>
> The AAC is covered by the RFC;  the example is there - mp4a.40.2.
>
>
> H.264 was recently covered by 3GPP.  See 26.244 V7.1.0 section A.2.2,
> available from www.3gpp.org.
>
>
> When the first element of a value is 'avc1', indicating H.264 (AVC)
> video [29], the second element is the hexadecimal representation of
> the following three bytes in the sequence parameter set NAL unit
> specified in [29]: 1) profile_idc, 2) a byte composed of the values of
> constraint_set0_flag, constraint_set1_flag, constraint_set2_flag,
> constraint_set3_flag, and reserved_zero_4bits in bit-significance
> order, starting from the most significant bit, and 3) level_idc. Note
> that reserved_zero_4bits is required to be equal to 0 in [29], but
> other values for it may be specified in the future by ITU-T or
> ISO/IEC.
>
>
> You don't give me a level number, so I put xx for those hex digits
below.
>
>
> Assuming we're simple baseline, and also main and extended compatible
> avc1.42E0xx
>
>
>
>
>  * H.264 Extended profile video and Low-Complexity AAC audio in MP4
> container. (video/mp4; .mp4
>
>  Extended profile has the value (decimal) 88, and typically Extended
> profile streams would be marked as Baseline compatible, at least.
> Here is an example for this AVC
>
>
> avc1.58A0xx
>
>
>
>  * H.264 Main profile video and Low-Complexity AAC audio in MP4
> container. (video/mp4; .mp4)
>
>  Main profile is decimal 77, so something like
> avc1.4D40xx
>
>  * H.264 High profile video and Low-Complexity AAC audio in MP4
> container. (video/mp4; .mp4)
>
>
> There are a number of high profiles, confusingly, though there is one
> called 'high', with value decimal 100.
> avc1.6400xx
>
>
> if it's not compatible with main, baseline, or extended profiles.
>
>
>  * MPEG-4 Simple Profile profile video and Low-Complexity AAC audio in
> MP4 container. (video/mp4; .mp4)
>
>
> Covered by the RFC:
>
>
> For example, MPEG-4 Visual Simple Profile Level 0 has the value 9,
>     so a complete string for MPEG-4 Visual Simple Profile Level 0 would
>     be "mp4v.20.9".
>
>
> Though when checking the next answer, I read in the spec. that we may
> have a typo here, it might be 8.  Ooops, if so.
>
>
> Simple Profile/Level 0  00001000
>  Reserved                        00001001 - 00001111
>
>
>
>  * MPEG-4 Advanced Simple Profile profile video and Low-Complexity AAC
> audio in MP4 container. (video/mp4; .mp4)
>
>  Advanced Simple  Profile/Level 0        11110000
>
>
> so, mp4v.20.240
>
>  * MPEG-4 Simple Profile profile video and AMR audio in 3GPP
> container. (video/3gpp; .3gp)
>
>
> Video we've covered.  AMR is in 26.244,
>
>
> samr
>
>
>
>
>  * WMV9 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)
>   * WMV8 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)
>
>
> These would be up to Microsoft to define, I assume.  I am not aware of
> a definition.
>
>
>  * VC-1 video and WMA 10 Pro audio in ASF container. (video/x-ms-wmv; .wmv)
>
>
> VC-1 is standardized by SMPTE, but again this container format is
Microsoft's.
>
>
>  * Real Video 10 video and High-Efficiency AAC audio in Real Media
> container. (application/vnd.rn-realmedia; rm)
>
>  We'd have to ask Real, but I don't think it is defined.
>
>  * XviD video and MP3 audio in AVI container. (video/x-msvideo; .avi)
>   * Motion-JPEG video and uncompressed PCM audio in AVI container.
> (video/x-msvideo; .avi)
>
>
> AVI I *think* is owned by Microsoft, and I *think* they deprecate it;
> it's up to the owner to define.  Again, I am not aware of a
> definition, but the 4CC style from MP4 might be appropriate.
>
>
>  * MPEG-1 video and MPEG-1 Audio Layer II audio in MPEG-1 program
> stream (video/mpeg; .mpg)
>
>
> MPEG has not defined the codecs parameter for these 'file' (stream)
> formats, as far as I know.
>
>
>
>  (That's a lot of cases and, yet, none are contrived.)
>
> I tried to figure this out on my own with RFC 4281 and concluded that
> this is not something that authors or even browser implementors are
> likely to get right without an expert-created lookup table. I see that
> at least of the RFC authors reads this mailing list. :-)
>
>  --
>
>
> David Singer
>  Apple Computer/QuickTime
>
>
>
>
> --
>     Charles Iliya Krempeaux, B.Sc.
>
>     charles @ reptile.ca
>     supercanadian @ gmail.com
>
>     developer weblog: http://ChangeLog.ca/
>

-- 
    Charles Iliya Krempeaux, B.Sc.

    charles @ reptile.ca
    supercanadian @ gmail.com

    developer weblog: http://ChangeLog.ca/

Thanks for informing us.

-Ivo