Speex dev - Aug 2004 - linux.conf.au and streaming (was Re: patch for libspeex)

> Now, if this were finalised before the conference then we could do
> a demo and use it for broadcasting the lectures streams around the
> world... What is currently the best way of doing this?
Well, the draft is just a spec... folks still have to code
it.  It wont be 'final' for awhile but it's fairly stable,
especially the SDP descriptions.  Sounds like someone needs to
hack 'rat' to use speex.

Greg

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'speex-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

Hi Jens,
> There is already streaming support in the xmms-plugin.
> The 0.6.0 version found at http://jzb.rapanden.dk/speex/ is hopelessly
> outdated, but the nightly version should work.
D'oh, should've looked at this before hand. Seems you've done more
or less what I've done and taken the streaming code from the
mpeg/vorbis plugins and meshed it in with yours.

I can't seem to get yours to work - it hangs at "CONNECTED: WAITING
FOR REPLY". My version is at
        http://dagobah.ucc.asn.au/speex-xmms-0.6.0brb.tar.gz

It hasn't yet been extensively tested (ie, all the streaming
options), but it streams :) Bugs and comments welcome please!

The other oddball thing that you've probably already found is that
when xmms interrogates each plugin asking if it can handle it, the
mpg123 plugin accepts all http streams that aren't oggs. Grrr.

The workaround is to get the speex plugin before the mpg123 plugin
in the chain, by giving it a name that comes before "MPEG..." -
I've
used " Speex..." (leading space).
> I haven't worked on it in a while since I'm really busy at school,
but if
> anyone makes any patches, I'll be happy to look at them.
In the process of merging your code with the http code, I've ripped
yours to shreds to fit... Hope you dont mind. Perhaps you could
bless it with a new version number? :)

Cheers,

Bernard.


-- 
 Bernard Blackham -- bernard at blackham dot com dot au
 <shameless plug follows/>
  Alan, Rasmus, HPA, Bdale, Rusty, Tridge, Hemos and more...
                    Four days. One place.
  How would you feel if you missed it? http://www.linux.conf.au/


-------------- next part --------------
A non-text attachment was scrubbed...
Name: part
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
Url :
http://lists.xiph.org/pipermail/speex-dev/attachments/20021218/15bfa00d/part-0001.pgp

> Otherwise Greg, can you send the latest version of the RTP draft so I
> can put it on the site (the current one is getting old)?
Attached for all to see.

Greg

-------------- next part --------------






Internet Engineering Task Force                          Greg Herlein
Internet Draft                                        Jean-Marc Valin
draft-herlein-speex-rtp-profile-06                       Simon Morlat
November, 2002
Expires: May, 2003


             RTP Payload Format for the Speex Codec

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress".

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   To view the list Internet-Draft Shadow Directories, see
   http://www.ietf.org/shadow.html.

Copyright Notice

   Copyright (C) The Internet Society (2002).  All Rights Reserved.


Abstract

Speex is an open-source, patent-free voice codec suitable for use in
Voice over IP (VoIP) type applications.  The Speex codec supports two
modes of operation: narrowband at a nominal 8kHz sample rate and
wideband at a nominal 16kHz sample rate.  Speex supports Voice
Activity Detection (VAD) and Variable Bit Rate (VBR).  This document
describes the payload format for Speex generated bit streams within an
RTP packet.  Also included here are the necessary details for the use
of Speex with MIME and SDP.


Herlein, Valin, etc                                         [Page 1]
^L
Internet-Draft    RTP Payload Format for the Speex Codec    Nov 2002


1. Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED",
"SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",
"MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC-2119 [5].

2. Overview of the Speex Codec

Speex is based on the CELP encoding technique with support for either
wideband (nominal 16kHz) or narrowband (nominal 8kHz) sampling.  The
main characteristics can be summerized as follows:

   o  Free software/open-source, royalty-free
   o  Integration of wideband and narrowband in the same bit-stream
   o  Wide range of bit-rates available
   o  Dynamic bit-rate switching and variable bit-rate (VBR)
   o  Voice Activity Detection (VAD, integrated with VBR)
   o  Variable complexity


3. RTP payload format for Speex

   Speex uses 20 ms frames and a variable sampling rate clock.  The
   RTP timestamp MUST be in units of 1/X of a second where X is the
   sample rate used.  Speex uses a nominal 8kHz sampling rate for
   narrowband use and a nominal 16kHz sampling rate for wideband use.

   The RTP payload for Speex has the format shown in Figure 1.  No
   additional header specific to this payload format is required.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                      RTP Header [2]                           |
      +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
      |                                                               |
      +                 one or more frames of Speex                   |
      |                             ....                            |p|
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                     Figure 1: RTP payload for Speex

   The encoding and decoding algorithm can change the bit rate at any
   20ms frame boundary but the bit rate change notification is provided
   in-band with the bit stream.  Each frame contains both 
   "mode" (narrowband or wideband) and "sub-mode" (bit-rate)
   information in the bit stream.  No out-of-band notification is
   required for the decoder to process changes in the bit rate sent by
   the encoder.

Herlein, Valin, etc                                         [Page 2]
^L
Internet-Draft    RTP Payload Format for the Speex Codec    Nov 2002


   For the purposes of packetizing the bit stream in RTP, it is only
   necessary to consider the sequence of bits as output by the Speex
   encoder, and present the same sequence to the decoder.  The payload
   format described here maintains this sequence.

   An RTP packet MAY contain Speex frames of the same bit rate or of
   varying bit rates, since the bit-rate for a frame is conveyed in
   band with the signal.

   It is RECOMMENDED that values of 8000 or 16000 be used for normal
   internet telephony applications, though the sample rate is
   supported at rates as low as 6000 Hz and as high as 22.05 kHz.

   The RTP payload MUST be padded to provide an integer number of
   octets as the payload length.  These padding bits MUST be all zero.
   This padding is only required for the last frame in the packet, and
   only to ensure the packet contents ends on an octet boundary.


3.1 RTP Payload Type Codes

   The RTP Audio-Visual Working Group will no longer issue static
   payload type codes for RTP (beyond those already assigned).
   Dynamic payload type codes MUST be negotiated 'out-of-band'
   for the assignment of a dynamic payload type from the
   range of 96-127.  Examples of this are shown in the section
   discussing the Session Description Protocol (SDP) below.


3.2 Multiple Speex frames in a RTP packet

   By default only one Speex frame is permitted in a single RTP
   packet.  When operating with multiple frames per packet then the
   end points MUST use out-of-band negotiation to determine the number
   of frames per packet.  See section 5 below for an example of how to
   do this with SDP [4].
  

3.3 Computing the number of Speex frames

   If using SDP [4] (see section 5 below for an example) this can be
   done using the "ptime" variable to denote the packetization
   interval (ie, how many milliseconds of audio is encoded in a single
   RTP packet).  Since Speex uses 20ms frames, ptime values of
   multiples of 20 denote multiple Speex frames per packet.  Values of
   ptime in other than multiples of 20 SHOULD be ignored and SHOULD
   use the default value of one instead.


4. MIME registration of Speex

   Full definition of the MIME type for Speex will be part of the Ogg
   Vorbis MIME type definition application.  The discussion here
   applies only to use within RTP applications.

   MIME media type name: audio

   MIME subtype: speex

   Required parameters:

Herlein, Valin, etc                                         [Page 3]
^L
Internet-Draft    RTP Payload Format for the Speex Codec    Nov 2002


   Optional parameters:

         ptime: duration of each packet in milliseconds.

	 sr:    actual sample rate in Hz.

	 ebw:   encoding bandwidth - either 'narrow' or 'wide' or 
                'ultra' (corresponds to nominal 8000, 16000, and
		32000 Hz sampling rates).

	 vbr:   variable bit rate  - either 'on' 'off' or
'vad'.

	 cng:   comfort noise generation - either 'on' or 'off'. 

	 mode:  speex encoding mode. Can be {1,2,3,4,5,any}
                defaults to 3.

	 penh:	use of perceptual enhancement. 1 indicates 
	 	to the decoder that perceptual enhancement is recommended,
		0 indicates that it is not. Defaults to no.


   Encoding considerations:
         This type is only defined for transfer via RTP as specified in
         a Work in Progress.

   Security Considerations:
         See Section 6 of RFC 3047.

   Interoperability considerations: none

   Published specification:

   Applications which use this media type:
         Audio and video streaming and conferencing tools

   Additional information: none

   Person & email address to contact for further information:
	 Greg Herlein <gherlein@herlein.com>
	 Jean-Marc Valin <jean-marc.valin@hermes.usherb.ca>


   Intended usage: COMMON

   Author/Change controller:
         Author:  Greg Herlein <gherlein@herlein.com>
         Change controller: Greg Herlein <gherlein@herlein.com>


Herlein, Valin, etc                                         [Page 4]
^L
Internet-Draft    RTP Payload Format for the Speex Codec    Nov 2002


5. SDP usage of Speex

   When conveying information by SDP [4], the encoding name SHALL be
   "speex" (the same as the MIME subtype).  An example of the media
   representation in SDP for offering a single channel of Speex at
   8000 samples per second might be:

	m=audio 8088 RTP/AVP 97
	a=rtpmap:97 speex/8000

   Note that the RTP payload type code of 97 is defined in this media
   definition to be 'mapped' to the speex codec at an 8kHz sampling
   frequency using the 'a=rtpmap' line.  Any number from 96 to 127
   could have been chosen (the allowed range for dynamic types).  The
   value of the sampling frequency is typically 8000 for narrow band
   operation, 16000 for wide band operation, and 32000 for ultra-wide
   band operation.

   If for some reason the offerer has bandwith limitations, he may use
   the "b=" header, as explained in SDP [4]. The following example
   illustrates the case where the offerer cannot receive more than
   10 kbit/s.

   	m=audio 8088 RTP/AVP 97
	b=AS:10
	a=rtmap:97 speex/8000

   In this case, if the remote part agrees, it should configure its
   speex encoder so that it does not use modes that produce more than
   10 kbit/s. Note that the "b=" constraint also applies on all
   payload types that may be proposed in the media line ("m=").

   An other way to make recommendations to the remote speex encoder
   is to use its specific parameters via the a=fmtp: directive.  The
   following parameters are defined for use in this way:

         ptime: duration of each packet in milliseconds.

	 sr:    actual sample rate in Hz.

	 ebw:   encoding bandwidth - either 'narrow' or 'wide' or 
                'ultra' (corresponds to nominal 8000, 16000, and
		32000 Hz sampling rates).

	 vbr:   variable bit rate  - either 'on' 'off' or 'vad'
		(defaults to off).  If on, variable bit rate is
		enabled.  If off, disabled.  If set to 'vad' then
		constant bit rate is used but silence will be encoded
		with special short frames to indicate a lack of voice
		for that period.

	 cng:   comfort noise generation - either 'on' or 'off'. If
		off then silence frames will be silent; if 'on' then
		those frames will be filled with comfort noise.

	 mode:  speex encoding mode. Can be {1,2,3,4,5,any}
                defaults to 3.

	 penh:	use of perceptual enhancement. 1 indicates 
	 	to the decoder that perceptual enhancement is recommended,
		0 indicates that it is not. Defaults to no.


Herlein, Valin, etc                                         [Page 5]
^L
Internet-Draft    RTP Payload Format for the Speex Codec    Nov 2002


   Examples:

   	m=audio 8008 RTP/AVP 97
	a=rtpmap:97 speex/8000
	a=fmtp:97 mode=4

   This examples illustrate an offerer that wishes to receive
   a speex stream at 8000Hz, but only using speex mode 3.
   
   The offerer may suggest to the remote decoder to activate
   its perceptual enhancement filter like this:
   
	m=audio 8088 RTP/AVP 97
	a=rtmap:97 speex/8000
	a=fmtp:97 penh=1 
	
   Several speex specific parameters can be given in a single
   a=fmtp line provided that they are separated by a semi-colon:
   
   	a=fmtp:97 mode=any;penh=1

   The offerer may indicate that it wishes to send variable bit rate
   frames with comfort noise:

	m=audio 8088 RTP/AVP 97
	a=rtmap:97 speex/8000
	a=fmtp:97 vbr=on;cng=on

	
   The use of a particular packetization interval may be
   suggested to the remote encoder using the ptime parameter:
   
   	m=audio 8008 RTP/AVP 97
	a=rtpmap:97 speex/8000
	a=ptime:40
	
   Note that the ptime parameter applies to all payloads listed
   in the media line and is not used as part of an a=fmtp directive.

   Speex can encode frames of 20 ms. Values of ptime not multiple
   of 20 ms are meaningless, so the receiver of such ptime values
   SHOULD ignore them.


Herlein, Valin, etc                                         [Page 6]
^L
Internet-Draft    RTP Payload Format for the Speex Codec    Nov 2002

6. ITU H.323/H.245 Use of Speex

   Application is underway to make Speex a standard ITU codec.
   However, until that is finalized, Speex MAY be used in H.323 by
   using a non-standard codec block definition in the H.245 codec
   capability negotiations.  


6.1  NonStandardMessage format

   For Speex use in H.245 based systems, the fields in the
   NonStandardMessage should be:

   t35CountryCode   = Hex: B5
   t35Extension     = Hex: 00
   manufacturerCode = Hex: 0026
   [Length of the Binary Sequence (8 bit number)]
   [Binary Sequence consisting of an ASCII string, no NULL terminator]

   The binary sequence is an ascii string merely for ease of use.
   The string is not null terminated.  The format of this string is

       speex [optional variables]
   
   The optional variables are identical to those used for the SDP
   a=fmtp strings discussed in section 5 above.  The string is built
   to be all on one line, each key-value pair seperated by a
   semi-colon.  The optional variables MAY be ommited, which causes
   the default values to be assumed.  They are:

       ebw=narrow;mode=3;vbr=off;cng=off;ptime=20;sr=8000;penh=no;

   The fifth byte of the block is the length of the binary sequence.

   NOTE:  this method can result in the advertising of a large number
   of Speex 'codecs' based on the number of variables possible.  For
   most VoIP applications, use of the defailt binary sequence of
   'speex' is RECOMMENDED to be used in addition to all other options.
   This maximizes the chances that two H.323 based applications that
   support Speex can find a mutual codec.   

6.2 RTP Payload Types

   Dynamic payload type codes MUST be negotiated 'out-of-band'
   for the assignment of a dynamic payload type from the
   range of 96-127.  H.323 applications MUST use the H.245
   H2250LogicalChannelParameters encoding to accomplish this.  


7. Security Considerations

   RTP packets using the payload format defined in this specification
   are subject to the security considerations discussed in the RTP
   specification [2], and any appropriate RTP profile.  This implies
   that confidentiality of the media streams is achieved by encryption.
   Because the data compression used with this payload format is applied
   end-to-end, encryption may be performed after compression so there is
   no conflict between the two operations.

   A potential denial-of-service threat exists for data encodings using
   compression techniques that have non-uniform receiver-end
   computational load.  The attacker can inject pathological datagrams
   into the stream which are complex to decode and cause the receiver to
   be overloaded.  However, this encoding does not exhibit any
   significant non-uniformity.

   As with any IP-based protocol, in some circumstances a receiver may
   be overloaded simply by the receipt of too many packets, either
   desired or undesired.  Network-layer authentication may be used to
   discard packets from undesired sources, but the processing cost of
   the authentication itself may be too high.  


8. References

   1. Bradner, S., "The Internet Standards Process -- Revision 3", BCP
      9, RFC 2026, October 1996.

   2. Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP:
      A Transport Protocol for real-time applications", RFC 1889,
      January 1996.  (Updated by a Work in Progress.)

   3. Freed, N. and N. Borenstein, "Multipurpose Internet Mail
      Extensions (MIME) Part One: Format of Internet Message Bodies",
      RFC 2045, November 1996.

   4. Handley, M. and V. Jacobson, "SDP: Session Description
Protocol",
      RFC 2327, April 1998.

   5. Bradner, S., "Key words for use in RFCs to Indicate Requirement
      Levels", BCP 14, RFC 2119, March 1997.

9. Acknowledgments

   The authors would like to thank Equivalence Pty Ltd of Australia
   for their assistance in standardizing the use of Speex in H.323
   applications, and for implementing Speex in their open source
   OpenH323 stack.  The authors would also like to thank Brian
   C. Wiles <brian@streamcomm.com> of StreamComm for his assistance in
   developing a standard for Speex use in H.323 applications.


10. Author's Address

   Greg Herlein <gherlein@herlein.com>
   2034 Filbert Street
   San Francisco, CA 
   United States 94123
   

   Jean-Marc Valin <jean-marc.valin@hermes.usherb.ca>
   Department of electrical and computer engineering
   University of Sherbrooke
   2500 blvd Université
   Sherbrooke, Quebec, Canada, J1K 2R1


   Simon MORLAT <simon.morlat@linphone.org>
   35, av de Vizille App 42
   38000 GRENOBLE
   FRANCE


   Roger Hardiman <roger@freebsd.org>
   49 Nettleton Road
   Cheltenham
   Gloucestershire
   GL51 6NR
   England


Herlein, Valin, etc                                         [Page 7]
^L
Internet-Draft    RTP Payload Format for the Speex Codec    Nov 2002

10. Full Copyright Statement

   Copyright (C) The Internet Society (2001).  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.




Herlein, Valin, etc                                         [Page 8]
^L

> Now, if this were finalised before the conference then we could do
> a demo and use it for broadcasting the lectures streams around the
> world... What is currently the best way of doing this?
The best demo right now is probably LinPhone (www.linphone.org). Simon
is currently working (in CVS) on making it up-to-date with the RTP spec
(Simon, is it usable yet?), but the earlier versions still work pretty
well with Speex (even at 8 kbps on a modem line).

Otherwise Greg, can you send the latest version of the RTP draft so I
can put it on the site (the current one is getting old)?

        Jean-Marc


-- 
Jean-Marc Valin, M.Sc.A.
LABORIUS (http://www.gel.usherb.ca/laborius)
Université de Sherbrooke, Québec, Canada

<p>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 242 bytes
Desc: signature.asc
Url :
http://lists.xiph.org/pipermail/speex-dev/attachments/20021218/90cb8b32/signature-0001.pgp

On Wed, Dec 18, 2002 at 04:11:39PM +0800, Bernard Blackham
wrote:
> I'm working on adding streaming support to the xmms plugin, then
> using http streaming. I haven't been able to find even Ogg streaming
> software (server/client) that isn't tightly integrated with Vorbis.
There is already streaming support in the xmms-plugin.
The 0.6.0 version found at http://jzb.rapanden.dk/speex/ is hopelessly
outdated, but the nightly version should work.

I haven't worked on it in a while since I'm really busy at school, but
if
anyone makes any patches, I'll be happy to look at them.
> 
> Bernard.
> 
/jens

<p>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: part
Type: application/pgp-signature
Size: 233 bytes
Desc: not available
Url :
http://lists.xiph.org/pipermail/speex-dev/attachments/20021218/fffb3fb0/part-0001.pgp

On Tue, Dec 17, 2002 at 11:55:21PM -0800, Greg Herlein
wrote:
> If such a thing happens, discussion of the RTP profile draft
> would be most welcome - please get responses back to the
> list!
Now, if this were finalised before the conference then we could do
a demo and use it for broadcasting the lectures streams around the
world... What is currently the best way of doing this?

I'm working on adding streaming support to the xmms plugin, then
using http streaming. I haven't been able to find even Ogg streaming
software (server/client) that isn't tightly integrated with Vorbis.
> Ah, if I could only be there... but it's a terribly long
> way to fly to.
See revised .sig :)

Bernard.


-- 
 Bernard Blackham -- bernard at blackham dot com dot au
  <shameless plug follows/>
  Alan, Rasmus, HPA, Bdale, Rusty, Tridge, Hemos and more...
  Four days. One place.
  How would you feel if you missed it? http://www.linux.conf.au/


-------------- next part --------------
A non-text attachment was scrubbed...
Name: part
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
Url :
http://lists.xiph.org/pipermail/speex-dev/attachments/20021218/653afae6/part-0001.pgp