Speex dev - Nov 2005 - Re: [iglance] iGlance audio/video pipeline

(Cross posted to speex-dev from iglance)

Enzo -- I haven't tried the fixed point engine, though I've considered 
it for the WinCE port.  For the desktop/laptop edition I'm assuming the 
slight short->float conversion cost will be made up by the improved 
performance of the floating point implementation.  But I could be wrong:

1) Can anyone recommend whether Speex performs better in fixed-point or 
floating point mode (assuming floating point is supported by the hardware)?

2) Does it perform so much better in floating point mode so as to 
warrant the conversion from short->float?

3) Has anyone been able to capture audio samples in Win32 in native 
floating-point format so the conversion can be eliminated?

-david

Enzo Michelangeli wrote:
> David,
>  
> You menntion conversion from short to float. I see that Speex 1.1.11 
> includes a fixed-point implementation, which would probably make that 
> codec viable for FPU-less CPU's such as the Broadcom mipsel models used
> in several embedded appliances (the floating-point Speex is unusable on 
> my OpenWRT-based Linksys WRT54GS). Have you given it a try so far?
>  
> Enzo
> 
>     ----- Original Message -----
>     *From:* David Barrett <mailto:dbarrett@quinthar.com>
>     *To:* iglance@yahoogroups.com <mailto:iglance@yahoogroups.com>
>     *Sent:* Saturday, November 26, 2005 1:04 PM
>     *Subject:* [iglance] iGlance audio/video pipeline
> 
>     So I've had some requests as for where to get started in the code,
and
>     #1 on that list is audio quality.  What I've got so far
"works", but it
>     really needs the following features:
> 
>     - denoising
>     - jitter buffer
>     - echo cancellation
> 
>     So the easiest way to improve audio quality across the board is through
>     "denoising".  Denoising is the process of
"cleaning" out from the input
>     signal before encoding.  This is a standard feature of Speex and
I'm
>     told is super easy to integrate, but I haven't done it yet.  Read
>     through the Speex website (http://www.speex.org/) and mailing list
>     (speex-dev@xiph.org) for details on the Speex part.
> 
>     As for iGlance, the audio pipeline goes as follows:
> 
>     1. Audio is captured in "GNonPortable_Win32.cpp".  Take a
look at
>     GAVCaptureOpen( ) and IGCaptureAudioListener -- this lower-level than
>     you need, but it's good to know the start of the chain.
> 
>     2. GVideoCaptureService manages a list of GVideoCaptureStreams (one
>     stream for each online buddy).  GVideoCaptureStream::onAudioFrame( )
>     listens for audio samples, and then encodes them with
>     GSpeexEncoder::encode( ).
> 
>     3. GSpeexEncoder is where the denoising will certainly go.  The
>     GSpeexEncoder constructor initializes Speex.  GSpeexEncoder::encode( )
>     does the follows:
>           a. Converts input samples from (short) to (float)
>           b. Encodes as many Speex frames as possible
>           c. Adds a terminator to finish the packet
>           d. Returns the encoded bits
> 
>     4. After encoding with Speex, GVideoCaptureService sends the encoded
>     audio to the remote peer.
> 
> 
>     I would recommend adding the denoising code between steps 3.a and
>     3.b --
>     basically, after converting from (short) to (float), but before
>     encoding
>     with Speex.  However, I don't know enough about the Speex denoiser
to
>     say more.
> 
>     So denoising is the first place to start, but there are certainly
>     others.  If you have other ideas of where you'd like to dig into
the
>     code and contribute patches, please don't hesitate to let me know
and
>     I'll point you in the right direction in the code.  Thanks!
> 
>     -david
> 
> 
> 
>     SPONSORED LINKS
>     Open source software
>    
<http://groups.yahoo.com/gads?t=ms&k=Open+source+software&w1=Open+source+software&w2=Open+source&c=2&s=43&.sig=b9lWSiIj5nN_PtXo3cYfcw>
>     	Open source
>    
<http://groups.yahoo.com/gads?t=ms&k=Open+source&w1=Open+source+software&w2=Open+source&c=2&s=43&.sig=WnjeMhRv3WTPBlFeLcniEA>
> 
> 
> 
>    
------------------------------------------------------------------------
>     YAHOO! GROUPS LINKS
> 
>         *  Visit your group "iglance
>           <http://groups.yahoo.com/group/iglance>" on the web.
>            
>         *  To unsubscribe from this group, send an email to:
>            iglance-unsubscribe@yahoogroups.com
>          
<mailto:iglance-unsubscribe@yahoogroups.com?subject=Unsubscribe>
>            
>         *  Your use of Yahoo! Groups is subject to the Yahoo! Terms of
>           Service <http://docs.yahoo.com/info/terms/>.
> 
> 
>    
------------------------------------------------------------------------
>

> 1) Can anyone recommend whether Speex performs better in fixed-point or 
> floating point mode (assuming floating point is supported by the hardware)?
Depends on the architecture. On x86, the float version is about 20-50%
faster.
> 2) Does it perform so much better in floating point mode so as to 
> warrant the conversion from short->float?
That's like asking whether using a fast_algorithm() instead of
algorithm() is worth the overhead of the 5 extra characters.
> 3) Has anyone been able to capture audio samples in Win32 in native 
> floating-point format so the conversion can be eliminated?
See above.

	Jean-Marc