Speex dev - Aug 2004 - Query r/e multiple decoder reset...

Hello!  I should have asked this earlier, but...  :)

We're returning to voice processing in our app now, in which we're
trying
to achieve n-way full duplex voice in a 3d environment.  Since Speex is a 
single voice stream CODEC, we've been looking at having each client upload 
its stream to the servers, which multiplex several spatially nearby active 
channels back down to the client.  (Like we did at OnLive where several of 
us once worked.)

To do this, we're looking at having ~4 decoders running per client to 
handle up to ~4 active downchannels.  The only catch is that these need to 
handle arbitrary sources. To achieve this, we seem to need to be able to 
reset a decoder to handle a stream from another source, ideally without 
having to free its resources, etc.

Is there any clean way to do this, and/or can such a function be added to 
the API?

Another query - we're currently running VAD in our layer above Speex.  Is 
there any way to query the encoder as to its VAD status after encoding?  It 
would be nice if we could leave all the raw data processing within Speex, 
all we're missing is a way to determine if a packet contained enough energy 
to warrant transmission.

Finally - has anyone looked at or be able to point me to papers on 
integration of vocal tract modeling into CELP systems?  I'm very interested 
in reproducing some of the intra-CODEC features we had at OnLive that 
allowed primitive mouth placement from the audio stream.  Its on our long 
term target list, and I was curious if anyone had any suggestions...

Thanks!

<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'speex-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

> To do this, we're looking at having ~4 decoders running per client to 
> handle up to ~4 active downchannels.  The only catch is that these need to 
> handle arbitrary sources. To achieve this, we seem to need to be able to 
> reset a decoder to handle a stream from another source, ideally without 
> having to free its resources, etc.
> 
> Is there any clean way to do this, and/or can such a function be added to 
> the API?
There is now a call (it's in CVS and will be in beta4 when it's released
in a couple days) that resets the decoder to its original state (after
creation):
speex_decoder_ctl(state, SPEEX_RESET_STATE, NULL);

Note however that because of the fact that the states contain a memory,
the first one or two frames (20-40ms) after the reset may be a bit more
noisy.
> Another query - we're currently running VAD in our layer above Speex. 
Is
> there any way to query the encoder as to its VAD status after encoding?  It
> would be nice if we could leave all the raw data processing within Speex, 
> all we're missing is a way to determine if a packet contained enough
energy
> to warrant transmission.
As it is in beta3, the Speex VAD is only used when in VBR mode, which
might not be what you want. In beta4, the VAD will be available
separately, even with DTX/comfort noise. There will be a way to query
the encoder for the VAD data.
> Finally - has anyone looked at or be able to point me to papers on 
> integration of vocal tract modeling into CELP systems?  I'm very
interested
> in reproducing some of the intra-CODEC features we had at OnLive that 
> allowed primitive mouth placement from the audio stream.  Its on our long 
> term target list, and I was curious if anyone had any suggestions...
I don't know much about that, but perhaps there would be an easy way to
reuse the LSP quantization from the decoder. It can be used to separate
the speech frames in 64 classes, based on the spectral envelope (vocal
tract). Then all that's needed is to assign mouth placement to each of
these classes. Of course that's in theory, since I've never tried that
:)

        Jean-Marc


-- 
Jean-Marc Valin, M.Sc.A.
LABORIUS (http://www.gel.usherb.ca/laborius)
Université de Sherbrooke, Québec, Canada

<p>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 242 bytes
Desc: signature.asc
Url :
http://lists.xiph.org/pipermail/speex-dev/attachments/20021218/0ed47caf/signature-0001.pgp