Speex dev - Dec 2004 - Jitter buffer

[sorry for the loss of proper attributions, this is from two messages]:

[Me]
>This is something I've encountered in trying to make a particular
> asterisk application handle properly IAX2 frames which contain either
> 20ms of 40ms of speex data.  For a CBR case, where the bitrate is
> known, this is fairly easy to do, especially if the frames _do_ always
> end on byte boundaries.  For a VBR case, it is more difficult, because
> it doesn't look like there's a way to just parse the speex
bitstream
> and break it up into the constituent 20ms frames.

[Jean Marc]
It would be possible, but unnecessarily messy. 




Jean-Marc Valin wrote:

[me]
>>I looked at nb_celp.c, and it seems that it would be pretty messy.
>>I'd need to implement a lot of the actual codec just to be able to
>>determine the number of frames in a packet.
>>    
>>
>
>  
>
[Jean-Marc]
>No, it's one step above nb_celp.c, all you need to implement is 8
>functions (init, destroy, process and ctl, for both encode and decode).
>It can be done fairly easily. Look at modes.c perhaps. The only struct
>that needs to be filled is SpeexMode. Even then, I'm willing to add an
>even simpler layer if necessary.
>  
>
I'm revisiting this issue now.  It looks like it would help to have the 
ability to:

1) Take a look at some speex data, and return the number of samples it 
contains.  This would go here in asterisk, for example:

asterisk/channels/chan_iax2.c:
static int get_samples(struct ast_frame *f)
{
        int samples=0;
        switch(f->subclass) {
        case AST_FORMAT_SPEEX:
                samples = 160;  /* XXX Not necessarily true XXX */
                break;
        case AST_FORMAT_G723_1:
                samples = 240 /* XXX Not necessarily true XXX */;
                break;
        case AST_FORMAT_ILBC:
                samples = 240 * (f->datalen / 50);
                break;
        case AST_FORMAT_GSM:
                samples = 160 * (f->datalen / 33);
                break;
        case AST_FORMAT_G729A:
                samples = 160 * (f->datalen / 20);
                break;
[...]
}

In this case, though, chan_iax2.c doesn't necessarily know if the speex 
codec is loaded..

And later, it might also be useful to have an API which takes a bunch of 
SpeexBits, and gives the caller a way to split up the SpeexBits into 
separate 20ms frames.  [The first API could be a subset of this].

The main API would be:

int speex_decode_bits(SpeexBits *inBits, SpeexBits*outBits). 

inBits is SpeexBits containing the bits we're interested in.
outBits may be NULL.  If not NULL, and inBits contains valid frames, 
they are written, one frame per call, to outBits.

it would return the same values as speex_decode(_int).


SpeexBits inBits, outBits;
void *state;

initialize:
    state = speex_decoder_init(&speex_nb_mode);


process:
    speex_bits_read_from(&inBits, inBuf, inlen);

    for(i=0; ;i+=160) {
       if(speex_decode_bits(&inBits,&outBits)) break;
      
        /* do something with outBits, if you want */
    }

    (i) now contains the number of samples contained in inBuf.


I think this is the simplest, most sensible API, no?

-SteveK


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/speex-dev/attachments/20041221/bb0addb8/attachment.html

Hi Steve,

Though it may work (haven't thought about all the details), I think it
would be much more messy than just using a codec abstraction layer (the
one in Speex or a custom one). I don't understand why you don't like
that idea.

	Jean-Marc
> And later, it might also be useful to have an API which takes a bunch
> of SpeexBits, and gives the caller a way to split up the SpeexBits
> into separate 20ms frames.  [The first API could be a subset of this].
> 
> The main API would be: 
> 
> int speex_decode_bits(SpeexBits *inBits, SpeexBits*outBits).  
> 
> inBits is SpeexBits containing the bits we're interested in.
> outBits may be NULL.  If not NULL, and inBits contains valid frames,
> they are written, one frame per call, to outBits.
> 
> it would return the same values as speex_decode(_int).
> 
> 
> SpeexBits inBits, outBits;
> void *state;
> 
> initialize:
>     state = speex_decoder_init(&speex_nb_mode);
> 
> 
> process:
>     speex_bits_read_from(&inBits, inBuf, inlen);
> 
>     for(i=0; ;i+=160) {
>        if(speex_decode_bits(&inBits,&outBits)) break;
>        
>         /* do something with outBits, if you want */
>     }
> 
>     (i) now contains the number of samples contained in inBuf.
> 
> 
> I think this is the simplest, most sensible API, no?
> 
> -SteveK
> 
> 
> _______________________________________________
> Speex-dev mailing list
> Speex-dev@xiph.org
> http://lists.xiph.org/mailman/listinfo/speex-dev
-- 
Jean-Marc Valin <Jean-Marc.Valin@USherbrooke.ca>
Universit? de Sherbrooke

On Dec 22, 2004, at 12:38 AM, Jean-Marc Valin wrote:
> Hi Steve,
>
> Though it may work (haven't thought about all the details), I think it
> would be much more messy than just using a codec abstraction layer (the
> one in Speex or a custom one). I don't understand why you don't
like
> that idea.
>
I guess I should have changed the subject of this message, because this 
API suggestion isn't really tied to the jitter buffer at all.

The codec abstraction in the jitterbuffer, though, assumes that the 
jitterbuffer will always be used for taking jittered input, and then 
decoding it to unjittered PCM.

That's not always what it's needed for, sometimes you just want to 
dejitter, and not decode.   Also, these operations often happen at 
differing places in a larger system, and coupling them so closely will 
require a much more complicated architecture.

Here's two examples of dejittering and not decoding:

1) iaxclient can handle an arbitrary number of calls, some of which may 
be active, and some of which may be "on hold".  For calls on hold, 
presently, we keep dejittering the input (so the buffer stays full, and 
we follow the network jitter), but we don't decode, because we're not 
playing the audio.

2) In asterisk, the jitterbuffer is presently at the network-protocol 
layer, but decoding is done much later.  Sometimes the call may be in 
an application which doesn't decode audio at all (i.e. it's playing you 
IVR prompts, or whatnot), or only decodes under certain circumstances 
(app_conference).  The network layer doesn't know anything about this.

-SteveK