Dear speex developers and users,
I'm considering adopting the speex jitter buffer for use with a
different codec in a voice conferencing system and would be very
grateful if those more acquainted with it could help me with some
questions.
The speex_jitter_buffer.c wrapper seems to maintain (buffer?) one
packet-frame ("current_packet") in addition to the packets already
tracked by the JitterBuffer itself. Why is this necessary?
Correspondingly, how is the internal JitterBuffer state affected by
calling jitter_buffer_get (and _tick) before jitter_buffer_put has a
chance to deliver the first packet of a stream? Should this be avoided?
jitter_buffer_get can return JITTER_BUFFER_INSERTION and
JITTER_BUFFER_MISSING. In both situations speex_jitter_buffer.c makes
the codec interpolate a frame. How are these conditions different, and
is there a better recommended strategy for handling one versus the
other?
Am I correct that calling jitter_buffer_update_delay might eventually
cause one or more buffered packets to be skipped? And in any case,
what are the implications of having auto_adjust change the delay on
every call to jitter_buffer_tick versus speex_jitter_buffer.c's
strategy of making it contingent upon the "activity level" of the last
frame? Is this equivalent to shrinking the buffer using some form of
silence detection?
Thank you very much in advance.
Zach
---------------------------------------
Zachary P. Schneirov
Software Developer
WCAS Multimedia Learning Center
Northwestern University
(847) 467-5655
Hi Zachary, Zachary Schneirov a ?crit :> The speex_jitter_buffer.c wrapper seems to maintain (buffer?) one > packet-frame ("current_packet") in addition to the packets already > tracked by the JitterBuffer itself. Why is this necessary?That's in case there's more than one frame per packet. That way, it finds the packet, decodes the first frame it contains, and keeps the rest for the next time. If you only use one frame per packet (which most people do), it's not necessary to do that.> Correspondingly, how is the internal JitterBuffer state affected by > calling jitter_buffer_get (and _tick) before jitter_buffer_put has a > chance to deliver the first packet of a stream? Should this be avoided?At this point, this should probably be avoided (I've heard of problems when doing that), but eventually this is a case the jitter buffer should be able to handle.> jitter_buffer_get can return JITTER_BUFFER_INSERTION and > JITTER_BUFFER_MISSING. In both situations speex_jitter_buffer.c makes > the codec interpolate a frame. How are these conditions different, and > is there a better recommended strategy for handling one versus the > other?At this point, the only way to do an insertion with libspeex is to pretend a packet was lost. But this is just an implementation issue, the two cases could be implemented differently.> Am I correct that calling jitter_buffer_update_delay might eventually > cause one or more buffered packets to be skipped? And in any case, > what are the implications of having auto_adjust change the delay on > every call to jitter_buffer_tick versus speex_jitter_buffer.c's > strategy of making it contingent upon the "activity level" of the last > frame? Is this equivalent to shrinking the buffer using some form of > silence detection?If you let the jitter manage the delay by itself, it will just insert/delete frames when needed without taking into account the audio. This can cause some quality degradation. On the other hand, you can look for silent periods and call jitter_buffer_update_delay() yourself, which will ensure that the delay is only updated when there's silence. Cheers, Jean-Marc> Thank you very much in advance. > > > Zach > > --------------------------------------- > Zachary P. Schneirov > > Software Developer > WCAS Multimedia Learning Center > Northwestern University > (847) 467-5655 > > _______________________________________________ > Speex-dev mailing list > Speex-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/speex-dev > >
Jean-Marc, Thank you very much for that most illuminating response. Zach On Jan 30, 2009, at 7:00 PM, Jean-Marc Valin wrote:> Hi Zachary, > > Zachary Schneirov a ?crit : >> The speex_jitter_buffer.c wrapper seems to maintain (buffer?) one >> packet-frame ("current_packet") in addition to the packets already >> tracked by the JitterBuffer itself. Why is this necessary? > > That's in case there's more than one frame per packet. That way, it > finds the packet, decodes the first frame it contains, and keeps the > rest for the next time. If you only use one frame per packet (which > most > people do), it's not necessary to do that. > >> Correspondingly, how is the internal JitterBuffer state affected by >> calling jitter_buffer_get (and _tick) before jitter_buffer_put has a >> chance to deliver the first packet of a stream? Should this be >> avoided? > > At this point, this should probably be avoided (I've heard of problems > when doing that), but eventually this is a case the jitter buffer > should > be able to handle. > >> jitter_buffer_get can return JITTER_BUFFER_INSERTION and >> JITTER_BUFFER_MISSING. In both situations speex_jitter_buffer.c makes >> the codec interpolate a frame. How are these conditions different, >> and >> is there a better recommended strategy for handling one versus the >> other? > > At this point, the only way to do an insertion with libspeex is to > pretend a packet was lost. But this is just an implementation issue, > the > two cases could be implemented differently. > >> Am I correct that calling jitter_buffer_update_delay might eventually >> cause one or more buffered packets to be skipped? And in any case, >> what are the implications of having auto_adjust change the delay on >> every call to jitter_buffer_tick versus speex_jitter_buffer.c's >> strategy of making it contingent upon the "activity level" of the >> last >> frame? Is this equivalent to shrinking the buffer using some form of >> silence detection? > > If you let the jitter manage the delay by itself, it will just > insert/delete frames when needed without taking into account the > audio. > This can cause some quality degradation. On the other hand, you can > look > for silent periods and call jitter_buffer_update_delay() yourself, > which > will ensure that the delay is only updated when there's silence. > > Cheers, > > Jean-Marc