Speex dev - Sep 2008 - Jitter Buffer issues

I am trying to using Speex for some Wireless voice transfers. I am using
the 8K narrow band but am feeding it 11025 sample rate. Each Speex frame
is about 14.5ms instead of 20ms. In order to minimize WiFi packet
traffic I am sending 7 Speex frames at a time for a packet rate of about
10 per second so each WiFi packet has about 101ms of voice data.

I have three modes
1. Simplex - voice messaging using TCP
2. Half Duplex - push to talk using TCP
3. Full Duplex - two way real time voice using UDP

I use TCP for the first two to insure all voice packets are delivered to
the user. I must use UDP for the Full Duplex link to minimize latency. 

When I use UDP I also use the Speex Jitter Buffer. I have run into some
issues when using it. 

My first problem is minor with jitter_buffer_init(int step_size). The
initialization of buffer_margin I think this should be 1 not 0. 

     /*FIXME: Should this be 0 or 1?*/
        jitter->buffer_margin = 0;

With it set to 0 there is a problem if jitter_buffer_get() is called
before jitter_buffer_put(). This cause all packets to be discarded due
to every new packet being late. With jitter->buffer_margin = 1; there is
no problem with the calling order.

My second problem happens when there is a dropout in the connection. If
the packets are postponed during the dropout and multiple packets with
different sequence arrive once the link is reestablished there is a
large latency delay introduced because the oldest packet is the first
one retrieved. This latency stays until tb_init() is called on the first
timing buffer. This can take as long as 12.3 seconds. When the first
timing buffer is cleared the next packet causes compute_opt_delay() to
return the correct delay since there is only one delay in the timing
buffer the jitter buffer is then shifted correctly. I think the problem
is in compute_opt_delay() in the way it finds the latest packet.
Currently it finds the most resent packet in the timing buffer not the
oldest packet. The oldest packet is found once the timing buffer is
reset and only one frame is in the tb[0]. 

   
      int latest = 32767;
	  /* Pick latest amoung all sub-windows */
      for (j=0;j<MAX_BUFFERS;j++)
      {
         if (pos[j] < tb[j].filled && tb[j].timing[pos[j]] <
latest)
         {
            next = j;
            latest = tb[j].timing[pos[j]];
         }
      }

 I think it should be 

      int latest = 0;
	  /* Pick latest among all sub-windows */
      for (j=0;j<MAX_BUFFERS;j++)
      {
         if (pos[j] < tb[j].filled && tb[j].timing[pos[j]] >
latest)
         {
            next = j;
            latest = tb[j].timing[pos[j]];
         }
      }

This cause the longest delay to be found immediately without waiting for
a timing buffer reset by  tb_init() to cause only one packet in TB(0).

> I think the problem
> is in compute_opt_delay() in the way it finds the latest packet.
> Currently it finds the most resent packet in the timing buffer not the
> oldest packet. The oldest packet is found once the timing buffer is
> reset and only one frame is in the tb[0]. 
> 
>    
>       int latest = 32767;
> 	  /* Pick latest amoung all sub-windows */
>       for (j=0;j<MAX_BUFFERS;j++)
>       {
>          if (pos[j] < tb[j].filled && tb[j].timing[pos[j]] <
latest)
>          {
>             next = j;
>             latest = tb[j].timing[pos[j]];
>          }
>       }
> 
>  I think it should be 
> 
>       int latest = 0;
> 	  /* Pick latest among all sub-windows */
>       for (j=0;j<MAX_BUFFERS;j++)
>       {
>          if (pos[j] < tb[j].filled && tb[j].timing[pos[j]] >
latest)
>          {
>             next = j;
>             latest = tb[j].timing[pos[j]];
>          }
>       }

Whatever this does, it's probably a side effect of being buggy. If you
look at what goes into timing, it's (timestemp_of_the_packet -
timestamp_we_re_playing) if a packet is late, it means that its
timestamp is smaller the the timestamp we're playing, which means that
"timing" is on the negative side. Thus, we're looking for the
smallest
timing we can find.

As for the margin, your mileage may vary. A margin of zero gives better
latency, but a bit more late packets.

	Jean-Marc