Icecast dev - Aug 2004 - Re: PATCH: increase network congestion resilience

On Friday 17 January 2003 20:17, Karl Heyes shaped the electrons to
say:
> I would suggest a slightly different approach.
>
> Instead of increasing the syscall overhead for all sockets, trapping
> for uncommon cases.  Try the sock_write_bytes and if that is
> continuously having to queue (ie not all data can be sent) then display
> the warning, maybe make it a run-time option (default enabled).
Hi Karl,
        I will study it later. Do you have commit privileges? If so, 
I can send you also a patch to solve a memory leak bug (refbufs 
not released) and propose a bigger change: move the logic of 
actually sending the buffers to another function, and also 
exploit it to do some flow control. 

It is working in my site since a year without a glitch. 
It's something like (for old icecast version):

<p>+int send_client_queue(client_t *client, int maxlag)
+{
+   refbuf_t *abuf;
+   long bytes, sbytes;
+   time_t now = time(NULL);
+
+   if (client->last_sent < (now - maxlag * 2)) {
+        client->con->error = 1; /* lame client or DoS */
+        return 1;
+   }
+   abuf = refbuf_queue_remove(&client->queue);
+   while (abuf) {
+       if (client->pos > 0)
+           bytes = abuf->len - client->pos;
+       else
+           bytes = abuf->len;

+       if( abuf->timestamp < (now - maxlag) /* packet too delayed,
discard */ ) {
+           client->pos = 0;
+           printf("DEBUG: discarding %d bytes from queue!!! LAG=%d
MAXLAG=%d\n", bytes, now - abuf->timestamp, maxlag);
+
+       } else {
+           sbytes = sock_write_bytes(client->con->sock,
&abuf->data[client->pos], bytes);
+           if (sbytes > 0)  {
+               client->con->sent_bytes += sbytes;
+               client->last_sent = now;
+           }
+           if (sbytes < bytes) {
+               if (sbytes < 0 && !sock_recoverable(sock_error())) {
+                   printf("SOURCE: Client had unrecoverable error (%d)
catching up (%ld/%ld)\n", sock_error(), sbytes, bytes);
+                   refbuf_release(abuf);
+                   client->con->error = 1;
+               } else {
+                   client->pos += sbytes>0?sbytes:0;
+                   /* put the refbuf back on top of the queue, since we
didn't finish with it */
+                   refbuf_queue_insert(&client->queue, abuf);
+               }
+               return 1;
+           }
+       }
+       /* we're done with that refbuf, release it and reset the pos */
+       refbuf_release(abuf);
+       client->pos = 0;
+
+       abuf = refbuf_queue_remove(&client->queue);
+   }
+   return 0;
+}
+

...
+           data_done = send_client_queue(client, maxlag);

<p>
-- 
  ricardo galli       GPG id C8114D34

--- >8 ----
List archives:  http://www.xiph.org/archives/
icecast project homepage: http://www.icecast.org/
To unsubscribe from this list, send a message to
'icecast-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

On Saturday 18 January 2003 06:55, Ricardo Galli wrote:
> On Friday 17 January 2003 20:17, Karl Heyes shaped the electrons to say:
> > I would suggest a slightly different approach.
> >
> > Instead of increasing the syscall overhead for all sockets, trapping
> > for uncommon cases.  Try the sock_write_bytes and if that is
> > continuously having to queue (ie not all data can be sent) then
display
> > the warning, maybe make it a run-time option (default enabled).
>
> Hi Karl,
> 	I will study it later. Do you have commit privileges? If so,
> I can send you also a patch to solve a memory leak bug (refbufs
> not released) and propose a bigger change: move the logic of
> actually sending the buffers to another function, and also
> exploit it to do some flow control.
>
> It is working in my site since a year without a glitch.
> It's something like (for old icecast version):
We can't just drop packets, the transmission model assumed by the format 
handlers (and required by at least one of them) will not permit us to drop 
any data. It'd be possible, in theory at least, to tag buffers with a flag
to
say that they can be dropped, but probably not worth the complexity.

Erroring out if the client hasn't received any data within some timeout may
be
a good idea, but needs some cleanup. 

Mike
--- >8 ----
List archives:  http://www.xiph.org/archives/
icecast project homepage: http://www.icecast.org/
To unsubscribe from this list, send a message to
'icecast-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

On Saturday 18 January 2003 03:37, Michael Smith shaped the electrons to 
shout:
> We can't just drop packets, the transmission model assumed by the
> format handlers (and required by at least one of them) will not permit
It's not vorbis. Does it?
> us to drop any data. It'd be possible, in theory at least, to tag
> buffers with a flag to say that they can be dropped, but probably not
> worth the complexity.
I think it's ok as a last resort technique for trying to maintain alive a 
connection. Dropping some packages is much better that just closing the 
connection. Indeed, I checked with few players and all of them recover 
just fine. You have to be concentrated to notice the drops.
> Erroring out if the client hasn't received any data within some timeout
> may be a good idea, but needs some cleanup.
That the reason I moved all client buffer delivery logic to just one 
function (now receiving from sources and delivery to clients are all 
embeded in just one function). I will give a shot again.


-- 
  ricardo galli       GPG id C8114D34

--- >8 ----
List archives:  http://www.xiph.org/archives/
icecast project homepage: http://www.icecast.org/
To unsubscribe from this list, send a message to
'icecast-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.