Icecast - Feb 2005 - relation between burst- and queue-size with low bitrate streams

Hi all,

I have a question about the relationship between the "queue-size" and
the
"burst-size" <limits> parameters.

	Q: When a client connects is the burst-size immediately added to the
queue-size?

If this is the case then the slack for a 16kpbs client goes down from 50 sec
(102400[queue-size] / 16kbps = 50 sec) to 18 seconds ( (102400 -
65536[burst-size]) / 16kpbs = 18 sec).

I'm experiencing quite a few of the following lines in the DEBUG logging. It
could be that the clients are really running too slowly, but chewing 32secs
of the slack probably woudn't help either.

Background info
==============I'm running an icecast-2.1 mp3 streaming server where a few
hundred
listeners connect to a few tens of streams at 16 and 24 kbit/s bitrate.

Listeners connect using a device which is dialing in using a 56kbps analog
modem. Some listeners experience poor stream stability at times, meaning
that the stream repeatedly skips (with pauses of around 20-30 seconds).

I've looked at various reasons for this behaviour such as bad connect speed
of the 56K modem and possible lack of bandwidth on the streaming server but
these do not seem to be the problem. The modem connect speeds are > 38Kbps
and there is enough bandwidth available to the streaming server.

I've switched on DEBUG logging and I am seeing these lines quite regularly:

[2005-02-09  20:38:57] DBUG source/send_to_listener Client has fallen too
far behind, removing

I have a hunch that the burst-size and queue-size might have something to do
with this. The limits section of my icecast.xml reads:

    <limits>
        <clients>2000</clients>
        <sources>100</sources>
        <threadpool>10</threadpool>
        <queue-size>102400</queue-size>
        <client-timeout>30</client-timeout>
        <header-timeout>15</header-timeout>
        <source-timeout>30</source-timeout>
        <!-- If enabled, this will provide a burst of data when a client
             first connects, thereby significantly reducing the startup
             time for listeners that do substantial buffering. However,
             it also significantly increases latency between the source
             client and listening client.  For low-latency setups, you
             might want to disable this. -->
        <burst-on-connect>1</burst-on-connect>
        <!-- same as burst-on-connect, but this allows for being more
             specific on how much to burst. Most people won't need to
             change from the default 64k. Applies to all mountpoints  -->
        <burst-size>65535</burst-size>
    </limits>

Any suggestions would be appreciated.

Regards,
Klaas Jan

On Wed, 2005-02-09 at 20:44, Klaas Jan Wierenga wrote:
> 	Q: When a client connects is the burst-size immediately added to the
> queue-size?
no.
> If this is the case then the slack for a 16kpbs client goes down from 50
sec
> (102400[queue-size] / 16kbps = 50 sec) to 18 seconds ( (102400 -
> 65536[burst-size]) / 16kpbs = 18 sec).
> 
> I'm experiencing quite a few of the following lines in the DEBUG
logging. It
> could be that the clients are really running too slowly, but chewing 32secs
> of the slack probably woudn't help either.
The minimum length of the queue is burst-size, and stays around whether
listening clients are connected or not. queue-size is the maximum amount
that will be stored in the queue. It's the same queue, just imposed
limits on it.

Usually burst-size bytes will be used as most clients are up to date in
the stream, but if a listening client is unable to take the stream data
then they move down the queue until the end is reached at which point
the client is dropped with the message "fallen too far behind"

> Listeners connect using a device which is dialing in using a 56kbps analog
> modem. Some listeners experience poor stream stability at times, meaning
> that the stream repeatedly skips (with pauses of around 20-30 seconds).
> 
> I've looked at various reasons for this behaviour such as bad connect
speed
> of the 56K modem and possible lack of bandwidth on the streaming server but
> these do not seem to be the problem. The modem connect speeds are >
38Kbps
> and there is enough bandwidth available to the streaming server.
Issues to consider with modems are error rate (eg ifconfig) and whether
the modems keeps retraining. A modem retraining will add latency to the
stream making the client fall behind in the queue. Icecast will try to
send enough data to the client so that it can catch up but it may not be
able to.

Other considerations are for bitrate calculations, an mp3 stream will
always take more bandwidth than stated, I don't know if frame overhead
is taken into account, but you do have protocol overhead and the
optional shoutcast metadata.   The other part is TCP throttling but the
bitrate is very low so it's unlikely to be an issue.
> I have a hunch that the burst-size and queue-size might have something to
do
> with this. The limits section of my icecast.xml reads:
Feel free to increase the queue-size if you think the problems are just
temporary and you need more tolerance for a short time, the queue will
shrink if the clients are catching up/dropped.

If your listeners are on dialup then reducing burst-size to something
much lower or 0 would be better. Their link will be getting saturated at
connection time, so reducing that is not a problem they wouldn't benefit
from the quick buffer fill anyway.
>     <limits>
...
>         <burst-on-connect>1</burst-on-connect>
you can drop this option, burst is on by default and burst-size of 64k
is what this does. 

If all listeners are disalup then try queue-size 200k and burst-size of
8k, dialups are generally only 3-5 kbyte/s anyway. You can state the
queue-size and burst-size per mount just in case you have low and high
bandwidth streams.


karl.

Hi Karl,

Many thanks for explanation and the useful suggestions.
I'll experiment with the settings and see what that does to the rate of
dropped connections I see.

Cheers,
KJ

-----Oorspronkelijk bericht-----
Van: Karl Heyes [mailto:karl@xiph.org]
Verzonden: woensdag 9 februari 2005 22:41
Aan: Klaas Jan Wierenga
CC: icecast
Onderwerp: Re: [Icecast] relation between burst- and queue-size with
lowbitrate streams


On Wed, 2005-02-09 at 20:44, Klaas Jan Wierenga wrote:
> 	Q: When a client connects is the burst-size immediately added to the
> queue-size?
no.
> If this is the case then the slack for a 16kpbs client goes down from 50
sec
> (102400[queue-size] / 16kbps = 50 sec) to 18 seconds ( (102400 -
> 65536[burst-size]) / 16kpbs = 18 sec).
>
> I'm experiencing quite a few of the following lines in the DEBUG
logging.
It
> could be that the clients are really running too slowly, but chewing
32secs
> of the slack probably woudn't help either.
The minimum length of the queue is burst-size, and stays around whether
listening clients are connected or not. queue-size is the maximum amount
that will be stored in the queue. It's the same queue, just imposed
limits on it.

Usually burst-size bytes will be used as most clients are up to date in
the stream, but if a listening client is unable to take the stream data
then they move down the queue until the end is reached at which point
the client is dropped with the message "fallen too far behind"

> Listeners connect using a device which is dialing in using a 56kbps analog
> modem. Some listeners experience poor stream stability at times, meaning
> that the stream repeatedly skips (with pauses of around 20-30 seconds).
>
> I've looked at various reasons for this behaviour such as bad connect
speed
> of the 56K modem and possible lack of bandwidth on the streaming server
but
> these do not seem to be the problem. The modem connect speeds are >
38Kbps
> and there is enough bandwidth available to the streaming server.
Issues to consider with modems are error rate (eg ifconfig) and whether
the modems keeps retraining. A modem retraining will add latency to the
stream making the client fall behind in the queue. Icecast will try to
send enough data to the client so that it can catch up but it may not be
able to.

Other considerations are for bitrate calculations, an mp3 stream will
always take more bandwidth than stated, I don't know if frame overhead
is taken into account, but you do have protocol overhead and the
optional shoutcast metadata.   The other part is TCP throttling but the
bitrate is very low so it's unlikely to be an issue.
> I have a hunch that the burst-size and queue-size might have something to
do
> with this. The limits section of my icecast.xml reads:
Feel free to increase the queue-size if you think the problems are just
temporary and you need more tolerance for a short time, the queue will
shrink if the clients are catching up/dropped.

If your listeners are on dialup then reducing burst-size to something
much lower or 0 would be better. Their link will be getting saturated at
connection time, so reducing that is not a problem they wouldn't benefit
from the quick buffer fill anyway.
>     <limits>
...
>         <burst-on-connect>1</burst-on-connect>
you can drop this option, burst is on by default and burst-size of 64k
is what this does.

If all listeners are disalup then try queue-size 200k and burst-size of
8k, dialups are generally only 3-5 kbyte/s anyway. You can state the
queue-size and burst-size per mount just in case you have low and high
bandwidth streams.


karl.