Icecast - Dec 2005 - Use of TCP_CORK instead of TCP_NODELAY

>
> p.s. For an in depth analysis of TCP_CORK read Christiopher Baus'
excelent
> article: http://www.baus.net/on-tcp_cork
Thanks for this pointer. I'd been meaning to reply on this thread, but
hadn't got around to it, primarily because I didn't really understand
TCP_CORK (the linux manpage is, as usual, fairly unclear on what
exactly it does). Now I understand!
>
> -----Oorspronkelijk bericht-----
> Van: icecast-bounces@xiph.org [mailto:icecast-bounces@xiph.org]Namens Henri
> Zikken
> Verzonden: zondag 25 december 2005 15:05
> Aan: icecast@xiph.org
> Onderwerp: [Icecast] Use of TCP_CORK instead of TCP_NODELAY
>
>
> We're abusing icecast in a true narrowcasting setup (personalized
stream per
> mountpoint). The streams itself are created in a piece of proprietory
> (spelling?, i'm dutch) software, icecast merely relays them.
>
> However, the intended endpoint is an embedded device. This device has
> trouble with tcp/ip packets not matching the max. packet size (MSS or MSS
> minus header). After eleborate testing, we found that using the sockopt
> 'TCP_CORK' instead of 'TCP_NODELAY' produces far better
results on the field
> on reconnects etc/. Also, with streaming media, TCP_CORK is more efficient
> than TCP_NODELAY.
This is pretty broken. There are really three possibilities. In order
of increasing maximum delay:
  1) TCP_NODELAY - what icecast uses, deliberately.
  2) default (Nagle) - we used to do this in icecast.
  3) TCP_CORK - what you've added

As a streaming server, it's fairly crucial for icecast to send out
data with as low a delay as possible (many clients don't care, but
some do). That's why we use TCP_NODELAY - we actually WANT to send out
data as soon as we can.

There are limits to how much overhead is reasonable to accept, which
is why we do some aggregation in userspace; this aggregation should
probably be tuned better, but it's important that icecast control it,
not the kernel.

You want TCP_CORK, it seems, because of bugs in your target devices -
well, whilst we're willing to make some concessions to broken clients,
an inability to speak TCP correctly is well outside what I consider
sensible, particularly given that it will degrade icecast's
performance for working clients (you remain welcome, of course, to
hack up your local copy). It's also very unportable.

Mike

Michael,

With regard to your comment below:
> As a streaming server, it's fairly crucial for icecast to send out
> data with as low a delay as possible (many clients don't care, but
> some do). That's why we use TCP_NODELAY - we actually WANT to send out
> data as soon as we can.
Can you explain how some clients depend on a low delay when receiving data
from icecast? How is the use of TCP_CORK going to break those? Would the
application level buffering currently used in format_mp3.c not cause the
same problem?

Is it that the (mp3/vorbis/...) frame rate should be as constant as possible
and maybe for VBR streams using TCP_CORK can cause problems in this area?

If I understand TCP_CORK correctly, it is not going to keep data from being
sent when it can fill a TCP packet to the MSS of the current connection. If
there is enough data in the kernel to fill a packet to the MSS, then it will
not be delayed because the socket has not been uncorked yet. I don't see a
difference with the application level buffering here.
> You (Henri) want TCP_CORK, it seems, because of bugs in your target
devices.

I know our device can handle any size TCP frame (upto MSS), but it would
certainly be more efficient if all frames were filled to MSS (536 for our
device).

Appreciate your comments.

KJ

p.s. Btw we're using a CBR stream as you might have guessed.


-----Oorspronkelijk bericht-----
Van: mlrsmith@gmail.com [mailto:mlrsmith@gmail.com]Namens Michael Smith
Verzonden: woensdag 28 december 2005 17:14
Aan: Klaas Jan Wierenga
CC: henri.zikken@deltasolutions.nl; icecast@xiph.org
Onderwerp: Re: [Icecast] Use of TCP_CORK instead of TCP_NODELAY

>
> p.s. For an in depth analysis of TCP_CORK read Christiopher Baus'
excelent
> article: http://www.baus.net/on-tcp_cork
Thanks for this pointer. I'd been meaning to reply on this thread, but
hadn't got around to it, primarily because I didn't really understand
TCP_CORK (the linux manpage is, as usual, fairly unclear on what
exactly it does). Now I understand!
>
> -----Oorspronkelijk bericht-----
> Van: icecast-bounces@xiph.org [mailto:icecast-bounces@xiph.org]Namens
Henri
> Zikken
> Verzonden: zondag 25 december 2005 15:05
> Aan: icecast@xiph.org
> Onderwerp: [Icecast] Use of TCP_CORK instead of TCP_NODELAY
>
>
> We're abusing icecast in a true narrowcasting setup (personalized
stream
per
> mountpoint). The streams itself are created in a piece of proprietory
> (spelling?, i'm dutch) software, icecast merely relays them.
>
> However, the intended endpoint is an embedded device. This device has
> trouble with tcp/ip packets not matching the max. packet size (MSS or MSS
> minus header). After eleborate testing, we found that using the sockopt
> 'TCP_CORK' instead of 'TCP_NODELAY' produces far better
results on the
field
> on reconnects etc/. Also, with streaming media, TCP_CORK is more efficient
> than TCP_NODELAY.
This is pretty broken. There are really three possibilities. In order
of increasing maximum delay:
  1) TCP_NODELAY - what icecast uses, deliberately.
  2) default (Nagle) - we used to do this in icecast.
  3) TCP_CORK - what you've added

As a streaming server, it's fairly crucial for icecast to send out
data with as low a delay as possible (many clients don't care, but
some do). That's why we use TCP_NODELAY - we actually WANT to send out
data as soon as we can.

There are limits to how much overhead is reasonable to accept, which
is why we do some aggregation in userspace; this aggregation should
probably be tuned better, but it's important that icecast control it,
not the kernel.

You want TCP_CORK, it seems, because of bugs in your target devices -
well, whilst we're willing to make some concessions to broken clients,
an inability to speak TCP correctly is well outside what I consider
sensible, particularly given that it will degrade icecast's
performance for working clients (you remain welcome, of course, to
hack up your local copy). It's also very unportable.

Mike


--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.371 / Virus Database: 267.14.8/215 - Release Date: 27-12-2005


--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.371 / Virus Database: 267.14.8/215 - Release Date: 27-12-2005

--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.371 / Virus Database: 267.14.8/215 - Release Date: 27-12-2005

>
> Can you explain how some clients depend on a low delay when receiving data
> from icecast? How is the use of TCP_CORK going to break those? Would the
> application level buffering currently used in format_mp3.c not cause the
> same problem?
I imagine it would, yes. I don't use icecast for mp3 streaming,
though, personally, as I don't have a licensed encoder.

I can't think of any case where TCP_CORK would be appropriate for
icecast, though removing TCP_NODELAY (thus getting the default Nagle
algorithm behaviour) might help in some cases.

Mike