Klaas Jan Wierenga wrote:> Hi Henri and others, > > Very interesting post about TCP_CORK. I would be very interested in having > it applied in the next version of Icecast.I'd be more interested in some figures to show there being a benefit, most examples talk about HTTP servers with short lived connections where sendfile(2) is used.> For low-bitrate streams the problem originates in the fact that the stream > source often produces many small packets (they should be using TCP_CORK > too!), which were passed on unchanged by icecast to each client (again as > small write() calls on each socket). This problem had been reduced a lot in > Icecast 2.3.0 as can be read from the release notes of 2.3.0:This is exactly why it was implemented, a few people complained about the overhead with large numbers of listeners, not only because of the TCP overhead but also the fact that it reduces the write syscall overhead. Will TCP_CORK (linux) and TCP_NOPUSH (BSD) give noticable benefits wrt icecast? It might prove helpful if available but more info is needed.> As far as I can see this fix has been applied only for the mp3 format. I > guess the problem still remains for other formats, correct me if I am wrong.It applies to all passthrough streams (ie mp3, aac*, nsv), Ogg is different as it has pages which are generally not really small. karl
> This is exactly why it was implemented, a few people complained about > the overhead with large numbers of listeners, not only because of the > TCP overhead but also the fact that it reduces the write syscall > overhead. Will TCP_CORK (linux) and TCP_NOPUSH (BSD) give noticable > benefits wrt icecast? It might prove helpful if available but more info > is needed.As Christopher Baus points out in his article http://www.baus.net/on-tcp_cork it is not so much the overhead of a few write calls that affects performance, but the overhead in sending more smaller packets. In my case sending small packets resulted in a lot of interrupts (one or two for each packet sent) for many listeners listening to low bitrate streams. This resulted in jerky playback because the machine simply couldn't keep up. With icecast 2.3.0 the interrupt rate at the same level of listeners has been reduced by more than 50%. The question that still remains for me is why some clients would have a problem with a lower packet rate of packets filled to the brim as opposed to more packets that are partially filled (with the TCP/IP PSH flag set). In our application it is extremely important to use the low bandwidth available as efficient as possible to prevent buffer underflow and connection stall. Regards, KJ -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.371 / Virus Database: 267.14.9/216 - Release Date: 29-12-2005
Klaas Jan Wierenga wrote:>>This is exactly why it was implemented, a few people complained about >>the overhead with large numbers of listeners, not only because of the >>TCP overhead but also the fact that it reduces the write syscall >>overhead. Will TCP_CORK (linux) and TCP_NOPUSH (BSD) give noticable >>benefits wrt icecast? It might prove helpful if available but more info >>is needed. > > > As Christopher Baus points out in his article > http://www.baus.net/on-tcp_cork it is not so much the overhead of a few > write calls that affects performance, but the overhead in sending more > smaller packets. In my case sending small packets resulted in a lot of > interrupts (one or two for each packet sent) for many listeners listening to > low bitrate streams. This resulted in jerky playback because the machine > simply couldn't keep up. With icecast 2.3.0 the interrupt rate at the same > level of listeners has been reduced by more than 50%.I understand what you are saying about the interrupt rate, that is why I added the batching up of the read data, which applies to all platforms. The article in question doesn't deal with the exact same situation as icecast, for instance we don't want icecast to do say 100 write syscalls where each one is something like 50 bytes, for each listener. TCP_CORK or not, it's still wasteful. TCP_CORK/TCP_NOPUSH may be useful (where available), but doing some tests would be the way forward, especially as 2.3 has helped your situation already. karl.