Ok I've been discovering many bugs in Icecast's YP implementation.. The most serious of which is, if a YP server is rejecting a stream the server has a memory leak which, as it continues to re-try to publish it's rejected streams, grows until the server segfaults. On our server, for instance, we're relaying shoutcast streams (which does not relay the stream name and other metadata) which are rejected from the YP server despite that they are comming from an Icecast2 server and using proper protocols because of the missing data. This grows over time and, as a result, our relay server continues to segfault. Also, last night I was working on the YP server just before our server suffered from some network issue and was taken offline (but still working). A stream source running from the box (from a playlist) continued to run to our main Icecast process which kept getting rejected from the YP server.. when we got the system back online I found this: 37088 nobody 2 0 213M 47696K RUN 55:12 0.00% 0.00% icecast Here are the bugs as I've found so far... 1) Icecast memory leak, specifically caused by rejected YP add/touches 2) Should relay icy/x-audiocast stream metadata when connecting to icecast1/shoutcast/etc so that this information is available to YP 3) Should remember that a stream was rejected by a YP server, and either not try to publish it again or wait a reasonable amount of time (ie, 15 minutes) before trying again, vs re-trying several times a minute The combination of these three bugs have made our setup unstable.. the only solution we've thought of is to move the streams from shoutcast to their own icecast2 process which doesnt publish to YP servers until this issue is corrected.. <p> -------------- next part -------------- A non-text attachment was scrubbed... Name: part Type: application/pgp-signature Size: 188 bytes Desc: not available Url : http://lists.xiph.org/pipermail/icecast-dev/attachments/20030624/37afb606/part.pgp
On Wednesday 25 June 2003 01:58, Arc shaped the electrons to say:> Ok I've been discovering many bugs in Icecast's YP implementation.. > > The most serious of which is, if a YP server is rejecting a stream the > server has a memory leak which, as it continues to re-try to publish > it's rejected streams, grows until the server segfaults. On our server, > for instance, we're relaying shoutcast streams (which does not relay the > stream name and other metadata) which are rejected from the YP server > despite that they are comming from an Icecast2 server and using proper > protocols because of the missing data. This grows over time and, as a > result, our relay server continues to segfault. > > Also, last night I was working on the YP server just before our server > suffered from some network issue and was taken offline (but still > working). A stream source running from the box (from a playlist) > continued to run to our main Icecast process which kept getting rejected > from the YP server.. when we got the system back online I found this: > > 37088 nobody 2 0 213M 47696K RUN 55:12 0.00% 0.00% icecast > > Here are the bugs as I've found so far... > > 1) Icecast memory leak, specifically caused by rejected YP add/touchesI've sent a patch ages ago (around september-october last year, check archives) that solves one side of this proble. It's a one-liner. Easealy reproductable: do a script that try to connect to a non-existing stream or that connects and disconnect inmediately. In few ours you will see the OOM killer. <p> -- ricardo galli GPG id C8114D34 --- >8 ---- List archives: http://www.xiph.org/archives/ icecast project homepage: http://www.icecast.org/ To unsubscribe from this list, send a message to 'icecast-dev-request@xiph.org' containing only the word 'unsubscribe' in the body. No subject is needed. Unsubscribe messages sent to the list will be ignored/filtered.
> 2) Should relay icy/x-audiocast stream metadata when connecting to > icecast1/shoutcast/etc so that this information is available to YPEh? It does. I even tested that with someone's shoutcast stream. It works. Just add <relay-shoutcast-metadata>1</relay-shoutcast-metadata> to the appropriate <relay> Mike --- >8 ---- List archives: http://www.xiph.org/archives/ icecast project homepage: http://www.icecast.org/ To unsubscribe from this list, send a message to 'icecast-dev-request@xiph.org' containing only the word 'unsubscribe' in the body. No subject is needed. Unsubscribe messages sent to the list will be ignored/filtered.
On Wed, Jun 25, 2003 at 11:34:07AM +1000, Michael Smith wrote:> > > 2) Should relay icy/x-audiocast stream metadata when connecting to > > icecast1/shoutcast/etc so that this information is available to YP > > Eh? It does. I even tested that with someone's shoutcast stream. It works. > Just add > <relay-shoutcast-metadata>1</relay-shoutcast-metadata> > to the appropriate <relay>It's not working... how long has this been in CVS? Actually all the relay entries, even the ones that are relaying Icecast2, have it. Here's the entry for "Enemy Combatant Radio"'s hifi MP3 that is not working: <relay> <server>radio.us2.indymedia.org</server> <port>8200</port> <mount>/</mount> <local-mount>/sf-hifi.mp3</local-mount> <relay-shoutcast-metadata>1</relay-shoutcast-metadata> </relay> Here's what's being sent to, and responding from, the YP CGI: Wed Jun 25 07:05:52 2003 < [MiniFieldStorage('action', 'add'), MiniFieldStorage('listenurl', 'http://liveradio.indymedia.org:8001/sf-hifi.mp3'), MiniFieldStorage('type', 'MP3 audio')]>YPResponse: 0 YPMessage: Stream name is required. And here's the header from the source server: icy-name:Enemy Combatant Radio icy-genre:Talk icy-url:http://sf.indymedia.org/ Content-Type:audio/mpeg icy-pub:1 icy-br:64 Clearly, the stream name is not being relayed from shoutcast. I compiled this icecast from CVS less than a week ago, I believe, in an attempt to fix YP and thus make it useable. What's the alternative to <relay-shoutcast-metadata> if you want to relay Icecast2 metadata vs shoutcast metadata, so I can fix the non-shoutcast entries? Do you use this for icecast1 servers as well? <p> -------------- next part -------------- A non-text attachment was scrubbed... Name: part Type: application/pgp-signature Size: 188 bytes Desc: not available Url : http://lists.xiph.org/pipermail/icecast-dev/attachments/20030625/80e5943b/part.pgp