Klaas Jan Wierenga wrote:> I suspect the Linux systems are running a nightly or weekly cron job > they weren't running before causing an excessive load on CPU or network > which caused the clients to disconnect. The nightly cron jobs are > generally located in /etc/cron.daily and run a 4am!. Check for any > changes in the nightly cron jobs.Thanks. I didn't know that 4 am was magic but had suspected it might be a cron job. Nothing would have been added recently. These have been in place varying lengths of time but are pretty much clones of each other with a little remote maintenance done on the first ones installed. But nothing has changed within the past 8-10 months. The only cron.daily that I put in was to do an rdate time synchronization with time-b.nist.gov. That's been in place since the beginning. But there were some default ones that are there from the initial FC1 installation. Maybe a yum or rpm? In any case, knowing the 4:00am magic time lets me know that it had nothing to do with Icecast. -- Regards Dick Trump Triad AV Services 1910 Ingersoll Ave. Des Moines, IA 50309 515-243-2125 515-243-2055 (fax) http://www.triadav.com dtrump1@triadav.com
Hi, Even excessive load shouldn't kick out clients. Although the playing of the audiostream may become a bit shaky, it wouldn't stop completely. Or, if it would indeed stop, it wouldn't restart without the computer being rebooted, and for sure not restart after 10 minutes. That doesn't make any sense. What's more: missing interrupts and shaky playback would only eventually get you into problems, since icecast has a large enough buffer to not kick out clients that are several tens of seconds behind. And IF icecast kicks out a client for being too far behind, it will mention that in the log file. So, I don't think this is the case here. Did you check the Linux machines for system and/or update logs at the given time? Maybe there was a power surge and the machines rebooted? Or maybe a network switch was at fault? Maybe a new kernel was installed that required a reboot? What I know of FC is that they do have kernel updates every once in a while, and those updates naturally require a reboot. (And, if your machine has been running for months, checking hard disks will probably take a few minutes, 10 wouldn't be that unusual.) The fact that icecast didn't kick out all clients at the same time, does not indicate the clients also went "down" at different times. Icecast "works in mysterious ways" when it comes to kicking clients. So you better check the Linux system's log files and uptime, maybe that can clear things up. (Also check /etc/crontab and if there's something in /etc/cron.d that would run around 04:00...) Regards, Maarten On Thu, 7 Sep 2006, Dick Trump wrote:> Klaas Jan Wierenga wrote: > > I suspect the Linux systems are running a nightly or weekly cron job > > they weren't running before causing an excessive load on CPU or network > > which caused the clients to disconnect. The nightly cron jobs are > > generally located in /etc/cron.daily and run a 4am!. Check for any > > changes in the nightly cron jobs. > > Thanks. I didn't know that 4 am was magic but had suspected it might be a cron job. Nothing would have been added recently. These have been in place varying lengths of time but are pretty much clones of each other with a little remote maintenance done on the first ones installed. But nothing has changed within the past 8-10 months. > > The only cron.daily that I put in was to do an rdate time synchronization with time-b.nist.gov. That's been in place since the beginning. But there were some default ones that are there from the initial FC1 installation. Maybe a yum or rpm? > > In any case, knowing the 4:00am magic time lets me know that it had nothing to do with Icecast.
Maarten makes some poses some valid questions:> Did you check the Linux machines for system and/or update logs at the > given time?I'm afraid I don't know all the places to look for logs that might tell me something. My script that keeps MPG123 running keeps a log of all reboots and outages. A reconnect that lasts less than one minute is counted as continuous outage. Looking more closely at my server's error.log, sorting the entries by client, during that 10 minute span (actually only 7 minute), the actual outages recorded as only a second long and from 2 to 4 drops per client. One of the drops occurred within a second of each other on 5 machines.> Maybe there was a power surge and the machines rebooted?No. The client machines were in 5 different locations, hundreds of miles apart.> Or maybe a network switch was at fault?The only switch in common was at the server. The three Windows clients that stayed up went through the same switch and were at 3 equally spread out locations.> Maybe a new kernel was installed that required a reboot?No reboot was logged in the log my script creates.> What I know of FC is that they do have kernel updates > every once in a while, and those updates naturally require a reboot.I'm on FC1. I don't think that kernel is in development. I'm sure they didn't reboot. I would have a log entry of that.> (And, if your machine has been running for months, checking hard disks > will probably take a few minutes, 10 wouldn't be that unusual.)But all on the same date? These machines were started on completely different dates in different cities. All are 13 GB drives with only 12% in use. I'm still focusing on a cron job being responsible. But it is a curiosity thing only. I don't see a long term problem. Based on the fact that the actual drops were so short, I'm guessing that some process that did run on a cron somehow fooled my detection procedure for a dropped connection. I know that it isn't perfect but it does work. I'm even further convinced that there was nothing in Icecast that was responsible. It has been incredibly reliable. Thanks for your input. Regards -- Dick dtrump1@triadav.com