Hiya, Currently running Centos 4.2 x86_64 dist on a dual 3G xeon, 2G ram, scsi setupand everythings been running fine on it for some time. Then at 4am last night something kicked in (have mrtg running monitoring when) and since then its been running a load of about 1.5 (normally around 0.4). CPU usage is Cpu(s): 1.1% us, 0.6% sy, 0.0% ni, 97.9% id, 0.2% wa, 0.1% hi, 0.1%si. Can't see any new processes that would cause the load, just wondering is there any way to try and track down whats actually causing this? It's not excessive load, but want to add some new services and wary now, its something that seems wrong given the sudden increase at 4am (think thats when some o.s housekeeping tasks are normally scheduled, but there's none running that I can see that started today). Just hoping someone may have some tips on checking whats always waiting or how to isolate whats happening. As said, ps -ef shows no new processes, and cpu usage is very low. Tia, Ian -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.centos.org/pipermail/centos/attachments/20060621/bf607c4d/attachment-0002.html>
On 21/06/06, Ian mu <mu.llamas at gmail.com> wrote:> Just hoping someone may have some tips on checking whats always waiting or > how to isolate whats happening. As said, ps -ef shows no new processes, and > cpu usage is very low.top vmstat 5 ... would be two good places to start. 4am sounds like about the time jobs from /etc/cron.daily would kick off. [root at willspc ~]# grep cron.daily /etc/crontab 02 4 * * * root run-parts /etc/cron.daily Have a look and see what's in cron.daily. On a recently built minimal install I have... 00-logwatch 0anacron prelink slocate.cron yum.cron 00-makewhatis.cron logrotate rpm tmpwatch It could just be updatedb building your slocate database? (Check /etc/updatedb.conf for DAILY_UPDATE= ). Will.
Ian mu wrote:> Hiya, > > Currently running Centos 4.2 x86_64 dist on a dual 3G xeon, 2G ram, > scsi setupand everythings been running fine on it for some time. Then > at 4am last night something kicked in (have mrtg running monitoring > when) and since then its been running a load of about 1.5 (normally > around 0.4). CPU usage is Cpu(s): 1.1% us, 0.6% sy, 0.0% ni, 97.9% > id, 0.2% wa, 0.1% hi, 0.1% si. > > Can't see any new processes that would cause the load, just wondering > is there any way to try and track down whats actually causing this? > It's not excessive load, but want to add some new services and wary > now, its something that seems wrong given the sudden increase at 4am > (think thats when some o.s housekeeping tasks are normally scheduled, > but there's none running that I can see that started today). > > Just hoping someone may have some tips on checking whats always > waiting or how to isolate whats happening. As said, ps -ef shows no > new processes, and cpu usage is very low. >Have you been up to date with patches? Have you tried running rkhunter and chkrootkit to see if you've been burgled? One of the first things a rootkit does is replace things like ps so it's processes become "invisible." Cheers,
On Wednesday 21 June 2006 12:17, Ian mu wrote:> Hiya, > > Currently running Centos 4.2 x86_64 dist on a dual 3G xeon, 2G ram, > scsi setupand everythings been running fine on it for some time. Then at > 4am last night something kicked in (have mrtg running monitoring when) and > since then its been running a load of about 1.5 (normally around 0.4). CPU > usage is Cpu(s): 1.1% us, 0.6% sy, 0.0% ni, 97.9% id, 0.2% wa, 0.1% > hi, 0.1%si. > > Can't see any new processes that would cause the load, just wondering is > there any way to try and track down whats actually causing this? It's not > excessive load, but want to add some new services and wary now, its > something that seems wrong given the sudden increase at 4am (think thats > when some o.s housekeeping tasks are normally scheduled, but there's none > running that I can see that started today). > > Just hoping someone may have some tips on checking whats always waiting or > how to isolate whats happening.hit it with the big sledgehammer, oprofile :-) that will very likely tell you if the kernel is doing something (besides the expected mwait_idle-ish...). rough guide (not for cut-n-paste): yum install oprofile kernel-smp-devel (and manual install of kernel-debuginfo) opcontrol --setup --vmlinux=...vmlinux from -debuginfo opcontrol --reset ; --start ; sleep ; --stop opreport -l -p /lib/modules/$uname -r | head /Peter> As said, ps -ef shows no new processes, and > cpu usage is very low. > > Tia, Ian-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 191 bytes Desc: not available URL: <http://lists.centos.org/pipermail/centos/attachments/20060621/ad1ae2d9/attachment-0002.sig>