thr3ads.net - freebsd stable - Weird system cpu usage [Mar 2008]

If this information is useful, please help other people find it:
Share via:

Charlie Root

2008-Mar-19 09:32 UTC

Weird system cpu usage

Hello,

I have to report, that I have a very strange cpu usage by system (as
the `top' reports).  The given box does not currently run any threaded
applications (only lighttpd and php-fcgi with 80 children, maybe 100
reqs/s), but I can see the same behavior on almost identical machine
which is running mysqld (Threads: 100  Questions: 36641597  Slow
queries: 620  Opens: 1139  Flush tables: 1  Open tables: 1112  Queries
per second avg: 584.349).

Attached, I am providing dmesg.boot of the given system and an output
of `top -CSaIud 10'.

I have to note, that the system does not have I/O problems, the only
thing that might be related is an interrupt strom on irq22, which is
being shared by em1 and atapci0 (but the top's interrupt column
doesn't report too high interrupt cpu usage).  Both of the machines
have this interrupt problem.

root@[ha-web1 ~]# vmstat -i
interrupt                          total       rate
irq1: atkbd0                          12          0
irq16: ohci0                           1          0
irq17: ohci1 ohci3                     1          0
irq18: ohci2 ohci4                     1          0
irq20: em0                      86255835       1361
irq22: em1 atapci0           18611379049     293795
cpu0: timer                    126702021       2000
cpu1: timer                    126701933       2000
Total                        18951038853     299157

Any hints on what might be going on are welcome.
Thank you.

-- 
Sincerely,
  Daniel Gerzo
-------------- next part --------------
last pid: 15637;  load averages:  5.04,  5.16,  6.19  up 0+17:16:40    09:36:52
166 processes: 3 running, 148 sleeping, 15 waiting

Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2554M Free
Swap: 8192M Total, 8K Used, 8192M Free


  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:19 33.98% [idle: cpu1]
   11      0      1 171 ki31     0K    16K CPU0   0 518:21 30.96% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 0  20:48  2.49%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15425     80      1   4    0   379M 24952K accept 1   0:20  1.37%
/usr/local/bin/php-cgi
15472     80      1   4    0   379M 24496K accept 1   0:18  1.37%
/usr/local/bin/php-cgi
15502     80      1   4    0   379M 24420K accept 1   0:17  1.37%
/usr/local/bin/php-cgi
15450     80      1   4    0   378M 24828K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15474     80      1   4    0   379M 24684K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15452     80      1   4    0   379M 25160K sbwait 0   0:18  1.27%
/usr/local/bin/php-cgi
15471     80      1   4    0   378M 24300K accept 0   0:18  1.27%
/usr/local/bin/php-cgi
15482     80      1   4    0   379M 25844K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15480     80      1   4    0   378M 23488K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15503     80      1   4    0   379M 24580K sbwait 1   0:17  1.27%
/usr/local/bin/php-cgi
15506     80      1   4    0   378M 23048K accept 1   0:16  1.27%
/usr/local/bin/php-cgi
15512     80      1   4    0   378M 23200K sbwait 0   0:16  1.27%
/usr/local/bin/php-cgi
15423     80      1   4    0   379M 23368K accept 1   0:22  1.17%
/usr/local/bin/php-cgi
15426     80      1   4    0   378M 22864K accept 1   0:21  1.17%
/usr/local/bin/php-cgi
15442     80      1   4    0   378M 23204K accept 0   0:19  1.17%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  5.04,  5.16,  6.19  up 0+17:16:42    09:36:54
166 processes: 4 running, 147 sleeping, 15 waiting
CPU states: 16.4% user,  0.0% nice, 44.2% system,  0.4% interrupt, 39.1% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2554M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:20 35.35% [idle: cpu1]
   11      0      1 171 ki31     0K    16K RUN    0 518:22 32.18% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 0  20:48  2.59%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15502     80      1   4    0   379M 24420K accept 1   0:17  1.46%
/usr/local/bin/php-cgi
15450     80      1   4    0   378M 24828K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15474     80      1   4    0   379M 24684K accept 0   0:19  1.27%
/usr/local/bin/php-cgi
15472     80      1   4    0   379M 24536K accept 0   0:18  1.27%
/usr/local/bin/php-cgi
15471     80      1   4    0   378M 24300K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15506     80      1   4    0   379M 23172K accept 1   0:17  1.27%
/usr/local/bin/php-cgi
15425     80      1   4    0   379M 24952K sbwait 0   0:20  1.17%
/usr/local/bin/php-cgi
15442     80      1   4    0   378M 23204K accept 1   0:19  1.17%
/usr/local/bin/php-cgi
15458     80      1   4    0   378M 23680K accept 1   0:18  1.17%
/usr/local/bin/php-cgi
15461     80      1   4    0   379M 24676K accept 0   0:18  1.17%
/usr/local/bin/php-cgi
15418     80      1   4    0   378M 23988K accept 1   0:21  1.07%
/usr/local/bin/php-cgi
15431     80      1   4    0   379M 26100K accept 1   0:21  1.07%
/usr/local/bin/php-cgi
15428     80      1   4    0   378M 23472K accept 1   0:21  1.07%
/usr/local/bin/php-cgi
15426     80      1   4    0   378M 22864K accept 1   0:21  1.07%
/usr/local/bin/php-cgi
15430     80      1   4    0   378M 23840K accept 1   0:20  1.07%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  5.12,  5.17,  6.18  up 0+17:16:44    09:36:56
166 processes: 10 running, 141 sleeping, 15 waiting
CPU states: 17.6% user,  0.0% nice, 62.9% system,  0.0% interrupt, 19.5% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2553M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:21 33.40% [idle: cpu1]
   11      0      1 171 ki31     0K    16K RUN    0 518:23 29.49% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 1  20:48  2.29%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15502     80      1   4    0   379M 24420K accept 1   0:17  1.46%
/usr/local/bin/php-cgi
15472     80      1   4    0   379M 24536K accept 0   0:18  1.37%
/usr/local/bin/php-cgi
15482     80      1   4    0   379M 25844K accept 1   0:18  1.37%
/usr/local/bin/php-cgi
15518     80      1   4    0   378M 24472K accept 1   0:16  1.37%
/usr/local/bin/php-cgi
15450     80      1   4    0   378M 24828K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15474     80      1   4    0   379M 24684K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15452     80      1  45    0   379M 25160K RUN    0   0:19  1.27%
/usr/local/bin/php-cgi
15458     80      1   4    0   378M 23680K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15503     80      1  44    0   379M 24580K RUN    0   0:17  1.27%
/usr/local/bin/php-cgi
15506     80      1   4    0   379M 23172K accept 0   0:17  1.27%
/usr/local/bin/php-cgi
15418     80      1   4    0   378M 23988K accept 0   0:21  1.17%
/usr/local/bin/php-cgi
15442     80      1   4    0   378M 23204K accept 1   0:20  1.17%
/usr/local/bin/php-cgi
15469     80      1   4    0   379M 24464K accept 1   0:19  1.17%
/usr/local/bin/php-cgi
15471     80      1   4    0   378M 24300K accept 1   0:18  1.17%
/usr/local/bin/php-cgi
15500     80      1   4    0   379M 25068K accept 1   0:17  1.17%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  5.12,  5.17,  6.18  up 0+17:16:46    09:36:58
166 processes: 6 running, 145 sleeping, 15 waiting
CPU states: 19.5% user,  0.0% nice, 52.7% system,  0.4% interrupt, 27.4% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2553M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:21 32.86% [idle: cpu1]
   11      0      1 171 ki31     0K    16K RUN    0 518:23 29.59% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 1  20:48  2.49%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15502     80      1   4    0   379M 24420K accept 0   0:17  1.37%
/usr/local/bin/php-cgi
15426     80      1  -4    0   378M 22864K RUN    1   0:21  1.27%
/usr/local/bin/php-cgi
15425     80      1   4    0   379M 24952K accept 1   0:21  1.27%
/usr/local/bin/php-cgi
15450     80      1   4    0   378M 24828K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15469     80      1   4    0   379M 24464K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15452     80      1   4    0   379M 25160K accept 0   0:19  1.27%
/usr/local/bin/php-cgi
15472     80      1   4    0   379M 24536K accept 0   0:18  1.27%
/usr/local/bin/php-cgi
15482     80      1   4    0   379M 25844K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15501     80      1   4    0   378M 24812K sbwait 1   0:18  1.27%
/usr/local/bin/php-cgi
15506     80      1   4    0   379M 23172K accept 1   0:17  1.27%
/usr/local/bin/php-cgi
15445     80      1   4    0   378M 23340K sbwait 0   0:19  1.17%
/usr/local/bin/php-cgi
15474     80      1   4    0   379M 24684K accept 1   0:19  1.17%
/usr/local/bin/php-cgi
15477     80      1   4    0   379M 25240K accept 1   0:18  1.17%
/usr/local/bin/php-cgi
15458     80      1   4    0   378M 23680K accept 1   0:18  1.17%
/usr/local/bin/php-cgi
15503     80      1   4    0   379M 24580K sbwait 1   0:17  1.17%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  5.12,  5.17,  6.18  up 0+17:16:48    09:37:00
166 processes: 4 running, 147 sleeping, 15 waiting
CPU states: 26.7% user,  0.0% nice, 62.2% system,  0.2% interrupt, 10.9% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2553M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:21 29.30% [idle: cpu1]
   11      0      1 171 ki31     0K    16K RUN    0 518:23 27.49% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 0  20:48  2.78%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15468     80      1   4    0   379M 24796K accept 0   0:19  1.56%
/usr/local/bin/php-cgi
15469     80      1   4    0   379M 24464K accept 1   0:19  1.46%
/usr/local/bin/php-cgi
15479     80      1   4    0   378M 23620K accept 0   0:18  1.46%
/usr/local/bin/php-cgi
15458     80      1   4    0   378M 23680K accept 1   0:18  1.37%
/usr/local/bin/php-cgi
15425     80      1   4    0   379M 24952K accept 0   0:21  1.27%
/usr/local/bin/php-cgi
15464     80      1   4    0   378M 22484K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15445     80      1   4    0   378M 23340K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15473     80      1   4    0   378M 23960K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15452     80      1   4    0   379M 25160K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15482     80      1   4    0   379M 25844K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15501     80      1   4    0   378M 24812K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15502     80      1   4    0   379M 24420K sbwait 1   0:17  1.27%
/usr/local/bin/php-cgi
15488     80      1   4    0   378M 21548K accept 1   0:17  1.27%
/usr/local/bin/php-cgi
15503     80      1   4    0   379M 24580K accept 0   0:17  1.27%
/usr/local/bin/php-cgi
15506     80      1   4    0   379M 23172K accept 0   0:17  1.27%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  4.95,  5.14,  6.17  up 0+17:16:50    09:37:02
166 processes: 3 running, 148 sleeping, 15 waiting
CPU states: 19.5% user,  0.0% nice, 50.6% system,  0.6% interrupt, 29.3% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2553M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:22 29.79% [idle: cpu1]
   11      0      1 171 ki31     0K    16K CPU0   0 518:24 27.49% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 0  20:48  2.59%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15506     80      1   4    0   379M 23172K accept 1   0:17  1.86%
/usr/local/bin/php-cgi
15469     80      1   4    0   379M 24464K accept 1   0:19  1.46%
/usr/local/bin/php-cgi
15468     80      1   4    0   379M 24796K accept 1   0:19  1.37%
/usr/local/bin/php-cgi
15479     80      1   4    0   378M 23620K accept 1   0:18  1.37%
/usr/local/bin/php-cgi
15482     80      1   4    0   379M 25844K accept 1   0:18  1.37%
/usr/local/bin/php-cgi
15515     80      1   4    0   378M 23916K accept 1   0:17  1.37%
/usr/local/bin/php-cgi
15426     80      1   4    0   378M 22864K accept 1   0:21  1.27%
/usr/local/bin/php-cgi
15425     80      1   4    0   379M 24952K accept 1   0:21  1.27%
/usr/local/bin/php-cgi
15454     80      1   4    0   379M 24356K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15458     80      1   4    0   378M 23680K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15472     80      1   4    0   379M 24536K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15451     80      1   4    0   379M 25916K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15516     80      1   4    0   379M 24636K accept 1   0:17  1.27%
/usr/local/bin/php-cgi
15512     80      1   4    0   378M 23720K accept 1   0:17  1.27%
/usr/local/bin/php-cgi
15430     80      1   4    0   378M 23840K accept 1   0:20  1.17%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  4.95,  5.14,  6.17  up 0+17:16:52    09:37:04
166 processes: 5 running, 146 sleeping, 15 waiting
CPU states: 17.0% user,  0.0% nice, 57.1% system,  1.5% interrupt, 24.3% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2553M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:23 29.59% [idle: cpu1]
   11      0      1 171 ki31     0K    16K RUN    0 518:24 26.56% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 1  20:49  2.88%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15469     80      1   4    0   379M 24464K accept 1   0:19  1.56%
/usr/local/bin/php-cgi
15506     80      1   4    0   379M 23172K accept 0   0:17  1.56%
/usr/local/bin/php-cgi
15454     80      1   4    0   379M 24356K accept 0   0:19  1.37%
/usr/local/bin/php-cgi
15472     80      1   4    0   379M 24536K accept 1   0:19  1.37%
/usr/local/bin/php-cgi
15479     80      1   4    0   378M 23620K accept 0   0:18  1.37%
/usr/local/bin/php-cgi
15516     80      1   4    0   379M 24636K accept 1   0:17  1.37%
/usr/local/bin/php-cgi
15425     80      1   4    0   379M 24952K accept 1   0:21  1.27%
/usr/local/bin/php-cgi
15463     80      1   4    0   379M 25348K sbwait 1   0:19  1.27%
/usr/local/bin/php-cgi
15468     80      1   4    0   379M 24796K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15474     80      1   4    0   379M 24772K accept 0   0:19  1.27%
/usr/local/bin/php-cgi
15458     80      1   4    0   378M 23680K accept 0   0:19  1.27%
/usr/local/bin/php-cgi
15482     80      1   4    0   379M 25844K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15512     80      1   4    0   378M 23720K accept 0   0:17  1.27%
/usr/local/bin/php-cgi
15430     80      1   4    0   378M 23840K accept 1   0:20  1.17%
/usr/local/bin/php-cgi
15456     80      1   4    0   379M 24012K accept 1   0:20  1.17%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  4.55,  5.05,  6.13  up 0+17:16:54    09:37:06
166 processes: 7 running, 144 sleeping, 15 waiting
CPU states: 18.4% user,  0.0% nice, 51.3% system,  1.9% interrupt, 28.4% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2553M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:23 29.30% [idle: cpu1]
   11      0      1 171 ki31     0K    16K RUN    0 518:25 27.20% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K RUN    1  20:49  3.27%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15469     80      1   4    0   379M 24464K accept 1   0:19  1.56%
/usr/local/bin/php-cgi
15506     80      1   4    0   379M 23172K accept 1   0:17  1.56%
/usr/local/bin/php-cgi
15473     80      1   4    0   378M 23960K accept 0   0:19  1.46%
/usr/local/bin/php-cgi
15479     80      1   4    0   378M 23620K accept 1   0:18  1.37%
/usr/local/bin/php-cgi
15516     80      1   4    0   379M 24636K accept 1   0:17  1.37%
/usr/local/bin/php-cgi
15456     80      1   4    0   379M 24012K accept 1   0:20  1.27%
/usr/local/bin/php-cgi
15454     80      1   4    0   379M 24356K sbwait 1   0:19  1.27%
/usr/local/bin/php-cgi
15468     80      1   4    0   379M 24796K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15474     80      1   4    0   379M 24772K accept 0   0:19  1.27%
/usr/local/bin/php-cgi
15452     80      1   4    0   379M 25160K accept 0   0:19  1.27%
/usr/local/bin/php-cgi
15458     80      1   4    0   378M 23680K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15472     80      1   4    0   379M 24536K accept 1   0:19  1.27%
/usr/local/bin/php-cgi
15482     80      1   4    0   379M 25848K accept 1   0:18  1.27%
/usr/local/bin/php-cgi
15422     80      1  45    0   379M 24648K RUN    0   0:23  1.17%
/usr/local/bin/php-cgi
15424     80      1   4    0   378M 23604K accept 0   0:21  1.17%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  4.55,  5.05,  6.13  up 0+17:16:56    09:37:08
166 processes: 18 running, 133 sleeping, 15 waiting
CPU states: 25.2% user,  0.0% nice, 69.0% system,  0.4% interrupt,  5.5% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2553M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:23 26.17% [idle: cpu1]
   11      0      1 171 ki31     0K    16K RUN    0 518:25 21.58% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 1  20:49  3.08%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15472     80      1   4    0   379M 24536K accept 1   0:19  1.66%
/usr/local/bin/php-cgi
15486     80      1  -4    0   378M 21440K RUN    0   0:18  1.66%
/usr/local/bin/php-cgi
15474     80      1   4    0   379M 24772K accept 1   0:19  1.56%
/usr/local/bin/php-cgi
15481     80      1   4    0   378M 23180K accept 1   0:18  1.56%
/usr/local/bin/php-cgi
15469     80      1   4    0   379M 24464K accept 1   0:19  1.46%
/usr/local/bin/php-cgi
15463     80      1   4    0   379M 25436K accept 1   0:19  1.46%
/usr/local/bin/php-cgi
15452     80      1   4    0   379M 25160K RUN    1   0:19  1.46%
/usr/local/bin/php-cgi
15506     80      1   4    0   379M 23172K accept 1   0:17  1.46%
/usr/local/bin/php-cgi
15445     80      1   4    0   378M 23340K accept 0   0:19  1.37%
/usr/local/bin/php-cgi
15447     80      1   4    0   378M 23800K accept 0   0:19  1.37%
/usr/local/bin/php-cgi
15473     80      1   4    0   378M 23960K accept 1   0:19  1.37%
/usr/local/bin/php-cgi
15477     80      1   4    0   379M 25240K accept 0   0:19  1.37%
/usr/local/bin/php-cgi
15476     80      1   4    0   378M 22972K sbwait 1   0:19  1.37%
/usr/local/bin/php-cgi
15482     80      1  -4    0   379M 25976K RUN    0   0:18  1.37%
/usr/local/bin/php-cgi
15423     80      1  -4    0   379M 23368K RUN    0   0:22  1.27%
/usr/local/bin/php-cgi

last pid: 15637;  load averages:  4.55,  5.05,  6.13  up 0+17:16:58    09:37:10
166 processes: 3 running, 148 sleeping, 15 waiting
CPU states: 16.3% user,  0.0% nice, 57.1% system,  0.4% interrupt, 26.2% idle
Mem: 317M Active, 740M Inact, 249M Wired, 48M Cache, 214M Buf, 2553M Free
Swap: 8192M Total, 8K Used, 8192M Free

  PID    UID    THR PRI NICE   SIZE    RES STATE  C   TIME    CPU COMMAND
   10      0      1 171 ki31     0K    16K RUN    1 501:24 26.46% [idle: cpu1]
   11      0      1 171 ki31     0K    16K CPU0   0 518:25 23.29% [idle: cpu0]
 1425     80      1   4    0 39968K 21568K kqread 0  20:49  2.39%
/usr/local/sbin/lighttpd -f /usr/local/etc/lighttpd.conf
15486     80      1   4    0   378M 21440K accept 0   0:18  1.86%
/usr/local/bin/php-cgi
15452     80      1   4    0   379M 25160K sbwait 1   0:19  1.56%
/usr/local/bin/php-cgi
15481     80      1   4    0   378M 23180K accept 1   0:18  1.56%
/usr/local/bin/php-cgi
15422     80      1   4    0   379M 24704K accept 1   0:23  1.37%
/usr/local/bin/php-cgi
15423     80      1   4    0   379M 23368K accept 1   0:22  1.37%
/usr/local/bin/php-cgi
15469     80      1   4    0   379M 24464K accept 1   0:19  1.37%
/usr/local/bin/php-cgi
15447     80      1   4    0   378M 23800K accept 1   0:19  1.37%
/usr/local/bin/php-cgi
15463     80      1   4    0   379M 25436K sbwait 1   0:19  1.37%
/usr/local/bin/php-cgi
15474     80      1   4    0   379M 24772K accept 1   0:19  1.37%
/usr/local/bin/php-cgi
15472     80      1   4    0   379M 24536K sbwait 1   0:19  1.37%
/usr/local/bin/php-cgi
15477     80      1   4    0   379M 25240K accept 0   0:19  1.37%
/usr/local/bin/php-cgi
15476     80      1   4    0   378M 22972K sbwait 1   0:19  1.37%
/usr/local/bin/php-cgi
15467     80      1   4    0   378M 23168K accept 1   0:19  1.37%
/usr/local/bin/php-cgi
15480     80      1   4    0   378M 23488K sbwait 0   0:18  1.37%
/usr/local/bin/php-cgi
15493     80      1   4    0   378M 24876K accept 1   0:18  1.37%
/usr/local/bin/php-cgi

-------------- next part --------------
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-STABLE #2: Tue Mar 18 16:16:43 CET 2008
    danger@ha-web1.hockeyarena.net:/usr/obj/usr/src/sys/ha-web1
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 5600+ (2799.98-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x40f33  Stepping = 3
 
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x2001<SSE3,CX16>
  AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x1f<LAHF,CMP,SVM,ExtAPIC,CR8>
  Cores per package: 2
usable memory = 4253347840 (4056 MB)
avail memory  = 4099985408 (3910 MB)
ACPI APIC Table: <M S I  OEMAPIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
ioapic0 <Version 2.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <M S I OEMRSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, ddf00000 (3) failed
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_throttle0: <ACPI CPU Throttling> on cpu0
acpi_throttle0: CLK_VAL field overlaps THT_EN bit
device_attach: acpi_throttle0 attach returned 6
cpu1: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xc000-0xc0ff mem
0xfc000000-0xfdffffff,0xfe9f0000-0xfe9fffff,0xfe800000-0xfe8fffff irq 18 at
device 5.0 on pci1
pci1: <multimedia> at device 5.2 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 7.0 on pci0
pci2: <ACPI PCI bus> on pcib2
atapci0: <ATI ATA controller> port
0xb000-0xb007,0xa000-0xa003,0x9000-0x9007,0x8000-0x8003,0x7000-0x700f mem
0xfe7ff800-0xfe7ffbff irq 22 at device 18.0 on pci0
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
ohci0: <OHCI (generic) USB controller> mem 0xfe7fe000-0xfe7fefff irq 16 at
device 19.0 on pci0
ohci0: [GIANT-LOCKED]
ohci0: [ITHREAD]
usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
ohci1: <OHCI (generic) USB controller> mem 0xfe7fd000-0xfe7fdfff irq 17 at
device 19.1 on pci0
ohci1: [GIANT-LOCKED]
ohci1: [ITHREAD]
usb1: OHCI version 1.0, legacy support
usb1: SMM does not respond, resetting
usb1: <OHCI (generic) USB controller> on ohci1
usb1: USB revision 1.0
uhub1: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
ohci2: <OHCI (generic) USB controller> mem 0xfe7fc000-0xfe7fcfff irq 18 at
device 19.2 on pci0
ohci2: [GIANT-LOCKED]
ohci2: [ITHREAD]
usb2: OHCI version 1.0, legacy support
usb2: SMM does not respond, resetting
usb2: <OHCI (generic) USB controller> on ohci2
usb2: USB revision 1.0
uhub2: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
uhub2: 2 ports with 2 removable, self powered
ohci3: <OHCI (generic) USB controller> mem 0xfe7fb000-0xfe7fbfff irq 17 at
device 19.3 on pci0
ohci3: [GIANT-LOCKED]
ohci3: [ITHREAD]
usb3: OHCI version 1.0, legacy support
usb3: SMM does not respond, resetting
usb3: <OHCI (generic) USB controller> on ohci3
usb3: USB revision 1.0
uhub3: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
uhub3: 2 ports with 2 removable, self powered
ohci4: <OHCI (generic) USB controller> mem 0xfe7fa000-0xfe7fafff irq 18 at
device 19.4 on pci0
ohci4: [GIANT-LOCKED]
ohci4: [ITHREAD]
usb4: OHCI version 1.0, legacy support
usb4: SMM does not respond, resetting
usb4: <OHCI (generic) USB controller> on ohci4
usb4: USB revision 1.0
uhub4: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb4
uhub4: 2 ports with 2 removable, self powered
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfe7ff000-0xfe7ff0ff irq
19 at device 19.5 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb5: EHCI version 1.0
usb5: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4
usb5: <EHCI (generic) USB 2.0 controller> on ehci0
usb5: USB revision 2.0
uhub5: <ATI EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb5
uhub5: 10 ports with 10 removable, self powered
pci0: <serial bus, SMBus> at device 20.0 (no driver attached)
atapci1: <ATI IXP600 UDMA133 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xff00-0xff0f at device 20.1 on pci0
ata0: <ATA channel 0> on atapci1
ata0: [ITHREAD]
isab0: <PCI-ISA bridge> at device 20.3 on pci0
isa0: <ISA bus> on isab0
pcib3: <ACPI PCI-PCI bridge> at device 20.4 on pci0
pci3: <ACPI PCI bus> on pcib3
em0: <Intel(R) PRO/1000 Network Connection Version - 6.7.3> port
0xe800-0xe83f mem 0xfebe0000-0xfebfffff,0xfebc0000-0xfebdffff irq 20 at device
2.0 on pci3
em0: Ethernet address: 00:1b:21:10:f7:d5
em0: [FILTER]
em1: <Intel(R) PRO/1000 Network Connection Version - 6.7.3> port
0xe400-0xe43f mem 0xfeb80000-0xfeb9ffff,0xfeb60000-0xfeb7ffff irq 22 at device
5.0 on pci3
em1: Ethernet address: 00:1b:21:10:f2:27
em1: [FILTER]
acpi_button0: <Power Button> on acpi0
sio0: configured irq 3 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: configured irq 3 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 flags 0x10 on
acpi0
sio0: type 16550A
sio0: [FILTER]
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
orm0: <ISA Option ROMs> at iomem 0xcd800-0xce7ff,0xce800-0xcf7ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
ad4: 381554MB <SAMSUNG HD403LJ CT100-12> at ata2-master UDMA33
ad6: 381554MB <SAMSUNG HD403LJ CT100-12> at ata3-master UDMA33
SMP: AP CPU #1 Launched!
GEOM_MIRROR: Device mirror/root launched (2/2).
GEOM_MIRROR: Device mirror/swap launched (2/2).
GEOM_MIRROR: Device mirror/var launched (2/2).
GEOM_MIRROR: Device mirror/usr launched (2/2).
GEOM_MIRROR: Device mirror/data launched (2/2).
Trying to mount root from ufs:/dev/mirror/root

Oliver Fromme

2008-Mar-19 17:53 UTC

head link

Weird system cpu usage

Charlie Root <root@ha-web1.hockeyarena.net> wrote:
 > [...]

It's preferable to send mail as a real user, not as root,
for various reasons.

 > I have to report, that I have a very strange cpu usage by system (as
 > the `top' reports).

You haven't mentioned what exactly you think is strange
in your top(1) output.  I think it looks pretty normal
under the given circumstances.

 > root@[ha-web1 ~]# vmstat -i
 > interrupt                          total       rate
 > irq1: atkbd0                          12          0
 > irq16: ohci0                           1          0
 > irq17: ohci1 ohci3                     1          0
 > irq18: ohci2 ohci4                     1          0
 > irq20: em0                      86255835       1361
 > irq22: em1 atapci0           18611379049     293795

Now that looks unusual indeed.  Do you get that rate
on irq22 right after boot, before the services have
started?  It looks like either hardware or driver
problems.  Do you have polling enabled on em1?

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch?ftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n-
chen, HRB 125758,  Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"Being really good at C++ is like being really good
at using rocks to sharpen sticks."
        -- Thant Tessman

Jeremy Chadwick

2008-Mar-19 18:06 UTC

head link

Weird system cpu usage

On Wed, Mar 19, 2008 at 06:53:36PM +0100, Oliver Fromme
wrote:> Charlie Root <root@ha-web1.hockeyarena.net> wrote:
>  > root@[ha-web1 ~]# vmstat -i
>  > interrupt                          total       rate
>  > irq1: atkbd0                          12          0
>  > irq16: ohci0                           1          0
>  > irq17: ohci1 ohci3                     1          0
>  > irq18: ohci2 ohci4                     1          0
>  > irq20: em0                      86255835       1361
>  > irq22: em1 atapci0           18611379049     293795
> 
> Now that looks unusual indeed.  Do you get that rate
> on irq22 right after boot, before the services have
> started?  It looks like either hardware or driver
> problems.  Do you have polling enabled on em1?
Also, I believe there was a report from another user who saw similar
issues with em(4), and found that disabling MSI fixed the storm in
question.  I believe you can disable MSI/MSIX by placing the following
in /boot/loader.conf, then reboot:

hw.pci.enable_msi="0"
hw.pci.enable_msix="0"

-- 
| Jeremy Chadwick                                    jdc at parodius.com |
| Parodius Networking                           http://www.parodius.com/ |
| UNIX Systems Administrator                      Mountain View, CA, USA |
| Making life hard for others since 1977.                  PGP: 4BD6C0CB |

freebsd stable - Mar 2008 - Weird system cpu usage

Weird system cpu usage

Weird system cpu usage

Weird system cpu usage