freebsd stable - Jul 2006 - Please confirm: top(1) cpu time broken on FreeBSD 6.1 (RELEASE/STABLE) SMP ?

Hi all,

I'm currently seeing strang things going on on all our
upgraded SMP 2 CPU boxes. 4 Boxes are upgraded from 5.5 to
6 STABLE, one box is running 6.1 RELEASE.

All of them have SMP kernels, just using the provided GENERIC and SMP
kernel config files. Mergemaster has been done on all
boxes.

If a box is full loaded top(1) shows:

CPU states: 41.7% user,  0.0% nice,  7.7% system,  0.6% interrupt, 50.0% idle

I can buildworld -j 100, do verything what I want. The
sum of idle time never goes below 50.0% idle ! And the sum
of total CPU time never goes over 50.0%.

So I wonder if just top is broken (in 6.1 RELEASE too), or if this
is a SCHED4BSD bug.

Can someone confirm this issue ? We see it on IBL X-Series 346 Boxes,
as on IBM 306M Dual Core SMP systems.

Martin

Martin Blapp, <mb@imp.ch> <mbr@FreeBSD.org>
------------------------------------------------------------------
ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: <finger -l mbr@freebsd.org>
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
------------------------------------------------------------------

#include <stdio.h>

main() {
         while(1) {}
}

cc -o test.c test

./test &
./test &
./test &
./test &
./test &
./test &
./test &
./test &

CPU states: 50.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 50.0% idle
Mem: 1160M Active, 988M Inact, 254M Wired, 34M Cache, 112M Buf, 573M Free
Swap: 8192M Total, 8192M Free

   PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
21651 root        1 104    0  1216K   420K RUN    2   0:03 28.85% test
21653 root        1 103    0  1216K   420K CPU0   0   0:02 28.26% test
21649 root        1 104    0  1216K   420K RUN    2   0:03 26.12% test
21647 root        1 103    0  1216K   420K RUN    2   0:02 25.04% test
21650 root        1 103    0  1216K   420K RUN    0   0:02 24.50% test
21645 root        1 103    0  1216K   420K RUN    0   0:03 23.31% test
21646 root        1 104    0  1216K   420K RUN    2   0:03 22.81% test
21638 root        1 103    0  1216K   420K RUN    2   0:03 21.79% test


Martin Blapp, <mb@imp.ch> <mbr@FreeBSD.org>
------------------------------------------------------------------
ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: <finger -l mbr@freebsd.org>
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
------------------------------------------------------------------

What does dmesg say looks like you may be running a P4 there
with HT disabled hence the results.

Martin Blapp wrote:
> 
> If a box is full loaded top(1) shows:
> 
> CPU states: 41.7% user,  0.0% nice,  7.7% system,  0.6% interrupt,
> 50.0% idle 
> 
> I can buildworld -j 100, do verything what I want. The
> sum of idle time never goes below 50.0% idle ! And the sum
> of total CPU time never goes over 50.0%.

===============================================This e.mail is private and
confidential between Multiplay (UK) Ltd. and the person or entity to whom it is
addressed. In the event of misdirection, the recipient is prohibited from using,
copying, printing or otherwise disseminating it or any information contained in
it.

In the event of misdirection, illegible or incomplete transmission please
telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.

Le Mar 18 juillet 2006 15:31, Martin Blapp a ?crit :
>
> Hi all,
hi,
> I'm currently seeing strang things going on on all our
> upgraded SMP 2 CPU boxes. 4 Boxes are upgraded from 5.5 to
> 6 STABLE, one box is running 6.1 RELEASE.
i'm running 6.1 RELEASE upgraded from 6.0 RELEASE.
> All of them have SMP kernels, just using the provided GENERIC and SMP
> kernel config files. Mergemaster has been done on all
> boxes.
>
> If a box is full loaded top(1) shows:
>
> CPU states: 41.7% user,  0.0% nice,  7.7% system,  0.6% interrupt, 50.0%
> idle
>
> I can buildworld -j 100, do verything what I want. The
> sum of idle time never goes below 50.0% idle ! And the sum
> of total CPU time never goes over 50.0%.
i ran make buildworld -j8 and i confirm. afair it was ok on 6.0-RELEASE.
> So I wonder if just top is broken (in 6.1 RELEASE too), or if this
> is a SCHED4BSD bug.
>
> Can someone confirm this issue ? We see it on IBL X-Series 346 Boxes,
> as on IBM 306M Dual Core SMP systems.
i have a DELL 1600SC (bi-xeons) with a SMP kernel and
machdep.hyperthreading_allowed=0. setting it to 1 made idle % to 0.

-- 
Matthieu Michaud
EPITA SRS 2007

Hi,

Ok, seems to be a HTT issue on a untuned system.

On two boxes I get with 'sysctl -a | grep smp | grep kern.smp.cpus'
kern.smp.cpus: 4

Here I see CPU-IDs 0 and 2 in top. No other CPUs seem to be
used. Here I get the broken CPU usage.

And yes, seeting hyperthreading_allowed=1 corrects the top usage.

On two other boxes where it works I get 'sysctl -a | grep smp | grep 
kern.smp.cpus' kern.smp.cpus: 2

Here I see CPU-ID's 0 and 1 in top. Here everything works as it should.

Martin

Martin Blapp, <mb@imp.ch> <mbr@FreeBSD.org>
------------------------------------------------------------------
ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: <finger -l mbr@freebsd.org>
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
------------------------------------------------------------------