search for: syscl

HI, T5220, S10U4 + patches mdb -k > ::memstat While above is working (takes some time, ideally ::memstat -n 4 to use 4 threads could be useful) mpstat 1 shows: CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 48 0 0 1922112 9 0 0 8 0 0 0 15254 6 94 0 0 So about 2mln xcalls per second. Let''s check with dtrace: dtrace -n sysinfo:::xcalls''{@=count();}'' -n tick-1s''{printa(@);clear(@);}'' dtrace: description...

In crossbow, each mac_srs has a kernel thread called "mac_rx_srs_poll_ring" to poll the hardware and crossbow will wakeup this thread to poll packets from the hardware automatically. Does crossbow provide any method to disable the polling mechanism, for example disabling the this kernel thread? Thanks Zhihui -------------- next part -------------- An HTML attachment was scrubbed... URL:

...48 675 22 0 0 0 0 0 0 0 84 1147 8616 2461 2 23 74 </pre> <br> Notice the run queue. Is there a DTrace script (from the DTT package) that I can use to figure out what is going on? <br><br> mpstat shows: <pre> CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 3511 48 97 784 197 1067 32 239 365 0 5814 5 43 0 52 1 1287 28 43 429 0 901 37 215 314 0 2821 3 40 0 57 2 2954 54 155 1442 1079 1176 26 241 339 0 4927 4 42 0 54 3 1364 20 886 167 16 655 32 184 299...

...39;m looking for a suggestion on a good way to hunt down the source of high context switching and mutex contentions... Is dtrace the way to go now, or should I stick with something like lockstat? Russ This is a 5 second interval for mpstat: CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 16 0 0 1115 1241 206 9095 912 2420 7393 0 12105 68 25 0 7 17 1 0 1054 1049 86 9290 921 2420 7355 0 12123 67 25 0 8 18 0 0 1004 984 6 9115 902 2551 7336 0 11843 66 25 0 9 19 0 0 926 1033 26 9405 927 2480 7413...

I just had a quick play with gzip compression on a filesystem and the result was the machine grinding to a halt while copying some large (.wav) files to it from another filesystem in the same pool. The system became very unresponsive, taking several seconds to echo keystrokes. The box is a maxed out AMD QuadFX, so it should have plenty of grunt for this. Comments? Ian

Recently I found that the mac_srs_rx_poll_ring thread may never stop in kernel, please see the following mpstat, cpu 2 is in 100% kernel usage, but no syscalls and no interrupts. CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 0 300 100 0 0 1 0 0 0 0 0 0 100 1 14 0 0 134 68 134 1 2 0 0 155 0 1 0 99 2 0 0 0 4 1 0 0 0 0 0 0 0 100 0 0...

...Hz 8-core, 32GB RAM, S10U3, zil_disable=1. Command ''zpool export f3-2'' is hung for 30 minutes now and still is going. Nothing else is running on the server. I can see one CPU being 100% in SYS like: bash-3.00# mpstat 1 [...] CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 67 220 110 20 0 0 0 0 0 0 0 0 100 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 100 2 0 0 0 2 0 2 0 0 0 0 5 0 0 0 100 3 0 0 0 1 0 0 0 0 0...

...''s what uptime says: Mon Nov 24 10:10:00 ARST 2008 10:10am up 10 day(s), 14:40, 6 users, load average: 325.52, 320.07, 318.72 (yes, load avg is almost 320, but the server was still usable) a little output from mpstat shows this: CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 531 259 150 46 27 17 43 0 29 99 1 0 0 1 0 0 3 66 0 124 65 36 10 0 86 99 1 0 0 2 0 0 0 23 0 34 22 17 9 0 8 100 0 0 0 3 0 0 0 13 0 13 12 6 9...

...ly receiving a bunch of multicast video streams and dumping them to the drives (at a rate of ~40MB/s). At an interval of roughly 1-2 minutes I get a storm of xcalls that completely eat one of the CPUs, so the mpstat line for the CPU looks like: CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 31 0 0 102191 1 0 0 0 0 0 0 0 0 100 0 0 100% busy in the system processing cross-calls. When I tried dtracing this issue, I found that this is the most likely culprit: dtrace -n ''sysinfo:::xcalls {@[stack()]=count();}'' un...

...s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 7.0 598.1 388.6 1832.9 0.0 2.0 0.0 3.4 0 93 c4t600C0FF00000000009258F3E4C4C5601d0 ^C bash-3.00# However for many seconds no IOs at all are issued. # mpstat 1 [...] CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 74 209 109 0 0 0 0 0 0 0 1 0 99 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 100 2 0 0 0 1 0 0 0 0 0 0 0 0 0 0 100 3 0 0 0 1 0 0 0 0 0...

...and two 4 core Xeons. The fileserver is attached by domain mode to Windows 2003 R2 SP2 with Services for Unix installed and we only have around 80 users (40-50 on the server at a time). Here's an example of the mpstat in the morning: CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 14 8910 8686 3612 12 1084 59403 316 921 1 65 0 35 1 0 0 7 5638 5536 5817 16 1214 66263 309 117 0 69 0 31 2 0 0 6 4646 4580 3404 6 1120 64217 324 74 0 62 0 38 3 0 0 8 3112 3006 3712 8 1131 61692...

...------------------------ -------------------------- Intel(R) Xeon(R) CPU X5570 @ 2.93GHz CPU 1 Intel(R) Xeon(R) CPU X5570 @ 2.93GHz CPU 2 [...] # uname -a SunOS XXXX 5.10 Generic_142901-03 i86pc i386 i86pc # mpstat 1 [...] CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 18 0 0 440 108 350 167 32 8170 23 5267 3 97 0 0 1 2 0 0 99 6 144 75 31 9119 15 4713 0 100 0 0 2 1 0 0 146 12 225 118 26 6509 3 3983 27 73 0 0 3 0 0 0 448 371 142 81 14 9800...

...I really mea nit, it''s not zero dot domething, its zero dot zero. - mpstat shows normal activity (almost nothing since this is a test machine, so only a few percent are used, but it still shows some activity and refreshes correctly) CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 125 428 109 113 1 4 0 0 251 2 0 0 98 1 0 0 20 56 16 44 2 2 1 0 277 11 1 0 88 2 0 0 163 152 13 309 1 3 0 0 1370 4 0 0 96 3 0 0 19 111 41 90 0 4...

...0.8 79 79 c2t1d0 (actv is less than 1 on c1 & c2 due to si3124 driver not supporting NCQ. I wonder what''s preventing ZFS from keeping the disks busy more than ~80% of the time. Maybe the CPU, see below.) "mpstat 2" CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 0 0 0 7275 7075 691 99 146 354 0 321 0 89 0 10 1 0 0 0 221 2 844 43 150 244 0 241 0 88 0 11 (The dual-core CPU is almost(?) a bottleneck at 90% utilization on both cores. For this reason I doubt such a read throughput c...