freebsd stable - Jan 2005 - Persisting troubles with periodic stalls every few minutes

Hi list,

I have been having a lot of trouble with performance on my laptop, which 
I first set up with 5.3-RELEASE and constantly keep up to date with 
5.3-STABLE. The box runs stable, but periodically, somewhere between 
every few minutes down to every few seconds, it stalls for 5 seconds. By 
that I mean that the screen is not being refreshed and all keyboard 
strokes go into some kind of buffer to get processed when the stall is 
over. Some key strokes also get lost or reversed in order, which makes 
this even more annoying.

I know that during the stalls, CPU usage goes up to 100%, so it is some 
process that periodically wakes up and hogs the CPU for a few seconds. 
Also, during the stalls, there is always a lot of disk activity. The 
only "special" thing about this machine is that /usr/home is GBDE 
encrypted. But even when I am not doing anything on that partition, 
stalls occur.

The rate of stalls varies a depending on what I am doing. Even when the 
box is idle, it does stall periodically. But when I am making world, the 
box becomes almost completely unusable as disk activity on /usr (which 
is not GBDE encrypted) triggers the same symptoms.

The trouble is that I cannot figure out how to find the responsible 
process. Tools such as top(1) update in one second intervals at best and 
as there are no screen updates during the stall, so they produce nothing 
useful. The only tool that gave me some kind of information was 
systat(1). When I invoke "systat -vmstat 1", I see the following:

When everything is working normally, the CPU is at:
15% system    30% user    65% idle

At the first screen update after a stall, the CPU is at:
15% system    85% user     0% idle

Also, the VM statistics such as "zfod" and "ofod" have
jumped from their
usual zero level to several thousand. And disk activity is reported as 
high, of course. A couple seconds after the stall is over, all 
statistics return to their normal values.

So, some user process is misbehaving. And has been doing so ever since 
this box was set up. Plus, it is somehow disk related and happens no 
matter what I am running or not, what I am doing or not.

Any ideas on how to debug this? How can I find the guilty process?

Thanks for any and all input,
- Bartosz Fabianowski

PS: According to sysctl, DMA is enabled on both ata and atapi so that is 
not the issue.

On Thu, Jan 20, 2005 at 02:36:30AM +0100, Bartosz Fabianowski wrote:
> When everything is working normally, the CPU is at:
> 15% system    30% user    65% idle
> 
> At the first screen update after a stall, the CPU is at:
> 15% system    85% user     0% idle
> 
> Also, the VM statistics such as "zfod" and "ofod" have
jumped from their
> usual zero level to several thousand. And disk activity is reported as 
> high, of course. A couple seconds after the stall is over, all 
> statistics return to their normal values.
> 
> So, some user process is misbehaving. And has been doing so ever since 
> this box was set up. Plus, it is somehow disk related and happens no 
> matter what I am running or not, what I am doing or not.
> 
> Any ideas on how to debug this? How can I find the guilty process?
Try top, ps, systat, etc.

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20050119/2bae8590/attachment.bin

On Thu, 2005-01-20 at 02:36 +0100, Bartosz Fabianowski
wrote:
> Hi list,
> 
> I have been having a lot of trouble with performance on my laptop, which 
> I first set up with 5.3-RELEASE and constantly keep up to date with 
> 5.3-STABLE. The box runs stable, but periodically, somewhere between 
> every few minutes down to every few seconds, it stalls for 5 seconds. By 
> that I mean that the screen is not being refreshed and all keyboard 
> strokes go into some kind of buffer to get processed when the stall is 
> over. Some key strokes also get lost or reversed in order, which makes 
> this even more annoying.
> 
> The trouble is that I cannot figure out how to find the responsible 
> process. Tools such as top(1) update in one second intervals at best and 
> as there are no screen updates during the stall, so they produce nothing 
> useful. The only tool that gave me some kind of information was 
> systat(1). When I invoke "systat -vmstat 1", I see the following:
Try running top with the -q option, and/or pressing space (which causes
top to try and refresh immediately).  If you still can't get any screen
updates to happen during the freeze, try
	top -qbSs 1 -d max > /var/tmp/somefile

And look through the file to see if you can establish what was actually
happening at the time of the freeze.

Gavin

> top -qbSs 1 -d max > /var/tmp/somefile
Thank you, that helped. Although top did not tell me which process was 
guilty directly, it allowed me to track it down. For the record:

The process hogging the entire CPU was Xorg, which in turn was being 
pushed by the KDE World Clock. At my screen resolution, the clock 
redraws itself every 45 minutes. Apparently, it does this in a very 
inefficient way, using a lot of CPU and making heavy use of the disk 
drive. I have filed a bug report on this with KDE:

http://bugs.kde.org/show_bug.cgi?id=97565

Thanks Gavin and Kris for your input,
- Bartosz