freebsd stable - Jan 2009 - Hangs, maybe a clue

I've had some mysterious hangs which I notice that several others have too.
Two of the machines in question are Soekris 4801's running as routers; this
is hard to handle ddb with (though possible for one of them...)  I started
noticing this sometime in December.  My laptop finally hung in a state where
I could do a ps (waiting a long time for the response.)  The strange and
likely related to the hang was softdepflush in R state with 43 MINUTES of
cpu.  (the machine has been up maybe an hour.)

This is 7.1-stable csup'd Jan 9.  I'm backing up to Dec 1 since I
didn't notice
it (maybe just didn't notice?).  The load average was about 10 but mostly
that
was lower-priority things ready but not run.

-- Pete

On Sun, Jan 18, 2009 at 09:30:05PM -0500, Pete Carah
wrote:
> I've had some mysterious hangs which I notice that several others have
too.
> Two of the machines in question are Soekris 4801's running as routers;
this
> is hard to handle ddb with (though possible for one of them...)  I started
> noticing this sometime in December.  My laptop finally hung in a state
where
> I could do a ps (waiting a long time for the response.)  The strange and
> likely related to the hang was softdepflush in R state with 43 MINUTES of
> cpu.  (the machine has been up maybe an hour.)
I'm seeing those hangs on Soekris 4801 routers running RELENG_7 as
well. The boxes are used as SoHo appliances and run mpd5, pf, named,
postfix, cyrus-imapd, lighttpd, openntpd and sshd. On all of them,
softupdates are enabled on all partitions except root and they use
real HDDs (not compact flash).

The hangs appear now every 2 or 3 days at different times. They don't
seem related to traffic type (heavy p2p, normal upload/download, or
idle) and also seem independent on disk activity (i.e. there's no more
during 3am than other time). They were less frequent before Dec 1st,
and IIRC the last RELENG_7 that was nearly hang-free was 2008-11-07.

IMHO, it could be some kind of resource leak (?), but I'm not sure.
Since the only serial port is used by getty, I'm not sure how to break
into the debugger and how to trace the problem (and I'm not experienced
enough for this). :(
> -- Pete
Regards,
-cpghost.

-- 
Cordula's Web. http://www.cordula.ws/