On Wed, Apr 14, 2010 at 06:44:58AM +0200, Gabor PALI
wrote:> Hello there,
>
> I am running a FreeBSD/amd64 8-STABLE with GENERIC kernel as of
> February 17 on my box (quad core, 2 GB RAM), and recently I spot some
> interesting problems in my logs. My machine runs two instances of a
> client in two separate chroot environments in parallel with 32-bit and
> 64-bit userlands respectively, doing a nightly building and testing.
> According to the logs it puts a nice load on my system, though it
> still seems to be working fine.
>
> Except one thing: it produces strange error messages on swap space
> without an apparent reason (at least to me):
>
> xxx# tail -f /var/log/messages
> Apr 14 05:26:44 xxx kernel: swap_pager_getswapspace(4): failed
> Apr 14 05:26:44 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:44 xxx kernel:
> Apr 14 05:26:44 xxx kernel: swap_pager_getswapspace(3): failed
> Apr 14 05:26:44 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:44 xxx kernel: swap_pager_getswapspace(3): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(12): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(2): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: pid 7388 (throwto003), uid 1001, was
> killed: out of swap space
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(8): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(8): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(12): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(9): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(9): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(3): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(16): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(9): failed
> Apr 14 05:26:45 xxx kernel: swap_pager_getswapspace(3): failed
> ^C
> xxx# swapinfo -h
> Device 1K-blocks Used Avail Capacity
> /dev/ad0s1b 4194304 112M 3.9G 3%
The swapinfo command you ran was not run at 05:26 in the morning. You
should probably set up a small script, run via cronjob, that logs
swapinfo -h output to a file somewhere (rotate it if you want via
newsyslog.conf).
You may have something running on the system that spirals out of
control, such as a web board script being pounded to death, or something
that's forking excessively.
I'd also recommend having the script output "top -b -o res 100",
which
will give you the top 100 processes on the machine sorted by RSS
(non-shared) memory usage. I don't know of a way to show "the amount
of
swap used by process N, or all processes", since it's transparently
handled by the VM. So I'm making the assumption RSS will be large.
--
| Jeremy Chadwick jdc@parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |