Some time ago I reported that pagezero kernel thread sometime takes (what seems to me) a too large chunk of available CPU (30%) on a very busy web server. There were no replies :( Since then, I've reconfigured apache to use PHP as a fastcgi module and the problem seems to have gotten worse - now it *always* takes ~25-30% CPU, and my "System" time stat shows it (its almost always ~30%). From what I can understand, pagezero thread fills kernel's pool of zeroed pages. This looks non-threteaning, but what would cause such high demand of zeroed pages on my system? The number of processes is almost constant, there are no frequent process spawnings or forks (top shows "last pid" to be almost constant). FastCGI processes communicate over UNIX sockets. Also, pagezero thread is (should be) executing at idle priority - does this mean it won't interfere much with machine's performance? Even if "0% idle" is not uncommon state? (This is FreeBSD 6.1-R, 2CPU SMP).
On 5/13/06, Ivan Voras <ivoras@fer.hr> wrote:> Some time ago I reported that pagezero kernel thread sometime takes > (what seems to me) a too large chunk of available CPU (30%) on a very > busy web server. There were no replies :( Since then, I've reconfigured > apache to use PHP as a fastcgi module and the problem seems to have > gotten worse - now it *always* takes ~25-30% CPU, and my "System" time > stat shows it (its almost always ~30%). > > From what I can understand, pagezero thread fills kernel's pool of > zeroed pages. This looks non-threteaning, but what would cause such high > demand of zeroed pages on my system? The number of processes is almost > constant, there are no frequent process spawnings or forks (top shows > "last pid" to be almost constant). FastCGI processes communicate over > UNIX sockets. > > Also, pagezero thread is (should be) executing at idle priority - does > this mean it won't interfere much with machine's performance? Even if > "0% idle" is not uncommon state? >How is the system, swapping-wise ? -- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it.
Vlad GALU wrote:> How is the system, swapping-wise ?Good: no swapping, 100+ MB free memory. Free memory varies between ~90 and ~110 MB - maybe something is returning memory to the OS and reclaiming it too often? (looking at top, it could be php-cgi, but is there a way to precisely track it? can it be stopped externally to the process?)
Ivan Voras wrote:> Some time ago I reported that pagezero kernel thread sometime takes > (what seems to me) a too large chunk of available CPU (30%) on a very > busy web server. There were no replies :( Since then, I've > reconfigured apache to use PHP as a fastcgi module and the problem > seems to have gotten worse - now it *always* takes ~25-30% CPU, and my > "System" time stat shows it (its almost always ~30%). > > From what I can understand, pagezero thread fills kernel's pool of > zeroed pages. This looks non-threteaning, but what would cause such > high demand of zeroed pages on my system?Generally, a whole bunch of processes that are demanding a lot or memory and/or swapping would do it. What does "vmstat 1" look like...? -- -Chuck
Ivan Voras wrote:> Some time ago I reported that pagezero kernel thread sometime takes > (what seems to me) a too large chunk of available CPU (30%) on a very > busy web server. There were no replies :( Since then, I've reconfigured > apache to use PHP as a fastcgi module and the problem seems to have > gotten worse - now it *always* takes ~25-30% CPU, and my "System" time > stat shows it (its almost always ~30%). > > From what I can understand, pagezero thread fills kernel's pool of > zeroed pages. This looks non-threteaning, but what would cause such high > demand of zeroed pages on my system? The number of processes is almost > constant, there are no frequent process spawnings or forks (top shows > "last pid" to be almost constant). FastCGI processes communicate over > UNIX sockets. > > Also, pagezero thread is (should be) executing at idle priority - does > this mean it won't interfere much with machine's performance? Even if > "0% idle" is not uncommon state? > > (This is FreeBSD 6.1-R, 2CPU SMP).Can you give the output of top(1)? Until this is fixed, you can disable pagezero with sysctl vm.idlezero_enable=0. -- Suleiman