dovecot - Dec 2008 - unkillable imap process(es) with high CPU-usage

Hello,

I am having a problem with my dovecot-daemon. It is forking one or more
(I saw up to perhaps 8 of them) imap processes under my user name. These
processes are consuming a lot of CPU time and are not killable:
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 8616 arno      20   0  2900 1600 1204 R   98  0.2   1196:38 imap
Stopping dovecot does not quit these processes. Killing them (even "kill
-9" as root) is not possible. The only solution to get rid of them is to
reboot.

I have found a mailing list post of one who seemed to have the same
problem. He solved it by upgrading to version 1.1. But in my case this
did not help.

Is this a known problem? What could I check or do?

My system:
very up to date debian/sid (sidux)
kernel 2.6.27-8.slh.1-sidux-686
CPU: Intel(R) Core(TM)2 CPU    4300  @ 1.80GHz
Filesystem: ext3fs

tried dovecot from sid: 1:1.0.15-2.3
tried dovecot from experimental: 1:1.1.2-3

My dovecot.conf is the original debian configuration with only one line
changed into:
  protocols = imaps

# dovecot -n
# 1.1.2: /etc/dovecot/dovecot.conf
log_timestamp: %Y-%m-%d %H:%M:%S
protocols: imaps
login_dir: /var/run/dovecot/login
login_executable: /usr/lib/dovecot/imap-login
mail_privileged_group: mail
auth default:
  passdb:
    driver: pam
  userdb:
    driver: passwd

I am using dovecot locally on my system with me as the only user. As
client I am using thunderbird alias icedove 2.0.0.17-1. Icedove
retrieves the Mails from another imap server and sorts them into
Maildir-folders in dovecot. I do not know when the imap-process are
going mad. It happens after the system (and the mail-client) is up for a
while.

Do you need more information?

Thanks,
Arno

On Thu, 2008-12-11 at 11:36 +0100, Arno Wald wrote:
> I am having a problem with my dovecot-daemon. It is forking one or more
> (I saw up to perhaps 8 of them) imap processes under my user name. These
> processes are consuming a lot of CPU time and are not killable:
> 
> >  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> > 8616 arno      20   0  2900 1600 1204 R   98  0.2   1196:38 imap
> 
> Stopping dovecot does not quit these processes. Killing them (even
"kill
> -9" as root) is not possible. The only solution to get rid of them is
to
> reboot.
If you can't kill a process with -9, the bug is in the kernel and
there's nothing Dovecot can do about it. User spaces processes can't
create unkillable processes unless something's broken.

Although you could see if "strace -p <pid>" prints something.
It's
doubtful though if you can't kill the process.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20081211/f3821f11/attachment-0002.bin>

Timo Sirainen wrote:
> If you can't kill a process with -9, the bug is in the kernel and
Do you have an idea, where and how I could report this to? (against the
kernel package?). Perhaps I try an original debian kernel instead the
sidux kernel first.
> there's nothing Dovecot can do about it. User spaces processes
can't
> create unkillable processes unless something's broken.
> 
> Although you could see if "strace -p <pid>" prints
something. It's
> doubtful though if you can't kill the process.
I have tried this already without success. The strace did not print
anything and hang, without being able to be stopped by CTRL-C.

Thank you for your answer,
Arno

> If you can't kill a process with -9, the bug is in the kernel and
> there's nothing Dovecot can do about it. User spaces processes
can't
> create unkillable processes unless something's broken.
It just means the process is doing an uninterruptable sleep (in BSD notation, a
tsleep() without PCATCH set). This may be an I/O operation, resource shortage
etc. and needn't be a kernel bug.

A new status report regarding this issue:

Dovecot on my PC in the office is still running fine with kernel 2.6.26.

Dovecot with the latest kernel 2.6.27-9.slh.1-sidux-686 on my PC at home 
did show the unkillable imap processes after a few minutes.

Now I am running dovecot compiled without inotify support on this kernel 
without any problems for about 70 minutes.


So I really think that the inotify stuff in kernel 2.6.27 does make the 
problem. (I will tell you if the imap process unexpectedly are making 
problems again in the current configuration.).

So where are kernel issues reported? I will try to find out.

Greetings,
Arno

I have running the older debian/sid dovecot 1:1.0.15-2.3 again, now with 
kernel 2.6.27-10.slh.1-sidux-686 and the issue seems to be fixed. So I 
recommend to use at least 2.6.27.10.

Bye, Arno.

I have this EXACT same problem after upgrading to SuSE 11.1, which uses this
exact kernel version!!

After reading this, I was excited to think that if I killed the nfsserver
daemon (which I had running for no good reason), that it would sort my
problem....

Sure enough, my computer - which up to now had been going unresponsive every
24 hours - was running fine for 72 hours and then BOOM... it happened again.

Just wanted to let people know that it seems that at the minute, the dovecot
that ships with SuSE and the kernel they are using in 11.1 exhibit this
problem.

Gino



Arno Wald wrote:
> 
> 
> I have running the older debian/sid dovecot 1:1.0.15-2.3 again, now with 
> kernel 2.6.27-10.slh.1-sidux-686 and the issue seems to be fixed. So I 
> recommend to use at least 2.6.27.10.
> 
> Bye, Arno.
> 
> 
> 
-- 
View this message in context:
http://www.nabble.com/unkillable-imap-process%28es%29-with-high-CPU-usage-tp20953049p22014569.html
Sent from the Dovecot mailing list archive at Nabble.com.

just again to the list wasnt in reply

Robert Schetterer schrieb:
> Hi Gene , small tips depending suse
> 
> Gene Steinberg schrieb:
>> On Feb 14, 2009, at 9:57 AM, agent59624285 wrote:
>>
>>> I have this EXACT same problem after upgrading to SuSE 11.1, which
>>> uses this
>>> exact kernel version!!
>>>
>>> After reading this, I was excited to think that if I killed the
nfsserver
>>> daemon (which I had running for no good reason), that it would sort
my
>>> problem....
>>>
>>> Sure enough, my computer - which up to now had been going
unresponsive
>>> every
>>> 24 hours - was running fine for 72 hours and then BOOM... it
happened
>>> again.
>>>
>>> Just wanted to let people know that it seems that at the minute,
the
>>> dovecot
>>> that ships with SuSE and the kernel they are using in 11.1 exhibit
this
>>> problem.
---my reply
> 
> Suse kernels are not like vanilla kernels,
> they are deeply patched an mostly have stuff from the next kernel
> version allready in
> you might use yast for upgrading latest suse kernel it might fixed allready
> a short look gives
>
http://download.opensuse.org/repositories/home:/tristanhoffmann/openSUSE_11.1/
> gives dovecot 1.1.11 which is latest
> also loadable by yast or just download rpm and upgrade wtih rpm -u
> 
> standart rep for mail stuff is
> http://download.opensuse.org/repositories/server:/mail/openSUSE_11.1/
> but here dovecot is not at latest
> but it shouldnt be any big problem downloading src and
> prepare the spec with latest src dovecot
> and compile by yourself
> which is the best way and gives you the suse splitted packs
> too and the advantage to see compile stuff
> for sure you need some others libs etc cause of dependencies
> befor but they are mostly downloadable also by yast
> 
> be aware that suse sometimes splits up funktions
> in own packs like
> dovecot-autocreate11, dovecot-antispam11
> etc this musnt be the same by versions compile from home suse users
> 
> also help full send a mail to the build service,
> normally it takes only days and the newest version gets compiled
> 
> 
> use http://software.opensuse.org/search
> if looking for newer versions of software
-----
> 
>>> Gino
>>>
>> This is sounding similar to the problem I have with my setup:
>>
>> 1. High CPU usage.
>>
>> 2. Can't kill IMAP.
>>
>> 3. Server becomes unresponsive.
>>
>> I'm using CentOS 5.2 64-bit version with the latest cPanel.
>>
>> So what am I missing, other than the problem nobody else is having is
>> clearly something they ARE having?
>>
>> Peace,
>> Gene
> 
> 

-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria