Hello, I originally just typed up a 5 paragraph essay on my problems but just deleted it all. basically I need to find out where to start looking to find out what is the problem with my dovecot pop3 setup. I switched from courier imap/pop3 to dovecot imap/pop3. I have had super high loads (40+ and overnight one server went to 511) and its been a nightmare fighting with this new setup for the last two weeks. I did manage to narrow the problem down to it being with the pop3 part of dovecot. I am not sure why the load keeps going so high. The first week I attributed it to users having to redownload mail and reindex (we were aware of the UIDL problem), but two weeks later one of my machines overnight went to 511. I have changed so many config options to try to make it work that I have become overwhelmed and feel like I have tried everything to get the right settings. I am running four 64bit machines with amd 9850 cpus, 4 gigs ram, the mailstore is nfs mounted, the nfs server is using xfs. Debian lenny/sid. I dont know what to do, I am almost debating going back to courier pop3 and going through the week of support calls of customers downloading mail again but that is a last ditch effort. I can understand if i just need to wait it out while things are indexed for dovecot and stuff, but machines falling over and loads of 40 to 500 just for dovecot seems not right. so, where do I start?
On Sun, 2008-10-26 at 09:57 -0700, Brandon Lamb wrote:> basically I need to find out where to start looking to find out what > is the problem with my dovecot pop3 setup. I switched from courier > imap/pop3 to dovecot imap/pop3. I have had super high loads (40+ and > overnight one server went to 511) and its been a nightmare fighting > with this new setup for the last two weeks. > > I did manage to narrow the problem down to it being with the pop3 part > of dovecot. I am not sure why the load keeps going so high.Usually this is because Dovecot wants to calculate the messages' virtual sizes. See maildir performance in http://wiki.dovecot.org/POP3Server. There were also some issues with this where Dovecot didn't always add the virtual size to dovecot-uidlist, but they were fixed in v1.1.5. So the initial high loads are expected if your file names don't contain ,W=size in them, but after that it should slow down. Of course if your new messages don't contain ,W= in them either, the load is still higher than it could be (but then again, Courier does the same thing so it shouldn't be higher than with Courier). One possibility would be to just make Dovecot violate POP3 spec and return file sizes as message sizes. Some servers like qmail-pop3d do that and they apparently work well enough.. This could be done by changing mail_get_virtual_size to mail_get_physical_size in src/pop3/client.c -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20081026/65c3d19a/attachment-0002.bin>
On Sun, Oct 26, 2008 at 10:15 AM, Timo Sirainen <tss at iki.fi> wrote:> On Sun, 2008-10-26 at 09:57 -0700, Brandon Lamb wrote: >> basically I need to find out where to start looking to find out what >> is the problem with my dovecot pop3 setup. I switched from courier >> imap/pop3 to dovecot imap/pop3. I have had super high loads (40+ and >> overnight one server went to 511) and its been a nightmare fighting >> with this new setup for the last two weeks. >> >> I did manage to narrow the problem down to it being with the pop3 part >> of dovecot. I am not sure why the load keeps going so high. > > Usually this is because Dovecot wants to calculate the messages' virtual > sizes. See maildir performance in http://wiki.dovecot.org/POP3Server. > There were also some issues with this where Dovecot didn't always add > the virtual size to dovecot-uidlist, but they were fixed in v1.1.5. > > So the initial high loads are expected if your file names don't > contain ,W=size in them, but after that it should slow down. Of course > if your new messages don't contain ,W= in them either, the load is still > higher than it could be (but then again, Courier does the same thing so > it shouldn't be higher than with Courier). > > One possibility would be to just make Dovecot violate POP3 spec and > return file sizes as message sizes. Some servers like qmail-pop3d do > that and they apparently work well enough.. This could be done by > changing mail_get_virtual_size to mail_get_physical_size in > src/pop3/client.cIm willing to try this, I really dont want to go back to courier, dovecot just seems like a way cleaner setup. I found the line to edit so I will recompile and give that a try. I did modify exim to add the W=size but it also required use_crlf and that seemed to break something else. I also found a couple posts saying to use INDEX=MEMORY for the pop3 part, but I didnt understand if that actually DISABLED pop3 using indexes or if it really stored something in memory, and if so what happens on a reboot, so I am confused on if i should tell the pop3 part to use the same /dovecot-index that I specified for imap. I also dont know if I made so many changes that I didnt give any of them time to work themselves out. We were just in panic mode trying to put out fires hehe.