I'm a new dovecot user, just began using it last week. I switched from courier because my research showed that Dovecot did indexes, and allows me to store them anywhere I chose. I did this, and it fixed some severe disk IO issues that my server had. (25k horde only webmail install using imap proxy). I'm using quotas, and by default had it set to 'dirsize'. my postfix is already creating 'maildirsize' files, so I opted to change dovecot to use this instead. However, when I do, I run into the "too many open files" issue that's been reported many times on the list. I've seen a lot of mention of too many open files, but I haven't seen it related to the quota type used. It's possible that simply my using 'dirsize' as the quota type slows things down enough to not have the issue? I don't really know. Anyhow. I doubled my max open files in FreeBSD's /boot/loader.conf to: kern.maxfiles="24656", but to no avail. When I have it set to maildirsize, the open files on the system grows uncontrollably. Under normal use, it's somewhere below 5k: sysctl -a | grep openfiles kern.openfiles: 4415 I looked into the patch mentioned here: - http://www.dovecot.org/list/dovecot/2006-November/017536.html - http://dovecot.org/list/dovecot-cvs/2006-November/006776.html Those patches already appear to be in rc15, which I'm already running (Via FreeBSD's ports). I also applied the patch mentioned here awhile back: - http://www.dovecot.org/list/dovecot/2005-June/007647.html and rebuilt, but it didn't help out either. There are also freebsd specific notes on forcing a limit on dovecot itsself using login.conf, but that wouldnt really fix anything, it would just cause dovecot to not interfere with other running daemons. - http://www.mail-archive.com/ports at openbsd.org/msg08646.html Any thoughts? TSS, you mention a few times in these notes that you're unable to reproduce this. Perhaps try with the maildir as quota? More details about my setup: - FreeBSD 4.8 (yea, I know) - Imapproxy (http://www.imapproxy.org/) - Postfix to create maildir++ maildirsize folders - MySQL backend The work of this is spread out between several servers, dovecot is running on the server that's the final SMTP delivery location, and has local disks with the maildirs. --falz -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://dovecot.org/pipermail/dovecot/attachments/20061215/fbd02da6/attachment-0002.html>
On 12/15/06, Ejay Hire <ehire at cbrweb.com> wrote:> > I'm looking at this too, for me it isn't releasing all of the index files consistently. > Two questions. > > 1. I have a dumb question. Does FreeBSD allow you to see /set the # of files a process can open with ulimit?Yep, in FreeBSD it's just "limits" which gives: Resource limits (current): cputime infinity secs filesize infinity kb datasize 524288 kb stacksize 65536 kb coredumpsize infinity kb memoryuse infinity kb memorylocked infinity kb maxprocesses 5547 openfiles 22190 sbsize infinity bytes vmemoryuse infinity kb 22190 is double the default. When I'm running without the issue (quota = dirsize) the # of open files is generally around 5k: # sysctl kern.openfiles kern.openfiles: 4910 When I change it, I can get it to go as high as I want (I then change dovecot back to dirsize for quota) sysctl kern.openfiles kern.openfiles: 19906> 2. can you do an lsof (ls open files) and send that (zipped!!) over so I can see what it has open?I just tried, but being a busy time right now, doing "lsof -p [PID] > lsof.txt" never returns before the # of open files gets maxed and the server goes to hell. I'll have to try again at a slower time to see if it will return- right now's about the busiest time of day on this box. Is there anything in particular you're looking for in there? Most of the lines are similar to the ones in the attached (small!) text file. Thanks, falz -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: lsof.txt URL: <http://dovecot.org/pipermail/dovecot/attachments/20061215/155b17c3/attachment-0002.txt>
On Fri, 2006-12-15 at 10:05 -0600, falz wrote:> I'm using quotas, and by default had it set to 'dirsize'. my postfix > is already creating 'maildirsize' files, so I opted to change dovecot > to use this instead. > > However, when I do, I run into the "too many open files" issue that's > been reported many times on the list.So you tried something like "quota=maildir:storage=123456"?> Anyhow. I doubled my max open files in FreeBSD's /boot/loader.conf to: > kern.maxfiles="24656", but to no avail. When I have it set to > maildirsize, the open files on the system grows uncontrollably.How quickly? By doing what exactly? Just logging in, opening a mailbox, saving a new mail, ..?> Any thoughts? TSS, you mention a few times in these notes that you're > unable to reproduce this. Perhaps try with the maildir as quota?The previous problems seemed to have been only with NetBSD 2.0 and kqueue enabled, and I think it's a NetBSD bug. I looked through the maildir quota code, and I do actually see one fd leak in there, but it happens only when deinitializing the quota, so practically it doesn't matter. You sent a lsof of the dovecot process, but assuming you hadn't dropped out thousands of lines from it, it probably wasn't the problematic process. Quota code is handled by imap process, so check that instead, and check also that lsof is returning thousands of lines. If it isn't, then it's something else that's eating the fds.. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20061216/20b89133/attachment.bin>