Hello, I'm preparing to convert from qpopper + UW-IMAP to dovecot.
So far testing has gone very well. One problem we haven't figured
out is that long-running POP sessions keep the mailbox locked, so that
the MDA times out while trying to deliver. We're using "maildrop"
as
our MDA if that matters.
We don't see this issue for dovecot IMAP sessions. During long-running
IMAP sessions, incoming messages are deilvered tot he user's INBOX
without delay. But with POP, the MDA times out and the messages go
back to sendmail's queue for later delivery.
We though this setting would address the issue:
# Keep the mailbox locked for the entire POP3 session.
pop3_lock_session = no
But it doesn't seem to.
Any ideas?
Here is our "dovecot -n":
----------------------------------------------------------------------
# 1.1.8: /usr/local/etc/dovecot.conf
# OS: FreeBSD 7.1-RELEASE-p2 i386
base_dir: /var/run/dovecot/
protocols: imap imaps pop3 pop3s
ssl_cert_file: /etc/mail/certs/mail.swcp.com.pem
ssl_key_file: /etc/mail/certs/mail.swcp.com.pem
ssl_cipher_list: ALL:!LOW:!SSLv2
disable_plaintext_auth: no
login_dir: /var/run/dovecot/login
login_executable(default): /usr/local/libexec/dovecot/imap-login
login_executable(imap): /usr/local/libexec/dovecot/imap-login
login_executable(pop3): /usr/local/libexec/dovecot/pop3-login
login_greeting_capability(default): yes
login_greeting_capability(imap): yes
login_greeting_capability(pop3): no
verbose_proctitle: yes
first_valid_uid: 1000
first_valid_gid: 100
mail_privileged_group: mail
mail_location: mbox:~/:INBOX=/var/mail/%u:INDEX=/var/dovecot/%u
mail_full_filesystem_access: yes
mmap_disable: yes
mail_nfs_storage: yes
mail_nfs_index: yes
lock_method: dotlock
mbox_lock_timeout: 120
dbox_rotate_days: 0
mail_executable(default): /usr/local/libexec/dovecot/imap
mail_executable(imap): /usr/local/libexec/dovecot/imap
mail_executable(pop3): /usr/local/libexec/dovecot/pop3
mail_plugin_dir(default): /usr/local/lib/dovecot/imap
mail_plugin_dir(imap): /usr/local/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3
imap_client_workarounds(default): delay-newmail netscape-eoh
tb-extra-mailbox-sep
imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep
imap_client_workarounds(pop3):
pop3_reuse_xuidl(default): no
pop3_reuse_xuidl(imap): no
pop3_reuse_xuidl(pop3): yes
pop3_client_workarounds(default):
pop3_client_workarounds(imap):
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
auth default:
mechanisms: plain login
username_format: %Lu
winbind_helper_path: /usr/local/bin/ntlm_auth
verbose: yes
debug: yes
debug_passwords: yes
passdb:
driver: pam
args: session=yes dovecot
userdb:
driver: passwd
args: blocking=yes
socket:
type: listen
client:
path: /var/run/dovecot/auth-client
mode: 432
master:
path: /var/run/dovecot/auth-master
mode: 384
----------------------------------------------------------------------
Thank,
Mark
--
Mark Costlow | Southwest Cyberport | Fax: +1-505-232-7975
cheeks at swcp.com | Web: www.swcp.com | Voice: +1-505-232-7992
abq-strange.com -- Interesting photos taken in Albuquerque, NM
Last post: Cruising San Mateo I - 2009-01-04 15:20:30
On Wed, 2009-02-04 at 11:17 -0700, Mark Costlow wrote:> Hello, I'm preparing to convert from qpopper + UW-IMAP to dovecot. > So far testing has gone very well. One problem we haven't figured > out is that long-running POP sessions keep the mailbox locked, so that > the MDA times out while trying to deliver. We're using "maildrop" as > our MDA if that matters...> We though this setting would address the issue: > > # Keep the mailbox locked for the entire POP3 session. > pop3_lock_session = no > > But it doesn't seem to. > > Any ideas?Switch to Maildir and the problem goes away. The issue is that the mbox file is read-locked when the first message is read. And since the POP3 client most likely just keeps reading messages for the entire session, the mbox file kept read-locked all the time. Can't really be fixed without some larger redesign (which is really not worth it). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20090204/158deca7/attachment-0002.bin>
On Wed, Feb 04, 2009 at 01:25:21PM -0500, Timo Sirainen wrote:> On Wed, 2009-02-04 at 11:17 -0700, Mark Costlow wrote: > > Hello, I'm preparing to convert from qpopper + UW-IMAP to dovecot. > > So far testing has gone very well. One problem we haven't figured > > out is that long-running POP sessions keep the mailbox locked, so that > > the MDA times out while trying to deliver. We're using "maildrop" as > > our MDA if that matters. > .. > > We though this setting would address the issue: > > > > # Keep the mailbox locked for the entire POP3 session. > > pop3_lock_session = no > > > > But it doesn't seem to. > > > > Any ideas? > > Switch to Maildir and the problem goes away. The issue is that the mbox > file is read-locked when the first message is read. And since the POP3 > client most likely just keeps reading messages for the entire session, > the mbox file kept read-locked all the time. Can't really be fixed > without some larger redesign (which is really not worth it).Point taken. Does this mean that pop3_lock_session is a NO-OP? Or does it have other implications? Thanks, Mark -- Mark Costlow | Southwest Cyberport | Fax: +1-505-232-7975 cheeks at swcp.com | Web: www.swcp.com | Voice: +1-505-232-7992 abq-strange.com -- Interesting photos taken in Albuquerque, NM Last post: Cruising San Mateo I - 2009-01-04 15:20:30
On Wed, 4 Feb 2009, Timo Sirainen wrote:> On Wed, 2009-02-04 at 11:17 -0700, Mark Costlow wrote: >> Hello, I'm preparing to convert from qpopper + UW-IMAP to dovecot. >> So far testing has gone very well. One problem we haven't figured >> out is that long-running POP sessions keep the mailbox locked, so that >> the MDA times out while trying to deliver. We're using "maildrop" as >> our MDA if that matters. > .. >> We though this setting would address the issue: >> >> # Keep the mailbox locked for the entire POP3 session. >> pop3_lock_session = no >> >> But it doesn't seem to. >> >> Any ideas? > > Switch to Maildir and the problem goes away. The issue is that the mbox > file is read-locked when the first message is read. And since the POP3 > client most likely just keeps reading messages for the entire session, > the mbox file kept read-locked all the time. Can't really be fixed > without some larger redesign (which is really not worth it).We see this as well with mbox and pop3 accesses where some pop3 clients do not logout (iphone's are the worst offenders) for 30-90 minutes it seems. Timeout settings in dovecot.conf do not seem to help. procmail backs up waiting to get access to the inbox to deliver mail. Is there a global timeout we can set that will close any pop3 connection after say 15 minutes? Maildir is not an option currently for many of our servers that use openwebmail, which does not support maildir currently. Not sure if these pop3 sessions are in a loop, just keeping the conenction open for a reason, etc... but they do not seem to do anything after the initial check besides not logging out... Rob
Rob Mangiafico <rmang at lexiconn.com> wrote:>On Wed, 4 Feb 2009, Timo Sirainen wrote: >> On Wed, 2009-02-04 at 11:17 -0700, Mark Costlow wrote: >>> Hello, I'm preparing to convert from qpopper + UW-IMAP to dovecot. >>> So far testing has gone very well. One problem we haven't figured >>> out is that long-running POP sessions keep the mailbox locked, so that >>> the MDA times out while trying to deliver. >> .. >> >> Switch to Maildir and the problem goes away. > > We see this as well with mbox and pop3 accesses where some pop3 clients do > not logout (iphone's are the worst offenders) for 30-90 minutes it seems. > Timeout settings in dovecot.conf do not seem to help. procmail backs up > waiting to get access to the inbox to deliver mail.We are having the same problem. It's not a problem with imap, only with pop3. We're using deliver as the MDA. We're also using flock() locking for the mbox: is that what everyone else is using as well? Migrating our servers to maildir globally is also not an option for us. In some cases, even convincing customers to use IMAP isn't going to work. One problem which might be making this worse than it needs to be, is the fact that mbox_lock_flock in mbox-lock.c is not using a blocking flock(); instead, it's polling for a non-blocking lock. This technique can cause lock starvation, if another process is dropping the lock and picking it back up again frequently: other processes will only see the lock as being available if they happen to poll for the lock at just the right instant. A better technique to use here, if it's adequately cross-platform, would be to set an alarm() for the max_wait_time, and use a blocking flock(). If the alarm times out and you don't have a lock, it's a timeout. In the meantime, you're guaranteed to eventually get the lock, if it is dropped. That said: I'm not sure whether this will solve our problem in practice. Why doesn't this happen with imap? Why can't we make pop3 do what imap does? Even if it's inefficient, it's better than hanging all incoming mail delivery while deliver eats up our local concurrency limits. Thanks! Alan Ferrency pair Networks, Inc. alan at pair.com