Hi
Throughout the v. 1.1 betas I have been receiving casual reports that
quota isn't handled correctly (running Maildir).
At the moment I'm running 1.1beta14 but this has been occurring in all
other 1.1 betas as well (I don't know for sure whether this was also an
issue in 1.0, but I think so).
In some earlier betas the issue where worse than is is now (due to
improved NFS locking I guess).
Apparently this occurs when the user shifts between pop3 and IMAP.
maildirsize gets updated whenever mails are coming in, no problems there.
But mails removed aren't always counted as removed - as a result some
users quota keep growing until they reach a limit and e-mails start
getting rejected.
I have been doing some testing and it appears that the updating of the
file maildirsize is sometimes delayed.
I figure this is because Dovecot collects theese changes in the
transactions or index file and then once in a while writes them to
maildirsize.
Could there be a week spot in the code somewhere that allow for this kind
of error when the use of POP3 and IMAP are mixed (I think this happens
even when POP3 and IMAP access aren't concurrent)?
Both my index files and mail storage are located on NFS.
Regards, Mikkel
This is the output of dovecot -n:
# 1.1.beta14: /local/config/dovecot.conf
Warning: fd limit 256 is lower than what Dovecot can use under full load
(more than 768). Either grow the limit or change login_max_processes_count
and max_mail_processes settings
log_path: /local/log/dovecot.run
info_log_path: /local/log/dovecot.run
protocols: imap pop3
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /opt/freeware/dovecot-1.1b14/var/run/dovecot/login
login_executable(default):
/opt/freeware/dovecot-1.1b14/libexec/dovecot/imap-login
login_executable(imap):
/opt/freeware/dovecot-1.1b14/libexec/dovecot/imap-login
login_executable(pop3):
/opt/freeware/dovecot-1.1b14/libexec/dovecot/pop3-login
login_process_per_connection: no
verbose_proctitle: yes
first_valid_uid: 105
first_valid_gid: 105
mmap_disable: yes
mail_nfs_storage: yes
mail_nfs_index: yes
mail_executable(default): /opt/freeware/dovecot-1.1b14/libexec/dovecot/imap
mail_executable(imap): /opt/freeware/dovecot-1.1b14/libexec/dovecot/imap
mail_executable(pop3): /opt/freeware/dovecot-1.1b14/libexec/dovecot/pop3
mail_plugins(default): quota imap_quota trash
mail_plugins(imap): quota imap_quota trash
mail_plugins(pop3): quota
mail_plugin_dir(default): /opt/freeware/dovecot-1.1b14/lib/dovecot/imap
mail_plugin_dir(imap): /opt/freeware/dovecot-1.1b14/lib/dovecot/imap
mail_plugin_dir(pop3): /opt/freeware/dovecot-1.1b14/lib/dovecot/pop3
imap_client_workarounds(default): outlook-idle delay-newmail
tb-extra-mailbox-sep
imap_client_workarounds(imap): outlook-idle delay-newmail
tb-extra-mailbox-sep
imap_client_workarounds(pop3):
pop3_client_workarounds(default):
pop3_client_workarounds(imap):
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
auth default:
mechanisms: plain login digest-md5 cram-md5 apop ntlm
cache_size: 4096
cache_ttl: 120
passdb:
driver: sql
args: /local/config/dovecot-sql.conf
userdb:
driver: prefetch
userdb:
driver: sql
args: /local/config/dovecot-sql.conf
socket:
type: listen
client:
path: /var/spool/postfix/private/auth
mode: 432
user: postfix
group: postfix
master:
path: /var/run/dovecot/auth-master
mode: 384
user: vmail
plugin:
quota: maildir
quota_rule2: Trash:storage=10M:messages=100
trash: /local/config/dovecot-trash.conf
On 2/21/2008, mikkel at euro123.dk (mikkel at euro123.dk) wrote:> At the moment I'm running 1.1beta14I'm sure Timo appreciates any/all help he can get, but if you are going to go through the trouble of running the development beta versions, don't you think it would make sense to always use the latest - and especially, don't report problems on anything but the latest? -- Best regards, Charles
On Thu, 2008-02-21 at 11:03 +0100, mikkel at euro123.dk wrote:> I have been doing some testing and it appears that the updating of the > file maildirsize is sometimes delayed.When do you see this delaying?> I figure this is because Dovecot collects theese changes in the > transactions or index file and then once in a while writes them to > maildirsize.POP3 is run in a single transaction, but the mails are expunged and maildirsize is updated only when QUIT is run. So you shouldn't be able to notice a situation where maildirsize doesn't match the maildir contents.> Could there be a week spot in the code somewhere that allow for this kind > of error when the use of POP3 and IMAP are mixed (I think this happens > even when POP3 and IMAP access aren't concurrent)?Both IMAP and POP3 use the same code to access mailboxes and update quota, so I can't really think of anything. Although it's of course possible to use different settings for POP3/IMAP.> mail_plugins(imap): quota imap_quota trash > mail_plugins(pop3): quotaOK, both use quota plugin..> plugin: > quota: maildir > quota_rule2: Trash:storage=10M:messages=100I guess quota_rule comes from userdb? Is it the same for both imap/pop3? Although that shouldn't matter since the maildirsize should be updated in any case.. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20080221/06453e17/attachment-0002.bin>
> On Thu, 2008-02-21 at 11:03 +0100, mikkel at euro123.dk wrote: >> I have been doing some testing and it appears that the updating of the >> file maildirsize is sometimes delayed. > > When do you see this delaying? >When I telnet manually to pop3 and imap and type commands to delete certain messages, while keeping an eye on the maildirsize file. This isn't always consistent, but it appears that the updates to maildirsize are grouped together before being committed.>> I figure this is because Dovecot collects theese changes in the >> transactions or index file and then once in a while writes them to >> maildirsize. > > POP3 is run in a single transaction, but the mails are expunged and > maildirsize is updated only when QUIT is run. So you shouldn't be able > to notice a situation where maildirsize doesn't match the maildir > contents.So what happens if QUIT is never run? If the the connection is broken before ending properly? Does the IMAP connection also have to be terminated properly before the updates are written? This may be the cause of the issue since some users in a production environment will always break the connection (loss of internet connectivity, the client program crashing or just generally badly behaving e-mail clients).> Both IMAP and POP3 use the same code to access mailboxes and update > quota, so I can't really think of anything. > > Although it's of course possible to use different settings for > POP3/IMAP. > >> mail_plugins(imap): quota imap_quota trash >> mail_plugins(pop3): quota > > OK, both use quota plugin.. >Yes and I know it works. When executing the pop3 remove commands myself I can see that changes are actually written to maildirsize.>> plugin: >> quota: maildir >> quota_rule2: Trash:storage=10M:messages=100 > > I guess quota_rule comes from userdb? Is it the same for both imap/pop3? > Although that shouldn't matter since the maildirsize should be updated > in any case.. >The queries are exactly alike for POP3 and IMAP. Thanks for looking into this. Regards, Mikkel