Baptiste Malguy
2010-Feb-19 16:17 UTC
[Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
Hello, I may have missed a point, but I could have found an issue between plugins quota and lazy_expunge. I have noticed that the "lazy_expunge"d mails are counted as part of the quota, while the documentation says it should not be. I have first tried to find out if this was a configuration mistake somewhere. Notably, you can notice that the expunged mails are in directory ~/expunged while the usual mail is in ~/Maildir : expunged mails are not in a subdirectory of ~/Maildir (this was just to make sure there were no side effect with this). I have deleted some expunged mails to be sure : at its next cycle, the file ~/Maildir/maildirsize contained expected value. So yes, I do know expunged mails in the expunged directory are counted (size, number of mails). So I've taken a look at the source code, adding some i_info(...) at different places of the quota and lazy_expunge plugins.>From the source code reading session, I understand that :1. In src/lazy-expunge/lazy-expunge-plugin.c 1.1. lazy_expunge_mail_storage_init() sets the expunged mails namespace flag with NAMESPACE_FLAG_NOQUOTA 1.2. lazy_expunge_mail_storage_init() is called by lazy_expunge_mail_storage_created() 1.3. lazy_expunge_mail_storage_created() is part of the callback list hook_mail_storage_create 2. In src/plugins/quota/quota-storage.c: 2.1. The only place in the whole dovecot source code where NAMESPACE_FLAG_NOQUOTA is checked is in function quota_mailbox_list_created() 3. In src/plugins/quota/quota-plugin.c: 3.1 quota_plugin_init() add quota_mailbox_list_created() to the callback list hook_mailbox_list_created.>From my observations :- Callback functions in quota_mailbox_list_created are called _before_ callback functions in hook_mail_storage_create. - This observation led me to move the piece of code that sets NAMESPACE_FLAG_NOQUOTA from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created(). - But there, lazy_expunge_mail_storage_created() is called after quota_mail_storage_created() because names of the library files are lib10_quota_plugin.so and lib02_lazy_expunge_plugin.so. - There I have also learnt that callback functions of the same list are called in the reverse order compared to the library filenames. So see if my observations were right, I have renamed lib10_quota_plugin.so to lib01_quota_plugin.so, then it worked. But I doubt this is the right way to fix it. This also reverses the order other callbacks between the two plugins are called, which migh not be expected at all. I'm not event sure moving some code from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created() was right : I've improvized the reading the source code and do not understand the whole logic, so I suppose this could break something else. Timo, what do you propose ? Thanks for your great software. -- Baptiste MALGUY PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2 -------------- next part -------------- # 1.2.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.30-bpo.1-amd64 x86_64 Debian 5.0.4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps managesieve ssl: required ssl_ca_file: /etc/ssl/certs/xxxxxxxxxxxx ssl_cert_file: /etc/ssl/certs/xxxxxxxxxxxxx ssl_key_file: /etc/ssl/private/xxxxxxxxxxxxxxxxx ssl_parameters_regenerate: 24 verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_greeting: XXXXXXXXXXXXXXXXX login_max_processes_count: 256 max_mail_processes: 1024 first_valid_uid: 500 last_valid_uid: 500 first_valid_gid: 501 last_valid_gid: 501 mail_privileged_group: mail mail_uid: vmail mail_gid: vmail mail_location: maildir:~/Maildir mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): autocreate expire fts fts_squat lazy_expunge quota imap_quota antispam acl imap_acl mail_plugins(imap): autocreate expire fts fts_squat lazy_expunge quota imap_quota antispam acl imap_acl mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: shared separator: / prefix: shared/%%u/ location: maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u list: yes namespace: type: private separator: / prefix: .ARCHIVES/ location: maildir:~/Maildir/archives list: yes subscriptions: yes namespace: type: private separator: / prefix: .EXPUNGED/ location: maildir:~/expunged list: yes subscriptions: yes lda: postmaster_address: postmaster at xxxxxxxxxx mail_plugins: sieve expire acl quota_full_tempfail: yes auth default: debug: yes passdb: driver: pam passdb: driver: ldap args: /etc/dovecot/dovecot-ldap-passdb.conf userdb: driver: passwd userdb: driver: ldap args: /etc/dovecot/dovecot-ldap-userdb.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: vmail plugin: quota: maildir: quota_rule: *:storage=2GB quota_rule2: Trash:storage=10%% quota_warning: storage=100%% /usr/local/bin/dovecot-quota-notify 100 FULL quota_warning2: storage=99%% /usr/local/bin/dovecot-quota-notify 100 FULL quota_warning3: storage=98%% /usr/local/bin/dovecot-quota-notify 98 CRITICAL quota_warning4: storage=97%% /usr/local/bin/dovecot-quota-notify 97 CRITICAL quota_warning5: storage=96%% /usr/local/bin/dovecot-quota-notify 96 CRITICAL quota_warning6: storage=95%% /usr/local/bin/dovecot-quota-notify 95 CRITICAL quota_warning7: storage=94%% /usr/local/bin/dovecot-quota-notify 94 CRITICAL quota_warning8: storage=93%% /usr/local/bin/dovecot-quota-notify 93 CRITICAL quota_warning9: storage=92%% /usr/local/bin/dovecot-quota-notify 92 CRITICAL quota_warning10: storage=91%% /usr/local/bin/dovecot-quota-notify 91 CRITICAL quota_warning11: storage=90%% /usr/local/bin/dovecot-quota-notify 90 CRITICAL quota_warning12: storage=87%% /usr/local/bin/dovecot-quota-notify 87 WARNING quota_warning13: storage=85%% /usr/local/bin/dovecot-quota-notify 85 WARNING quota_warning14: storage=80%% /usr/local/bin/dovecot-quota-notify 80 WARNING quota_warning15: storage=75%% /usr/local/bin/dovecot-quota-notify 75 WARNING acl: vfile:/etc/dovecot/dovecot-acls:cache_secs=5 acl_shared_dict: file:/var/vmail/shared-mailboxes.db expire: Trash 60 Trash/* 60 Corbeille 60 Corbeille/* 60 Spam 30 Spam/* 30 Junk 30 Junk/* 30 expire_dict: proxy::expire lazy_expunge: .EXPUNGED/ autocreate: Trash autocreate2: Sent autocreate3: Spam autosubscribe: Trash autosubscribe2: Sent autosubscribe3: Spam fts: squat fts_squat: partial=4 full=10 sieve: ~/.dovecot.sieve sieve_dir: ~/sieve sieve_global_path: /etc/dovecot/sieve/default.sieve sieve_global_dir: /etc/dovecot/sieve/global.d/ sieve_before: /etc/dovecot/sieve/before.d/ sieve_after: /etc/dovecot/sieve/after.d/ antispam_trash: trash;Trash;TRASH;Deleted Items;Deleted Messages;Corbeille antispam_spam: spam;Spam;SPAM;junk;Junk;Junk mail;Junk E-mail;Courrier ind&AOk-sirable antispam_allow_append_to_spam: no antispam_spool2dir_spam: /var/spool/antispam/spam/%%020lu-%u-%%05lu-spam antispam_spool2dir_notspam: /var/spool/antispam/notspam/%%020lu-%u-%%05lu-ham dict: expire: mysql:/etc/dovecot/dovecot-dict-expire.conf
Baptiste Malguy
2010-Feb-22 14:21 UTC
[Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
Hello, In case my nasty changes are right, here comes the patch (including Makefile.in & Makefile.am ... I'm not really good with automake/autoconf) I've applied to dovecot 1.2.10 (from Debian backport repository for Lenny, source package version 1:1.2.10-1~bpo50+1, expecting my changes won't be different from the vanilla version). Regards, -- Baptiste MALGUY PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2 -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-lazy-expunge-quota.patch Type: text/x-patch Size: 11222 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20100222/1a63d5ae/attachment-0002.bin>
Timo Sirainen
2010-May-26 17:35 UTC
[Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
On Fri, 2010-02-19 at 17:17 +0100, Baptiste Malguy wrote:> - But there, lazy_expunge_mail_storage_created() is called after > quota_mail_storage_created() because names of the library files are > lib10_quota_plugin.so and lib02_lazy_expunge_plugin.so.Uh oh. This is completely the reverse of how they were supposed to be working. I'm surprised that there haven't been more problems. But it's probably a bit too dangerous to change it anymore now in v1.x. It should be fixed already with v2.0.