dovecot - Aug 2012 - quota: ignore deleted messages (?)

Hi!

I am currently in the process of preparing a migration of our old
Courier-based IMAP/POP server setup to a Dovecot-based one.

During this process I came across the following problem with the
difference Courier and Dovecot handle deleted messages and mail quota.

Quote from http://www.courier-mta.org/imap/README.maildirquota.html:

,----
| The default application configuration that uses this maildirquota
| library does not count deleted messages, and any contents of the Trash
| folder, against the quota. Messages that are marked as deleted (but not
| yet actually removed), or messages that are moved to the Trash folder
| (which is subject to automatic purging) do not count towards the set
| quota.
`----

Ignoring the content (or increasing the quota) of the Trash folder is
easy and no problem, but ignoring deleted messages seems impossible
without changes to the code.

While deleted messages are still stored on the server and still take up
space until they are expunged, counting them against the quota is
somewhat counter-intuitive because most clients don't show those mails
and the normal user is unaware the mails he deleted are still there and
take up space. Worse yet, if the client uses move-to-Trash, a user
deleting mails will double the used space (until Expunge is used) and
this may push him over his quota and thus cause any new mail delivery to
fail.

Unfortunately as our users are used to the courier way of handling the
quota this will cause trouble after the migration.

And this setup is run at a University, so I don't have any control over
the clients a user uses and the behavior of said clients so I am not
able to disable the move-to-Trash feature or force an immediate Expunge
after a delete.

So I propose an additional flag for the quota_rule config option to be
able to enable a more lax interpretation of the quota enforcement:

  quota_rule = *:storage=1G:ignoredeleted
  quota_rule2 = Trash:storage=+100M

Of course I would then have a nightly cronjob which force-expunges all
deleted messages so that users can't store mails infinitely in their
mailboxes.

Thanks for your time and Gr??e,
Sven.


-- 
Sigmentation fault. Core dumped.

On 24.8.2012, at 15.02, Sven Hartge wrote:
> Ignoring the content (or increasing the quota) of the Trash folder is
> easy and no problem, but ignoring deleted messages seems impossible
> without changes to the code.
..
> So I propose an additional flag for the quota_rule config option to be
> able to enable a more lax interpretation of the quota enforcement:
> 
>  quota_rule = *:storage=1G:ignoredeleted
>  quota_rule2 = Trash:storage=+100M
This can't be implemented race-condition-free without huge changes to code.

Sven Hartge <sven at svenhartge.de> wrote:
> Nope, does not compile (dovecot-dev headers are installed):
OK, trying to fix this, without having any deeper knowlege of C
(anymore):
> deleted-to-trash-plugin.c: In function ?mailbox_open_or_create?:
> deleted-to-trash-plugin.c:79: error: ?MAILBOX_FLAG_KEEP_RECENT? undeclared
(first use in this function)
MAILBOX_FLAG_KEEP_RECENT is not present in 2.1, seems to me it was made
the default and MAILBOX_FLAG_DROP_RECENT was introduced as its
counterpart.

I removed the flag from the call to mailbox_alloc() in

 72 static struct mailbox *
 73 mailbox_open_or_create(struct mailbox_list *list, const char *name,
 74                        const char **error_r)
 75 {
 76         struct mailbox *box;
 77         enum mail_error error;
 78 
 79         box = mailbox_alloc(list, name, MAILBOX_FLAG_NO_INDEX_FILES);
 80         if (mailbox_open(box) == 0) {
 81                 *error_r = NULL;
 82                 return box;
 83         }
 84 

and retried to compile:

cc    \
              -fPIC -shared -Wall \
              -I/usr/include/dovecot \
              -I/usr/include/dovecot/src \
              -I/usr/include/dovecot/src/lib \
              -I/usr/include/dovecot/src/lib-storage \
              -I/usr/include/dovecot/src/lib-mail \
              -I/usr/include/dovecot/src/lib-imap \
              -I/usr/include/dovecot/src/lib-index \
              -DHAVE_CONFIG_H \
              deleted-to-trash-plugin.c -o lib_deleted_to_trash_plugin.so
deleted-to-trash-plugin.c: In function ?copy_deleted_mail_to_trash?:
deleted-to-trash-plugin.c:135: warning: passing argument 1 of
?mailbox_keywords_unref? from incompatible pointer type
/usr/include/dovecot/mail-storage.h:612: note: expected ?struct mail_keywords
**? but argument is of type ?struct mailbox *?
deleted-to-trash-plugin.c:135: error: too many arguments to function
?mailbox_keywords_unref?
make: *** [lib_deleted_to_trash_plugin.so] Error 1

_and_ now I am at the end of my wisdom. Pointer magic in C has always
been a dark dark mystery to me (I learned programming in Pascal, Ada95
and later Perl ...).

Help, anybody?

Gr??e,
Sven.

-- 
Sigmentation fault. Core dumped.

Timo Sirainen <tss at iki.fi> wrote:
> On 25.8.2012, at 18.28, Sven Hartge wrote:
>> But because of performance considerations I want to use LMTP to deliver
>> the mails. Would this nested plugin configuration work:
>> (sorry, not at work, cannot test directly)
>> 
>> protocol lmtp {
>>  mail_plugins = $mail_plugins sieve
>>  plugin {
>>    quota = dict:Benutzer-Quota::noenforcing:proxy::quota
>>  }
>> }
> That should work, yes.
Very good, thank you for your fast answers.

Gr??e,
Sven.

-- 
Sigmentation fault. Core dumped.