Dovecot will soon have a plugin which allows running a nightly cronjob
to expunge mails from configured mailboxes which have been in there for
a configurable amount of time. For example the configuration could be:
plugin {
# Trash 7d, Spam 30d
expire = Trash 7 Spam 30
}
It also supports using Dovecot's lib-dict to keep track of the mailboxes
and their oldest mail's timestamp, so that the nightly cronjob can
quickly go through only those mailboxes that have those old mails.
Berkeley DB support has been added to lib-dict for this purpose, but you
could use SQL as well.
There's however one problem with it that I could use opinions on.
Currently there's no way to know how long a mail has been in a mailbox.
There are pretty much 3 ways to handle this:
1) When copying to those Trash/Spam mailboxes the message's INTERNALDATE
could be replaced with the current time. That way the INTERNALDATE tells
how long the message has been in there. The obvious downside of this is
that if the user wants to move the message back to another mailbox, the
original INTERNALDATE is lost (INTERNALDATE means the time when the
message was received and is often the timestamp used to sort messages).
2) Preserve the INTERNALDATE while copying, but still use it anyway.
Often the messages are moved to Trash (and especially to Spam) soon
after they have been received. If not, then the user has time until the
next nightly cronjob to move the messages back before they're lost.
3) Add a new field to index file which contains the date when the
message was copied. This could be done only for those Trash/Spam boxes
so they don't waste space for other mailboxes. The problem with this is
that it won't work if there are no index files. To overcome that problem
the time could be stored in yet another place, such as maildir filename
and some new mbox header. I don't see supporting those as worth the
trouble though, so the fallback could be either the date index file is
created or the messages' INTERNALDATE.
I think either 1) or 3) would be best solutions. 3) probably, but if
index files don't exist nearly permanently, it's not very reliable..
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://dovecot.org/pipermail/dovecot/attachments/20060608/ff80810f/attachment.pgp
Timo Sirainen wrote:> 3) Add a new field to index file which contains the date when the > message was copied.This is the best solution, to my mind, as it has the extreme benefit of doing what makes the most sense to the enduser. In other words, I receive a message today and move it to Trash immediately. My sysadmin has helpfully told me that messages older than 7 days will be deleted from Trash. The act of copying it to Trash starts the clock running. Moving the file out of Trash and back in the next day restarts the clock. Anything that messes with INTERNALDATE will inevitably be wrong for the indecisive user. I was going to argue for option #2, but then I realized that if I sit on a message for weeks and then Trash it, it would make more sense that it would be deleted 7 days hence, not that night. Sites running without indexes shouldn't use this feature (or possibly this feature should be disabled if the appropriate index doesn't exist). John -- John Peacock Director of Information Research and Technology Rowman & Littlefield Publishing Group 4501 Forbes Boulevard Suite H Lanham, MD 20706 301-459-3366 x.5010 fax 301-429-5748
On Thu, 8 Jun 2006, Timo Sirainen wrote:> 3) Add a new field to index file which contains the date when the > message was copied. This could be done only for those Trash/Spam boxes > so they don't waste space for other mailboxes. The problem with this is > that it won't work if there are no index files. To overcome that problem > the time could be stored in yet another place, such as maildir filename > and some new mbox header. I don't see supporting those as worth the > trouble though, so the fallback could be either the date index file is > created or the messages' INTERNALDATE.Hmm, I'd prefer variant 3) with: if there _should_ be an index (aka it was destroyed and recreated): restart the timer; if there is NO index configured at all, use the INTERNALDATE. However, I'd store all information only found in the indexes into the Maildir filenames as well, so the indexes can be recreated from the filenames, incl. keywords and such. Bye, -- Steffen Kaiser
On Thu, 2006-06-08 at 17:33 +0300, Timo Sirainen wrote:> 3) Add a new field to index file which contains the date when the > message was copied. This could be done only for those Trash/Spam boxes > so they don't waste space for other mailboxes. The problem with this is > that it won't work if there are no index files. To overcome that problem > the time could be stored in yet another place, such as maildir filename > and some new mbox header. I don't see supporting those as worth the > trouble though, so the fallback could be either the date index file is > created or the messages' INTERNALDATE.Actually for maildir the time is in file's ctime, so that would be a good fallback. For dbox I could add a new field if people think it is useful (might be useful for other purposes also?). mbox users are out of luck though. For them the fallback could be the current date, so as long as the indexes usually stay for longer than the expire days, this works well enough. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 191 bytes Desc: This is a digitally signed message part Url : http://dovecot.org/pipermail/dovecot/attachments/20060608/199c67f0/attachment-0001.pgp
On 6/8/06, Timo Sirainen <tss@iki.fi> wrote:> Dovecot will soon have a plugin which allows running a nightly cronjob > to expunge mails from configured mailboxes which have been in there for > a configurable amount of time. For example the configuration could be: > > plugin { > # Trash 7d, Spam 30d > expire = Trash 7 Spam 30 > }Just wanted to let you know how grateful I am when this comes in! Thanks a million Timo for making such a great package, and bringing useful functions in that most people want! Tim /me goes to work out how to add a message above Spam and Trash folder in SM, to reflect this new plugin -- Linux Counter user #273956
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Timo Sirainen wrote:> Dovecot will soon have a plugin which allows running a nightly cronjob > to expunge mails from configured mailboxes which have been in there for > a configurable amount of time. For example the configuration could be: > > plugin { > # Trash 7d, Spam 30d > expire = Trash 7 Spam 30 > } > > It also supports using Dovecot's lib-dict to keep track of the mailboxes > and their oldest mail's timestamp, so that the nightly cronjob can > quickly go through only those mailboxes that have those old mails. > Berkeley DB support has been added to lib-dict for this purpose, but you > could use SQL as well. > > There's however one problem with it that I could use opinions on. > Currently there's no way to know how long a mail has been in a mailbox. > There are pretty much 3 ways to handle this: > > 1) When copying to those Trash/Spam mailboxes the message's INTERNALDATE > could be replaced with the current time. That way the INTERNALDATE tells > how long the message has been in there. The obvious downside of this is > that if the user wants to move the message back to another mailbox, the > original INTERNALDATE is lost (INTERNALDATE means the time when the > message was received and is often the timestamp used to sort messages). > > 2) Preserve the INTERNALDATE while copying, but still use it anyway. > Often the messages are moved to Trash (and especially to Spam) soon > after they have been received. If not, then the user has time until the > next nightly cronjob to move the messages back before they're lost. > > 3) Add a new field to index file which contains the date when the > message was copied. This could be done only for those Trash/Spam boxes > so they don't waste space for other mailboxes. The problem with this is > that it won't work if there are no index files. To overcome that problem > the time could be stored in yet another place, such as maildir filename > and some new mbox header. I don't see supporting those as worth the > trouble though, so the fallback could be either the date index file is > created or the messages' INTERNALDATE. > > I think either 1) or 3) would be best solutions. 3) probably, but if > index files don't exist nearly permanently, it's not very reliable..Timo, I think #3 is the closet to a good solution. I read through the thread but decided to reply to the original post. In the case of the mbox backend, when a message is deleted, the actual contents of the message has to be altered to change the status: flag, right? (it's been a long time since i've used mbox, so i could be off base on this) ... what if you (optionally) added an X-Dovecot-Reserved: header to the email with the last relevant date information to use as a backup if the indexes don't exist? I think the maildir format issues have been pretty well hashed out and using ctime (mtime?) as the fallback would probably be reasonable. I really like this idea as it could help me help my users manage their quotas by forcing the trashes to be deleted on a regular basis. my 2yen worth Alan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEiLybE2gsBSKjZHQRAu7UAKCgxwUKdKGDj2WvMJZGW6cAj6hD0ACfaj6x VpXJ9LrEa2YI/0gphDKiu7I=8r0f -----END PGP SIGNATURE-----
Timo Sirainen wrote:> Dovecot will soon have a plugin which allows running a nightly cronjob > to expunge mails from configured mailboxes which have been in there for > a configurable amount of time. For example the configuration could be: > > plugin { > # Trash 7d, Spam 30d > expire = Trash 7 Spam 30 > }I'm looking forward to this. In the meantime, for messages in Maildir format, is it safe to delete message files using a simple cron script based on "find" with a specified mtime? For mbox style I've used the common scripts "archivemail" and "expiremail.pl", and I'm looking for something similar to use with Maildir. Thanks, Mark
On Thu, 8 Jun 2006, Timo Sirainen wrote:> Dovecot will soon have a plugin which allows running a nightly cronjob > to expunge mails from configured mailboxes which have been in there for > a configurable amount of time. For example the configuration could be: > > plugin { > # Trash 7d, Spam 30d > expire = Trash 7 Spam 30 > }It would be good to have user-specific settings for this. E.g. our users uses plenty of MUAs, all of them with their own mind of where to store SPAM, Junk, etc.pp. (I also would like to configure some users to expire _all_ folders or at least the INBOX :-) Bye, -- Steffen Kaiser
On 6/8/06, Timo Sirainen <tss at iki.fi> wrote:> Dovecot will soon have a plugin which allows running a nightly cronjob > to expunge mails from configured mailboxes which have been in there for > a configurable amount of time. For example the configuration could be: > > plugin { > # Trash 7d, Spam 30d > expire = Trash 7 Spam 30 > }Timo, how is progress on this? Is there anything we can do to help, cause I'm really interested in this. Thanks Tim -- Linux Counter user #273956