dovecot - Jul 2018 - doveadm expunge didn't clear Trash mailbox

On Mon, 23 Jul 2018, Michael Wagner wrote:
> here works a dovecot 2.2.27 on a raspberrypi and the behaviour is as
> expected.
> 
> doveadm -f tab fetch -u <user> "uid date.saved" mailbox
Trash
> uid     date.saved
> 314     2018-06-23 00:35:59
> 315     2018-06-23 12:39:10
> 316     2018-06-24 10:32:43
> ...
> 
> And I have a cron script that expunges the mails older than 30 days.
> /usr/bin/doveadm expunge -u <user> mailbox Trash savedbefore 30d
Hi Michael,

Thanks for your observation.  I think you probably use maildir format,
yes?

I think my problem stems from the fact I use mbox, so one file contains
many messages, whereas maildir uses one file for each message.  Dovecot,
from my understanding, will use the message file's mtime for date.saved
if it doesn't have it in the cached.  This is probably why I am seeing
multiple messages with the same value.

What's confusing is that if I query a message's date.saved, that
value -- whether from the cache or the mailbox's mtime -- remains
in the cache, so the next query will give me that value.

So running "doveadm fetch ... date.saved", will get the mailbox's
mtime at
the time I fetched those values.  In order to get near accurate date.saved
values, I would have to run "doveadm fetch ... date.saved" frequently.
(I think this behaviour must have changed recently because this no
longer works consistently.)

However, if each message is stored in a separate file (like maildir does),
then each message can have an accurate date.saved(=mtime).  Once you
have accurate date.saved, then the "expunge savedbefore" works
correctly.

Another reason I really ought to switch to maildir.

Joseph Tam <jtam.home at gmail.com>

Hello Joseph,

On Jul 24, 2018 um 13:12:05, Joseph Tam wrote:
> On Mon, 23 Jul 2018, Michael Wagner wrote:
> 
> > here works a dovecot 2.2.27 on a raspberrypi and the behaviour is as
> > expected.
> > 
> > doveadm -f tab fetch -u <user> "uid date.saved"
mailbox Trash
> > uid     date.saved
> > 314     2018-06-23 00:35:59
> > 315     2018-06-23 12:39:10
> > 316     2018-06-24 10:32:43
> > ...
> > 
> > And I have a cron script that expunges the mails older than 30 days.
> > /usr/bin/doveadm expunge -u <user> mailbox Trash savedbefore 30d
> 
> Thanks for your observation.  I think you probably use maildir format,
> yes?
> 
> I think my problem stems from the fact I use mbox, so one file contains
> many messages, whereas maildir uses one file for each message.  Dovecot,
> from my understanding, will use the message file's mtime for date.saved
> if it doesn't have it in the cached.  This is probably why I am seeing
> multiple messages with the same value.
yes, I use here the maildir format and I think you are right that 
dovecot uses the mtime and this is set every time when you write to the 
mbox file. If you use linux, you can use mb2md for the converting to 
maildir.

apt show mb2md
Description: Convert Mbox mailboxes to Maildir format
 mb2md (mbox to maildir) takes one or more Mbox format mailbox files
 in a directory and convert them to Maildir format mailboxes.

 As the Mbox format has some drawbacks, D. J. Bernstein created the
 Maildir format when he wrote Qmail. With the Mbox format all mail of
 a specific folder is stored as one large text file. The Maildir
 format stores each mail as a separate file. It is a faster and more
 efficient way to store mail. It works particularly well over NFS,
 which has a long history of locking-related woes.

 The Mbox format is used by many POP3/IMAP servers, most mail servers
 (MTAs) and mail readers (MUAs). The Maildir format is used by Qmail,
 Courier-MTA and can be also used as a alternative mail storage format
 by Postfix and Exim or any MTA which can use procmail as
 MDA. POP3/IMAP servers which support Maildirs are e.g. Courier IMAP
 and Dovecot.

 mb2md does not only convert Mbox mailbox files into a Maildir but can
 also convert the /var/spool/mail/$USER mailspool file. It is smart
 enough to not transfer a dummy message such as the UW IMAPD puts at
 the start of Mbox mailboxes - and you could add your own search terms
 into the script to make it ignore other forms of dummy first message.

I never repented the converting to maildir.

Michael

PS: Sorry for the mail I sent private to you, it was a misconfiguration 
of mutt.

-- 
BOFH excuse #446:
Mailer-daemon is busy burning your message in hell.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 906 bytes
Desc: not available
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20180725/53c3d339/attachment.sig>