dovecot - Jan 2013 - Access to deleted mails in mdbox storage

Hi Timo,

we are currently running dovecot 2.1.10 using mdbox storage. Sometimes we 
get requests from users to restore accidentally deleted mails. If the 
mails got deleted before a backup was made, such mails could of course not 
be restored from backup. One option to help users in this respect would be 
to use the lazy expunge plugin.

On the other hand I have seen, that the deleted mails stay intact in the 
mdbox storage until a doveadm purge has been performed. Therefore without 
using the plugin it is in principle possible to get access to any deleted 
mails. As in our case the expunge runs once a week while the backup is 
done daily we can basically recover all mails ever received.

My question: doveadm has currently no command similar to doveadm search or 
fetch to get to deleted mails. Would it be useful to have such a command?

As a proof of concept I wrote a little script to mimic the doveadm fetch 
as closely as possible given the limited amount of information in the 
mdbox/storage/m.* files and could indeed get access to deleted mails. Do 
you regard such a procedure (within doveadm) useful or would you recommend 
nevertheless using the lazy expunge plugin?

I am aware of the fact that this is a solution for mdbox storage only.
-- 
Wolfgang Friebel                   Deutsches Elektronen-Synchrotron DESY
Phone/Fax:  +49 33762 77372/216    Platanenallee 6
Mail: Wolfgang.Friebel AT desy.de  D-15738 Zeuthen  Germany

Hi,

On 8.1.2013, at 11.24, Wolfgang.Friebel at desy.de wrote:
> we are currently running dovecot 2.1.10 using mdbox storage. Sometimes we
get requests from users to restore accidentally deleted mails. If the mails got
deleted before a backup was made, such mails could of course not be restored
from backup. One option to help users in this respect would be to use the lazy
expunge plugin.
Yes, lazy_expunge is the best way to do this. When expunging a mail with mdbox
it does a quick copy operation to lazy_expunge namespace, so it's only a
quick index update.
> On the other hand I have seen, that the deleted mails stay intact in the
mdbox storage until a doveadm purge has been performed. Therefore without using
the plugin it is in principle possible to get access to any deleted mails. As in
our case the expunge runs once a week while the backup is done daily we can
basically recover all mails ever received.
> 
> My question: doveadm has currently no command similar to doveadm search or
fetch to get to deleted mails. Would it be useful to have such a command?
> 
> As a proof of concept I wrote a little script to mimic the doveadm fetch as
closely as possible given the limited amount of information in the
mdbox/storage/m.* files and could indeed get access to deleted mails. Do you
regard such a procedure (within doveadm) useful or would you recommend
nevertheless using the lazy expunge plugin?
> 
> I am aware of the fact that this is a solution for mdbox storage only.
One way that you could already do this would be to copy the mdbox elsewhere,
delete index files and let Dovecot rebuild all the indexes. Then it adds back
all the deleted mails as well. But of course you can't easily tell which
ones are the deleted mails and which ones aren't.

Maybe one way to implement this that wouldn't be horribly ugly would be to
have some kind of an in-memory "deleted mails" mailbox, which contains
all mails that have refcount=0. It could be named e.g. "//deleted",
which wouldn't be a valid normal mailbox name (assuming separator=/).
Probably shouldn't be accessible via IMAP. Anyway, it probably would be more
than a little bit of work..