dovecot - Nov 2020 - Recovering expunged but not purged e-mails from mdbox with zlib compression

Yesterday I had to recover an e-mail which a user had deleted. If I 
understand this correctly the message was expunged but not purged 
(doveadm purge had not been run).

This e-mail was clearly still in the mdbox stored with zlib compression 
as I could tell using the doveadm dump command (doveadm dump -t dbox 
filename).

I could however not reveal the e-mail with the normal doveadm fetch -u 
username "body" guid (guid from dump output)

In the end I was able to recover the e-mail by cutting it out from the 
mdbox and running gunzip over it, but this method was very fiddly and 
would not have worked for a bulk job.? I appreciate I could have just 
deleted the index files and gotten all the deleted messages back by 
having the index rebuilt but that would have been the proverbial "hammer 
to crack a nut".

Fortunately this e-mail had no attachment for me to worry about, as 
those are also detached for sis by dovecot.

Can anyone tell me of a quick and easy way to recover one or multiple 
e-mails marked as expunged but not purged which are stored in a mdbox 
with zlib compression?? I'm sure I'll have to do this again the future 
and my method wouldn't work with a folder!

Clearly if I was not using zlib compression I could just have read the 
contents of the mdbox without any complication.


-- 
Daniel Sch?tze
Director

------------------

CWA International Ltd
5th Floor, 42 Trinity Square
London
EC3N 4DJ

(t) + 44 (0)20 7242 8444
(e) dms at cwa.uk.com
(w) http://www.cwa.international/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20201110/812bff5e/attachment.html>

You can use mdbox_deleted driver to access mails with refcount 0. See
https://wiki2.dovecot.org/MailboxFormat/dbox

Aki

On 10.11.2020 12.42, Daniel Sch?tze wrote:
>
> Yesterday I had to recover an e-mail which a user had deleted.? If I
> understand this correctly the message was expunged but not purged
> (doveadm purge had not been run).
>
> This e-mail was clearly still in the mdbox stored with zlib
> compression as I could tell using the doveadm dump command (doveadm
> dump -t dbox filename).
>
> I could however not reveal the e-mail with the normal doveadm fetch -u
> username "body" guid (guid from dump output)
>
> In the end I was able to recover the e-mail by cutting it out from the
> mdbox and running gunzip over it, but this method was very fiddly and
> would not have worked for a bulk job.? I appreciate I could have just
> deleted the index files and gotten all the deleted messages back by
> having the index rebuilt but that would have been the proverbial
> "hammer to crack a nut".
>
> Fortunately this e-mail had no attachment for me to worry about, as
> those are also detached for sis by dovecot.
>
> Can anyone tell me of a quick and easy way to recover one or multiple
> e-mails marked as expunged but not purged which are stored in a mdbox
> with zlib compression?? I'm sure I'll have to do this again the
future
> and my method wouldn't work with a folder!
>
> Clearly if I was not using zlib compression I could just have read the
> contents of the mdbox without any complication.
>
>
> -- 
> Daniel Sch?tze
> Director
>
> ------------------
>
> CWA International Ltd
> 5th Floor, 42 Trinity Square
> London
> EC3N 4DJ
>
> (t) + 44 (0)20 7242 8444
> (e) dms at cwa.uk.com
> (w) http://www.cwa.international/

Dear Aki

Thank you.? Unfortunately I'm struggling to get the right syntax for 
this as it looks like someone else was here too

https://dovecot.org/pipermail/dovecot/2018-July/112441.html

The location in my dovecot.conf is

mdbox:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/%%n

I'm trying to fetch a message (for testing purposes now) based on it's 
guid as that is available from doveadm dump.

So my syntax is (based on the previous person who didn't get it to work).

doveadm -o 
"mail_location=mdbox_deleted:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/%%n"
fetch "body" guid (msg.guid from doveadm dump)

But that's getting no response.? I've tried putting in the hard path in 
case there is any trouble with the %%h etc but that doesn't help either.

I've also tried the fetch with the subject of a known deleted message 
and also adding mailbox Drafts (as I know that's where it is or rather 
was) but to no avail.

Given doveadm dump gives msg.uid and no subject I'd prefer to use that 
in any case.

There are no error messages, I do with doveadm was a little more verbose 
to help over these bumps!

Any chance of a helping hand?




Daniel Sch?tze
Director

------------------

CWA International Ltd
5th Floor, 42 Trinity Square
London
EC3N 4DJ

(t) + 44 (0)20 7242 8444
(e) dms at cwa.uk.com
(w) http://www.cwa.international/
On 10/11/20 10:53, Aki Tuomi wrote:
> You can use mdbox_deleted driver to access mails with refcount 0. See
> https://wiki2.dovecot.org/MailboxFormat/dbox
>
> Aki
>
> On 10.11.2020 12.42, Daniel Sch?tze wrote:
>> Yesterday I had to recover an e-mail which a user had deleted.? If I
>> understand this correctly the message was expunged but not purged
>> (doveadm purge had not been run).
>>
>> This e-mail was clearly still in the mdbox stored with zlib
>> compression as I could tell using the doveadm dump command (doveadm
>> dump -t dbox filename).
>>
>> I could however not reveal the e-mail with the normal doveadm fetch -u
>> username "body" guid (guid from dump output)
>>
>> In the end I was able to recover the e-mail by cutting it out from the
>> mdbox and running gunzip over it, but this method was very fiddly and
>> would not have worked for a bulk job.? I appreciate I could have just
>> deleted the index files and gotten all the deleted messages back by
>> having the index rebuilt but that would have been the proverbial
>> "hammer to crack a nut".
>>
>> Fortunately this e-mail had no attachment for me to worry about, as
>> those are also detached for sis by dovecot.
>>
>> Can anyone tell me of a quick and easy way to recover one or multiple
>> e-mails marked as expunged but not purged which are stored in a mdbox
>> with zlib compression?? I'm sure I'll have to do this again the
future
>> and my method wouldn't work with a folder!
>>
>> Clearly if I was not using zlib compression I could just have read the
>> contents of the mdbox without any complication.
>>
>>
>> -- 
>> Daniel Sch?tze
>> Director
>>
>> ------------------
>>
>> CWA International Ltd
>> 5th Floor, 42 Trinity Square
>> London
>> EC3N 4DJ
>>
>> (t) + 44 (0)20 7242 8444
>> (e) dms at cwa.uk.com
>> (w) http://www.cwa.international/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20201110/1815749d/attachment.html>