dovecot - Dec 2017 - recover missing messages - files still present in storage

Hi,

I have a user account that had almost 20GB of emails and now they're
missing.
Only a few are available trough IMAP or doveadm.

I can see in /path/to/mailbox/storage that thousands of "m." files are
still there, summing up 19GB of files.

doveconf -n http://termbin.com/7lgc

I have tried:
accessing via IMAP
accessing via doveadm search/fetch/mailbox status
doveadm index
doveadm force-resync

doveadm dump:

-- INDEX:
/srv/dovecot2/index/DOMAIN_SUPRESSED/ACCOUNT_SUPRESSED/storage/dovecot.map.index
version .................. = 7.3
base header size ......... = 120
header size .............. = 176
record size .............. = 20
compat flags ............. = 1
index id ................. = 1500105889 (2017-07-15 05:04:49)
flags .................... = 0
uid validity ............. = 1500105889 (2017-07-15 05:04:49)
next uid ................. = 26514
messages count ........... = 26513
seen messages count ...... = 0
deleted messages count ... = 0
first recent uid ......... = 1
first unseen uid lowwater  = 1
first deleted uid lowwater = 26446
log file seq ............. = 47
log file tail offset ..... = 15656
log file head offset ..... = 15656
log2 rotate time ......... = 1512558922 (2017-12-06 09:15:22)
last temp file scan ...... = 0 (1969-12-31 21:00:00)
day stamp ................ = 1512604800 (2017-12-06 22:00:00)
day first uid[0] ......... = 26450
day first uid[1] ......... = 26395
day first uid[2] ......... = 26328
day first uid[3] ......... = 26252
day first uid[4] ......... = 26234
day first uid[5] ......... = 26216
day first uid[6] ......... = 26135
day first uid[7] ......... = 26055
-- Extension 0 --
name ........ = map
hdr_size .... = 8
reset_id .... = 0
record_offset = 8
record_size . = 12
record_align  = 4
header ........ = 6111000001000000
-- Extension 1 --
name ........ = ref
hdr_size .... = 0
reset_id .... = 0
record_offset = 6
record_size . = 2
record_align  = 2
-- Keywords --

-- CACHE:
/srv/dovecot2/index/DOMAIN_SUPRESSED/ACCOUNT_SUPRESSED/storage/dovecot.map.index.cache
cache is unusable


Regards,

Webert Lima
DevOps Engineer at MAV Tecnologia
*Belo Horizonte - Brasil*

Have you attempted doveadm force-resync -u SUPPRESSED_VICTIM "*"?

Aki
> On December 7, 2017 at 7:00 PM Webert de Souza Lima <webert.boss at
gmail.com> wrote:
> 
> 
> Hi,
> 
> I have a user account that had almost 20GB of emails and now they're
> missing.
> Only a few are available trough IMAP or doveadm.
> 
> I can see in /path/to/mailbox/storage that thousands of "m."
files are
> still there, summing up 19GB of files.
> 
> doveconf -n http://termbin.com/7lgc
> 
> I have tried:
> accessing via IMAP
> accessing via doveadm search/fetch/mailbox status
> doveadm index
> doveadm force-resync
> 
> doveadm dump:
> 
> -- INDEX:
>
/srv/dovecot2/index/DOMAIN_SUPRESSED/ACCOUNT_SUPRESSED/storage/dovecot.map.index
> version .................. = 7.3
> base header size ......... = 120
> header size .............. = 176
> record size .............. = 20
> compat flags ............. = 1
> index id ................. = 1500105889 (2017-07-15 05:04:49)
> flags .................... = 0
> uid validity ............. = 1500105889 (2017-07-15 05:04:49)
> next uid ................. = 26514
> messages count ........... = 26513
> seen messages count ...... = 0
> deleted messages count ... = 0
> first recent uid ......... = 1
> first unseen uid lowwater  = 1
> first deleted uid lowwater = 26446
> log file seq ............. = 47
> log file tail offset ..... = 15656
> log file head offset ..... = 15656
> log2 rotate time ......... = 1512558922 (2017-12-06 09:15:22)
> last temp file scan ...... = 0 (1969-12-31 21:00:00)
> day stamp ................ = 1512604800 (2017-12-06 22:00:00)
> day first uid[0] ......... = 26450
> day first uid[1] ......... = 26395
> day first uid[2] ......... = 26328
> day first uid[3] ......... = 26252
> day first uid[4] ......... = 26234
> day first uid[5] ......... = 26216
> day first uid[6] ......... = 26135
> day first uid[7] ......... = 26055
> -- Extension 0 --
> name ........ = map
> hdr_size .... = 8
> reset_id .... = 0
> record_offset = 8
> record_size . = 12
> record_align  = 4
> header ........ = 6111000001000000
> -- Extension 1 --
> name ........ = ref
> hdr_size .... = 0
> reset_id .... = 0
> record_offset = 6
> record_size . = 2
> record_align  = 2
> -- Keywords --
> 
> -- CACHE:
>
/srv/dovecot2/index/DOMAIN_SUPRESSED/ACCOUNT_SUPRESSED/storage/dovecot.map.index.cache
> cache is unusable
> 
> 
> Regards,
> 
> Webert Lima
> DevOps Engineer at MAV Tecnologia
> *Belo Horizonte - Brasil*

On Thu, Dec 7, 2017 at 3:05 PM, Aki Tuomi <aki.tuomi at dovecot.fi> wrote:
> Have you attempted doveadm force-resync -u SUPPRESSED_VICTIM "*"?
>
>
Hello Aki,

yes I did that, but I didn't remove the map files first (I don't know if
that's required).
I can do it again if needed.

Regards,

Webert Lima
DevOps Engineer at MAV Tecnologia
*Belo Horizonte - Brasil*