dovecot - Feb 2019 - Multi-dbox storage space

Hi, I originally picked mdbox because I had the impression from reading 
about it on the mailing list that it was more performant and that it 
would conserve disc space.

But lately i found mailboxes have nearly double the storage as reported. 
I mean, IMAP QUOTA reports around 900MB for one account that has over 
2GB when I inspect the mail location of the user using the linux command 
`du`

Another account has around 750MB according to IMAP but using du I find 
1.5GB. This is for message files and index files combined.

Is this normal? I tried to read the MailboxFormat/dbox wiki page and it 
does not directly address this topic but it does reference "doveadm 
purge". Do I need to use "doveadm purge"?

> 
> On 03 Feb 2019, at 22:07, MRob <mrobti at insiberia.net> wrote:
> 
> Hi, I originally picked mdbox because I had the impression from reading
about it on the mailing list that it was more performant and that it would
conserve disc space.
> 
> But lately i found mailboxes have nearly double the storage as reported. I
mean, IMAP QUOTA reports around 900MB for one account that has over 2GB when I
inspect the mail location of the user using the linux command `du`
> 
> Another account has around 750MB according to IMAP but using du I find
1.5GB. This is for message files and index files combined.
> 
> Is this normal? I tried to read the MailboxFormat/dbox wiki page and it
does not directly address this topic but it does reference "doveadm
purge". Do I need to use "doveadm purge"?
From https://wiki2.dovecot.org/MailboxFormat/dbox:
> Expunging a message only decreases the message's refcount. The space is
later freed in "purge" step.
Mails are not directly removed from disk. You should probably setup the nightly
cronjob to clean your mailbox. Check the mailbox size once after running doveadm
purge.


Philon

On 2019-02-03 21:12, Philon wrote:
>> 
>> On 03 Feb 2019, at 22:07, MRob <mrobti at insiberia.net> wrote:
>> 
>> Hi, I originally picked mdbox because I had the impression from 
>> reading about it on the mailing list that it was more performant and 
>> that it would conserve disc space.
>> 
>> But lately i found mailboxes have nearly double the storage as 
>> reported. I mean, IMAP QUOTA reports around 900MB for one account that 
>> has over 2GB when I inspect the mail location of the user using the 
>> linux command `du`
>> 
>> Another account has around 750MB according to IMAP but using du I find 
>> 1.5GB. This is for message files and index files combined.
>> 
>> Is this normal? I tried to read the MailboxFormat/dbox wiki page and 
>> it does not directly address this topic but it does reference
"doveadm
>> purge". Do I need to use "doveadm purge"?
> 
> From https://wiki2.dovecot.org/MailboxFormat/dbox:
> 
>> Expunging a message only decreases the message's refcount. The
space
>> is later freed in "purge" step.
Thats what I read but I had a hard time to believe I could understand it 
was a critical maintenance because it was not highlighted in the docs.

I ran purge and find now "du" reporting about 60% the size of IMAP
QUOTA
so mdbox now is saving space and I'm very happy for Dovecot again is 
good software. But may I ask developers it is not common belief of 
administrators to know they must purge message storage - can you please 
make this a warning in highlight and big text on the mailformat 
documentation? I think this is a ciritcal oversight

PS for 10.000+ users how heavy is this process on performance? what if 
100.000+ users?