Greetings, I have several users who have inboxes that are over 20 GB. Lately I have noticed Dovecot logs say it's taking over 30 seconds to sync their mailboxes. As email admins,? how do you handle inboxes that are so large? Do you use mailbox types that have better performance like dbox? We're using maildir. What's a reasonable inbox size?? Is 20+ GB reasonable and nothing to worry about? Thanks for any insight here. -- Asai7
On 5/7/2020 11:39 AM, Asai wrote:> What's a reasonable inbox size?? Is 20+ GB reasonable and nothing to > worry about? >Great question. At my firm, we wrote rotation tools that work for mbox format to rotate inboxes monthly if they are over a certain size.? We also do the sent items folders. We find that large inboxes are bad for the server and bad for the client because the MUAs just don't handle it well either.? 1 or 2GBs and you start to see issues. After a little bit of user training, they like it.? Part of the routine maintenance they need. Regards, KAM
Thanks for your response, So, how do those rotation scripts work in concept? People are still able to access their old inboxes, but it just moves them to an archived state? On 5/7/2020 9:40 AM, Kevin A. McGrail wrote:> On 5/7/2020 11:39 AM, Asai wrote: >> What's a reasonable inbox size?? Is 20+ GB reasonable and nothing to >> worry about? >> > Great question. > > At my firm, we wrote rotation tools that work for mbox format to rotate > inboxes monthly if they are over a certain size.? We also do the sent > items folders. > > We find that large inboxes are bad for the server and bad for the client > because the MUAs just don't handle it well either.? 1 or 2GBs and you > start to see issues. > > After a little bit of user training, they like it.? Part of the routine > maintenance they need. > > Regards, > KAM > >
> On 7. May 2020, at 18.39, Asai <asai at globalchangemusic.org> wrote: > > Greetings, > > I have several users who have inboxes that are over 20 GB. > > Lately I have noticed Dovecot logs say it's taking over 30 seconds to sync their mailboxes. > > As email admins, how do you handle inboxes that are so large? Do you use mailbox types that have better performance like dbox? We're using maildir. > > What's a reasonable inbox size? Is 20+ GB reasonable and nothing to worry about? > > Thanks for any insight here.It's more like huge number of files in single folder than the pure size of the mails combined. For filesystem based storage formats number of files is always a challenge. In your case mdbox probably would work better as it stores multiple mails in single file. The only downside is that it requires periodic purge operations to remove deleted mails from middle of the mail bundles. Sami
On Thu, 7 May 2020, Asai wrote:> I have several users who have inboxes that are over 20 GB. > > As email admins, how do you handle inboxes that are so large? Do you use > mailbox types that have better performance like dbox? We're using maildir. > > What's a reasonable inbox size? Is 20+ GB reasonable and nothing to worry > about?It depends on what you consider reasonable. The processing time of file operation that iterates through a mailbox will generally go up proportinately with size. If you do a text search without some indexing system like Solr, it will take a very long time. If the mailbox is just some archive that you pile up and forget about it except for once in a blue moon retrieval, then it might be reasonable. If it's an active mailbox, it will be a pain to navigate, in the same way a single folder with 100K files or a file cabinet with huge stacks of envelopes. I would guess some partioning of the large mailboxes into smaller mailboxes would help with active mailboxes. Most people spend most of their time on new/recent messages, so making time or size or subject based volmes wouldn't be a bad idea. If the bulk of the size are redundant copies of attachments, then Dovecot's *dbox support de-duping which would aso help. Joseph Tam <jtam.home at gmail.com>
On 07 May 2020, at 10:40, Kevin A. McGrail <KMcGrail at pccc.com> wrote:> We find that large inboxes are bad for the server and bad for the client > because the MUAs just don't handle it well either. 1 or 2GBs and you > start to see issues.Which is a good reason to move off mbox. There are several other much superior choices and as far as I know there is no reason to use mbox. -- Minds are like parachutes, they only work when they are open.
> It depends on what you consider reasonable. > > The processing time of file operation that iterates through a mailbox > will generally go up proportinately with size.? If you do a text search > without some indexing system like Solr, it will take a very long time. > > If the mailbox is just some archive that you pile up and forget about it > except for once in a blue moon retrieval, then it might be reasonable. > > If it's an active mailbox, it will be a pain to navigate, in the same > way a single folder with 100K files or a file cabinet with huge stacks > of envelopes. > > I would guess some partioning of the large mailboxes into smaller > mailboxes would help with active mailboxes.? Most people spend most of > their time on new/recent messages, so making time or size or subject > based volmes wouldn't be a bad idea. > > If the bulk of the size are redundant copies of attachments, then > Dovecot's > *dbox support de-duping which would aso help. >So, generally speaking, you don't want to have inboxes that just sync all day long, due to massive amounts of small files in the inbox.? This may be OK in the case of a rarely accessed archive folder, but not good for regularly accessed inboxes, etc.?