On 16 Aug 2017, at 23.59, Joseph Tam <jtam.home at gmail.com>
wrote:>
>
> Timo Sirainen wrote:
>
>> There are various changes in this release that can be used to
significantly reduce disk IO with:
>> 1) NFS storage especially, but I guess also other remote filesystems
and even some with local disks
>> 2) When mail storage and INDEX storage are separated
>
> Thanks for these changes! Big win for my setup. My servers are not
> overly stressed, but how much performance gain (using any metric you
> choose) can one expect from these changes?
Depends a lot of things, but for the one specific NFS installation we reduced
NFS IOPS by something like 70% And actually they haven't finished setting up
the local LISTINDEX changes yet, so maybe even more.
>> + mail_location can now include VOLATILEDIR=<path> parameter.
This
>> is used for creating lock files and in future potentially other
>> files that don't need to exist permanently. The path could point
to
>> tmpfs for example. This is especially useful to avoid creating lock
>> files to NFS or other remote filesystems. For example:
>> mail_location=sdbox:~/sdbox:VOLATILEDIR=/tmp/volatile/%2.256Nu/%u
>
> Is "/tmp/volatile" auto-created, or must be pre-created?
Autocreated.
>> + mail_location's LISTINDEX=<path> can now contain a full
path.
>> This allows storing mailbox list index to a different storage
>> than the rest of the indexes, for example to tmpfs.
>
> Is this in any way related to VOLATILEDIR? Can they be set to the same
> value without problems?
Yeah, can be set to the same directory. Maybe even a good idea.
>> + mail_location can now include NO-NOSELECT parameter. This
>> automatically deletes any \NoSelect mailboxes that have no children.
>> These mailboxes are sometimes confusing to users.
>
> Sorry for my IMAP ignorance, but how can this situation come about?
a CREATE noselectbox/
--> With this change it's the same as CREATE "noselectbox" =
will become selectable.
or
a CREATE noselectbox/childbox
b DELETE noselectbox/childbox
--> noselectbox is left behind. With this change it will be auto-deleted.
>> + mail_location can now include ITERINDEX parameter. This tells Dovecot
>> to perform mailbox listing from the INDEX path instead of from the
>> mail root path. It's mainly useful when the INDEX storage is on a
>> faster storage.
>> + If mailbox_list_index_very_dirty_syncs=yes, the list index is no
>> longer refreshed against filesystem when listing mailboxes. This
>> allows the mailbox listing to be done entirely by only reading the
>> mailbox list index.
>> + Added mailbox_list_index_include_inbox setting to control whether
>> INBOX's STATUS information should be cached in the mailbox list
>> index. The default is "no", but it may be useful to change
it to
>> "yes", especially if LISTINDEX points to tmpfs.
>
> So as I understand it, the optimzation comes about from segregating mail
> data information into 3 parts: raw mail, indices, and volatile components,
> putting them into increasingly better performing storage media.
>
> How do these I/O optimizations affect the client's view of a mailbox
> if their mailbox is subject to modification outside the dovecot system
> (e.g. procmail, mail readers directly modifies mailbox files)? Is there
> a trade-off between metadata consistency and performance, or it's a
> win-win all around?
I think already even with mailbox_list_index=yes it won't notice external
changes done to the mailboxes when doing STATUS calls, so better to avoid that.
It does get fixed automatically once the folder is selected though.