http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16. Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements. The largest changes since v2.2.15: * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes dovecot.index.cache file. The cache file was rarely the problem so this just caused unnecessary slowness. * Mailbox name limits changed during mailbox creation: Each part of a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 chars long (instead of 200). This also reduces the max number of hierarchical levels to 16 (instead of 20) to keep the maximum name length 4096 (a common PATH_MAX limit). The 255 char limit is hopefully large enough for migrations from all existing systems. It's also the limit on many filesystems. + director: Added director_consistent_hashing setting to enable consistent hashing (instead of the mostly-random MD5 hashing). This causes fewer user moves between backends when backend counts are changed, which may improve performance (mainly due to caching). + director: Added support for "tags", which allows one director ring to serve multiple backend clusters with different sets of users. + LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user. + LMTP server: Added support for STARTTLS command. + If logging data is generated faster than it can be written, log a warning about it and show information about it in log process's process title in ps output. Also don't allow a single service to flood too long at the cost of delaying other services' logging. + stats: Added support for getting global statistics. + stats: Use the same session IDs as the rest of Dovecot. + stats: Plugins can now create their own statistics fields + doveadm server: Non-mail related commands can now also be used via doveadm server (TCP socket). + doveadm proxying: passdb lookup can now override doveadm_port and change the username. + doveadm: Search query supports now "oldestonly" parameter to stop immediately on the first non-match. This can be used to optimize: doveadm expunge mailbox Trash savedbefore 30d oldestonly + doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve). + doveadm fetch: Added body.snippet field, which returns the first 100 chars of a message without whitespace or HTML tags. The result is stored into dovecot.index.cache, so it can be fetched efficiently. + dsync: Added -t <timestamp> parameter to sync only mails newer than the given received-timestamp. + dsync: Added -F [-]<flag> parameter to sync only mails with[out] the given flag/keyword. + dsync: Added -a <mailbox> parameter to specify the virtual mailbox containing user's all mails. If this mailbox is already found to contain the wanted mail (by its GUID), the message is copied from there instead of being re-saved. (This isn't efficient enough yet for incremental replication.) + dsync: -m parameter can now specify \Special-use names for mailboxes. + imapc: Added imapc_features=gmail-migration to help migrations from GMail. See http://wiki2.dovecot.org/Migration/Gmail + imapc: Added imapc_features=search to support IMAP SEARCH command. (Currently requires ESEARCH support from remote server.) + expire plugin: Added expire_cache=yes setting to cache most of the database lookups in dovecot index files. + quota: If overquota-flag in userdb doesn't match the current quota usage, execute a configured script. + redis dict: Added support for expiring keys (:expire_secs=n) and specifying the database number (:db=n) - auth: Don't crash if master user login is attempted without any configured master=yes passdbs - Parsing UTF-8 text for mails could have caused broken results sometimes if buffering was split in the middle of a UTF-8 character. This affected at least searching messages. - String sanitization for some logged output wasn't done properly: UTF-8 text could have been truncated wrongly or the truncation may not have happened at all. - fts-lucene: Lookups from virtual mailbox consisting of over 32 physical mailboxes could have caused crashes.
* Timo Sirainen <dovecot at dovecot.org>:> + LMTP server: Added lmtp_user_concurrency_limit setting to limit how > many LMTP deliveries can be done concurrently for a single user.That's broken! 0 doesn't remove the limit :( -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 06 Mar 2015, at 18:47, Ralf Hildebrandt <r at sys4.de> wrote:> > * Timo Sirainen <dovecot at dovecot.org>: > >> + LMTP server: Added lmtp_user_concurrency_limit setting to limit how >> many LMTP deliveries can be done concurrently for a single user. > > That's broken! 0 doesn't remove the limit :(Should have been fixed in the rc1 already: http://hg.dovecot.org/dovecot-2.2/rev/9c69c197b527
On 3/6/2015 7:53 AM, Timo Sirainen wrote:> http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz > http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig > > Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16. > > Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements.Great to hear both Dovecot and you are doing well. I do need to ask you to check the list for two threads: mdbox attachment errors Rebuilding SIS attachment links from log A few of us have been having SIS problems. -- Daniel
On 06 Mar 2015, at 21:58, Daniel Miller <dmiller at amfes.com> wrote:> > On 3/6/2015 7:53 AM, Timo Sirainen wrote: >> http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz >> http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig >> >> Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16. >> >> Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements. > > Great to hear both Dovecot and you are doing well. I do need to ask you to check the list for two threads: > > mdbox attachment errors > Rebuilding SIS attachment links from log > > A few of us have been having SIS problems.Unless there's a way to reproduce a bug I don't think I can do anything about it (I could spend hours looking at the code or trying to reproduce it and come up with nothing). But a while ago I did think about a SIS redesign that would make it much less likely to break - just need to get it actually implemented: Currently single instance storage works by having one global directory that contains all the attachments. They are hashed by the attachment content, so for example /var/attachments/ac/7d/ac7d1274891248912489124 would be the attachment. Then each instance would have its own hard link to it, e.g. /var/attachments/ac/7d/hashes/ac7d1274891248912489124-1234567890. sdbox and mdbox can use these by containing the "ac7d1274891248912489124-1234567890" in the header metadata. When mail is deleted, the hard link is deleted. If the link count had been 2, the original attachment file was deleted also. (There's of course some race conditions here, but in those rare situations the attachment would just be duplicated, which isn't too bad.) The main problem with the old design is that all the users' attachments are dumped into a single global directory. It's difficult to take backups and in general it seems too difficult to manage correctly so I haven't really recommended using it in any bigger installations. So here's the new idea, which is nearly the same as the old, but with a small change that makes it much nicer I think: Instead of storing the attachment hard links to a global dir, store the hard links under the user's mail dir. This way taking backups doesn't require anything complicated, just tar the user's mail dir. You can rm -rf the user without forever leaving the user's attachments lying around in the global dir (assuming there's a job that periodically cleans out attachments with link count=1). In general there's no easy way to accidentally break things. The only new complication here is that if users are split to multiple filesystems, hard linking across them isn't going to work. So this would then require not only having a per-user mail directory but also per-user attachment directory (which would actually be the per-filesystem attachment dir). The SIS is implemented as lib-fs backend wrapper, so a new one could be implemented easily without breaking the old one.
This easy to correct errors is unfortunately still exists. I once hung my solution, but I can only test on a system. # dovecot --version 2.2.16.rc1 # doveadm -f table mailbox status "messages guid" -u flor_hardy "*" mailbox messages guid Trash 0 60a619171f66fd54035800004b126f5b Entw?rfe 0 62a619171f66fd54035800004b126f5b INBOX 0 61a619171f66fd54035800004b126f5b -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.2.16rc1-table.diff Type: text/x-patch Size: 1426 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20150309/364467d5/attachment.bin> -------------- next part -------------- An embedded message was scrubbed... From: Hardy Flor <HFlor at gmx.de> Subject: incorrect column width for multi-byte characters by'doveadm -f table' Date: Wed, 03 Dec 2014 22:57:24 +0100 Size: 5012 URL: <http://dovecot.org/pipermail/dovecot/attachments/20150309/364467d5/attachment.mht>
On 09 Mar 2015, at 13:07, Hardy Flor <HFlor at gmx.de> wrote:> > This easy to correct errors is unfortunately still exists. > I once hung my solution, but I can only test on a system. > > > # dovecot --version > 2.2.16.rc1 > # doveadm -f table mailbox status "messages guid" -u flor_hardy "*" > mailbox messages guid > Trash 0 60a619171f66fd54035800004b126f5b > Entw?rfe 0 62a619171f66fd54035800004b126f5b > INBOX 0 61a619171f66fd54035800004b126f5b > > <dovecot-2.2.16rc1-table.diff><Nachricht als Anhang.eml>Thanks, committed with some coding style changes: http://hg.dovecot.org/dovecot-2.2/rev/8780ca0fbf22 <http://hg.dovecot.org/dovecot-2.2/rev/8780ca0fbf22>