http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz.sig A few fixes and some imapc improvements since the release candidate. * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes dovecot.index.cache file. The cache file was rarely the problem so this just caused unnecessary slowness. * Mailbox name limits changed during mailbox creation: Each part of a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 chars long (instead of 200). This also reduces the max number of hierarchical levels to 16 (instead of 20) to keep the maximum name length 4096 (a common PATH_MAX limit). The 255 char limit is hopefully large enough for migrations from all existing systems. It's also the limit on many filesystems. + director: Added director_consistent_hashing setting to enable consistent hashing (instead of the mostly-random MD5 hashing). This causes fewer user moves between backends when backend counts are changed, which may improve performance (mainly due to caching). + director: Added support for "tags", which allows one director ring to serve multiple backend clusters with different sets of users. + LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user. + LMTP server: Added support for STARTTLS command. + If logging data is generated faster than it can be written, log a warning about it and show information about it in log process's process title in ps output. Also don't allow a single service to flood too long at the cost of delaying other services' logging. + stats: Added support for getting global statistics. + stats: Use the same session IDs as the rest of Dovecot. + stats: Plugins can now create their own statistics fields + doveadm server: Non-mail related commands can now also be used via doveadm server (TCP socket). + doveadm proxying: passdb lookup can now override doveadm_port and change the username. + doveadm: Search query supports now "oldestonly" parameter to stop immediately on the first non-match. This can be used to optimize: doveadm expunge mailbox Trash savedbefore 30d oldestonly + doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve). + doveadm fetch: Added body.snippet field, which returns the first 100 chars of a message without whitespace or HTML tags. The result is stored into dovecot.index.cache, so it can be fetched efficiently. + dsync: Added -t <timestamp> parameter to sync only mails newer than the given received-timestamp. + dsync: Added -F [-]<flag> parameter to sync only mails with[out] the given flag/keyword. + dsync: Added -a <mailbox> parameter to specify the virtual mailbox containing user's all mails. If this mailbox is already found to contain the wanted mail (by its GUID), the message is copied from there instead of being re-saved. (This isn't efficient enough yet for incremental replication.) + dsync: -m parameter can now specify \Special-use names for mailboxes. + imapc: Added imapc_features=gmail-migration to help migrations from GMail. See http://wiki2.dovecot.org/Migration/Gmail + imapc: Added imapc_features=search to support IMAP SEARCH command. (Currently requires ESEARCH support from remote server.) + expire plugin: Added expire_cache=yes setting to cache most of the database lookups in dovecot index files. + quota: If overquota-flag in userdb doesn't match the current quota usage, execute a configured script. + redis dict: Added support for expiring keys (:expire_secs=n) and specifying the database number (:db=n) - auth: Don't crash if master user login is attempted without any configured master=yes passdbs - Parsing UTF-8 text for mails could have caused broken results sometimes if buffering was split in the middle of a UTF-8 character. This affected at least searching messages. - String sanitization for some logged output wasn't done properly: UTF-8 text could have been truncated wrongly or the truncation may not have happened at all. - fts-lucene: Lookups from virtual mailbox consisting of over 32 physical mailboxes could have caused crashes.
For this no man-page available. Am 12.03.2015 um 18:30 schrieb Timo Sirainen:> + doveadm: Added "save" command to directly save mails to specified > mailbox (bypassing Sieve).
/usr/lib64/dovecot/stats/libstats_mail.so why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files? Am 12.03.2015 um 18:30 schrieb Timo Sirainen:> http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz > http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz.sig > > A few fixes and some imapc improvements since the release candidate. > > * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes > dovecot.index.cache file. The cache file was rarely the problem > so this just caused unnecessary slowness. > * Mailbox name limits changed during mailbox creation: Each part of > a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 > chars long (instead of 200). This also reduces the max number of > hierarchical levels to 16 (instead of 20) to keep the maximum name > length 4096 (a common PATH_MAX limit). The 255 char limit is > hopefully large enough for migrations from all existing systems. > It's also the limit on many filesystems. > > + director: Added director_consistent_hashing setting to enable > consistent hashing (instead of the mostly-random MD5 hashing). > This causes fewer user moves between backends when backend counts > are changed, which may improve performance (mainly due to caching). > + director: Added support for "tags", which allows one director ring > to serve multiple backend clusters with different sets of users. > + LMTP server: Added lmtp_user_concurrency_limit setting to limit how > many LMTP deliveries can be done concurrently for a single user. > + LMTP server: Added support for STARTTLS command. > + If logging data is generated faster than it can be written, log a > warning about it and show information about it in log process's > process title in ps output. Also don't allow a single service to > flood too long at the cost of delaying other services' logging. > + stats: Added support for getting global statistics. > + stats: Use the same session IDs as the rest of Dovecot. > + stats: Plugins can now create their own statistics fields > + doveadm server: Non-mail related commands can now also be used > via doveadm server (TCP socket). > + doveadm proxying: passdb lookup can now override doveadm_port and > change the username. > + doveadm: Search query supports now "oldestonly" parameter to stop > immediately on the first non-match. This can be used to optimize: > doveadm expunge mailbox Trash savedbefore 30d oldestonly > + doveadm: Added "save" command to directly save mails to specified > mailbox (bypassing Sieve). > + doveadm fetch: Added body.snippet field, which returns the first > 100 chars of a message without whitespace or HTML tags. The result > is stored into dovecot.index.cache, so it can be fetched efficiently. > + dsync: Added -t <timestamp> parameter to sync only mails newer than > the given received-timestamp. > + dsync: Added -F [-]<flag> parameter to sync only mails with[out] the > given flag/keyword. > + dsync: Added -a <mailbox> parameter to specify the virtual mailbox > containing user's all mails. If this mailbox is already found to > contain the wanted mail (by its GUID), the message is copied from > there instead of being re-saved. (This isn't efficient enough yet > for incremental replication.) > + dsync: -m parameter can now specify \Special-use names for mailboxes. > + imapc: Added imapc_features=gmail-migration to help migrations from > GMail. See http://wiki2.dovecot.org/Migration/Gmail > + imapc: Added imapc_features=search to support IMAP SEARCH command. > (Currently requires ESEARCH support from remote server.) > + expire plugin: Added expire_cache=yes setting to cache most of the > database lookups in dovecot index files. > + quota: If overquota-flag in userdb doesn't match the current quota > usage, execute a configured script. > + redis dict: Added support for expiring keys (:expire_secs=n) and > specifying the database number (:db=n) > - auth: Don't crash if master user login is attempted without > any configured master=yes passdbs > - Parsing UTF-8 text for mails could have caused broken results > sometimes if buffering was split in the middle of a UTF-8 character. > This affected at least searching messages. > - String sanitization for some logged output wasn't done properly: > UTF-8 text could have been truncated wrongly or the truncation may > not have happened at all. > - fts-lucene: Lookups from virtual mailbox consisting of over 32 > physical mailboxes could have caused crashes.-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://dovecot.org/pipermail/dovecot/attachments/20150312/ec689cc4/attachment.sig>
On 12 Mar 2015, at 21:09, Reindl Harald <h.reindl at thelounge.net> wrote:> > /usr/lib64/dovecot/stats/libstats_mail.so > > why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files?So that external plugins can add more files in there and extend the available statistics.
This happened after upgrading to 2.2.16. Running on Debian wheezy amd64.
We never had such a crash on this server before. However, this too was a
single incident.
All accounts use Maildir.
Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: Log
synchronization error at seq=0,offset=0 for
/stor/mail/domains/----/----/Maildir/dovecot.index: Append with UID
6684, but next_uid = 6685
Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error:
/stor/mail/domains/----/----/Maildir/dovecot.index view syncing failed
to apply changes
Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Panic: file
index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion
failed: (range->seq2 <= ctx->messages_count)
Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f34cd27c57f] ->
/usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f34cd27c5de] ->
/usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f34cd22b0d5] ->
/usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f34cd5a6391] ->
dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes
waiting](imap_sync_more+0xce) [0x7f34cda5ad2e] -> dovecot/imap
[---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](+0x22344)
[0x7f34cda5b344] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH -
2373 bytes waiting](cmd_sync_delayed+0x1bf) [0x7f34cda5b67f] ->
dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes
waiting](client_output+0xe0) [0x7f34cda52910] ->
/usr/lib/dovecot/libdovecot.so.0(+0x98a85) [0x7f34cd299a85] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f34cd28e80b]
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb)
[0x7f34cd28f88b] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9)
[0x7f34cd28e899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f34cd28e918] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f34cd2307d3] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH -
2373 bytes waiting](main+0x2b7) [0x7f34cda46997] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f34cce94ead]
-> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes
waiting](+0xdb09) [0x7f34cda46b09]
Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Fatal: master:
service(imap): child 30144 killed with signal 6 (core dumps disabled)
# doveconf -n
# 2.2.16: /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.7
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8
auth_master_user_separator = *
auth_mechanisms = plain login cram-md5
auth_verbose = yes
auth_verbose_passwords = plain
dict {
expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
disable_plaintext_auth = no
login_greeting = Dovecot ready
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l
pip=%{real_rip} mpid=%e %c session=<%{session}>
mail_gid = vmail
mail_location = /nowhere
mail_plugins = quota expire listescape
mail_uid = vmail
managesieve_sieve_capability = fileinto envelope encoded-character
subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables mailbox date index ihave duplicate
namespace inbox {
inbox = yes
location mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix separator = /
type = private
}
passdb {
args = /etc/dovecot/master-users
driver = passwd-file
master = yes
pass = yes
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
antispam_backend = pipe
antispam_pipe_program = /usr/bin/ssh
antispam_pipe_program_args =
-l;spamd;-i;/etc/dovecot/sareport.key;mx1.xxxxx
antispam_pipe_program_notspam_arg = revoke
antispam_pipe_program_spam_arg = report
antispam_pipe_tmpdir = /tmp
antispam_spam = Junk
antispam_trash_pattern_ignorecase = trash;Deleted *
expire = Trash
expire2 = Trash/*
expire3 = Junk
expire_dict = proxy::expire
quota = dict:user::proxy::quota
quota_rule = *:storage=2G
quota_rule2 = Trash:storage=+250M
quota_rule3 = Junk:ignore
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=85%% quota-warning 85 %u
sieve = ~/.dovecot.sieve
sieve_before = /etc/dovecot/sieve-global/fileinto-spam.sieve
sieve_dir = ~/sieve
sieve_extensions = -vacation -body -reject -enotify -environment
-virustest -spamtest
sieve_max_actions = 32
sieve_max_redirects = 4
sieve_max_script_size = 10K
sieve_quota_max_scripts = 100
sieve_quota_max_storage = 4M
}
pop3_no_flag_updates = yes
protocols = imap pop3 lmtp sieve
service auth-worker {
user = $default_internal_user
}
service auth {
client_limit = 2448
unix_listener auth-userdb {
group = root
mode = 0600
user = vmail
}
}
service dict {
unix_listener dict {
group = root
mode = 0600
user = vmail
}
}
service imap-login {
process_min_avail = 4
service_count = 0
vsz_limit = 192 M
}
service imap-postlogin {
executable = script-login /usr/local/bin/postlogin
unix_listener imap-postlogin {
group = root
mode = 0600
user = vmail
}
user = root
}
service imap {
executable = imap imap-postlogin
process_limit = 1024
vsz_limit = 128 M
}
service lmtp {
inet_listener lmtp {
address = 10.44.23.1
port = 7025
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
service_count = 0
vsz_limit = 64 M
}
service managesieve {
process_limit = 20
}
service pop3-login {
process_min_avail = 4
service_count = 0
vsz_limit = 192 M
}
service pop3 {
executable = pop3 imap-postlogin
process_limit = 1024
}
service quota-warning {
executable = script /usr/local/bin/quota-warning
unix_listener quota-warning {
user = vmail
}
user = dovecot
}
ssl_cert = </etc/dovecot/dovecot.pem
ssl_key = </etc/dovecot/private/dovecot.pem
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
verbose_proctitle = yes
protocol lmtp {
mail_plugins = quota expire listescape sieve
plugin {
quota = dict:user::noenforcing:proxy::quota
}
}
protocol lda {
mail_plugins = quota expire listescape sieve
}
protocol imap {
mail_max_userip_connections = 12
mail_plugins = quota expire listescape imap_quota antispam
ssl_cert = </etc/dovecot/xxxxxx.crt
ssl_key = </etc/dovecot/private/xxxxx.key
}
protocol sieve {
mail_max_userip_connections = 5
}
protocol pop3 {
mail_max_userip_connections = 10
ssl_cert = </etc/dovecot/xxxxx.crt
ssl_key = </etc/dovecot/private/xxxxx.key
}
On 03/23/2015 12:10 AM, Gedalya wrote:> This happened after upgrading to 2.2.16. Running on Debian wheezy amd64. > We never had such a crash on this server before. However, this too was > a single incident. > All accounts use Maildir. > > > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: Log > synchronization error at seq=0,offset=0 for > /stor/mail/domains/----/----/Maildir/dovecot.index: Append with UID > 6684, but next_uid = 6685 > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: > /stor/mail/domains/----/----/Maildir/dovecot.index view syncing failed > to apply changes > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Panic: file > index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion > failed: (range->seq2 <= ctx->messages_count) > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Error: Raw > backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f34cd27c57f] > -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f34cd27c5de] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f34cd22b0d5] -> > /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f34cd5a6391] -> > dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes > waiting](imap_sync_more+0xce) [0x7f34cda5ad2e] -> dovecot/imap > [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting](+0x22344) > [0x7f34cda5b344] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - > 2373 bytes waiting](cmd_sync_delayed+0x1bf) [0x7f34cda5b67f] -> > dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - 2373 bytes > waiting](client_output+0xe0) [0x7f34cda52910] -> > /usr/lib/dovecot/libdovecot.so.0(+0x98a85) [0x7f34cd299a85] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) > [0x7f34cd28e80b] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) > [0x7f34cd28f88b] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) > [0x7f34cd28e899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f34cd28e918] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f34cd2307d3] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - > 2373 bytes waiting](main+0x2b7) [0x7f34cda46997] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) > [0x7f34cce94ead] -> dovecot/imap [---- at ---.com xx.xx.xx.xx UID FETCH - > 2373 bytes waiting](+0xdb09) [0x7f34cda46b09] > Mar 21 09:32:02 imap1 dovecot: imap(---- at ---.com): Fatal: master: > service(imap): child 30144 killed with signal 6 (core dumps disabled) >And now it just happened, again, same user! The previous dovecot version before the upgrade was 2.2.12 Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Error: Log synchronization error at seq=0,offset=0 for /stor/mail/domains/----.com/----/Maildir/dovecot.index: Append with UID 12794, but next_uid = 12795 Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Error: /stor/mail/domains/----.com/----/Maildir/dovecot.index view syncing failed to apply changes Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Panic: file index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion failed: (range->seq2 <= ctx->messages_count) Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f136acdc57f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f136acdc5de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f136ac8b0d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f136b006391] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](imap_sync_more+0xce) [0x7f136b4bad2e] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](+0x22344) [0x7f136b4bb344] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](cmd_sync_delayed+0x1bf) [0x7f136b4bb67f] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](client_handle_input+0x1d5) [0x7f136b4b2725] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](client_input+0x75) [0x7f136b4b2a35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f136acee80b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f136acef88b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f136acee899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f136acee918] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f136ac907d3] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](main+0x2b7) [0x7f136b4a6997] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f136a8f4ead] -> dovecot/imap [---- at ----.com xx.xx.xx.xx UID FETCH](+0xdb09) [0x7f136b4a6b09] Mar 28 07:51:17 imap1 dovecot: imap(---- at ----.com): Fatal: master: service(imap): child 12444 killed with signal 6 (core dumps disabled)