dovecot - Mar 2014 - Panic: file ostream-lzma.c: line 147: unreached. Dovecot 2.2.12 with zlib/XZ compression

While migrating users from Cyrus IMAP v2.3.14 to Dovecot 2.2.12 
(compiled from source) dsync aborts with a backtrace on some (maybe 15 
out of 800) mail accounts:

(same error happens wiht or without -f flag)

dsync -D -v -o mail_fsync=never mirror -f -R -u user at domain imapc:

<snip>
dsync(user at domain.com): Debug: brain M: in state=sync_mails
dsync(user at domain.com): Debug: brain M: in box 'Privat/Jerry' 
recv_state=mails send_state=done
dsync(user at domain.com): Debug: brain M: import mail uid 55 guid
dsync(user at domain.com): Debug: brain M: Import Privat/Jerry: Import 
mail body for GUID= UID=55
dsync(user at domain.com): Panic: file ostream-lzma.c: line 147: unreached
dsync(user at domain.com): Error: Raw backtrace: 
/usr/local/lib/dovecot/libdovecot.so.0(+0x68d4f) [0x7fb79ac26d4f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) 
[0x7fb79ac26e2a] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7fb79abe03b9] -> /usr/local/lib/dovecot/lib20_zlib_plugin.so(+0x5ca7) 
[0x7fb799fd0ca7] -> 
/usr/local/lib/dovecot/libdovecot.so.0(o_stream_flush+0x4d) 
[0x7fb79ac4039d] -> 
/usr/local/lib/dovecot/libdovecot.so.0(o_stream_nflush+0xf) 
[0x7fb79ac4085f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(o_stream_nfinish+0x9) 
[0x7fb79ac408a9] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x2b1) 
[0x7fb79aedb771] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x64) 
[0x7fb79af0ed44] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xa2) 
[0x7fb79af06c72] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(maildir_copy+0x51) 
[0x7fb79aed7b81] -> 
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5e) 
[0x7fb79af0eeae] -> dsync() [0x430dc2] -> 
dsync(dsync_brain_sync_mails+0x276) [0x42f256] -> 
dsync(dsync_brain_run+0x353) [0x42b8c3] -> dsync() [0x4296b2] -> dsync() 
[0x411dc7] -> dsync(doveadm_mail_try_run+0x260) [0x412a90] -> 
dsync(main+0x3f0) [0x4119b0] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fb79a81f76d] 
-> dsync() [0x411bad]
Aborted

I tried reconstructing the users mailbox on the cyrus server but it 
made no difference to the sync

Dovecot compiled with:

./configure --with-ssl --with-mysql --with-solr --with-zlib 
--with-bzlib --with-libcap --with-lzma --with-ldap --with-pam

As you can see, I have made very few configuration changes to get this 
far:

# 2.2.12: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-60-virtual x86_64 Ubuntu 12.04.4 LTS xfs
imapc_features = rfc822.size fetch-headers
imapc_host = oldmail.domain.com
imapc_master_user = <adminuser>
imapc_password = <password>
imapc_user = %u
mail_gid = vmail
mail_location = maildir:/var/vmail/domains/%d/%1n/%n/Maildir
mail_plugins = zlib
mail_prefetch_count = 50
mail_uid = vmail
mailbox_list_index = yes
namespace inbox {
   inbox = yes
   location    mailbox Drafts {
     special_use = \Drafts
   }
   mailbox Junk {
     special_use = \Junk
   }
   mailbox Sent {
     special_use = \Sent
   }
   mailbox "Sent Messages" {
     special_use = \Sent
   }
   mailbox Trash {
     special_use = \Trash
   }
   prefix    separator = /
}
passdb {
   args = /etc/dovecot/dovecot-ldap.conf.ext
   driver = ldap
}
plugin {
   zlib_save = xz
   zlib_save_level = 6
}
service auth-worker {
   user = $default_internal_user
}
ssl_cert = </etc/ssl/certs/<cert>
ssl_key = </etc/ssl/private/<cert>
userdb {
   driver = prefetch
}
userdb {
   args = /etc/dovecot/dovecot-ldap.conf.ext
   driver = ldap
}

I have looked at the source code referenced in the error and my limited 
knowledge got me looking for LZMA compress error codes but I couldn't 
find anything really :(

Please help!

Am 28.03.2014 12:22, schrieb Alan McGinlay:
> While migrating users from Cyrus IMAP v2.3.14 to Dovecot 2.2.12
are you on latest patchlevel ?

see

http://hg.dovecot.org/dovecot-2.2/

there are some dsync fixes since 2.2.12

> (compiled from source) dsync aborts with a backtrace on some (maybe 15
> out of 800) mail accounts:
> 
> (same error happens wiht or without -f flag)
> 
> dsync -D -v -o mail_fsync=never mirror -f -R -u user at domain imapc:
> 
> <snip>
> dsync(user at domain.com): Debug: brain M: in state=sync_mails
> dsync(user at domain.com): Debug: brain M: in box 'Privat/Jerry'
> recv_state=mails send_state=done
> dsync(user at domain.com): Debug: brain M: import mail uid 55 guid
> dsync(user at domain.com): Debug: brain M: Import Privat/Jerry: Import mail
> body for GUID= UID=55
> dsync(user at domain.com): Panic: file ostream-lzma.c: line 147: unreached
> dsync(user at domain.com): Error: Raw backtrace:
> /usr/local/lib/dovecot/libdovecot.so.0(+0x68d4f) [0x7fb79ac26d4f] ->
> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a)
> [0x7fb79ac26e2a] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
> [0x7fb79abe03b9] -> /usr/local/lib/dovecot/lib20_zlib_plugin.so(+0x5ca7)
> [0x7fb799fd0ca7] ->
> /usr/local/lib/dovecot/libdovecot.so.0(o_stream_flush+0x4d)
> [0x7fb79ac4039d] ->
> /usr/local/lib/dovecot/libdovecot.so.0(o_stream_nflush+0xf)
> [0x7fb79ac4085f] ->
> /usr/local/lib/dovecot/libdovecot.so.0(o_stream_nfinish+0x9)
> [0x7fb79ac408a9] ->
> /usr/local/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x2b1)
[0x7fb79aedb771]
> ->
> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x64)
> [0x7fb79af0ed44] ->
> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xa2)
> [0x7fb79af06c72] ->
> /usr/local/lib/dovecot/libdovecot-storage.so.0(maildir_copy+0x51)
> [0x7fb79aed7b81] ->
> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5e)
> [0x7fb79af0eeae] -> dsync() [0x430dc2] ->
> dsync(dsync_brain_sync_mails+0x276) [0x42f256] ->
> dsync(dsync_brain_run+0x353) [0x42b8c3] -> dsync() [0x4296b2] ->
dsync()
> [0x411dc7] -> dsync(doveadm_mail_try_run+0x260) [0x412a90] ->
> dsync(main+0x3f0) [0x4119b0] ->
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fb79a81f76d]
> -> dsync() [0x411bad]
> Aborted
> 
> I tried reconstructing the users mailbox on the cyrus server but it made
> no difference to the sync
> 
> Dovecot compiled with:
> 
> ./configure --with-ssl --with-mysql --with-solr --with-zlib --with-bzlib
> --with-libcap --with-lzma --with-ldap --with-pam
> 
> As you can see, I have made very few configuration changes to get this far:
> 
> # 2.2.12: /usr/local/etc/dovecot/dovecot.conf
> # OS: Linux 3.2.0-60-virtual x86_64 Ubuntu 12.04.4 LTS xfs
> imapc_features = rfc822.size fetch-headers
> imapc_host = oldmail.domain.com
> imapc_master_user = <adminuser>
> imapc_password = <password>
> imapc_user = %u
> mail_gid = vmail
> mail_location = maildir:/var/vmail/domains/%d/%1n/%n/Maildir
> mail_plugins = zlib
> mail_prefetch_count = 50
> mail_uid = vmail
> mailbox_list_index = yes
> namespace inbox {
>   inbox = yes
>   location >   mailbox Drafts {
>     special_use = \Drafts
>   }
>   mailbox Junk {
>     special_use = \Junk
>   }
>   mailbox Sent {
>     special_use = \Sent
>   }
>   mailbox "Sent Messages" {
>     special_use = \Sent
>   }
>   mailbox Trash {
>     special_use = \Trash
>   }
>   prefix >   separator = /
> }
> passdb {
>   args = /etc/dovecot/dovecot-ldap.conf.ext
>   driver = ldap
> }
> plugin {
>   zlib_save = xz
>   zlib_save_level = 6
> }
> service auth-worker {
>   user = $default_internal_user
> }
> ssl_cert = </etc/ssl/certs/<cert>
> ssl_key = </etc/ssl/private/<cert>
> userdb {
>   driver = prefetch
> }
> userdb {
>   args = /etc/dovecot/dovecot-ldap.conf.ext
>   driver = ldap
> }
> 
> I have looked at the source code referenced in the error and my limited
> knowledge got me looking for LZMA compress error codes but I couldn't
> find anything really :(
> 
> Please help!


Best Regards
MfG Robert Schetterer

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstra?e 15, 81669 M?nchen

Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein