Hello Timo,
I've successfully setup xz compression for my Dovecot installation using
the version 2.2.12 from Debian unstable.
Together with the activation of the "Single Instance Attachment
Storage", this reduced the mail storage utilization for 20 mail accounts
by 74%.
*My question:*
(For this test just one user was logged in. OS = Debian Wheezy, AMD64)
After conversion of my mail storage, IMAP read by Thunderbird worked
perfectly. But IMAP write and LMTP write was not possible.*
*xz compression seems to need a higher vsz_limit than its default value
of 256M without compression.
Before the conversion, 256M was enough with the plugins "**acl fts
fts_solr" and " imap_zlib".
512M was still not enough for xz compression, 1024M works OK but I'm not
sure since this is a relative large increase.
My new setting: /etc/dovecot/conf.d/10-master.conf: service imap {
vsz_limit=1024M }
The absolute minimum to successfully send an empty mail is 710MB (700MB
still produces out of memory)!
_*Is this normal behaviour? What is a recommended value for "service
imap {vsz_limit = }"?*_
I recommend to add this "_vsz_limit"_-increase to the wiki
(http://wiki2.dovecot.org/Plugins/Zlib) to make others life easier.
Error log below.
*
**My config & how I converted:*
/etc/dovecot/conf.d/10-mail.conf:
mail_plugins = *zlib *acl fts fts_solr
/etc/dovecot/conf.d/90-plugin.conf:
# Enable these only if you want compression while saving:
zlib_save_level = 9 # 1..9
zlib_save = xz
For "Single Instance Attachment Storage":
/etc/dovecot/conf.d/10-mail.conf:
# Directory root where to store mail attachments. Disabled, if empty.
mail_attachment_dir = /srv/vmail/mail_attachments
For compressed IMAP access ( to reduce the bandwidth usage of IMAP )
(has nothing to do with storage space):
/etc/dovecot/conf.d/20-imap.conf:
protocol imap {
mail_plugins = $mail_plugins *imap_zlib*
}
Previous storage format was standard mdbox Format.
To convert a single user, I used dsync from the command line:
dsync -u user at domain.de mirror mdbox:/srv/converted_vmail/user at domain.de/
After conversion (~2 hours per account), I moved the original mail
folder of the user to a backup location and replaced it by the converted
folders from "/srv/converted_vmail/user at domain.de/".
(Pay attention that the directory structure matches!, I had to adjust ist.)
*My Problem:*
*After conversion IMAP read by Thunderbird worked perfectly**. But IMAP
write and **LMTP write was not possible**.**
**/var/log/dovecot.log showed that vsz_limit**was to restrictive:*
Log:
Apr 03 14:13:14 imap(user at domain.de): Fatal: lzma: Out of memory
Apr 03 14:13:14 imap(user at domain.de): Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(+0x674f0) [0x7f8538fd54f0]
-> /usr/lib/dovecot/libdovecot.so.0(+0x675ce)
[0x7f8538fd55ce]
-> /usr/lib/dovecot/libdovecot.so.0(i_error+0)
[0x7f8538f90678]
->
/usr/lib/dovecot/modules/lib20_zlib_plugin.so(o_stream_create_lzma+0xeb)
[0x7f8538393ecb]
->
/usr/lib/dovecot/modules/lib20_zlib_plugin.so(+0x2c14) [0x7f8538390c14]
->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x43)
[0x7f85392bdc23]
-> dovecot/imap [user at domain.de 192.168.1.42
append](+0xd824) [0x7f8539788824]
-> dovecot/imap [user at domain.de 192.168.1.42
append](cmd_append+0x10e) [0x7f8539788a6e]
-> dovecot/imap [user at domain.de 192.168.1.42
append](command_exec+0x3c) [0x7f853979390c]
-> dovecot/imap [user at domain.de 192.168.1.42
append](+0x178ff) [0x7f85397928ff]
-> dovecot/imap [user at domain.de 192.168.1.42
append](+0x179b5) [0x7f85397929b5]
-> dovecot/imap [user at domain.de 192.168.1.42
append](client_handle_input+0x125) [0x7f8539792cd5]
-> dovecot/imap [user at domain.de 192.168.1.42
append](client_input+0x75) [0x7f8539793065]
->
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x27) [0x7f8538fe55c7]
->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7f8538fe6357]
->
/usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f8538fe5168]
->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f8538f95723]
-> dovecot/imap [user at domain.de 192.168.1.42
append](main+0x2c4) [0x7f8539787324]
->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f8538be6b45]
-> dovecot/imap [user at domain.de 192.168.1.42
append](+0xc490) [0x7f8539787490]
Apr 03 14:13:14 imap-login: Debug: SSL alert: close notify [192.168.1.42]
Apr 03 14:13:14 imap(user at domain.de): Fatal: master: service(imap):
child 23995 returned error 83 (Out of memory (service imap {
vsz_limit=256 MB },
you may need to increase it) - set
CORE_OUTOFMEM=1 environment to get core dump)
==============================================