benjamin at godbersen.info
2019-Mar-14 22:28 UTC
Dovecot crashing when attempting to search in virtual folder with fts_squat activated
Hi everyone, I am running into a problem when trying to use fts_squat in a virtual folder. Without fts_squat plugin the search (from, subject...) works in all folders. With activated fts the search on the inbox folders works expectedly well but any attempt to search anything in any virtual folder leads to the following error. Similarly when attempting "doveadm fts lookup". I also noticed that no search index for the virtual folders gets build - is this expected behaviour? Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. Panic: file mail-storage.c: line 1913 (mailbox_get_open_status): assertion failed: (box->opened) Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. Error: Raw backtrace: /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(+0xba731) [0x7f553a7ff731] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(+0xba7fa) [0x7f553a7ff7fa] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(i_fatal+0) [0x7f553a771638] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot-storage.so.0(mailbox_get_open_status+0x68) [0x7f553aae4a78] -> /usr/lib/dovecot/modules/lib21_fts_squat_plugin.so(+0x3684) [0x7f553677a684] -> /usr/lib/dovecot/modules/lib21_fts_squat_plugin.so(+0x3820) [0x7f553677a820] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_lookup_multi+0x163) [0x7f5539b016a3] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd728) [0x7f5539b06728] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_lookup+0xeb) [0x7f5539b06bbb] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xf8b8) [0x7f5539b088b8] -> dovecot/imap(imap_search_start+0x6a) [0x5654cb5a0d6a] -> dovecot/imap(cmd_sort+0x293) [0x5654cb593553] -> dovecot/imap(command_exec+0x64) [0x5654cb599874] -> dovecot/imap(+0x1bd22) [0x5654cb597d22] -> dovecot/imap(+0x1bdbc) [0x5654cb597dbc] -> dovecot/imap(client_handle_input+0x1b5) [0x5654cb5981c5] -> dovecot/imap(client_input+0xa4) [0x5654cb5987e4] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f553a8174a9] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e) [0x7f553a818d1e] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f553a8175ac] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f553a8177b8] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f553a7940a3] -> dovecot/imap(main+0x339) [0x5654cb58a539] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f553a375b97] -> dovecot/imap(_start+0x2a) [0x5654cb58a71a] Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. Fatal: master: service(imap): child 6436 killed with signal 6 (core dumps disabled) This is my config: # 2.3.0.1 (ffd8a29): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.0.1 (d33dca20) # OS: Linux 4.15.0-46-generic x86_64 Ubuntu 18.04.2 LTS ext4 auth_mechanisms = plain login digest-md5 cram-md5 apop auth_username_chars abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890&.-_@' default_vsz_limit = 8096 M disable_plaintext_auth = no first_valid_uid = 30 imap_client_workarounds = delay-newmail imap_logout_format = rcvd=%i, sent=%o mail_home = /var/qmail/mailnames/%Ld/%Ln mail_location = maildir:/var/qmail/mailnames/%Ld/%Ln/Maildir mail_log_prefix = "service=%s, user=%u, ip=[%r]. " mail_max_userip_connections = 100 mail_plugins = quota fts fts_squat virtual managesieve_logout_format = rcvd=%i, sent=%o managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeri$ namespace inbox { ? inbox = yes ? location ? prefix = INBOX. ? separator = . } namespace virtual { ? hidden = no ? inbox = no ? list = yes ? location virtual:/var/qmail/mailnames/%Ld/%Ln/virtual/:INDEX=/var/qmail/mailnames/%Ld/%Ln/virtual/ ? prefix ? separator = . } passdb { ? driver = plesk } plugin { ? fts = squat ? fts_squat = partial=4 full=10 ? quota = maildir:User quota ? quota_grace = 0 ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve ? sieve_extensions = +notify +imapflags } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = rcvd=%i, sent=%o, top=%t/%p, retr=%r/%b, del=%d/%m, size=%s protocols = imap pop3 sieve service auth-worker { ? group ? user } service auth { ? group ? unix_listener auth-userdb { ??? group = popuser ??? mode = 0600 ??? user = popuser ? } ? user } service imap-login { ? process_limit = 2048 ? service_count = 1 } service imap { ? process_limit = 2048 ? service_count = 1 } service pop3-login { ? process_limit = 2048 ? service_count = 1 } service pop3 { ? process_limit = 2048 ? service_count = 1 } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_cipher_list = HIGH:!aNULL:!MD5 ssl_dh =? # hidden, use -P to show it ssl_key =? # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { ? args = uid=popuser gid=popuser ? driver = static } protocol imap { ? mail_plugins = quota fts fts_squat virtual imap_quota } protocol pop3 { ? pop3_uidl_format = UID%u-%v } protocol sieve { ? mail_plugins = quota fts fts_squat virtual } protocol lda { ? mail_plugins = quota fts fts_squat virtual sieve } dovecot --version -> 2.3.0.1 (ffd8a29) I will be greatful for any tips hinting me on how to resolve this behaviour. Thank you very much in advance Benjamin -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20190314/0d0ef86e/attachment-0001.html>
Benjamin Godbersen
2019-Mar-20 19:21 UTC
Dovecot crashing when attempting to search in virtual folder with fts_squat activated
Hi everyone, I have now updated to dovecot 2.3.4.1 - unfortunately the issue still persists. Can anyone help me figure out if this is due to a misconfiguration on my part or another error? Any help is greatly appreciated! Cheers Benjamin Am 14.03.2019 um 23:28 schrieb benjamin at godbersen.info:> Hi everyone, > > I am running into a problem when trying to use fts_squat in a virtual > folder. Without fts_squat plugin the search (from, subject...) works > in all folders. With activated fts the search on the inbox folders > works expectedly well but any attempt to search anything in any > virtual folder leads to the following error. Similarly when attempting > "doveadm fts lookup". I also noticed that no search index for the > virtual folders gets build - is this expected behaviour? > > Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. > Panic: file mail-storage.c: line 1913 (mailbox_get_open_status): > assertion failed: (box->opened) > Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. > Error: Raw backtrace: > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(+0xba731) > [0x7f553a7ff731] -> > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(+0xba7fa) > [0x7f553a7ff7fa] -> > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f553a771638] -> > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot-storage.so.0(mailbox_get_open_status+0x68) > [0x7f553aae4a78] -> > /usr/lib/dovecot/modules/lib21_fts_squat_plugin.so(+0x3684) > [0x7f553677a684] -> > /usr/lib/dovecot/modules/lib21_fts_squat_plugin.so(+0x3820) > [0x7f553677a820] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_lookup_multi+0x163) > [0x7f5539b016a3] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd728) > [0x7f5539b06728] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_lookup+0xeb) > [0x7f5539b06bbb] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xf8b8) > [0x7f5539b088b8] -> dovecot/imap(imap_search_start+0x6a) > [0x5654cb5a0d6a] -> dovecot/imap(cmd_sort+0x293) [0x5654cb593553] > -> dovecot/imap(command_exec+0x64) [0x5654cb599874] -> > dovecot/imap(+0x1bd22) [0x5654cb597d22] -> dovecot/imap(+0x1bdbc) > [0x5654cb597dbc] -> dovecot/imap(client_handle_input+0x1b5) > [0x5654cb5981c5] -> dovecot/imap(client_input+0xa4) > [0x5654cb5987e4] -> > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_call_io+0x69) > [0x7f553a8174a9] -> > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e) > [0x7f553a818d1e] -> > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) > [0x7f553a8175ac] -> > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f553a8177b8] -> > /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f553a7940a3] -> dovecot/imap(main+0x339) [0x5654cb58a539] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) > [0x7f553a375b97] -> dovecot/imap(_start+0x2a) [0x5654cb58a71a] > Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. > Fatal: master: service(imap): child 6436 killed with signal 6 > (core dumps disabled) > > This is my config: > > # 2.3.0.1 (ffd8a29): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.0.1 (d33dca20) > # OS: Linux 4.15.0-46-generic x86_64 Ubuntu 18.04.2 LTS ext4 > auth_mechanisms = plain login digest-md5 cram-md5 apop > auth_username_chars > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890&.-_@' > default_vsz_limit = 8096 M > disable_plaintext_auth = no > first_valid_uid = 30 > imap_client_workarounds = delay-newmail > imap_logout_format = rcvd=%i, sent=%o > mail_home = /var/qmail/mailnames/%Ld/%Ln > mail_location = maildir:/var/qmail/mailnames/%Ld/%Ln/Maildir > mail_log_prefix = "service=%s, user=%u, ip=[%r]. " > mail_max_userip_connections = 100 > mail_plugins = quota fts fts_squat virtual > managesieve_logout_format = rcvd=%i, sent=%o > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeri$ > namespace inbox { > ? inbox = yes > ? location > ? prefix = INBOX. > ? separator = . > } > namespace virtual { > ? hidden = no > ? inbox = no > ? list = yes > ? location > virtual:/var/qmail/mailnames/%Ld/%Ln/virtual/:INDEX=/var/qmail/mailnames/%Ld/%Ln/virtual/ > ? prefix > ? separator = . > } > passdb { > ? driver = plesk > } > plugin { > ? fts = squat > ? fts_squat = partial=4 full=10 > ? quota = maildir:User quota > ? quota_grace = 0 > ? sieve = ~/.dovecot.sieve > ? sieve_dir = ~/sieve > ? sieve_extensions = +notify +imapflags > } > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_logout_format = rcvd=%i, sent=%o, top=%t/%p, retr=%r/%b, > del=%d/%m, size=%s > protocols = imap pop3 sieve > service auth-worker { > ? group > ? user > } > service auth { > ? group > ? unix_listener auth-userdb { > ??? group = popuser > ??? mode = 0600 > ??? user = popuser > ? } > ? user > } > service imap-login { > ? process_limit = 2048 > ? service_count = 1 > } > service imap { > ? process_limit = 2048 > ? service_count = 1 > } > service pop3-login { > ? process_limit = 2048 > ? service_count = 1 > } > service pop3 { > ? process_limit = 2048 > ? service_count = 1 > } > ssl_cert = </etc/dovecot/private/dovecot.pem > ssl_cipher_list = HIGH:!aNULL:!MD5 > ssl_dh =? # hidden, use -P to show it > ssl_key =? # hidden, use -P to show it > ssl_prefer_server_ciphers = yes > userdb { > ? args = uid=popuser gid=popuser > ? driver = static > } > protocol imap { > ? mail_plugins = quota fts fts_squat virtual imap_quota > } > protocol pop3 { > ? pop3_uidl_format = UID%u-%v > } > protocol sieve { > ? mail_plugins = quota fts fts_squat virtual > } > protocol lda { > ? mail_plugins = quota fts fts_squat virtual sieve > } > > dovecot --version -> 2.3.0.1 (ffd8a29) > > I will be greatful for any tips hinting me on how to resolve this > behaviour. Thank you very much in advance > Benjamin > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20190320/ce267c4b/attachment.html>
Michael Slusarz
2019-Mar-20 19:31 UTC
Dovecot crashing when attempting to search in virtual folder with fts_squat activated
fts_squat was deprecated in 2.1. There's a high likelihood it is buggy in a variety of ways in any recent Dovecot release. michael> On March 20, 2019 at 1:21 PM Benjamin Godbersen via dovecot <dovecot at dovecot.org> wrote: > > Hi everyone, > > I have now updated to dovecot 2.3.4.1 - unfortunately the issue still persists. Can anyone help me figure out if this is due to a misconfiguration on my part or another error? > > Any help is greatly appreciated! > > Cheers > Benjamin > > Am 14.03.2019 um 23:28 schrieb benjamin at godbersen.info mailto:benjamin at godbersen.info : > > > > Hi everyone, > > > > I am running into a problem when trying to use fts_squat in a virtual folder. Without fts_squat plugin the search (from, subject...) works in all folders. With activated fts the search on the inbox folders works expectedly well but any attempt to search anything in any virtual folder leads to the following error. Similarly when attempting "doveadm fts lookup". I also noticed that no search index for the virtual folders gets build - is this expected behaviour? > > > > > > > Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. Panic: file mail-storage.c: line 1913 (mailbox_get_open_status): assertion failed: (box->opened) > > > Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. Error: Raw backtrace: /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(+0xba731) [0x7f553a7ff731] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(+0xba7fa) [0x7f553a7ff7fa] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(i_fatal+0) [0x7f553a771638] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot-storage.so.0(mailbox_get_open_status+0x68) [0x7f553aae4a78] -> /usr/lib/dovecot/modules/lib21_fts_squat_plugin.so(+0x3684) [0x7f553677a684] -> /usr/lib/dovecot/modules/lib21_fts_squat_plugin.so(+0x3820) [0x7f553677a820] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_lookup_multi+0x163) [0x7f5539b016a3] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd728) [0x7f5539b06728] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_search_lookup+0xeb) [0x7f5539b06bbb] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xf8b8) [0x7f5539b088b8] -> dovecot/imap(imap_search_start+0x6a) [0x5654cb5a0d6a] -> dovecot/imap(cmd_sort+0x293) [0x5654cb593553] -> dovecot/imap(command_exec+0x64) [0x5654cb599874] -> dovecot/imap(+0x1bd22) [0x5654cb597d22] -> dovecot/imap(+0x1bdbc) [0x5654cb597dbc] -> dovecot/imap(client_handle_input+0x1b5) [0x5654cb5981c5] -> dovecot/imap(client_input+0xa4) [0x5654cb5987e4] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f553a8174a9] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e) [0x7f553a818d1e] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f553a8175ac] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f553a8177b8] -> /usr/lib/x86_64-linux-gnu/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f553a7940a3] -> dovecot/imap(main+0x339) [0x5654cb58a539] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f553a375b97] -> dovecot/imap(_start+0x2a) [0x5654cb58a71a]> > > Mar 14 23:14:58 *** dovecot: service=imap, user=***, ip=[::1]. Fatal: master: service(imap): child 6436 killed with signal 6 (core dumps disabled) > > > > > > > > This is my config: > > > > > > > # 2.3.0.1 (ffd8a29): /etc/dovecot/dovecot.conf > > > # Pigeonhole version 0.5.0.1 (d33dca20) > > > # OS: Linux 4.15.0-46-generic x86_64 Ubuntu 18.04.2 LTS ext4 > > > auth_mechanisms = plain login digest-md5 cram-md5 apop > > > auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890&.-_@' > > > default_vsz_limit = 8096 M > > > disable_plaintext_auth = no > > > first_valid_uid = 30 > > > imap_client_workarounds = delay-newmail > > > imap_logout_format = rcvd=%i, sent=%o > > > mail_home = /var/qmail/mailnames/%Ld/%Ln > > > mail_location = maildir:/var/qmail/mailnames/%Ld/%Ln/Maildir > > > mail_log_prefix = "service=%s, user=%u, ip=[%r]. " > > > mail_max_userip_connections = 100 > > > mail_plugins = quota fts fts_squat virtual > > > managesieve_logout_format = rcvd=%i, sent=%o > > > managesieve_notify_capability = mailto > > > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeri$ > > > namespace inbox { > > > inbox = yes > > > location > > > prefix = INBOX. > > > separator = . > > > } > > > namespace virtual { > > > hidden = no > > > inbox = no > > > list = yes > > > location = virtual:/var/qmail/mailnames/%Ld/%Ln/virtual/:INDEX=/var/qmail/mailnames/%Ld/%Ln/virtual/ > > > prefix > > > separator = . > > > } > > > passdb { > > > driver = plesk > > > } > > > plugin { > > > fts = squat > > > fts_squat = partial=4 full=10 > > > quota = maildir:User quota > > > quota_grace = 0 > > > sieve = ~/.dovecot.sieve > > > sieve_dir = ~/sieve > > > sieve_extensions = +notify +imapflags > > > } > > > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > > > pop3_logout_format = rcvd=%i, sent=%o, top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > > > protocols = imap pop3 sieve > > > service auth-worker { > > > group > > > user > > > } > > > service auth { > > > group > > > unix_listener auth-userdb { > > > group = popuser > > > mode = 0600 > > > user = popuser > > > } > > > user > > > } > > > service imap-login { > > > process_limit = 2048 > > > service_count = 1 > > > } > > > service imap { > > > process_limit = 2048 > > > service_count = 1 > > > } > > > service pop3-login { > > > process_limit = 2048 > > > service_count = 1 > > > } > > > service pop3 { > > > process_limit = 2048 > > > service_count = 1 > > > } > > > ssl_cert = </etc/dovecot/private/dovecot.pem > > > ssl_cipher_list = HIGH:!aNULL:!MD5 > > > ssl_dh = # hidden, use -P to show it > > > ssl_key = # hidden, use -P to show it > > > ssl_prefer_server_ciphers = yes > > > userdb { > > > args = uid=popuser gid=popuser > > > driver = static > > > } > > > protocol imap { > > > mail_plugins = quota fts fts_squat virtual imap_quota > > > } > > > protocol pop3 { > > > pop3_uidl_format = UID%u-%v > > > } > > > protocol sieve { > > > mail_plugins = quota fts fts_squat virtual > > > } > > > protocol lda { > > > mail_plugins = quota fts fts_squat virtual sieve > > > } > > > > > > > > dovecot --version -> 2.3.0.1 (ffd8a29) > > > > I will be greatful for any tips hinting me on how to resolve this behaviour. Thank you very much in advance > > Benjamin > > > > > > > > > > > > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20190320/82c4dec1/attachment-0001.html>
Possibly Parallel Threads
- Dovecot crashing when attempting to search in virtual folder with fts_squat activated
- Dovecot crashing when attempting to search in virtual folder with fts_squat activated
- Dovecot crashing when attempting to search in virtual folder with fts_squat activated
- Dovecot crashing when attempting to search in virtual folder with fts_squat activated
- (message_parse_header_next): assertion failed:, +(IS_LWSP(line->value[0])) 1.1beta14