PGNet Dev
2020-Oct-31 03:57 UTC
v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed)
On 10/18/20 10:28 PM, Aki Tuomi wrote:>> doveadm(myuser at example.com): Panic: file mail-storage.c: line 2112 (mailbox_get_open_status): assertion failed: (box->opened)...> I can reproduce your problem with the `fts lookup` command. Luckily it's equivalent to running `doveadm search`. I'll open a bug about this.Can you provide any status on the bug/fix? Thanks.
John Fawcett
2020-Oct-31 16:55 UTC
v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]
On 31/10/2020 04:57, PGNet Dev wrote:> On 10/18/20 10:28 PM, Aki Tuomi wrote: >>> ????doveadm(myuser at example.com): Panic: file mail-storage.c: line >>> 2112 (mailbox_get_open_status): assertion failed: (box->opened) > ... > >> I can reproduce your problem with the `fts lookup` command. Luckily >> it's equivalent to running `doveadm search`. I'll open a bug about this. > > Can you provide any status on the bug/fix? > > Thanks.I can contribute a patch that solves the segfault. Unfortunately though fts search may be more broken than this. It does not give me search results, even though I see it querying solr and getting hits. diff -ur dovecot-2.3.11.3-orig/src/plugins/fts/doveadm-fts.c dovecot-2.3.11.3-patch/src/plugins/fts/doveadm-fts.c --- dovecot-2.3.11.3-orig/src/plugins/fts/doveadm-fts.c??? 2020-08-12 14:20:41.000000000 +0200 +++ dovecot-2.3.11.3-patch/src/plugins/fts/doveadm-fts.c??? 2020-10-31 17:52:09.019388695 +0100 @@ -47,6 +47,14 @@ ???? i_array_init(&result.scores, 16); ? ???? box = mailbox_alloc(info->ns->list, info->vname, 0); +??? mailbox_set_reason(box,"fts search"); +??? if (mailbox_open(box) < 0) { +??? ??? i_error("Couldn't open mailbox: %s", +??? ??? ??? mailbox_get_last_internal_error(box, NULL)); +??? ??? doveadm_mail_failed_error(ctx, MAIL_ERROR_TEMP); +??? ??? return -1; +??? } + ???? if (fts_backend_lookup(backend, box, ctx->search_args->args, ???? ??? ??? ??? ????? FTS_LOOKUP_FLAG_AND_ARGS, &result) < 0) { ???? ??? i_error("fts lookup failed"); On a more minor issue, with this patch if you search for a non existent mailbox, it does give a segfault for a different assert, in mail-user.c (*user)->refcount == 1. doveadm(john at voipsupport.it): Error: Couldn't open mailbox: Mailbox doesn't exist: inboxx doveadm(john at voipsupport.it): Panic: file mail-user.c: line 229 (mail_user_deinit): assertion failed: ((*user)->refcount == 1) doveadm(john at voipsupport.it): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f35c44c3ee2] -> /usr/local/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f35c44c3fee] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xec53e) [0x7f35c44ce53e] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xec581) [0x7f35c44ce581] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f35c44254ea] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x56d87) [0x7f35c47e4d87] -> doveadm(+0x2cb28) [0x55c0eaa57b28] -> doveadm(+0x2d77c) [0x55c0eaa5877c] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) [0x55c0eaa5960d] -> doveadm(doveadm_cmd_run_ver2+0x472) [0x55c0eaa6a372] -> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55c0eaa6a497] -> doveadm(main+0x1d4) [0x55c0eaa47c54] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f35c3d34555] -> doveadm(+0x1d0ef) [0x55c0eaa480ef] Aborted John
PGNet Dev
2020-Oct-31 21:01 UTC
v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]
On 10/31/20 9:55 AM, John Fawcett wrote:> I can contribute a patch that solves the segfault. Unfortunately though > fts search may be more broken than this. It does not give me search > results, even though I see it querying solr and getting hits.Thx -- hopefully it moves this in the right direction. Also on the 'good news' page, it appears there's been some progress on Thunderbird's use of backend/server search, TBird "search on server" doesn't -- NO comm with backend IMAP/SOLR; appears to be local-only search https://bugzilla.mozilla.org/show_bug.cgi?id=1673928 "A fix for this is upcoming." Remains to be seen if the doveadm search issues, and implications on backend problems, have any effect on the Thunderbird searches.
John Fawcett
2020-Nov-02 12:18 UTC
v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]
On 31/10/2020 17:55, John Fawcett wrote:> On 31/10/2020 04:57, PGNet Dev wrote: >> On 10/18/20 10:28 PM, Aki Tuomi wrote: >>>> ????doveadm(myuser at example.com): Panic: file mail-storage.c: line >>>> 2112 (mailbox_get_open_status): assertion failed: (box->opened) >> ... >> >>> I can reproduce your problem with the `fts lookup` command. Luckily >>> it's equivalent to running `doveadm search`. I'll open a bug about this. >> Can you provide any status on the bug/fix? >> >> Thanks. > I can contribute a patch that solves the segfault. Unfortunately though > fts search may be more broken than this. It does not give me search > results, even though I see it querying solr and getting hits. > > diff -ur dovecot-2.3.11.3-orig/src/plugins/fts/doveadm-fts.c > dovecot-2.3.11.3-patch/src/plugins/fts/doveadm-fts.c > --- dovecot-2.3.11.3-orig/src/plugins/fts/doveadm-fts.c??? 2020-08-12 > 14:20:41.000000000 +0200 > +++ dovecot-2.3.11.3-patch/src/plugins/fts/doveadm-fts.c??? 2020-10-31 > 17:52:09.019388695 +0100 > @@ -47,6 +47,14 @@ > ???? i_array_init(&result.scores, 16); > ? > ???? box = mailbox_alloc(info->ns->list, info->vname, 0); > +??? mailbox_set_reason(box,"fts search"); > +??? if (mailbox_open(box) < 0) { > +??? ??? i_error("Couldn't open mailbox: %s", > +??? ??? ??? mailbox_get_last_internal_error(box, NULL)); > +??? ??? doveadm_mail_failed_error(ctx, MAIL_ERROR_TEMP); > +??? ??? return -1; > +??? } > + > ???? if (fts_backend_lookup(backend, box, ctx->search_args->args, > ???? ??? ??? ??? ????? FTS_LOOKUP_FLAG_AND_ARGS, &result) < 0) { > ???? ??? i_error("fts lookup failed"); > > On a more minor issue, with this patch if you search for a non existent > mailbox, it does give a segfault for a different assert, in mail-user.c > (*user)->refcount == 1. > > doveadm(john at voipsupport.it): Error: Couldn't open mailbox: Mailbox > doesn't exist: inboxx > doveadm(john at voipsupport.it): Panic: file mail-user.c: line 229 > (mail_user_deinit): assertion failed: ((*user)->refcount == 1) > doveadm(john at voipsupport.it): Error: Raw backtrace: > /usr/local/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) > [0x7f35c44c3ee2] -> > /usr/local/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) > [0x7f35c44c3fee] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xec53e) > [0x7f35c44ce53e] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xec581) > [0x7f35c44ce581] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f35c44254ea] -> > /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x56d87) > [0x7f35c47e4d87] -> doveadm(+0x2cb28) [0x55c0eaa57b28] -> > doveadm(+0x2d77c) [0x55c0eaa5877c] -> > doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) [0x55c0eaa5960d] -> > doveadm(doveadm_cmd_run_ver2+0x472) [0x55c0eaa6a372] -> > doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55c0eaa6a497] -> > doveadm(main+0x1d4) [0x55c0eaa47c54] -> > /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f35c3d34555] -> > doveadm(+0x1d0ef) [0x55c0eaa480ef] > Aborted > > John >After further testing, I believe the fix for doveadm fts lookup command is ok (at least on a virtual users setup - I don't have system users to test this with). What I stated above is wrong: it does produce search results - I missed them in the debugging because the format is a bit different to doveadm search, but they are there. So unless someone with system users reports something not working, I'd recommend the above patch to resolve the mailbox not open assert/segfault issue. John
Possibly Parallel Threads
- v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed)
- v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]
- v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]
- v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]
- v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed)