thr3ads.net - dovecot - Error: fts_solr: received invalid uid (John Fawcett) [Jan 2022]

If this information is useful, please help other people find it:
Share via:

oliver.krone at snoog.ch

2022-Jan-03 00:18 UTC

Error: fts_solr: received invalid uid (John Fawcett)

Hi John,

You are right this was not a dovecot solr query. It is really strange 
everything seems to work with Dovecot 2.3.4. E.g. search and  rebuilding 
index from scratch using doveadm. As soon as I use version  2.3.13 
things are getting wild: dovecot crashed when I re-scan the index using 
doveadm and search does not work anymore, after re-scan:

Panic: file http-client-request.c: line 1240 
(http_client_request_send_more): assertion failed: (req->payload_input 
!= NULL)
doveadm(krone at invectrix.ch): Error: Raw backtrace: 
/usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7fc54cace4e2] 
-> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7fc54cace5fe] 
-> /usr/lib/dovecot/libdovecot.so.0(+0xfc49b) [0x7fc54cada49b] -> 
/usr/lib/dovecot/libdovecot.so.0(+0xfc4d1) [0x7fc54cada4d1] -> 
/usr/lib/dovecot/libdovecot.so.0(+0x53aee) [0x7fc54ca31aee] -> 
/usr/lib/dovecot/libdovecot.so.0(+0x4aec2) [0x7fc54ca28ec2] -> 
/usr/lib/dovecot/libdovecot.so.0(http_client_connection_output+0xee) 
[0x7fc54ca7ebde] -> /usr/lib/dovecot/libdovecot.so.0(+0x122171) 
[0x7fc54cb00171] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7fc54caeff59] 
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) 
[0x7fc54caf1592] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) 
[0x7fc54caf0000] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) 
[0x7fc54caf01c0] -> /usr/lib/dovecot/libdovecot.so.0(+0x9c4cd) 
[0x7fc54ca7a4cd] -> 
/usr/lib/dovecot/libdovecot.so.0(http_client_request_finish_payload+0x2c) 
[0x7fc54ca7a6dc] -> 
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xf243) [0x7fc54c0a6243] 
-> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_parser_more+0x25) 
[0x7fc54c0a5345] -> 
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xbfcf) [0x7fc54c0a2fcf] 
-> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0xa27) 
[0x7fc54c0a3a87] -> 
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11f0b) [0x7fc54c0a8f0b] 
-> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) 
[0x7fc54cbf14ae] -> doveadm(+0x3761f) [0x55d18bdbf61f] -> 
doveadm(+0x31bad) [0x55d18bdb9bad] -> doveadm(+0x32860) [0x55d18bdba860] 
-> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x22d) [0x55d18bdbb4ad] 
-> doveadm(doveadm_cmd_run_ver2+0x4c8) [0x55d18bdcbb88] -> 
doveadm(doveadm_cmd_try_run_ver2+0x3a) [0x55d18bdcbbda] -> 
doveadm(main+0x1d0) [0x55d18bdaa450] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fc54c69fd0a] 
-> doveadm(_start+0x2a) [0x55d18bdaa92a]

and solr:

2022-01-03 00:13:20.829 ERROR (qtp1991278377-21) [   x:dovecot] 
o.a.s.s.HttpSolrCall null:[com.ctc.wstx.exc.WstxLazyException] 
com.ctc.wstx.exc.WstxIOException: Early EOF

....

Caused by: org.eclipse.jetty.io.EofException: Early EOF
	at org.eclipse.jetty.server.HttpInput$3.getError(HttpInput.java:1143)
	at org.eclipse.jetty.server.HttpInput$3.noContent(HttpInput.java:1131)
	at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:318)

Seems like dovecot is sending an empty query?

Oliver
> 
> Hi Oliver
> 
> unless I'm missing something that doesn't look like the type of
query
> dovecot launches - dovecot sends various parameters including the
> request for xml results. The thing that looks odd to me is that the
> results fields should be single values not arrays enclosed in [], ie I
> would have expected
> 
> "uid":21,
> 
> instead of
> 
> "uid":[21],
> 
> I'd be interested to see the equivalent xml output produced by running 
> a
> query that dovecot sends.
> 
> John
>

oliver.krone at snoog.ch

2022-Jan-03 00:23 UTC

head link

Error: fts_solr: received invalid uid

Hi John,

You are right this was not a dovecot solr query. It is really strange 
everything seems to work with Dovecot 2.3.4. E.g. search and  rebuilding 
index from scratch using doveadm. As soon as I use version  2.3.13 
things are getting wild: dovecot crashed when I re-scan the index using 
doveadm and search does not work anymore, after re-scan:

Panic: file http-client-request.c: line 1240 
(http_client_request_send_more): assertion failed: (req->payload_input 
!= NULL)
doveadm(krone at invectrix.ch): Error: Raw backtrace: 
/usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7fc54cace4e2] 
-> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7fc54cace5fe] 
-> /usr/lib/dovecot/libdovecot.so.0(+0xfc49b) [0x7fc54cada49b] -> 
/usr/lib/dovecot/libdovecot.so.0(+0xfc4d1) [0x7fc54cada4d1] -> 
/usr/lib/dovecot/libdovecot.so.0(+0x53aee) [0x7fc54ca31aee] -> 
/usr/lib/dovecot/libdovecot.so.0(+0x4aec2) [0x7fc54ca28ec2] -> 
/usr/lib/dovecot/libdovecot.so.0(http_client_connection_output+0xee) 
[0x7fc54ca7ebde] -> /usr/lib/dovecot/libdovecot.so.0(+0x122171) 
[0x7fc54cb00171] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7fc54caeff59] 
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) 
[0x7fc54caf1592] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) 
[0x7fc54caf0000] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) 
[0x7fc54caf01c0] -> /usr/lib/dovecot/libdovecot.so.0(+0x9c4cd) 
[0x7fc54ca7a4cd] -> 
/usr/lib/dovecot/libdovecot.so.0(http_client_request_finish_payload+0x2c) 
[0x7fc54ca7a6dc] -> 
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xf243) [0x7fc54c0a6243] 
-> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_parser_more+0x25) 
[0x7fc54c0a5345] -> 
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xbfcf) [0x7fc54c0a2fcf] 
-> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0xa27) 
[0x7fc54c0a3a87] -> 
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11f0b) [0x7fc54c0a8f0b] 
-> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) 
[0x7fc54cbf14ae] -> doveadm(+0x3761f) [0x55d18bdbf61f] -> 
doveadm(+0x31bad) [0x55d18bdb9bad] -> doveadm(+0x32860) [0x55d18bdba860] 
-> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x22d) [0x55d18bdbb4ad] 
-> doveadm(doveadm_cmd_run_ver2+0x4c8) [0x55d18bdcbb88] -> 
doveadm(doveadm_cmd_try_run_ver2+0x3a) [0x55d18bdcbbda] -> 
doveadm(main+0x1d0) [0x55d18bdaa450] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fc54c69fd0a] 
-> doveadm(_start+0x2a) [0x55d18bdaa92a]

and solr:

2022-01-03 00:13:20.829 ERROR (qtp1991278377-21) [   x:dovecot] 
o.a.s.s.HttpSolrCall null:[com.ctc.wstx.exc.WstxLazyException] 
com.ctc.wstx.exc.WstxIOException: Early EOF

....

Caused by: org.eclipse.jetty.io.EofException: Early EOF
     at 
org.eclipse.jetty.server.HttpInput$3.getError(HttpInput.java:1143)
     at 
org.eclipse.jetty.server.HttpInput$3.noContent(HttpInput.java:1131)
     at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:318)

Seems like dovecot is sending an empty query?

Oliver
> Hi Oliver
> 
> unless I'm missing something that doesn't look like the type of
query
> dovecot launches - dovecot sends various parameters including the
> request for xml results. The thing that looks odd to me is that the
> results fields should be single values not arrays enclosed in [], ie I
> would have expected
> 
> "uid":21,
> 
> instead of
> 
> "uid":[21],
> 
> I'd be interested to see the equivalent xml output produced by running 
> a
> query that dovecot sends.
> 
> John-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20220103/ef33c2e4/attachment.htm>

John Fawcett

2022-Jan-03 00:35 UTC

head link

Error: fts_solr: received invalid uid (John Fawcett)

On 03/01/2022 01:18, oliver.krone at snoog.ch wrote:> Hi John,
>
> You are right this was not a dovecot solr query. It is really strange 
> everything seems to work with Dovecot 2.3.4. E.g. search and? 
> rebuilding index from scratch using doveadm. As soon as I use version? 
> 2.3.13 things are getting wild: dovecot crashed when I re-scan the 
> index using doveadm and search does not work anymore, after re-scan:
>
> Panic: file http-client-request.c: line 1240 
> (http_client_request_send_more): assertion failed: (req->payload_input 
> != NULL)
> doveadm(krone at invectrix.ch): Error: Raw backtrace: 
> /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) 
> [0x7fc54cace4e2] -> 
> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7fc54cace5fe] 
> -> /usr/lib/dovecot/libdovecot.so.0(+0xfc49b) [0x7fc54cada49b] -> 
> /usr/lib/dovecot/libdovecot.so.0(+0xfc4d1) [0x7fc54cada4d1] -> 
> /usr/lib/dovecot/libdovecot.so.0(+0x53aee) [0x7fc54ca31aee] -> 
> /usr/lib/dovecot/libdovecot.so.0(+0x4aec2) [0x7fc54ca28ec2] -> 
> /usr/lib/dovecot/libdovecot.so.0(http_client_connection_output+0xee) 
> [0x7fc54ca7ebde] -> /usr/lib/dovecot/libdovecot.so.0(+0x122171) 
> [0x7fc54cb00171] -> 
> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) 
> [0x7fc54caeff59] -> 
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) 
> [0x7fc54caf1592] -> 
> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50) 
> [0x7fc54caf0000] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) 
> [0x7fc54caf01c0] -> /usr/lib/dovecot/libdovecot.so.0(+0x9c4cd) 
> [0x7fc54ca7a4cd] -> 
> /usr/lib/dovecot/libdovecot.so.0(http_client_request_finish_payload+0x2c) 
> [0x7fc54ca7a6dc] -> 
> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xf243) [0x7fc54c0a6243] 
> -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_parser_more+0x25) 
> [0x7fc54c0a5345] -> 
> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xbfcf) [0x7fc54c0a2fcf] 
> -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0xa27) 
> [0x7fc54c0a3a87] -> 
> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11f0b) 
> [0x7fc54c0a8f0b] -> 
> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) 
> [0x7fc54cbf14ae] -> doveadm(+0x3761f) [0x55d18bdbf61f] -> 
> doveadm(+0x31bad) [0x55d18bdb9bad] -> doveadm(+0x32860) 
> [0x55d18bdba860] -> 
> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x22d) [0x55d18bdbb4ad] 
> -> doveadm(doveadm_cmd_run_ver2+0x4c8) [0x55d18bdcbb88] -> 
> doveadm(doveadm_cmd_try_run_ver2+0x3a) [0x55d18bdcbbda] -> 
> doveadm(main+0x1d0) [0x55d18bdaa450] -> 
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) 
> [0x7fc54c69fd0a] -> doveadm(_start+0x2a) [0x55d18bdaa92a]
>
> and solr:
>
> 2022-01-03 00:13:20.829 ERROR (qtp1991278377-21) [?? x:dovecot] 
> o.a.s.s.HttpSolrCall null:[com.ctc.wstx.exc.WstxLazyException] 
> com.ctc.wstx.exc.WstxIOException: Early EOF
>
> ....
>
> Caused by: org.eclipse.jetty.io.EofException: Early EOF
> ????at org.eclipse.jetty.server.HttpInput$3.getError(HttpInput.java:1143)
> ????at 
> org.eclipse.jetty.server.HttpInput$3.noContent(HttpInput.java:1131)
> ????at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:318)
>
> Seems like dovecot is sending an empty query?
>
> Oliver 
Hi Oliver

I remember there being some patches to dovecot for using solr that 
addressed errors like this and also an additional patch for using solr 
and tika. You could try looking back in the archives and applying them. 
But a quicker way if it's open is to use a later version. I am sure that 
by 2.3.16 all the fixes were already incorporated by the development team.

However, I'm not sure if this is related to your original issue. The 
response from solr looks strange to me. But you'll need to get a working 
version before progressing that issue (and maybe in doing so you'll find 
it resolved by itself).

John

dovecot - Jan 2022 - Error: fts_solr: received invalid uid (John Fawcett)

Error: fts_solr: received invalid uid (John Fawcett)

Error: fts_solr: received invalid uid

Error: fts_solr: received invalid uid (John Fawcett)