dovecot - Oct 2020 - v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed)

I've since rebuilt/reconfig'd all parts of my setup from scratch; some
good cleanup along the way.

Atm, my entire system for send/recv, store/retrieve, + rules & search is
working as I intend.  Ok, mostly ...

Except for this accented-character search mystery.  I've got a _lot_ of mail
with various languages in bodies, so _do_ need to get this sorted.
> On 10/18/20 2:58 PM, John Fawcett wrote:
...
> silly question
...

hardly!


creating 2 messages

	(1)
	Subject: tambien
	Body:    tambien

	(2)
	Subject: tambi?n
	Body:    tambi?n

and two more, two avoid known stop words

	(3)
	Subject: aausdfrhyetdwgyatrdf
	Body:    aausdfrhyetdwgyatrdf

	(4)
	Subject: aausdfrhy?tdwgyatrdf
	Body:    aausdfrhy?tdwgyatrdf


1st,

	doveadm fts rescan -u myuser at example.com
	doveadm index      -u myuser at example.com -q '*'

TBird/solr searches,

	Subject: tambien  ==> FOUND
	Subject: tambi?n  ==> FOUND
	Subject: aausdfrhyetdwgyatrdf  ==> FOUND
	Subject: aausdfrhy?tdwgyatrdf  ==> FOUND

	Body:    tambien  ==> FOUND
	Body:    tambi?n  ==> (empty)
	Body:    aausdfrhyetdwgyatrdf  ==> FOUND
	Body:    aausdfrhy?tdwgyatrdf  ==>  (empty)

suggests it's _not_ (just) an existing-stopword problem

notable/odd that subject searches are OK, but not body.



On 10/18/20 2:58 PM, Shawn Heisey wrote:
...
> If you are using something like the following schema:
>
https://raw.githubusercontent.com/dovecot/core/master/doc/solr-schema-7.7.0.xml
I am
> Solr does have a set of ICU filters, which I would recommend using rather
than the lowercase filter
I'll give that a try ; haven't used solr outside of the dovecot context
-- so need to find a doc/example on how, exactly, that's done correctly.
> I cannot say much about the panic you're getting when using the doveadm
command.  The stacktrace says it is happening in
> dovecot code, not Solr code.  And it looks like the panic had nothing to do
with FTS or Solr ... what I see points to
> mailbox storage code. 
again/still

	doveadm fts lookup -u myuser at example.com <any key> "<any
str>"

_all_ panic, as above,


	doveadm(myuser at example.com): Panic: file mail-storage.c: line 2112
(mailbox_get_open_status): assertion failed: (box->opened)
	doveadm(myuser at example.com): Error: Raw backtrace:
/usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x7f61bba4ecc6]
	-> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x7f61bba4ede2]
	-> /usr/lib64/dovecot/libdovecot.so.0(+0x10025b) [0x7f61bba5825b]
	-> /usr/lib64/dovecot/libdovecot.so.0(+0x100297) [0x7f61bba58297]
	-> /usr/lib64/dovecot/libdovecot.so.0(+0x59bc6) [0x7f61bb9b1bc6]
	-> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x4779e) [0x7f61bbb6579e]
	-> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x5849) [0x7f61bb5b7849]
	-> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_backend_lookup+0x51)
[0x7f61bb1d9491]
	-> /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so(+0x3280)
[0x7f61bb14b280]
	-> doveadm(+0x343cd) [0x55f5def873cd]
	-> doveadm(+0x34fe0) [0x55f5def87fe0]
	-> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x22d) [0x55f5def88e2d]
	-> doveadm(doveadm_cmd_run_ver2+0x4e8) [0x55f5def998d8]
	-> doveadm(doveadm_cmd_try_run_ver2+0x3e) [0x55f5def9992e]
	-> doveadm(main+0x1d4) [0x55f5def77cf4]
	-> /lib64/libc.so.6(__libc_start_main+0xf2) [0x7f61bb613042]
	-> doveadm(_start+0x2e) [0x55f5def781ce]
	Aborted

Hopefully dovecot devs might comment further.

I'll see what I find with using the ICU filters -- if perhaps anything
changes

> On 19/10/2020 02:49 PGNet Dev <pgnet.dev at gmail.com> wrote:
> 
>  
> I've since rebuilt/reconfig'd all parts of my setup from scratch;
some good cleanup along the way.
> 
> Atm, my entire system for send/recv, store/retrieve, + rules & search
is working as I intend.  Ok, mostly ...
> 
> Except for this accented-character search mystery.  I've got a _lot_ of
mail with various languages in bodies, so _do_ need to get this sorted.
> 
> > On 10/18/20 2:58 PM, John Fawcett wrote:
> ...
> > silly question
> ...
> 
> hardly!
> 
> 
> creating 2 messages
> 
> 	(1)
> 	Subject: tambien
> 	Body:    tambien
> 
> 	(2)
> 	Subject: tambi?n
> 	Body:    tambi?n
> 
> and two more, two avoid known stop words
> 
> 	(3)
> 	Subject: aausdfrhyetdwgyatrdf
> 	Body:    aausdfrhyetdwgyatrdf
> 
> 	(4)
> 	Subject: aausdfrhy?tdwgyatrdf
> 	Body:    aausdfrhy?tdwgyatrdf
> 
> 
> 1st,
> 
> 	doveadm fts rescan -u myuser at example.com
> 	doveadm index      -u myuser at example.com -q '*'
> 
> TBird/solr searches,
> 
> 	Subject: tambien  ==> FOUND
> 	Subject: tambi?n  ==> FOUND
> 	Subject: aausdfrhyetdwgyatrdf  ==> FOUND
> 	Subject: aausdfrhy?tdwgyatrdf  ==> FOUND
> 
> 	Body:    tambien  ==> FOUND
> 	Body:    tambi?n  ==> (empty)
> 	Body:    aausdfrhyetdwgyatrdf  ==> FOUND
> 	Body:    aausdfrhy?tdwgyatrdf  ==>  (empty)
> 
> suggests it's _not_ (just) an existing-stopword problem
> 
> notable/odd that subject searches are OK, but not body.
> 
> 
> 
> On 10/18/20 2:58 PM, Shawn Heisey wrote:
> ...
> > If you are using something like the following schema:
> >
https://raw.githubusercontent.com/dovecot/core/master/doc/solr-schema-7.7.0.xml
> 
> I am
> 
> > Solr does have a set of ICU filters, which I would recommend using
rather than the lowercase filter
> 
> I'll give that a try ; haven't used solr outside of the dovecot
context -- so need to find a doc/example on how, exactly, that's done
correctly.
> 
> > I cannot say much about the panic you're getting when using the
doveadm command.  The stacktrace says it is happening in
> > dovecot code, not Solr code.  And it looks like the panic had nothing
to do with FTS or Solr ... what I see points to
> > mailbox storage code. 
> 
> again/still
> 
> 	doveadm fts lookup -u myuser at example.com <any key> "<any
str>"
> 
> _all_ panic, as above,
> 
> 
> 	doveadm(myuser at example.com): Panic: file mail-storage.c: line 2112
(mailbox_get_open_status): assertion failed: (box->opened)
> 	doveadm(myuser at example.com): Error: Raw backtrace:
/usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x7f61bba4ecc6]
> 	-> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x22)
[0x7f61bba4ede2]
> 	-> /usr/lib64/dovecot/libdovecot.so.0(+0x10025b) [0x7f61bba5825b]
> 	-> /usr/lib64/dovecot/libdovecot.so.0(+0x100297) [0x7f61bba58297]
> 	-> /usr/lib64/dovecot/libdovecot.so.0(+0x59bc6) [0x7f61bb9b1bc6]
> 	-> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x4779e)
[0x7f61bbb6579e]
> 	-> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x5849)
[0x7f61bb5b7849]
> 	-> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_backend_lookup+0x51)
[0x7f61bb1d9491]
> 	-> /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so(+0x3280)
[0x7f61bb14b280]
> 	-> doveadm(+0x343cd) [0x55f5def873cd]
> 	-> doveadm(+0x34fe0) [0x55f5def87fe0]
> 	-> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x22d) [0x55f5def88e2d]
> 	-> doveadm(doveadm_cmd_run_ver2+0x4e8) [0x55f5def998d8]
> 	-> doveadm(doveadm_cmd_try_run_ver2+0x3e) [0x55f5def9992e]
> 	-> doveadm(main+0x1d4) [0x55f5def77cf4]
> 	-> /lib64/libc.so.6(__libc_start_main+0xf2) [0x7f61bb613042]
> 	-> doveadm(_start+0x2e) [0x55f5def781ce]
> 	Aborted
> 
> Hopefully dovecot devs might comment further.
> 
> I'll see what I find with using the ICU filters -- if perhaps anything
changes
Hi!

I can reproduce your problem with the `fts lookup` command. Luckily it's
equivalent to running `doveadm search`. I'll open a bug about this.

Dovecot FTS tokenization is not done, unless you have `use_libfts` in fts_solr
setting, in your case

fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts

Without this, everything is sent to to solr as-is, which is then expected to do
all the work.

Aki

> On 19/10/2020 08:28 Aki Tuomi <aki.tuomi at open-xchange.com> wrote:
> 
>  
> > On 19/10/2020 02:49 PGNet Dev <pgnet.dev at gmail.com> wrote:
> > 
> >  
> > I've since rebuilt/reconfig'd all parts of my setup from
scratch; some good cleanup along the way.
> > 
> > Atm, my entire system for send/recv, store/retrieve, + rules &
search is working as I intend.  Ok, mostly ...
> > 
> > Except for this accented-character search mystery.  I've got a
_lot_ of mail with various languages in bodies, so _do_ need to get this sorted.
> > 
> > > On 10/18/20 2:58 PM, John Fawcett wrote:
> > ...
> > > silly question
> > ...
> > 
> > hardly!
> > 
> > 
> > creating 2 messages
> > 
> > 	(1)
> > 	Subject: tambien
> > 	Body:    tambien
> > 
> > 	(2)
> > 	Subject: tambi?n
> > 	Body:    tambi?n
> > 
> > and two more, two avoid known stop words
> > 
> > 	(3)
> > 	Subject: aausdfrhyetdwgyatrdf
> > 	Body:    aausdfrhyetdwgyatrdf
> > 
> > 	(4)
> > 	Subject: aausdfrhy?tdwgyatrdf
> > 	Body:    aausdfrhy?tdwgyatrdf
> > 
> > 
> > 1st,
> > 
> > 	doveadm fts rescan -u myuser at example.com
> > 	doveadm index      -u myuser at example.com -q '*'
> > 
> > TBird/solr searches,
> > 
> > 	Subject: tambien  ==> FOUND
> > 	Subject: tambi?n  ==> FOUND
> > 	Subject: aausdfrhyetdwgyatrdf  ==> FOUND
> > 	Subject: aausdfrhy?tdwgyatrdf  ==> FOUND
> > 
> > 	Body:    tambien  ==> FOUND
> > 	Body:    tambi?n  ==> (empty)
> > 	Body:    aausdfrhyetdwgyatrdf  ==> FOUND
> > 	Body:    aausdfrhy?tdwgyatrdf  ==>  (empty)
> > 
> > suggests it's _not_ (just) an existing-stopword problem
> > 
> > notable/odd that subject searches are OK, but not body.
> > 
> > 
> > 
> > On 10/18/20 2:58 PM, Shawn Heisey wrote:
> > ...
> > > If you are using something like the following schema:
> > >
https://raw.githubusercontent.com/dovecot/core/master/doc/solr-schema-7.7.0.xml
> > 
> > I am
> > 
> > > Solr does have a set of ICU filters, which I would recommend
using rather than the lowercase filter
> > 
> > I'll give that a try ; haven't used solr outside of the
dovecot context -- so need to find a doc/example on how, exactly, that's
done correctly.
> > 
> > > I cannot say much about the panic you're getting when using
the doveadm command.  The stacktrace says it is happening in
> > > dovecot code, not Solr code.  And it looks like the panic had
nothing to do with FTS or Solr ... what I see points to
> > > mailbox storage code. 
> > 
> > again/still
> > 
> > 	doveadm fts lookup -u myuser at example.com <any key>
"<any str>"
> > 
> > _all_ panic, as above,
> > 
> > 
> > 	doveadm(myuser at example.com): Panic: file mail-storage.c: line 2112
(mailbox_get_open_status): assertion failed: (box->opened)
> > 	doveadm(myuser at example.com): Error: Raw backtrace:
/usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x7f61bba4ecc6]
> > 	-> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x22)
[0x7f61bba4ede2]
> > 	-> /usr/lib64/dovecot/libdovecot.so.0(+0x10025b) [0x7f61bba5825b]
> > 	-> /usr/lib64/dovecot/libdovecot.so.0(+0x100297) [0x7f61bba58297]
> > 	-> /usr/lib64/dovecot/libdovecot.so.0(+0x59bc6) [0x7f61bb9b1bc6]
> > 	-> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x4779e)
[0x7f61bbb6579e]
> > 	-> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x5849)
[0x7f61bb5b7849]
> > 	-> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_backend_lookup+0x51)
[0x7f61bb1d9491]
> > 	-> /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so(+0x3280)
[0x7f61bb14b280]
> > 	-> doveadm(+0x343cd) [0x55f5def873cd]
> > 	-> doveadm(+0x34fe0) [0x55f5def87fe0]
> > 	-> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x22d)
[0x55f5def88e2d]
> > 	-> doveadm(doveadm_cmd_run_ver2+0x4e8) [0x55f5def998d8]
> > 	-> doveadm(doveadm_cmd_try_run_ver2+0x3e) [0x55f5def9992e]
> > 	-> doveadm(main+0x1d4) [0x55f5def77cf4]
> > 	-> /lib64/libc.so.6(__libc_start_main+0xf2) [0x7f61bb613042]
> > 	-> doveadm(_start+0x2e) [0x55f5def781ce]
> > 	Aborted
> > 
> > Hopefully dovecot devs might comment further.
> > 
> > I'll see what I find with using the ICU filters -- if perhaps
anything changes
> 
> Hi!
> 
> I can reproduce your problem with the `fts lookup` command. Luckily
it's equivalent to running `doveadm search`. I'll open a bug about this.
> 
> Dovecot FTS tokenization is not done, unless you have `use_libfts` in
fts_solr setting, in your case
> 
> fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
> 
> Without this, everything is sent to to solr as-is, which is then expected
to do all the work.
> 
> Aki
Also, I noticed you had soft_commit=yes on separate line, that also needs to be
on *same* line as fts_solr, as these settings are passed along to solr plugin
for parsing.

Aki

On 19/10/2020 01:49, PGNet Dev wrote:
>
> I've since rebuilt/reconfig'd all parts of my setup from scratch;
some
> good cleanup along the way.
>
> Atm, my entire system for send/recv, store/retrieve, + rules & search
> is working as I intend.? Ok, mostly ...
>
> Except for this accented-character search mystery.? I've got a _lot_
> of mail with various languages in bodies, so _do_ need to get this
> sorted.
>
>> On 10/18/20 2:58 PM, John Fawcett wrote:
> ...
>> silly question
> ...
>
> hardly!
>
>
> creating 2 messages
>
> ????(1)
> ????Subject: tambien
> ????Body:??? tambien
>
> ????(2)
> ????Subject: tambi?n
> ????Body:??? tambi?n
>
> and two more, two avoid known stop words
>
> ????(3)
> ????Subject: aausdfrhyetdwgyatrdf
> ????Body:??? aausdfrhyetdwgyatrdf
>
> ????(4)
> ????Subject: aausdfrhy?tdwgyatrdf
> ????Body:??? aausdfrhy?tdwgyatrdf
>
>
> 1st,
>
> ????doveadm fts rescan -u myuser at example.com
> ????doveadm index????? -u myuser at example.com -q '*'
>
> TBird/solr searches,
>
> ????Subject: tambien? ==> FOUND
> ????Subject: tambi?n? ==> FOUND
> ????Subject: aausdfrhyetdwgyatrdf? ==> FOUND
> ????Subject: aausdfrhy?tdwgyatrdf? ==> FOUND
>
> ????Body:??? tambien? ==> FOUND
> ????Body:??? tambi?n? ==> (empty)
> ????Body:??? aausdfrhyetdwgyatrdf? ==> FOUND
> ????Body:??? aausdfrhy?tdwgyatrdf? ==>? (empty)
>
> suggests it's _not_ (just) an existing-stopword problem
>
> notable/odd that subject searches are OK, but not body.
>
The explanation for the different behaviour between headers and bodies
is the following setting:

fts_enforced = body

I believe your header searches are not being sent to solr. See the
following for different values.

https://doc.dovecot.org/settings/plugin/fts-plugin/#fts-plugin

If you're the only one doing searches at the moment you should be able
to confirm that by tailing the access_log file on solr server and see
that no access is being made for header searches only for body searches.

If you want to use solr for all searches then fts_enforced should be set
to yes.

Another point as mentioned by Aki is that you did not have soft_commit
correctly configured. That has the effect of not opening a new solr
searcher after each update of the index. So your tests may be invalid
and in any case leave room for doubt as to whether the index updates
were visible or not at the moment of your test.

I would recommend you to redo the tests after correcting the
configuration. To be doubly sure you can include accented and unique non
accented text in the same email and search for both. If the non accented
text is found you know you've searching against the updated index and
the fact that accented text is not found is not simply because the index
updates are not visible.

John

On 10/18/20 10:28 PM, Aki Tuomi wrote:
> I can reproduce your problem with the `fts lookup` command. Luckily
it's equivalent to running `doveadm search`. I'll open a bug about this.
thx!
> Dovecot FTS tokenization is not done, unless you have `use_libfts` in
fts_solr setting, in your case
> fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
> 
> Without this, everything is sent to to solr as-is, which is then expected
to do all the work.
missed that one. fixed.
> Also, I noticed you had soft_commit=yes on separate line, that also needs
to be on *same* line as fts_solr, as these settings are passed along to solr
plugin for parsing.

yup.  found/fixed that already after  last post ...

now, I've

   fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
soft_commit=yes batch_size=250

On 10/18/20 10:28 PM, Aki Tuomi wrote:
> Dovecot FTS tokenization is not done, unless you have `use_libfts` in
fts_solr setting, in your case
> 
> fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
changing

-	fts_solr = url=https://solr.example.com:8984/solr/dovecot/ soft_commit=yes
batch_size=250
+	fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
soft_commit=yes batch_size=250

exec of

	doveadm fts rescan -u testuser at example.com

returns

	doveadm(testuser at example.com): Error: fts-solr: fts_filter_normalizer_icu:
libicu support not built in
	doveadm(testuser at example.com): Error: fts: Failed to initialize backend
'solr': Invalid fts_solr setting
	doveadm(testuser at example.com): Error: fts not enabled for user's
namespace INBOX


in my current package src

	https://src.fedoraproject.org/rpms/dovecot/blob/master/f/dovecot.spec

config,

     --with-solr                  \

exists, and

	--with-icu              Build with libicu support (for FTS normalization)
							(auto)

_should_ be picked up automatically

checking,

	ldd `locate lib64 | grep fts.*so$` | grep icu
		(empty)

packaging issue, then?

or additional config @ dovecot needed?

On 10/19/20 1:18 AM, John Fawcett wrote:
>> notable/odd that subject searches are OK, but not body.
>>
> The explanation for the different behaviour between headers and bodies
> is the following setting:
> 
> fts_enforced = body
> 
> I believe your header searches are not being sent to solr. See the
> following for different values.
ah.  I'd (mis)understood that the "FTS index updated" for '=
body' searched, but didn't update -- and changed failure mode.
> https://doc.dovecot.org/settings/plugin/fts-plugin/#fts-plugin
RE-reading helps!
> If you want to use solr for all searches then fts_enforced should be set
> to yes.
set now to ' = yes'



thx
> I would recommend you to redo the tests after correcting the
> configuration. To be doubly sure you can include accented and unique non
> accented text in the same email and search for both. If the non accented
> text is found you know you've searching against the updated index and
> the fact that accented text is not found is not simply because the index
> updates are not visible.
good point.  will do, as soon as i find/fix my latest libicu issue ...

On 10/19/20 1:18 AM, John Fawcett wrote:
> I would recommend you to redo the tests after correcting the
> configuration. To be doubly sure you can include accented and unique non
> accented text in the same email and search for both. If the non accented
> text is found you know you've searching against the updated index and
> the fact that accented text is not found is not simply because the index
> updates are not visible.
temp changing,

	fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
soft_commit=yes batch_size=250
-	fts_enforced = body
+	fts_enforced = yes
-	fts_filters = normalizer-icu snowball stopwords
+	fts_filters = lowercase snowball stopwords

now,

	doveadm fts rescan -u testuser at example.com
	doveadm index      -u testuser at example.com -q '*'

, no errors.

on receipt of a test email with

	subject: aausdfrhy?tdwgyatrdf aausdfrhyetdwgyatrdf
	body:    aausdfrhy?tdwgyatrdf aausdfrhyetdwgyatrdf

i see a solr auto-scan,

	2020-10-19 14:41:50.628 INFO 
(searcherExecutor-15-thread-1-processing-x:dovecot) [   x:dovecot]
o.a.s.c.SolrCore [dovecot]  Registered new searcher autowarm time: 1 ms
	2020-10-19 14:41:50.634 INFO  (qtp1533985074-18) [   x:dovecot]
o.a.s.u.p.LogUpdateProcessorFactory [dovecot]  webapp=/solr path=/update
params={}{commit=} 0 155
	2020-10-19 14:41:51.571 INFO  (qtp1533985074-24) [   x:dovecot]
o.a.s.u.p.LogUpdateProcessorFactory [dovecot]  webapp=/solr path=/update
params={}{add=[135785/c92f64f79f0d1ed01e6d5b314f04886c/testuser at example.com
(1680991596381732864)]} 0 9

	==> /var/log/solr/solr_gc.log <=
[2020-10-19T07:41:51.612-0700][146823.946s] GC(343) Pause Young (Normal) (G1
Evacuation Pause)
	[2020-10-19T07:41:51.613-0700][146823.947s] GC(343) Using 2 workers of 2 for
evacuation
	[2020-10-19T07:41:51.635-0700][146823.969s] GC(343)   Pre Evacuate Collection
Set: 0.3ms
	[2020-10-19T07:41:51.636-0700][146823.970s] GC(343)   Merge Heap Roots: 0.2ms
	[2020-10-19T07:41:51.636-0700][146823.970s] GC(343)   Evacuate Collection Set:
18.9ms
	[2020-10-19T07:41:51.637-0700][146823.971s] GC(343)   Post Evacuate Collection
Set: 2.0ms
	[2020-10-19T07:41:51.637-0700][146823.971s] GC(343)   Other: 1.5ms
	[2020-10-19T07:41:51.637-0700][146823.971s] GC(343) Eden regions:
238->0(244)
	[2020-10-19T07:41:51.638-0700][146823.972s] GC(343) Survivor regions:
4->2(31)
	[2020-10-19T07:41:51.638-0700][146823.972s] GC(343) Old regions: 189->189
	[2020-10-19T07:41:51.638-0700][146823.973s] GC(343) Archive regions: 2->2
	[2020-10-19T07:41:51.639-0700][146823.973s] GC(343) Humongous regions: 10->9
	[2020-10-19T07:41:51.639-0700][146823.973s] GC(343) Metaspace:
61564K(78028K)->61564K(78028K) NonClass: 55348K(65024K)->55348K(65024K)
Class: 6216K(13004K)->6216K(13004K)
	[2020-10-19T07:41:51.640-0700][146823.974s] GC(343) Pause Young (Normal) (G1
Evacuation Pause) 441M->200M(512M) 27.372ms
	[2020-10-19T07:41:51.640-0700][146823.974s] GC(343) User=0.01s Sys=0.01s
Real=0.03s

	==> /var/log/solr/solr.log <=	2020-10-19 14:41:51.702 INFO 
(searcherExecutor-15-thread-1-processing-x:dovecot) [   x:dovecot]
o.a.s.c.SolrCore [dovecot]  Registered new searcher autowarm time: 0 ms
	2020-10-19 14:41:51.705 INFO  (qtp1533985074-18) [   x:dovecot]
o.a.s.u.p.LogUpdateProcessorFactory [dovecot]  webapp=/solr path=/update
params={}{commit=} 0 127


search in TBird

	subject: aausdfrhyetdwgyatrdf  => FOUND
	body:    aausdfrhy?tdwgyatrdf  => FOUND

	subject: aausdfrhyetdwgyatrdf  => FOUND
	body:    aausdfrhy?tdwgyatrdf  => (emtpy)

on header search, I'm _not_ seeing any additional activity in solr.log

so, either i'm looking in the wrong place, haven't turned on appropriate
logging, or i'm still not searching via solr ...

separately,

	doveadm fts lookup ...

still panics; Aki's bug will hopefully deal with that

On 10/18/20 10:28 PM, Aki Tuomi wrote:
>> 	doveadm(myuser at example.com): Panic: file mail-storage.c: line 2112
(mailbox_get_open_status): assertion failed: (box->opened)
...
> I can reproduce your problem with the `fts lookup` command. Luckily
it's equivalent to running `doveadm search`. I'll open a bug about this.
Can you provide any status on the bug/fix?

Thanks.