dovecot - Oct 2020 - v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed)

On 10/18/20 10:28 PM, Aki Tuomi wrote:
> Dovecot FTS tokenization is not done, unless you have `use_libfts` in
fts_solr setting, in your case
> 
> fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
changing

-	fts_solr = url=https://solr.example.com:8984/solr/dovecot/ soft_commit=yes
batch_size=250
+	fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
soft_commit=yes batch_size=250

exec of

	doveadm fts rescan -u testuser at example.com

returns

	doveadm(testuser at example.com): Error: fts-solr: fts_filter_normalizer_icu:
libicu support not built in
	doveadm(testuser at example.com): Error: fts: Failed to initialize backend
'solr': Invalid fts_solr setting
	doveadm(testuser at example.com): Error: fts not enabled for user's
namespace INBOX


in my current package src

	https://src.fedoraproject.org/rpms/dovecot/blob/master/f/dovecot.spec

config,

     --with-solr                  \

exists, and

	--with-icu              Build with libicu support (for FTS normalization)
							(auto)

_should_ be picked up automatically

checking,

	ldd `locate lib64 | grep fts.*so$` | grep icu
		(empty)

packaging issue, then?

or additional config @ dovecot needed?

On 19/10/2020 16:20, PGNet Dev wrote:
> On 10/18/20 10:28 PM, Aki Tuomi wrote:
>> Dovecot FTS tokenization is not done, unless you have `use_libfts` in
>> fts_solr setting, in your case
>>
>> fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts
>
> changing
>
> -??? fts_solr = url=https://solr.example.com:8984/solr/dovecot/
> soft_commit=yes batch_size=250
> +??? fts_solr = url=https://solr.example.com:8984/solr/dovecot/
> use_libfts soft_commit=yes batch_size=250
>
> exec of
>
> ????doveadm fts rescan -u testuser at example.com
>
> returns
>
> ????doveadm(testuser at example.com): Error: fts-solr:
> fts_filter_normalizer_icu: libicu support not built in
> ????doveadm(testuser at example.com): Error: fts: Failed to initialize
> backend 'solr': Invalid fts_solr setting
> ????doveadm(testuser at example.com): Error: fts not enabled for user's
> namespace INBOX
>
>
> in my current package src
>
> ????https://src.fedoraproject.org/rpms/dovecot/blob/master/f/dovecot.spec
>
> config,
>
> ??? --with-solr????????????????? \
>
> exists, and
>
> ????--with-icu????????????? Build with libicu support (for FTS
> normalization)
> ??????????????????????????? (auto)
>
> _should_ be picked up automatically
>
> checking,
>
> ????ldd `locate lib64 | grep fts.*so$` | grep icu
> ??????? (empty)
>
> packaging issue, then?
>
> or additional config @ dovecot needed?
--with-icu should be sufficient, actually on centos 7 I got libuci
compiled in without setting the explicit flag.

config.log will tell more about whether it was successful if you're
compiling yourself (which if I remember requires the development header
files for the library).

Here's my ldd, which is under /usr/local/lib/dovecot

ldd /usr/local/lib/dovecot/libdovecot-fts.so
??? linux-vdso.so.1 =>? (0x00007fff7e4c7000)
??? libicui18n.so.50 => /lib64/libicui18n.so.50 (0x00007facd048d000)
??? libicuuc.so.50 => /lib64/libicuuc.so.50 (0x00007facd0114000)
??? libicudata.so.50 => /lib64/libicudata.so.50 (0x00007facceb41000)
??? libdovecot.so.0 => /usr/local/lib/dovecot/libdovecot.so.0
(0x00007facce7a2000)
??? libc.so.6 => /lib64/libc.so.6 (0x00007facce3d4000)
??? libpthread.so.0 => /lib64/libpthread.so.0 (0x00007facce1b8000)
??? libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007faccdeb1000)
??? libm.so.6 => /lib64/libm.so.6 (0x00007faccdbaf000)
??? libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007faccd999000)
??? libdl.so.2 => /lib64/libdl.so.2 (0x00007faccd795000)
??? /lib64/ld-linux-x86-64.so.2 (0x00007facd0ab6000)

John

On 10/19/20 9:48 AM, John Fawcett wrote:
> --with-icu should be sufficient, actually on centos 7 I got libuci
> compiled in without setting the explicit flag.
> Here's my ldd, which is under /usr/local/lib/dovecot
> 
> ldd /usr/local/lib/dovecot/libdovecot-fts.so
noted. as suspected. thx.
> config.log will tell more about whether it was successful if you're
> compiling yourself (which if I remember requires the development header
> files for the library).


i'm not.  yet.



the pkg bld logs,

 
https://kojipkgs.fedoraproject.org//packages/dovecot/2.3.11.3/5.fc32/data/logs/x86_64/build.log



_do_ show

	make[4]: Entering directory
'/builddir/build/BUILD/dovecot-2.3.11.3/src/lib-fts'
	...
	... -c -o fts-filter-normalizer-icu.lo fts-filter-normalizer-icu.c
	...


but at 1st glance, i don't see any explicit inclusion of the icu
libs/headers

checking the spec,

	https://src.fedoraproject.org/rpms/dovecot/blob/master/f/dovecot.spec

there's no:

	BuildRequires: libicu-devel

either.  Dunno yet if it's pulled in otherwise ... but I'm suspicious.

i've cc'd in the pkg maintainer; see if they've got a comment