search for: use_libfts

> On 02/11/2020 15:11 PGNet Dev <pgnet.dev at gmail.com> wrote: > > > On 11/2/20 12:44 AM, Aki Tuomi wrote: > > you should try removing use_libfts from your config line and let solr do that part. > > sry, i'm a bit confused. > > you'd suggested I _add_ it, > > https://dovecot.org/pipermail/dovecot/2020-October/120258.html > > > I can reproduce your problem with the `fts lookup` command. Luckily it's...

given existing problems with fts_solr = ... use_libfts ... i've rm'd it from dovecot config, and am moving all tokenization to solr config. iiuc, in dovecot + solr context, that's correctly config'd in solr/data/dovecot/conf/schema.xml dovecot release ships with 'starter configs', cp -af /usr/share/doc/dovecot/solr-config...

.... > ??????? SOLR_AUTH_TYPE=basic > ??????? SOLR_AUTHENTICATION_OPTS="-Dbasicauth=myuser:my at pass" > > and > > ????dovecot/conf.d/10-master.cf > > ??????? fts_solr = > url=https://"myuser":"my%40pass"@solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > > changing all > > -??? my at pass > +??? my_pass > > so that > > ??????? fts_solr = > url=https://"myuser":"my_pass"@solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > > does...

On 10/18/20 10:28 PM, Aki Tuomi wrote: > Dovecot FTS tokenization is not done, unless you have `use_libfts` in fts_solr setting, in your case > > fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts changing - fts_solr = url=https://solr.example.com:8984/solr/dovecot/ soft_commit=yes batch_size=250 + fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts soft_com...

...>> But if no hits are found, then dovecot cannot be expected to display > >> results. It still may be an indexing problem though. > > > > my current config has soft_commit enabled, > > > > ? fts_solr = url=https://solr.example.com:8984/solr/dovecot/ > > use_libfts soft_commit=yes batch_size=250 > > > > i'll see abt getting some clearer test results ... > > Yes, getting more data about any potential problem would be useful. > > Just to clarify: I have a fully working search setup for some time now > over various dovecot releas...

On 13/11/2020 19:55, PGNet Dev wrote: > On 11/13/20 9:56 AM, PGNet Dev wrote: >>> (2) see here: https://wiki.dovecot.org/Plugins/FTS/Solr >>> >>> two useful settings are debug and rawlog_dir=whatever to be added in >>> the same line as fts_solr > > with > > ????fts_solr = ... debug > > line #35 @ https://pastebin.com/9ecLQspD _looks_ like

On 13/11/2020 22:04, PGNet Dev wrote: >> I guess you didn't need to enclose username and password in quotes, i.e. >> >> fts_solr = >> url=https://myuser:my%40pass at solr.example.com:8984/solr/dovecot/ >> use_libfts soft_commit=yes batch_size=250 > > On 11/13/20 12:56 PM, John Fawcett wrote: > >> I guess you didn't need to enclose username and password in quotes, i.e. > >> > >> fts_solr = > >> url=https://myuser:my%40pass at solr.example.com:8984/solr/dovecot/ >...

I've since rebuilt/reconfig'd all parts of my setup from scratch; some good cleanup along the way. Atm, my entire system for send/recv, store/retrieve, + rules & search is working as I intend. Ok, mostly ... Except for this accented-character search mystery. I've got a _lot_ of mail with various languages in bodies, so _do_ need to get this sorted. > On 10/18/20 2:58 PM,

...mentation... What should I add >> there anbd where? > > fts_languages = de,en > seems to do the trick. But I wonder if there's something like "all" > and what exactly this setting does. That code shouldn't have run at all (unless you had configured fts_lucene = use_libfts). It's still work in progress. I'll get rid of the errors.

On 19/10/2020 16:20, PGNet Dev wrote: > On 10/18/20 10:28 PM, Aki Tuomi wrote: >> Dovecot FTS tokenization is not done, unless you have `use_libfts` in >> fts_solr setting, in your case >> >> fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts > > changing > > -??? fts_solr = url=https://solr.example.com:8984/solr/dovecot/ > soft_commit=yes batch_size=250 > +??? fts_solr = url=https://solr....

..."admin" } } } and /etc/default/solr.in.sh ... SOLR_AUTH_TYPE=basic SOLR_AUTHENTICATION_OPTS="-Dbasicauth=myuser:my at pass" and dovecot/conf.d/10-master.cf fts_solr = url=https://"myuser":"my%40pass"@solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 changing all - my at pass + my_pass so that fts_solr = url=https://"myuser":"my_pass"@solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 does the trick. fts/solr is indexing, solr's logging the activity, and t...

> I guess you didn't need to enclose username and password in quotes, i.e. > > fts_solr = > url=https://myuser:my%40pass at solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 On 11/13/20 12:56 PM, John Fawcett wrote: > I guess you didn't need to enclose username and password in quotes, i.e. > > fts_solr = > url=https://myuser:my%40pass at solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=...

...> I'll see what I find with using the ICU filters -- if perhaps anything changes Hi! I can reproduce your problem with the `fts lookup` command. Luckily it's equivalent to running `doveadm search`. I'll open a bug about this. Dovecot FTS tokenization is not done, unless you have `use_libfts` in fts_solr setting, in your case fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts Without this, everything is sent to to solr as-is, which is then expected to do all the work. Aki

...amp;rows=3202&wt=xml} > hits=3 status=0 QTime=3 > > But if no hits are found, then dovecot cannot be expected to display > results. It still may be an indexing problem though. my current config has soft_commit enabled, fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 i'll see abt getting some clearer test results ...

...eatures for example it was > stopping you indexing excluded words like tambien). On my setup without > lib-fts everything goes to solr which does the work of indexing without > all the features of lib-fts. withOUT libfts - fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 + fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ soft_commit=yes batch_size=250 and unmodified dovecot-provided schema/config, /bin/cp -af /usr/share/doc/dovecot/solr-config-7.7.0.xml /path/to/solr/data/dovecot/conf/solrconfig.xml /bin/cp -af /u...

...earch for both. If the non accented > text is found you know you've searching against the updated index and > the fact that accented text is not found is not simply because the index > updates are not visible. temp changing, fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 - fts_enforced = body + fts_enforced = yes - fts_filters = normalizer-icu snowball stopwords + fts_filters = lowercase snowball stopwords now, doveadm fts rescan -u testuser at example.com doveadm index -u testuser at example.com -q '*' , no errors....

I don't find "fts_languages" in the documentation... What should I add there anbd where? doveadm(restore): Error: fts-dovecot: fts_languages setting is missing - disabling -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer

...>> there anbd where? > > > > fts_languages = de,en > > seems to do the trick. But I wonder if there's something like "all" > > and what exactly this setting does. > > That code shouldn't have run at all (unless you had configured fts_lucene = use_libfts). It's still work in progress. I'll get rid of the errors. oops :) -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsi...

...opping you indexing excluded words like tambien). On my setup without >> lib-fts everything goes to solr which does the work of indexing without >> all the features of lib-fts. > > withOUT libfts > > -??? fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > +??? fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ > soft_commit=yes batch_size=250 > > and unmodified dovecot-provided schema/config, > > ????/bin/cp -af /usr/share/doc/dovecot/solr-config-7.7.0.xml > /path/to/solr/data/dove...

...ts=3 status=0 QTime=3 >> >> But if no hits are found, then dovecot cannot be expected to display >> results. It still may be an indexing problem though. > > my current config has soft_commit enabled, > > ? fts_solr = url=https://solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > > i'll see abt getting some clearer test results ... Yes, getting more data about any potential problem would be useful. Just to clarify: I have a fully working search setup for some time now over various dovecot releases, so no patches needed to get it...