search for: soft_commit

...??? SOLR_AUTH_TYPE=basic > ??????? SOLR_AUTHENTICATION_OPTS="-Dbasicauth=myuser:my at pass" > > and > > ????dovecot/conf.d/10-master.cf > > ??????? fts_solr = > url=https://"myuser":"my%40pass"@solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > > changing all > > -??? my at pass > +??? my_pass > > so that > > ??????? fts_solr = > url=https://"myuser":"my_pass"@solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > > does the trick.?...

...ng indexes? i hadn't yet seriously considered _circumventing_ fts plugin; if this^ does get resolved soonish, then it's not a big deal. if not, an fts-plugin-less setup would be interesting to know more abt! > Can you get evidence of things not working? For example tests run with > soft_commit configured - that's important since without it the updates > don't show up immediately in searches, that do show that the update is > happening in solr via solr log, but then search is not working on > accented characters, despite it working on other text in the same > message?...

On 13/11/2020 19:55, PGNet Dev wrote: > On 11/13/20 9:56 AM, PGNet Dev wrote: >>> (2) see here: https://wiki.dovecot.org/Plugins/FTS/Solr >>> >>> two useful settings are debug and rawlog_dir=whatever to be added in >>> the same line as fts_solr > > with > > ????fts_solr = ... debug > > line #35 @ https://pastebin.com/9ecLQspD _looks_ like

...9;t yet seriously considered _circumventing_ fts plugin; if this^ > does get resolved soonish, then it's not a big deal.? if not, an > fts-plugin-less setup would be interesting to know more abt! > >> Can you get evidence of things not working? For example tests run with >> soft_commit configured - that's important since without it the updates >> don't show up immediately in searches, that do show that the update is >> happening in solr via solr log, but then search is not working on >> accented characters, despite it working on other text in the same &g...

On 13/11/2020 22:04, PGNet Dev wrote: >> I guess you didn't need to enclose username and password in quotes, i.e. >> >> fts_solr = >> url=https://myuser:my%40pass at solr.example.com:8984/solr/dovecot/ >> use_libfts soft_commit=yes batch_size=250 > > On 11/13/20 12:56 PM, John Fawcett wrote: > >> I guess you didn't need to enclose username and password in quotes, i.e. > >> > >> fts_solr = > >> url=https://myuser:my%40pass at solr.example.com:8984/solr/dovecot/ > >> us...

...red _circumventing_ fts plugin; if this^ > > does get resolved soonish, then it's not a big deal.? if not, an > > fts-plugin-less setup would be interesting to know more abt! > > > >> Can you get evidence of things not working? For example tests run with > >> soft_commit configured - that's important since without it the updates > >> don't show up immediately in searches, that do show that the update is > >> happening in solr via solr log, but then search is not working on > >> accented characters, despite it working on other text...

On 10/18/20 10:28 PM, Aki Tuomi wrote: > Dovecot FTS tokenization is not done, unless you have `use_libfts` in fts_solr setting, in your case > > fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts changing - fts_solr = url=https://solr.example.com:8984/solr/dovecot/ soft_commit=yes batch_size=250 + fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 exec of doveadm fts rescan -u testuser at example.com returns doveadm(testuser at example.com): Error: fts-solr: fts_filter_normalizer_icu: libicu support not built in do...

On 10/31/20 9:55 AM, John Fawcett wrote: > I can contribute a patch that solves the segfault. Unfortunately though > fts search may be more broken than this. It does not give me search > results, even though I see it querying solr and getting hits. Thx -- hopefully it moves this in the right direction. Also on the 'good news' page, it appears there's been some progress on

> On 02/11/2020 15:11 PGNet Dev <pgnet.dev at gmail.com> wrote: > > > On 11/2/20 12:44 AM, Aki Tuomi wrote: > > you should try removing use_libfts from your config line and let solr do that part. > > sry, i'm a bit confused. > > you'd suggested I _add_ it, > > https://dovecot.org/pipermail/dovecot/2020-October/120258.html > > > I

...example it was > stopping you indexing excluded words like tambien). On my setup without > lib-fts everything goes to solr which does the work of indexing without > all the features of lib-fts. withOUT libfts - fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 + fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ soft_commit=yes batch_size=250 and unmodified dovecot-provided schema/config, /bin/cp -af /usr/share/doc/dovecot/solr-config-7.7.0.xml /path/to/solr/data/dovecot/conf/solrconfig.xml /bin/cp -af /usr/share/doc...

I've since rebuilt/reconfig'd all parts of my setup from scratch; some good cleanup along the way. Atm, my entire system for send/recv, store/retrieve, + rules & search is working as I intend. Ok, mostly ... Except for this accented-character search mystery. I've got a _lot_ of mail with various languages in bodies, so _do_ need to get this sorted. > On 10/18/20 2:58 PM,

..." } } } and /etc/default/solr.in.sh ... SOLR_AUTH_TYPE=basic SOLR_AUTHENTICATION_OPTS="-Dbasicauth=myuser:my at pass" and dovecot/conf.d/10-master.cf fts_solr = url=https://"myuser":"my%40pass"@solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 changing all - my at pass + my_pass so that fts_solr = url=https://"myuser":"my_pass"@solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 does the trick. fts/solr is indexing, solr's logging the activity, and there appear...

> I guess you didn't need to enclose username and password in quotes, i.e. > > fts_solr = > url=https://myuser:my%40pass at solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 On 11/13/20 12:56 PM, John Fawcett wrote: > I guess you didn't need to enclose username and password in quotes, i.e. > > fts_solr = > url=https://myuser:my%40pass at solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 heh, q...

...gt; Dovecot FTS tokenization is not done, unless you have `use_libfts` in >> fts_solr setting, in your case >> >> fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts > > changing > > -??? fts_solr = url=https://solr.example.com:8984/solr/dovecot/ > soft_commit=yes batch_size=250 > +??? fts_solr = url=https://solr.example.com:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > > exec of > > ????doveadm fts rescan -u testuser at example.com > > returns > > ????doveadm(testuser at example.com): Error: fts-solr: &gt...

...indexing excluded words like tambien). On my setup without >> lib-fts everything goes to solr which does the work of indexing without >> all the features of lib-fts. > > withOUT libfts > > -??? fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ > use_libfts soft_commit=yes batch_size=250 > +??? fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ > soft_commit=yes batch_size=250 > > and unmodified dovecot-provided schema/config, > > ????/bin/cp -af /usr/share/doc/dovecot/solr-config-7.7.0.xml > /path/to/solr/data/dovecot/conf/sol...

...less errors using http than https. Relevant config: OS: CentOS6, fully updated plugin { ? fts = solr ? batch_size = 1 ? fts_solr = url=https://username:password at solr-01.vevida.net:443/solr/dovecot/ ? #fts_tika = https://solr-01.vevida.net:443/tika/ ? batch_size = 1000 ? fts_autoindex=yes ? soft_commit=no } # dovecot --version 2.3.9.3 (9f41b88fa) # Configure options: ??? --docdir=%{_docdir}/dovecot? \ ??? --disable-static???????????? \ ??? --with-nss?????????????????? \ ??? --with-shadow??????????????? \ ??? --with-pam?????????????????? \ ??? --with-gssapi=plugin???????? \ ??? --with-ldap=plug...

...oth. If the non accented > text is found you know you've searching against the updated index and > the fact that accented text is not found is not simply because the index > updates are not visible. temp changing, fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 - fts_enforced = body + fts_enforced = yes - fts_filters = normalizer-icu snowball stopwords + fts_filters = lowercase snowball stopwords now, doveadm fts rescan -u testuser at example.com doveadm index -u testuser at example.com -q '*' , no errors. on receipt...

...should be able to confirm that by tailing the access_log file on solr server and see that no access is being made for header searches only for body searches. If you want to use solr for all searches then fts_enforced should be set to yes. Another point as mentioned by Aki is that you did not have soft_commit correctly configured. That has the effect of not opening a new solr searcher after each update of the index. So your tests may be invalid and in any case leave room for doubt as to whether the index updates were visible or not at the moment of your test. I would recommend you to redo the tests aft...

...= imap_search($conn, 'BODY "tambi?n"', SE_UID); print_r($uids); Only thing I cannot vouch for is bringing dovecot fts library and config into the equation because my setup delegates almost everything to solr. Can you get evidence of things not working? For example tests run with soft_commit configured - that's important since without it the updates don't show up immediately in searches, that do show that the update is happening in solr via solr log, but then search is not working on accented characters, despite it working on other text in the same message? The solr logs also s...

...gt;> I guess this kinda confirms your guess, but interestingly shows less >> errors than my previous test with a large header 136KB Mail. >> >> >> Patrik >> > I did notice that changing the batch_size in > > fts_solr = url=http://solr.example.org:8983/solr/ soft_commit=yes batch_size=1000 > > does have an influence in how often the error occurs. Setting it to 1 > or some huge number like 10000 reduces the chances quite a bit but not > completely and also causes lots of small , or few but quite large > requests to Solr, so its not a practical workar...