search for: softcommits

...ut the </div> <div> softCommit gets sent once per mailbox folder at the end of all the </div> <div> requests for that folder. </div> <div> <br> </div> <div> I immagine that one of the reasons dovecot sends softCommits is because </div> <div> without autoindex active and even if mailboxes are periodically indexed </div> <div> from cron, the last emails received with be indexed at the moment of the </div> <div> search. So while sending softCommit h...

On 4/4/19 6:47 PM, dovecot-request at dovecot.org wrote: > For a typical Solr index, 60 seconds is an eternity. Most people aim > for query times of 100 milliseconds or less, and they often achieve > that goal. I'm pretty sure I get these while indexing, not querying. Apr 04 16:44:50 host dovecot[114690]: indexer-worker(me at example.com): Error: fts_solr: Indexing failed: Request

On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: > If this value was made configurable people could set it to what they > want. However the underlying problem is likely on solr configuration. The Jetty that is included in Solr has its idle timeout set to 50 seconds. But in practice, I have not seen this timeout trigger ... and if the OP is seeing a 60 second timeout, then the 50

...> Error: fts_solr: Indexing failed: Request timed out (Request queued > 66.015 secs ago, 1 attempts in 66.005 secs, 63.146 in http ioloop, 0.000 > in other ioloops, connected 94.903 secs ago) Doing a TCP dump on indexing operations which consistently fail, I see that there's a lot of softCommits which never get an HTTP answer: ============================================== POST /solr/dovebody/update HTTP/1.1 Host: localhost:8983 Date: Wed, 10 Apr 2019 14:22:29 GMT Expect: 100-continue Content-Length: 47 Connection: Keep-Alive Content-Type: text/xml HTTP/1.1 100 Continue <commit softC...

...fts-solr plugin is not sending softCommit every 1000 emails. Emails from a single folder are batched in up to a maximum of 1000 emails per request, but the softCommit gets sent once per mailbox folder at the end of all the requests for that folder. I immagine that one of the reasons dovecot sends softCommits is because without autoindex active and even if mailboxes are periodically indexed from cron, the last emails received with be indexed at the moment of the search.? So while sending softCommit has the advantage of including recent mails in searches, it means that softCommits are being done upon use...

...m the TCP windows running full and Dovecot backing of ... until it times out and close the connection. >From my understanding of how Dovecot operates towards Solr it will flush ~1000 documents towards Solr in /update request until it has traversed the mailbox (let's say 20.000 mails), doing softCommits after each. But is it really reasonable for Dovecot to expect that no request will take more than 60s to process by Solr? It doesn't seem like my Solr can handle that, although it does process documents and it does reasonably fast clear pending documents after Dovecot closes the connection. O...

...index data before a much larger heap >> is required. I run with SOLR_JAVA_MEM="-Xmx8g -Xms2g" > I looked into the code (version 2.3.5.1): This is 2.2.35. I haven't checked the source difference to 2.3.x I must admit. > I immagine that one of the reasons dovecot sends softCommits is because > without autoindex active and even if mailboxes are periodically indexed > from cron, the last emails received with be indexed at the moment of the > search.? I expect that dovecot has to because of it's default behavior by only bringing the index up-to-date just before s...

On 28 May 2018, at 13.28, kfx <kadafax at gmail.com> wrote: > > >> Especially what is in the "fts" header vs. "next uid" header? Does the UID in "fts" header keep changing every time you save a new mail? I suppose it will. > > Diff between 2 emails: > next uid = 30104 | next uid = 30105 > last_indexed_uid = 30103 |

...>> is required. > I run with > > SOLR_JAVA_MEM="-Xmx8g -Xms2g" > >> I looked into the code (version 2.3.5.1): > This is 2.2.35. I haven't checked the source difference to 2.3.x I must > admit. > >> I immagine that one of the reasons dovecot sends softCommits is because >> without autoindex active and even if mailboxes are periodically indexed >> from cron, the last emails received with be indexed at the moment of the >> search.? > I expect that dovecot has to because of it's default behavior by only > bringing the index up-...

On 28/05/2018 13:23, kfx wrote: > On 28/05/2018 13:04, Timo Sirainen wrote: >> On 28 May 2018, at 13.28, kfx <kadafax at gmail.com >> <mailto:kadafax at gmail.com>> wrote: >>> >>> >>>> Especially what is in the "fts" header vs. "next uid" header? Does >>>> the UID in "fts" header keep changing

I've built a new dovecot + fts-solr instance; I've now picked up & am running the recently released solr 8.7.0. In a test account, I've one message -- in the 'Drafts' folder. On exec of fts 'index' doveadm index -u testuser at example.com -q '*' I get a 401 unauthorized indexing failure, only for the one dir+message in 'Drafts', 2020-11-13

...nning full and Dovecot backing of ... until it times out > and close the connection. > > From my understanding of how Dovecot operates towards Solr it will flush > ~1000 documents towards Solr in /update request until it has traversed > the mailbox (let's say 20.000 mails), doing softCommits after each. > > But is it really reasonable for Dovecot to expect that no request will > take more than 60s to process by Solr? > It doesn't seem like my Solr can handle that, although it does process > documents and it does reasonably fast clear pending documents after > Dove...

On 28/05/2018 13:04, Timo Sirainen wrote: > On 28 May 2018, at 13.28, kfx <kadafax at gmail.com > <mailto:kadafax at gmail.com>> wrote: >> >> >>> Especially what is in the "fts" header vs. "next uid" header? Does >>> the UID in "fts" header keep changing every time you save a new mail? >>> I suppose it will.

...index data before a much larger heap >> is required. I run with SOLR_JAVA_MEM="-Xmx8g -Xms2g" > I looked into the code (version 2.3.5.1): This is 2.2.35. I haven't checked the source difference to 2.3.x I must admit. > I immagine that one of the reasons dovecot sends softCommits is because > without autoindex active and even if mailboxes are periodically indexed > from cron, the last emails received with be indexed at the moment of the > search.? I expect that dovecot has to because of it's default behavior by only bringing the index up-to-date just before s...

On 11/13/20 9:46 AM, John Fawcett wrote: >> (1) Can anyone yet verify Doveoct/fts-solr working with solr 8.7.x?>> (2) What logging config in Dovecot gets me more/detailed output for the fts_solr fail? >> (3) Any obvious clues as to what, specifically, is the source of this prob? >> > (1) I can't, I'm still on an earlier version I'll look for the 8.6.3,

On 13/11/2020 14:56, PGNet Dev wrote: > I've built a new dovecot + fts-solr instance; I've now picked up & am > running the recently released solr 8.7.0. > > In a test account, I've one message -- in the 'Drafts' folder. > > On exec of fts 'index' > > ????doveadm index -u testuser at example.com -q '*' > > I get a > >

On 21 May 2018, at 14.11, kadafax at gmail.com wrote: > > Le 21/05/2018 ? 12:38, Aki Tuomi a ?crit : >> can you try turning on pluign { fts_enforced = yes } and repeat your test? > > Same (wrong) result: > 1. Send an email with "too6Ouka" in the body > > 2. Search against "too6Ouka": > # doveadm search -u username mailbox INBOX body too6Ouka