search for: autosoftcommit

I am testing Solr FTS on dovecot. Read online that some suggested to run cronjob commit every minute, and optimize once a day. I am using Solr 7.1.0 and I see some configurations: In /etc/default/solr.in.sh: #SOLR_OPTS="$SOLR_OPTS -Dsolr.autoSoftCommit.maxTime=3000" #SOLR_OPTS="$SOLR_OPTS -Dsolr.autoCommit.maxTime=60000" Also in solrconfig.xml: ??? <autoCommit> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> ????? <openSearcher>false</openSearcher> ??? </autoCommit> ?? <autoSoftCommi...

...t really > concerned about qucik visibility but just getting things into the index > efficiently, a better approach would be for dovecot plugin not to send > any commit or softCommit (or waitSearcher either) and that should speed > things up. You'd need to configure solr with a long autoSoftCommit > maxTime and a reasonable autoCommit maxTime, which you could then > reconfigure when the load was done. Solr ships with autoCommit set to 15 seconds and openSearcher set to false on the autoCommit. The autoSoftCommit setting is not enabled by default, but depending on how the index was...

...>> efficiently, a better approach would be for dovecot plugin not to send </div> <div> >> any commit or softCommit (or waitSearcher either) and that should speed </div> <div> >> things up. You'd need to configure solr with a long autoSoftCommit </div> <div> >> maxTime and a reasonable autoCommit maxTime, which you could then </div> <div> >> reconfigure when the load was done. </div> <div> > </div> <blockquote type="cite"> <d...

...I got distracted half way and forgot to put a meaningfull subject so the archive could figure out the thread. - resending. On 4/14/19 4:04 PM, dovecot-request at dovecot.org wrote: >> Solr ships with autoCommit set to 15 seconds and openSearcher set to >> false on the autoCommit.? The autoSoftCommit setting is not enabled by >> default, but depending on how the index was created, Solr might try to >> set autoSoftCommit to 3 seconds ... which is WAY too short. I just run with the default. 15s autoCommit and no autoSoftCommit >> This thread says that dovecot is sending explic...

Looking further at tcpdumps of the Dovecot->Solr traffic and Solr metrics it doesn't seem like there's anything suspicious apart from the TCP windows running full and Dovecot backing of ... until it times out and close the connection. >From my understanding of how Dovecot operates towards Solr it will flush ~1000 documents towards Solr in /update request until it has traversed the

...; concerned about qucik visibility but just getting things into the index >> efficiently, a better approach would be for dovecot plugin not to send >> any commit or softCommit (or waitSearcher either) and that should speed >> things up. You'd need to configure solr with a long autoSoftCommit >> maxTime and a reasonable autoCommit maxTime, which you could then >> reconfigure when the load was done. > > Solr ships with autoCommit set to 15 seconds and openSearcher set to > false on the autoCommit.? The autoSoftCommit setting is not enabled by > default, but depend...

...and forgot to put a meaningfull > subject so the archive could figure out the thread. - resending. > > On 4/14/19 4:04 PM, dovecot-request at dovecot.org wrote: > >>> Solr ships with autoCommit set to 15 seconds and openSearcher set to >>> false on the autoCommit.? The autoSoftCommit setting is not enabled by >>> default, but depending on how the index was created, Solr might try to >>> set autoSoftCommit to 3 seconds ... which is WAY too short. > I just run with the default. 15s autoCommit and no autoSoftCommit > >>> This thread says that dovec...

...y minute via cron so the only consequence of this is that you >cannot search for messages delivered before less then a minute (which >you, usually, don't need to do anyway). > >While googling i also find out that Solr supports autoCommit function >(and from version 4.0 also autoSoftCommit), so there's no reason for >Dovecot to handle this on it's own (and potentially doing hundreds or >thousands of soft commits every second) - you can just set Solr to, >for example, do autoSoftCommit every second and autoCommit every minute: >https://cwiki.apache.org/conflu...

On 4/14/19 4:04 PM, dovecot-request at dovecot.org wrote: >> Solr ships with autoCommit set to 15 seconds and openSearcher set to >> false on the autoCommit.? The autoSoftCommit setting is not enabled by >> default, but depending on how the index was created, Solr might try to >> set autoSoftCommit to 3 seconds ... which is WAY too short. I just run with the default. 15s autoCommit and no autoSoftCommit >> This thread says that dovecot is sending explic...

...hard commits every minute via cron so the only consequence of this is that you cannot search for messages delivered before less then a minute (which you, usually, don't need to do anyway). While googling i also find out that Solr supports autoCommit function (and from version 4.0 also autoSoftCommit), so there's no reason for Dovecot to handle this on it's own (and potentially doing hundreds or thousands of soft commits every second) - you can just set Solr to, for example, do autoSoftCommit every second and autoCommit every minute: https://cwiki.apache.org/confluence/display/sol...

Hello, it seems an mismatch of schema file provided by dovecot-2.3.5.2 [root at mail conf]# pwd /var/solr/data/dovecot/conf # Below solrconfig.xml is from Solr-8.0.0. [root at mail conf]# grep text_general solrconfig.xml <str name="queryAnalyzerFieldType">text_general</str> field types. Text content will be indexed as "text_general" as <str

...ionBuckets">${solr.ulog.numVersionBuckets:65536}</int> > </updateLog> > <autoCommit> > <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> > <openSearcher>false</openSearcher> > </autoCommit> > <autoSoftCommit> > <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime> > </autoSoftCommit> > </updateHandler> > <query> > <maxBooleanClauses>${solr.max.booleanClauses:1024}</maxBooleanClauses> > <filterCache class="solr.F...

...<int name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}</int> </updateLog> <autoCommit> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> <openSearcher>false</openSearcher> </autoCommit> <autoSoftCommit> <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime> </autoSoftCommit> </updateHandler> <query> <maxBooleanClauses>${solr.max.booleanClauses:1024}</maxBooleanClauses> <filterCache class="solr.FastLRUCache" size="5...

...ng, where you are not really concerned about qucik visibility but just getting things into the index efficiently, a better approach would be for dovecot plugin not to send any commit or softCommit (or waitSearcher either) and that should speed things up. You'd need to configure solr with a long autoSoftCommit maxTime and a reasonable autoCommit maxTime, which you could then reconfigure when the load was done. If you're using dovecot built from source code it should be possible to test that by some minor modification of the code in? fts-backend-solr.c. John

...m on Ubuntu 18.04. My Java version is OpenJDK 10.0.2. Attached is my complete Solr config. Try one more time - stop the server, delete the data folder, unpack the attached into the conf folder - and restart. I also have > > /etc/default/solr.in.sh: > SOLR_OPTS="$SOLR_OPTS -Dsolr.autoSoftCommit.maxTime=3000" > SOLR_OPTS="$SOLR_OPTS -Dsolr.autoCommit.maxTime=60000" > SOLR_PID_DIR=/run/solr > SOLR_HOME=/usr/local/lib > Adjust the above folders as appropriate - or don't use them at all if you're using the defaults. > > /etc/systemd/system/solr.servi...

...ing - I'm on Ubuntu 18.04.? My Java version is OpenJDK 10.0.2.? Attached is my complete Solr config.? Try one more time - stop the server, delete the data folder, unpack the attached into the conf folder - and restart.? I also have /etc/default/solr.in.sh: SOLR_OPTS="$SOLR_OPTS -Dsolr.autoSoftCommit.maxTime=3000" SOLR_OPTS="$SOLR_OPTS -Dsolr.autoCommit.maxTime=60000" SOLR_PID_DIR=/run/solr SOLR_HOME=/usr/local/lib Adjust the above folders as appropriate - or don't use them at all if you're using the defaults. /etc/systemd/system/solr.service: # put this file in /etc/...

...m on Ubuntu 18.04. My Java version is OpenJDK 10.0.2. Attached is my complete Solr config. Try one more time - stop the server, delete the data folder, unpack the attached into the conf folder - and restart. I also have > > /etc/default/solr.in.sh: > SOLR_OPTS="$SOLR_OPTS -Dsolr.autoSoftCommit.maxTime=3000" > SOLR_OPTS="$SOLR_OPTS -Dsolr.autoCommit.maxTime=60000" > SOLR_PID_DIR=/run/solr > SOLR_HOME=/usr/local/lib > Adjust the above folders as appropriate - or don't use them at all if you're using the defaults. > > /etc/systemd/system/solr.servi...

Dear Daniel. Thank you for your kind reply. Regarding NFS, no, there is nothing like this in my setup. Deleteing SOLR and recreating it, I did it so many times already. I started with *your* setup in the first place, as FTS_squat (which actually works very well and very straightforward, I have no clue why going for SOlr which is just a pain and not maintaining squat), and it leads to