On 12/4/2018 10:40 AM, Joan Moreau via dovecot wrote:> > In the Wiki, ( https://wiki.dovecot.org/Plugins/FTS/Solr ), it would > nice to stipulate to the reader? to type the command : > > sudo -u solr /opt/solr/bin/solr create -c dovecot # to create the > dovecot instance > > before updating the schema.xml . > > Also,? schema.xml is in /opt/solr/server/solr/dovecot/conf for > archlinux users > > Additionaly, the url is http://(solr_ > <http://(solr>server):8983/solr/dovecot/ (error in wiki) >After installing Solr, wherever the installation sets up there should a folder similar to: <your prefix>/solr/server/solr/configsets If you look there, you'll probably see folders like '_default' and 'sample_techproducts_configs'.? I haven't played with the 'techproducts' sample.? Copy the '_default' folder, with all its contents, to a 'dovecot' folder.? In the new dovecot folder, replace the 'managed-schema' file with the file from the Dovecot Wiki https://wiki.dovecot.org/Plugins/FTS/Solr?action=AttachFile&do=view&target=solr-7.x-schema.xml after that, you should be able to run 'solr /opt/solr/bin/solr create -c dovecot' to create the instance.? If things still don't work let us know. The schema is one I've tweaked and updated during my own migrations since Solr 3.3.? It's possible there's something else in my config that needs documenting - but having experienced Solr search against my mailstore I never want to be without it. Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20181210/05e89246/attachment.html>
Hi Daniel, THere is no need of all this, just the command (on Solr 7.5) "create -c dovecot " is enough The chema.xml provided on the wiki basically does not work on 7.5 Here the latest one I am working on , but nothing works properly (bad search results, errors in ftp_solr, etc..) <?xml version="1.0" encoding="UTF-8"?> <schema name="dovecot" version="2.0"> <uniqueKey>id</uniqueKey> <types> <fieldType name="string" class="solr.StrField" /> <fieldType name="long" class="solr.LongPointField" positionIncrementGap="0" /> <fieldType name="text" class="solr.TextField" autoGeneratePhraseQueries="true" positionIncrementGap="100"> <analyzer type="index"> <tokenizer class="solr.StandardTokenizerFactory"/> <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/> <filter class="solr.WordDelimiterGraphFilterFactory" generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1" splitOnNumerics="1" catenateWords="1" catenateNumbers="1" catenateAll="1"/> <filter class="solr.FlattenGraphFilterFactory"/> <!-- required on index analyzers after graph filters --> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.NGramFilterFactory" minGramSize="3" maxGramSize="15" /> <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> <filter class="solr.PorterStemFilterFactory"/> </analyzer> <analyzer type="query"> <tokenizer class="solr.StandardTokenizerFactory"/> <filter class="solr.SynonymGraphFilterFactory" expand="true" ignoreCase="true" synonyms="synonyms.txt"/> <filter class="solr.FlattenGraphFilterFactory"/> <!-- required on index analyzers after graph filters --> <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/> <filter class="solr.WordDelimiterGraphFilterFactory" generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1" splitOnNumerics="1" catenateWords="1" catenateNumbers="1" catenateAll="1"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.NGramFilterFactory" minGramSize="3" maxGramSize="15" /> <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> <filter class="solr.PorterStemFilterFactory"/> </analyzer> </fieldType> </types> <fields> <field name="_version_" type="long" indexed="true" stored="true"/> <field name="bcc" type="text" indexed="true" stored="false"/> <field name="body" type="text" indexed="true" stored="false"/> <field name="box" type="string" indexed="false" required="true" stored="true"/> <field name="hdr" type="text" indexed="true" stored="false"/> <field name="cc" type="text" indexed="true" stored="false"/> <field name="from" type="text" indexed="true" stored="false"/> <field name="id" type="string" indexed="true" required="true" stored="true"/> <field name="subject" type="text" indexed="true" stored="false"/> <field name="to" type="text" indexed="true" stored="false"/> <field name="uid" type="long" indexed="true" required="true" stored="true"/> <field name="user" type="string" indexed="true" required="true" stored="true"/> </fields> </schema> On 2018-12-10 21:17, Daniel Miller via dovecot wrote:> On 12/4/2018 10:40 AM, Joan Moreau via dovecot wrote: > >> In the Wiki, ( https://wiki.dovecot.org/Plugins/FTS/Solr ), it would nice to stipulate to the reader to type the command : >> >> sudo -u solr /opt/solr/bin/solr create -c dovecot # to create the dovecot instance >> >> before updating the schema.xml . >> >> Also, schema.xml is in /opt/solr/server/solr/dovecot/conf for archlinux users >> >> Additionaly, the url is http://(solr_ [1]server):8983/solr/dovecot/ (error in wiki) > > After installing Solr, wherever the installation sets up there should a folder similar to: > > <your prefix>/solr/server/solr/configsets > > If you look there, you'll probably see folders like '_default' and 'sample_techproducts_configs'. I haven't played with the 'techproducts' sample. Copy the '_default' folder, with all its contents, to a 'dovecot' folder. In the new dovecot folder, replace the 'managed-schema' file with the file from the Dovecot Wiki > > https://wiki.dovecot.org/Plugins/FTS/Solr?action=AttachFile&do=view&target=solr-7.x-schema.xml > > after that, you should be able to run 'solr /opt/solr/bin/solr create -c dovecot' to create the instance. If things still don't work let us know. > > The schema is one I've tweaked and updated during my own migrations since Solr 3.3. It's possible there's something else in my config that needs documenting - but having experienced Solr search against my mailstore I never want to be without it. > > DanielLinks: ------ [1] http://(solr -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20181210/c517582d/attachment.html>
The one on the Wiki is mine...which I'm using now.? So it certainly does work - but perhaps there's a setting you have differently from me. Performing a "create -c dovecot" creates a Solr instance *named* dovecot - that does *not* initialize it with the necessary schema.? You need to specify "-d dovecot", with a dovecot configset already setup, to do that. The other choice is to create the instance as you show, ensure Solr is stopped, delete the "<prefix>/solr/dovecot/data" folder, and copy the managed-schema file to "<prefix>/solr/dovecot/conf".? Again, the filename saved in the /conf folder needs to be "managed-schema" - no ".xml" suffix. If that doesn't work for you - please share the errors. Daniel On 12/10/2018 11:40 AM, Joan Moreau wrote:> > Hi Daniel, > > THere is no need of all this, just the command (on Solr 7.5) "create > -c dovecot " is enough > > The chema.xml provided on the wiki basically does not work on 7.5 > > > Here the latest one I am working on , but nothing works properly (bad > search results, errors in ftp_solr, etc..) > > > <?xml version="1.0" encoding="UTF-8"?> > <schema name="dovecot" version="2.0"> > <uniqueKey>id</uniqueKey> > <types> > <fieldType name="string" class="solr.StrField" /> > <fieldType name="long" class="solr.LongPointField" > positionIncrementGap="0" /> > <fieldType name="text" class="solr.TextField" > autoGeneratePhraseQueries="true" positionIncrementGap="100"> > <analyzer type="index"> > <tokenizer class="solr.StandardTokenizerFactory"/> > <filter class="solr.StopFilterFactory" words="stopwords.txt" > ignoreCase="true"/> > <filter class="solr.WordDelimiterGraphFilterFactory" > generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1" > splitOnNumerics="1" catenateWords="1" catenateNumbers="1" > catenateAll="1"/> > <filter class="solr.FlattenGraphFilterFactory"/> <!-- required on > index analyzers after graph filters --> > <filter class="solr.LowerCaseFilterFactory"/> > <filter class="solr.NGramFilterFactory" minGramSize="3" > maxGramSize="15" /> > <filter class="solr.KeywordMarkerFilterFactory" > protected="protwords.txt"/> > <filter class="solr.PorterStemFilterFactory"/> > </analyzer> > <analyzer type="query"> > <tokenizer class="solr.StandardTokenizerFactory"/> > <filter class="solr.SynonymGraphFilterFactory" expand="true" > ignoreCase="true" synonyms="synonyms.txt"/> > <filter class="solr.FlattenGraphFilterFactory"/> <!-- required on > index analyzers after graph filters --> > <filter class="solr.StopFilterFactory" words="stopwords.txt" > ignoreCase="true"/> > <filter class="solr.WordDelimiterGraphFilterFactory" > generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1" > splitOnNumerics="1" catenateWords="1" catenateNumbers="1" > catenateAll="1"/> > <filter class="solr.LowerCaseFilterFactory"/> > <filter class="solr.NGramFilterFactory" minGramSize="3" > maxGramSize="15" /> > <filter class="solr.KeywordMarkerFilterFactory" > protected="protwords.txt"/> > <filter class="solr.PorterStemFilterFactory"/> > </analyzer> > </fieldType> > </types> > <fields> > <field name="_version_" type="long" indexed="true" stored="true"/> > <field name="bcc" type="text" indexed="true" stored="false"/> > <field name="body" type="text" indexed="true" stored="false"/> > <field name="box" type="string" indexed="false" required="true" > stored="true"/> > <field name="hdr" type="text" indexed="true" stored="false"/> > <field name="cc" type="text" indexed="true" stored="false"/> > <field name="from" type="text" indexed="true" stored="false"/> > <field name="id" type="string" indexed="true" required="true" > stored="true"/> > <field name="subject" type="text" indexed="true" stored="false"/> > <field name="to" type="text" indexed="true" stored="false"/> > <field name="uid" type="long" indexed="true" required="true" > stored="true"/> > <field name="user" type="string" indexed="true" required="true" > stored="true"/> > </fields> > </schema> > > > > On 2018-12-10 21:17, Daniel Miller via dovecot wrote: > >> On 12/4/2018 10:40 AM, Joan Moreau via dovecot wrote: >>> >>> In the Wiki, ( https://wiki.dovecot.org/Plugins/FTS/Solr ), it would >>> nice to stipulate to the reader? to type the command : >>> >>> sudo -u solr /opt/solr/bin/solr create -c dovecot # to create the >>> dovecot instance >>> >>> before updating the schema.xml . >>> >>> Also,? schema.xml is in /opt/solr/server/solr/dovecot/conf for >>> archlinux users >>> >>> Additionaly, the url is http://(solr_ >>> <http://(solr>server):8983/solr/dovecot/ (error in wiki) >>> >> After installing Solr, wherever the installation sets up there should >> a folder similar to: >> >> <your prefix>/solr/server/solr/configsets >> >> If you look there, you'll probably see folders like '_default' and >> 'sample_techproducts_configs'.? I haven't played with the >> 'techproducts' sample.? Copy the '_default' folder, with all its >> contents, to a 'dovecot' folder.? In the new dovecot folder, replace >> the 'managed-schema' file with the file from the Dovecot Wiki >> >> https://wiki.dovecot.org/Plugins/FTS/Solr?action=AttachFile&do=view&target=solr-7.x-schema.xml >> >> after that, you should be able to run 'solr /opt/solr/bin/solr create >> -c dovecot' to create the instance. If things still don't work let us >> know. >> >> The schema is one I've tweaked and updated during my own migrations >> since Solr 3.3.? It's possible there's something else in my config >> that needs documenting - but having experienced Solr search against >> my mailstore I never want to be without it. >> >> Daniel >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20181210/3781d999/attachment.html>
Additionally, here the errors I get in logs: Dovecot: Dec 09 09:21:09 imap(jom at grosjo.net)<3349><DiRnXpN8Lux/AAAB>: Error: fts_solr: received invalid uid '0' Dec 09 09:21:10 imap(jom at grosjo.net)<3349><DiRnXpN8Lux/AAAB>: Error: fts_solr: received invalid uid '0' or 11 03:36:03 indexer-worker(jom at grosjo.net)<2093><icWMJaIwD1znEgAA0thIag:GPBOCKMwD1wtCAAA0thIag>: Error: fts_solr: Indexing failed: 500 Server Error Solr: CAUSED BY: ORG.APACHE.LUCENE.STORE.ALREADYCLOSEDEXCEPTION: THIS INDEXWRITER IS CLOSED Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:679) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:693) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4929) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:279) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:235) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2039) Dec 11 06:00:14 gjserver solr[16761]: ... 51 more DEC 11 06:00:14 GJSERVER SOLR[16761]: CAUSED BY: JAVA.LANG.ARRAYINDEXOUTOFBOUNDSEXCEPTION: -65536 Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.FreqProxTermsWriterPerField.writeProx(FreqProxTermsWriterPerField.java:80) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:184) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) On 2018-12-10 21:40, Joan Moreau wrote:> Hi Daniel, > > THere is no need of all this, just the command (on Solr 7.5) "create -c dovecot " is enough > > The chema.xml provided on the wiki basically does not work on 7.5 > > Here the latest one I am working on , but nothing works properly (bad search results, errors in ftp_solr, etc..) > > <?xml version="1.0" encoding="UTF-8"?> > <schema name="dovecot" version="2.0"> > <uniqueKey>id</uniqueKey> > <types> > <fieldType name="string" class="solr.StrField" /> > <fieldType name="long" class="solr.LongPointField" positionIncrementGap="0" /> > <fieldType name="text" class="solr.TextField" autoGeneratePhraseQueries="true" positionIncrementGap="100"> > <analyzer type="index"> > <tokenizer class="solr.StandardTokenizerFactory"/> > <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/> > <filter class="solr.WordDelimiterGraphFilterFactory" generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1" splitOnNumerics="1" catenateWords="1" catenateNumbers="1" catenateAll="1"/> > <filter class="solr.FlattenGraphFilterFactory"/> <!-- required on index analyzers after graph filters --> > <filter class="solr.LowerCaseFilterFactory"/> > <filter class="solr.NGramFilterFactory" minGramSize="3" maxGramSize="15" /> > <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> > <filter class="solr.PorterStemFilterFactory"/> > </analyzer> > <analyzer type="query"> > <tokenizer class="solr.StandardTokenizerFactory"/> > <filter class="solr.SynonymGraphFilterFactory" expand="true" ignoreCase="true" synonyms="synonyms.txt"/> > <filter class="solr.FlattenGraphFilterFactory"/> <!-- required on index analyzers after graph filters --> > <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/> > <filter class="solr.WordDelimiterGraphFilterFactory" generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1" splitOnNumerics="1" catenateWords="1" catenateNumbers="1" catenateAll="1"/> > <filter class="solr.LowerCaseFilterFactory"/> > <filter class="solr.NGramFilterFactory" minGramSize="3" maxGramSize="15" /> > <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> > <filter class="solr.PorterStemFilterFactory"/> > </analyzer> > </fieldType> > </types> > <fields> > <field name="_version_" type="long" indexed="true" stored="true"/> > <field name="bcc" type="text" indexed="true" stored="false"/> > <field name="body" type="text" indexed="true" stored="false"/> > <field name="box" type="string" indexed="false" required="true" stored="true"/> > <field name="hdr" type="text" indexed="true" stored="false"/> > <field name="cc" type="text" indexed="true" stored="false"/> > <field name="from" type="text" indexed="true" stored="false"/> > <field name="id" type="string" indexed="true" required="true" stored="true"/> > <field name="subject" type="text" indexed="true" stored="false"/> > <field name="to" type="text" indexed="true" stored="false"/> > <field name="uid" type="long" indexed="true" required="true" stored="true"/> > <field name="user" type="string" indexed="true" required="true" stored="true"/> > </fields> > </schema> > > On 2018-12-10 21:17, Daniel Miller via dovecot wrote: On 12/4/2018 10:40 AM, Joan Moreau via dovecot wrote: > > In the Wiki, ( https://wiki.dovecot.org/Plugins/FTS/Solr ), it would nice to stipulate to the reader to type the command : > > sudo -u solr /opt/solr/bin/solr create -c dovecot # to create the dovecot instance > > before updating the schema.xml . > > Also, schema.xml is in /opt/solr/server/solr/dovecot/conf for archlinux users > > Additionaly, the url is http://(solr_ [1]server):8983/solr/dovecot/ (error in wiki) > > After installing Solr, wherever the installation sets up there should a folder similar to: > > <your prefix>/solr/server/solr/configsets > > If you look there, you'll probably see folders like '_default' and 'sample_techproducts_configs'. I haven't played with the 'techproducts' sample. Copy the '_default' folder, with all its contents, to a 'dovecot' folder. In the new dovecot folder, replace the 'managed-schema' file with the file from the Dovecot Wiki > > https://wiki.dovecot.org/Plugins/FTS/Solr?action=AttachFile&do=view&target=solr-7.x-schema.xml > > after that, you should be able to run 'solr /opt/solr/bin/solr create -c dovecot' to create the instance. If things still don't work let us know. > > The schema is one I've tweaked and updated during my own migrations since Solr 3.3. It's possible there's something else in my config that needs documenting - but having experienced Solr search against my mailstore I never want to be without it. > > DanielLinks: ------ [1] http://(solr -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20181211/d874f5e0/attachment-0001.html>