search for: fts_tika

...cted by ip restrictions via a proxy in front of it. I've configured a tika server behind an apache proxy which enforces basic auth, but sending basic auth credentials for a tika server is not currently supported by Dovecot. The following patch allows to have user and password specified in the fts_tika url in much the same way you can for fts_solr. fts_tika = https://user:password at tika_host/tika John --- dovecot-2.3.11.3-orig/src/plugins/fts/fts-parser-tika.c???? 2020-08-12 14:20:41.000000000 +0200 +++ dovecot-2.3.11.3/src/plugins/fts/fts-parser-tika.c? 2020-11-15 15:18:24.351281064 +0100 @...

...ed up. > (ya-request for a proper @dovecot public bug/issue queue!) > > have you found any other 'magic required' to get solr & tika indexing > text/attachments, respectively, in Dovecot context? > is it as straightforward as spec'ing the 'fts_solr' & 'fts_tika' urls, > and Dovecot does the passing-around correctly? I've just started using tika myself, but from my tests, it's as simple as adding fts_tika to a working solr integration. John

Hi, I?ve set up in dovecot 2.2.24-1~auto+49 (from dovecot repo) fts_solr and fts_tika - jetti8 (from Debian Jessie) and latest tika-server running on a seperate machine. But if I want to rescan all messages for reindexing for instance all attachments with "doveadm -v index -u user at domain.tld INBOX" with 3137 mail in the INBOX it counts and then by 2900 mails the do...

Hi, I played around a bit and tried to get tika to run with dovecot. In the end I was at least a bit successful. However, when I tried to index my inbox with "doveadm index -A '*'" I get: doveadm(infoomatic): Error: fts_tika: PUT http://localhost:8081/tika failed: 500 Server Error doveadm(infoomatic): Warning: I/O leak: 0x7f4f697bb170 (line 127, fd 24) doveadm(infoomatic): Panic: file ioloop-iolist.c: line 22: unreached doveadm(infoomatic): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x67f30) [0x7f4f697e8f3...

...I have already mentioned this in http://www.dovecot.org/pipermail/dovecot/2014-November/098592.html I could reproduce the errors above in a self-compiled v2.2.18 and the prebuilt packages from xi.rename-it.nl (in addition to version 2.2.15 mentioned in the link) The problem occurs when enabling fts_tika in the plugins (tried tika 1.6, 1.7 and 1.8). I tried to move a folder of my mailbox with about 2000 mails to my server (no users, modern hardware). At some point I get an error and from this time on dovecot keeps repeating the last lines with every mail that comes in ... see [1] I then get kernel...

...request for a proper @dovecot public bug/issue queue!) >> >> have you found any other 'magic required' to get solr & tika indexing >> text/attachments, respectively, in Dovecot context? >> is it as straightforward as spec'ing the 'fts_solr' & 'fts_tika' urls, >> and Dovecot does the passing-around correctly? > I've just started using tika myself, but from my tests, it's as simple > as adding fts_tika to a working solr integration. > > John > > Just a couple of updates about Tika and Solr together. 1. On mass r...

...ds of documents, sometimes after a few. Usually after a few hundred. It appears there are less errors using http than https. Relevant config: OS: CentOS6, fully updated plugin { ? fts = solr ? batch_size = 1 ? fts_solr = url=https://username:password at solr-01.vevida.net:443/solr/dovecot/ ? #fts_tika = https://solr-01.vevida.net:443/tika/ ? batch_size = 1000 ? fts_autoindex=yes ? soft_commit=no } # dovecot --version 2.3.9.3 (9f41b88fa) # Configure options: ??? --docdir=%{_docdir}/dovecot? \ ??? --disable-static???????????? \ ??? --with-nss?????????????????? \ ??? --with-shadow???????????????...

...ent mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.pipe vnd.dovecot.execute mbox_write_locks = fcntl passdb { args = failure_show_msg=yes dovecot driver = pam } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://localhost:4949/solr/dovecot/ fts_tika = http://localhost:9998/tika sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve-execute sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_plugins = sieve_extprograms...

...r: service(indexe r-worker): child 24003 killed with signal 6 (core dumps disabled) Mar 12 20:49:01 dsync-local(laeeth at laeeth.com): Error: Couldn't lock /home/mail/ laeeth_laeeth_com/.dovecot-sync.lock: Timed out after 30 seconds Mar 12 20:49:16 indexer-worker(laeeth at laeeth.com): Error: fts_tika: PUT http://l ocalhost:9997/tika/ failed: 500 Server Error Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Warning: I/O leak: 0x7fc47d60fcf0 (line 127, fd 25) Mar 12 20:49:17 indexer-worker(rosie at kaleidicassociates.com): Panic: file ioloop .c: line 39 (io_add_file): asserti...

...b { args = user=%Ln noauthenticate driver = static skip = authenticated } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam skip = authenticated } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ fts_tika = http://localhost:9998/tika/ imapsieve_mailbox1_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = SPAM imapsieve_mailbox2_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-ham.sieve imapsieve_...

...h ? } ? prefix = ? separator = / } passdb { ? args = /etc/dovecot/master-users ? driver = passwd-file ? master = yes ? pass = yes } passdb { ? args = /etc/dovecot/dovecot-sql.conf.ext ? driver = sql } plugin { ? fts = solr ? fts_autoindex = yes ? fts_solr = url=http://127.0.0.1:8983/solr/dovecot/ ? fts_tika = http://127.0.0.1:9998/tika/ ? imapsieve_mailbox1_before = file:/usr/local/lib/imapsieve/report-spam.sieve ? imapsieve_mailbox1_causes = COPY ? imapsieve_mailbox1_name = Junk ? imapsieve_mailbox2_before = file:/usr/local/lib/imapsieve/report-ham.sieve ? imapsieve_mailbox2_causes = COPY ? imapsieve...

...b { args = user=%Ln noauthenticate driver = static skip = authenticated } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam skip = authenticated } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ fts_tika = http://localhost:9998/tika/ imapsieve_mailbox1_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = SPAM imapsieve_mailbox2_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-ham.sieve imapsieve_...

...gt; > args = failure_show_msg=yes session=yes max_requests=20 > > driver = pam > > skip = authenticated > > } > > plugin { > > fts = solr > > fts_autoindex = yes > > fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ > > fts_tika = http://localhost:9998/tika/ > > imapsieve_mailbox1_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-spam.sieve > > imapsieve_mailbox1_causes = COPY > > imapsieve_mailbox1_name = SPAM > > imapsieve_mailbox2_before = file:/usr/local/share/dovecot-pigeon...

...e connections hanging arond for a long time. (Affected Dovecot v1.1+) + mdbox: Added mdbox_purge_preserve_alt setting to keep the file within alt storage during purge. (Should become enforced in v2.3.0?) + fts: Added support for parsing attachments via Apache Tika. Enable with: plugin { fts_tika = http://tikahost:9998/tika/ } + virtual plugin: Delay opening backend mailboxes until it's necessary. This requires mailbox_list_index=yes to work. (Currently IMAP IDLE command still causes all backend mailboxes to be opened.) + mail_never_cache_fields=* means now to disable all cachin...

...e connections hanging arond for a long time. (Affected Dovecot v1.1+) + mdbox: Added mdbox_purge_preserve_alt setting to keep the file within alt storage during purge. (Should become enforced in v2.3.0?) + fts: Added support for parsing attachments via Apache Tika. Enable with: plugin { fts_tika = http://tikahost:9998/tika/ } + virtual plugin: Delay opening backend mailboxes until it's necessary. This requires mailbox_list_index=yes to work. (Currently IMAP IDLE command still causes all backend mailboxes to be opened.) + mail_never_cache_fields=* means now to disable all cachin...

...driver = static skip = authenticated } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam override_fields = domain=lerctr.org skip = authenticated } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ fts_tika = http://localhost:9998/tika/ imapsieve_mailbox1_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = SPAM imapsieve_mailbox2_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-ham.sieve imapsieve_...

...er piping the attachment through a shellscript as done with fts_decoder. Having this feature enabled though I sometimes do see the situation that the tika service does not reply in time and it looks like the whole FTS transaction gets aborted: dovecot: indexer-worker(eggs at localhost): Error: fts_tika: PUT http://localhost:9998/tika/ failed: Request timed out (Request queued 60.145 secs ago, 1 attempts in 60.134 secs, 60.034 in http ioloop, 0.000 in other ioloops, connected 1739.235 secs ago) dovecot: indexer-worker(eggs at localhost): Error: Mailbox INBOX.junk: Mail search failed: Internal...

...mise/patch will get picked up. (ya-request for a proper @dovecot public bug/issue queue!) have you found any other 'magic required' to get solr & tika indexing text/attachments, respectively, in Dovecot context? is it as straightforward as spec'ing the 'fts_solr' & 'fts_tika' urls, and Dovecot does the passing-around correctly?

...ist/solr-cell-*| solr-4.7.2/contrib/extraction/lib/* to ||/var/lib/tomcat6/webapps/solr/WEB-INF/lib/ how do I get dovecot to index attachments? Do I just need to add| fts-solr =/index-attachments url=http://localhost:8080/solr-4.7.2/ /to the 90-plugin.conf? Or do I need to so something like fts_tika =/http://localhost:8080/solr-4.7.2// IE the same uri I have for the solr? Thanks Alex

...gt; > args = failure_show_msg=yes session=yes max_requests=20 > > driver = pam > > skip = authenticated > > } > > plugin { > > fts = solr > > fts_autoindex = yes > > fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ > > fts_tika = http://localhost:9998/tika/ > > imapsieve_mailbox1_before = file:/usr/local/share/dovecot- > > pigeonhole/sieve/report-spam.sieve > > imapsieve_mailbox1_causes = COPY > > imapsieve_mailbox1_name = SPAM > > imapsieve_mailbox2_before = file:/usr/local/share/dov...