search for: fts_backend

Op 06/01/2019 om 01:00 schreef Joan Moreau: > Anyone willing to explain those functions ? > > Most notably " get_last_uid" From src/plugins/fts/fts-api.h: /* Get the last_uid for the mailbox. */ int fts_backend_get_last_uid(struct fts_backend *backend, struct mailbox *box, ??? ??? ??? ???? uint32_t *last_uid_r); The solr sources ( src/plugins/fts-solr/fts-backend-solr.c:213) tell me this returns the last UID added to the index for the given mailbox and FTS index. > "set_build_key" Fro...

...gt; > On 2019-01-06 08:43, Stephan Bosch wrote: > > Op 06/01/2019 om 01:00 schreef Joan Moreau: Anyone willing to explain those functions ? > > Most notably " get_last_uid" > From src/plugins/fts/fts-api.h: > > /* Get the last_uid for the mailbox. */ > int fts_backend_get_last_uid(struct fts_backend *backend, struct mailbox *box, > uint32_t *last_uid_r); > > The solr sources ( src/plugins/fts-solr/fts-backend-solr.c:213) tell me this returns the last UID added to the index for the given mailbox and FTS index. > > "set_build_key" >...

For "rescan " and "optimize", wouldn't it be the dovecot core who indicate which are to be dismissed (expunged), or re-ask for indexing a particular (or all) uid ? WHy would the backend be aware of the transactions on the mailbox ??? There is alredy "fts_backend_xxx_update_expunge", so I beleive the management of the expunged messages is *NOT* in the backend, right ? On 2019-01-06 15:41, Joan Moreau wrote: > also, for fts_backend_solr_update_set_build_key -> where is the data (of the hdr_name or the body) ? > > On 2019-01-06 14:10, Joa...

Anyone willing to explain those functions ? On January 5, 2019 14:23:10 Joan Moreau via dovecot <dovecot at dovecot.org> wrote: > Thank Stephan > I basically need to know the role/description of each of the functions of > the fts_backend: > > struct fts_backend fts_backend_xapian = { > .name = "xapian", > .flags = FTS_BACKEND_FLAG_NORMALIZE_INPUT, -> what other flags ? > { > fts_backend_xapian_alloc, > fts_backend_xapian_init, > fts_backend_xapian_deinit, > fts_backend_xapian_get_last_uid, &g...

and finally , for fts_backend_xxxx_lookup_multi, why is that backend dependent ? Would- nt the below function below be the same for any backend ? Waiting fro your feedback on all those questions Thank you JM ----------------- static int fts_backend_xapian_lookup_multi(struct fts_backend *_backend, struct mailbox *co...

...08:43, Stephan Bosch wrote: > Op 06/01/2019 om 01:00 schreef Joan Moreau: > >> Anyone willing to explain those functions ? >> >> Most notably " get_last_uid" > > From src/plugins/fts/fts-api.h: > > /* Get the last_uid for the mailbox. */ > int fts_backend_get_last_uid(struct fts_backend *backend, struct mailbox *box, > uint32_t *last_uid_r); > > The solr sources ( src/plugins/fts-solr/fts-backend-solr.c:213) tell me this returns the last UID added to the index for the given mailbox and FTS index. > >> "set_build_key" &g...

also, for fts_backend_solr_update_set_build_key -> where is the data (of the hdr_name or the body) ? On 2019-01-06 14:10, Joan Moreau wrote: > for the "last uid"-> this is not the last added, but the maximum of the UID in the indexed emails, right ? > > On 2019-01-06 11:53, Joan Moreau via...

...What is the real meaning of this function ? Q5 : Rescan : is it just a bout remonving all indexes for a specific mailbox ? Q6 : lokkup_multi : isn't the function the same for all plugnins (see below) ? THank you On 2019-01-06 16:50, Joan Moreau via dovecot wrote: > and finally , for fts_backend_xxxx_lookup_multi, why is that backend dependent ? > > Would- nt the below function below be the same for any backend ? > > Waiting fro your feedback on all those questions > > Thank you > > JM > > ----------------- > > static int fts_backend_xapian_l...

for fts_backend_xxx_lookup, where is specidifed in which field (to, cc, subject, body, from, all) to lookup ? On 2019-01-06 16:03, Joan Moreau wrote: > For "rescan " and "optimize", wouldn't it be the dovecot core who indicate which are to be dismissed (expunged), or re-ask for indexin...

Op 04/01/2019 om 11:17 schreef Joan Moreau via dovecot: > > Why not, but please guide me about the core structure (mandatory > funcitons, etc..) of a typical Dovecot FTS plugin > The Dovecot API documentation is not exhaustive everywhere, but the basics are documented. The remaining questions can be answered by looking at examples found in similar plugins or the relevant API

...solr/fts-backend-solr.c 2019-04-14 14:40:12.513938845 +0200 @@ -28,8 +28,6 @@ #define SOLR_HEADER_LINE_MAX_TRUNC_SIZE 1024 #define SOLR_QUERY_MAX_MAILBOX_COUNT 10 -/* How often to flush indexing request to Solr before beginning a new one. */ -#define SOLR_MAIL_FLUSH_INTERVAL 1000 struct solr_fts_backend { struct fts_backend backend; @@ -392,6 +390,9 @@ (struct solr_fts_backend_update_context *)_ctx; struct solr_fts_backend *backend = (struct solr_fts_backend *)_ctx->backend; + struct fts_backend *_backend = + (struct solr_fts_backend *)_ctx->backend; + struct fts_solr_user *fuser...

Thank Stephan I basically need to know the role/description of each of the functions of the fts_backend: struct fts_backend fts_backend_xapian = { .name = "xapian", .flags = FTS_BACKEND_FLAG_NORMALIZE_INPUT, -> WHAT OTHER FLAGS ? { fts_backend_xapian_alloc, fts_backend_xapian_init, fts_backend_xapian_deinit, fts_backend_xapian_get_last_uid, fts_backend_xapian_update_init, fts_backend...

...ast_uid" "set_build_key" "build_more" , what is refresh versus rescan ? On January 5, 2019 14:23:10 Joan Moreau via dovecot <dovecot at dovecot.org> wrote: > Thank Stephan > I basically need to know the role/description of each of the functions of > the fts_backend: > > struct fts_backend fts_backend_xapian = { > .name = "xapian", > .flags = FTS_BACKEND_FLAG_NORMALIZE_INPUT, -> what other flags ? > { > fts_backend_xapian_alloc, > fts_backend_xapian_init, > fts_backend_xapian_deinit, > fts_backend_xapian_get_last_uid, &g...

On 4/13/2019 4:29 AM, John Fawcett via dovecot wrote: > If this value was made configurable people could set it to what they > want. However the underlying problem is likely on solr configuration. The Jetty that is included in Solr has its idle timeout set to 50 seconds. But in practice, I have not seen this timeout trigger ... and if the OP is seeing a 60 second timeout, then the 50

A starting point would be to have a look at the current FTS plugins: https://github.com/dovecot/core/tree/master/src/plugins/fts-solrandhttps://github.com/dovecot/core/tree/master/src/plugins/fts-squat -M Am Freitag, den 04.01.2019, 18:17 +0800 schrieb Joan Moreau via dovecot: > Why not, but please guide me about the core structure (mandatory > funcitons, etc..) of a typical Dovecot FTS

...rns immediately here. When I perform `doveadm index INBOX` for my test user, I do see a lot of fts and HTTP activity. THE SOLR PLUGIN IS NOT CODED ENTIRELY, REFRESH AND RESCAN FUNCTIONS ARE MISSING : https://github.com/dovecot/core/blob/master/src/plugins/fts-solr/fts-backend-solr.c static int fts_backend_solr_refresh(struct fts_backend *backend ATTR_UNUSED) { return 0; } static int fts_backend_solr_rescan(struct fts_backend *backend) { /* FIXME: proper rescan needed. for now we'll just reset the last-uids */ return fts_backend_reset_last_uids(backend); } > *- Bugs so far* > > -&gt...

...y in the index. fts-lucene implements this, but other FTS backends are lazy and simply rebuild all mails. Actually fts-solr is bad because it doesn't even delete the extra mails. > Q6 : lokkup_multi : isn't the function the same for all plugnins (see below) ? >> and finally , for fts_backend_xxxx_lookup_multi, why is that backend dependent ? This function is called only when searching in virtual folders. So for example the virtual "All mails" folder, which would contain all mails in all folders. In that case the boxes[] would contain a list of user's all folders, except...

...he index. > > fts-lucene implements this, but other FTS backends are lazy and simply rebuild all mails. Actually fts-solr is bad because it doesn't even delete the extra mails. > > Q6 : lokkup_multi : isn't the function the same for all plugnins (see below) ? and finally , for fts_backend_xxxx_lookup_multi, why is that backend dependent ? This function is called only when searching in virtual folders. So for example the virtual "All mails" folder, which would contain all mails in all folders. In that case the boxes[] would contain a list of user's all folders, except...

...dotlock *) 0x0 changed = true ret = 0 #2 0x000000004101b4c9 in squat_trie_open (trie=0x40c5a800) at squat-trie.c:242 No locals. #3 0x000000004101f908 in squat_trie_get_last_uid (trie=0x40c5a800, last_uid_r=0x7fffffffce5c) at squat-trie.c:1725 No locals. #4 0x000000004101a79c in fts_backend_squat_get_last_uid (_backend=0x40c2b150, last_uid_r=0x7fffffffce5c) at fts-backend-squat.c:104 backend = (struct squat_fts_backend *) 0x40c2b150 #5 0x0000000040e0f822 in fts_backend_get_last_uid (backend=0x40c2b150, last_uid_r=0x7fffffffce5c) at fts-api.c:80 No locals. #6 0x0000000040e1...

Yes but: 1 - is there a documentation of the main object ? (fts_backend, mail_user, mailbox, etc..) 2 - What are the mandatory functions ? 3 - Search : Supposedly, the FTS shall have several parameters : the keyword(s), the user & mailbox, and the fields (to, from, body, etc..) to be includude in the search. What is the function called in the plugin ? 4 - Ind...