dovecot - Nov 2021 - Solr FTS - message deletes not working as expected

> On 11/03/2021 12:56 PM Shawn Heisey <elyograg at elyograg.org> wrote:
> 
> > Thunderbird does NOT necessarily process expunges immediately. 
Depends on what else it is doing in the background.  So you can't click
delete in the UI and not immediately see anything on the backend and
definitively correlate the two.
> 
> The message is deleted by dovecot immediately.? I double checked this by 
> purging a message on my Linux client and saw the message immediately 
> disappear on my Windows client.? It happened even faster than I would 
> have expected.? IMAP seems to be a very good protocol!? I can't say it 
> definitively without more evidence, but the problem *seems* to be in 
> FTS, not imap or core dovecot.
For Solr, there's a code path in the FTS expunge code that will silently
toss expunge requests:

        if (ctx->last_indexed_uid == 0 ||
            uid > ctx->last_indexed_uid + 100) {
                /* don't waste time asking Solr to expunge a message that is
                   highly unlikely to be indexed at this time. */
                return;
        }

So it's possible you are running into that.

michael

On 11/3/21 1:09 PM, Michael Slusarz wrote:
> For Solr, there's a code path in the FTS expunge code that will
silently toss expunge requests:
>
>          if (ctx->last_indexed_uid == 0 ||
>              uid > ctx->last_indexed_uid + 100) {
>                  /* don't waste time asking Solr to expunge a message
that is
>                     highly unlikely to be indexed at this time. */
>                  return;
>          }
>
> So it's possible you are running into that.
Interesting.? I don't know dovecot code well enough to figure out if 
maybe *every* expunge that happens is classified by that code as 
"unlikely to be indexed at this time."? For the situations I have
tried,
I know that the message HAS been indexed and should be deleted from the 
index.

I was going to try removing that whole code construct and running it to 
see what happens, but it looks like the dovecot apt repo does not have 
source packages.? When I add a deb-src line to the dovecot repo config, 
then do "apt update" (which works without complaint) followed by 
"apt-get source dovecot-solr" it pulls source packages from the Ubuntu
repo, not the dovecot repo, so the source I end up with is for version 
2.3.7, not the 2.3.17 that I am running.

Maybe I can do a custom compile of the source code and replace the 
/usr/lib/dovecot/modules/lib21_fts_solr_plugin.so file with what the 
compile produces.? I'm going to try that, and see if it explodes. :)

Thanks,
Shawn