dovecot - Nov 2021 - Solr FTS - message deletes not working as expected

On 11/3/21 12:38 PM, Michael Slusarz wrote:
> Have you tried another client?
I only have two clients configured:? Thunderbird 78.13.0 on Linux and 
Thunderbird 91.2.1 on Windows.? Behavior is the same on both.

I will see if I can get another client configured.? Windows Mail is 
included on Windows 10, so I can try that.? I have a client system 
running Ubuntu, and I bet that there are a LOT of mail clients available 
there.
> Thunderbird does NOT necessarily process expunges immediately.  Depends on
what else it is doing in the background.  So you can't click delete in the
UI and not immediately see anything on the backend and definitively correlate
the two.
The message is deleted by dovecot immediately.? I double checked this by 
purging a message on my Linux client and saw the message immediately 
disappear on my Windows client.? It happened even faster than I would 
have expected.? IMAP seems to be a very good protocol!? I can't say it 
definitively without more evidence, but the problem *seems* to be in 
FTS, not imap or core dovecot.
> Another option is to ensure debug logging is enabled for Dovecot so you can
see what the FTS code is doing.  "log_debug = category=fts" and/or
"mail_debug = yes" will help in that regard.

When I have some real time available, I will look into debug logging.? 
I'm going to have to research exactly where to put those config options.

Thanks,
Shawn

> On 11/03/2021 12:56 PM Shawn Heisey <elyograg at elyograg.org> wrote:
> 
> > Thunderbird does NOT necessarily process expunges immediately. 
Depends on what else it is doing in the background.  So you can't click
delete in the UI and not immediately see anything on the backend and
definitively correlate the two.
> 
> The message is deleted by dovecot immediately.? I double checked this by 
> purging a message on my Linux client and saw the message immediately 
> disappear on my Windows client.? It happened even faster than I would 
> have expected.? IMAP seems to be a very good protocol!? I can't say it 
> definitively without more evidence, but the problem *seems* to be in 
> FTS, not imap or core dovecot.
For Solr, there's a code path in the FTS expunge code that will silently
toss expunge requests:

        if (ctx->last_indexed_uid == 0 ||
            uid > ctx->last_indexed_uid + 100) {
                /* don't waste time asking Solr to expunge a message that is
                   highly unlikely to be indexed at this time. */
                return;
        }

So it's possible you are running into that.

michael