dovecot - Nov 2020 - dovecot fts-solr + solr 8.7.0 upgrade: "Indexing failed: 401 Unauthorized" + "Transaction commit failed: FTS transaction commit failed: backend deinit" ?

I've built a new dovecot + fts-solr instance; I've now picked up &
am running the recently released solr 8.7.0.

In a test account, I've one message -- in the 'Drafts' folder.

On exec of fts 'index'

	doveadm index -u testuser at example.com -q '*'

I get a

	401 unauthorized

indexing failure, only for the one dir+message in 'Drafts',

	2020-11-13 05:43:21 indexer-worker(testuser at
example.com)<8MKVNHmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Queue
	2020-11-13 05:43:21 indexer-worker(testuser at
example.com)<OHfzNHmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in testing
	2020-11-13 05:43:21 indexer-worker(testuser at
example.com)<yMxJNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Junk
	2020-11-13 05:43:21 indexer-worker(testuser at
example.com)<uDCbNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Trash
	2020-11-13 05:43:21 indexer-worker(testuser at
example.com)<kA/uNXmNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in Sent
	2020-11-13 05:43:22 indexer-worker(testuser at
example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: fts_solr: Indexing failed:
401 Unauthorized
	2020-11-13 05:43:22 indexer-worker(testuser at
example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: fts_solr: Indexing failed:
401 Unauthorized
	2020-11-13 05:43:22 indexer-worker(testuser at
example.com)<cEw/NnmNrl/DJgEA+IOfAw>: Error: Mailbox Drafts: Transaction
commit failed: FTS transaction commit failed: backend deinit (attempted to index
1 messages (UIDs 3..3))
	2020-11-13 05:43:22 indexer-worker(testuser at
example.com)<2OUMCHqNrl/DJgEA+IOfAw>: Info: Indexed 0 messages in INBOX

I get NO output atm in my solr logs @ this 'index'; just, atm, in the
dovecot logs, above

I've checked all my usual perms.
Indexing _was_ working with a prior solr v8.6.3 ...

Either I've misconfigured some perm, and am missing it, or new-solr-version
v8.7.0 is causing me some grief.

(1) Can anyone yet verify Doveoct/fts-solr working with solr 8.7.x?
(2) What logging config in Dovecot gets me more/detailed output for the fts_solr
fail?
(3) Any obvious clues as to what, specifically, is the source of this prob?

On 13/11/2020 14:56, PGNet Dev wrote:
> I've built a new dovecot + fts-solr instance; I've now picked up
& am
> running the recently released solr 8.7.0.
>
> In a test account, I've one message -- in the 'Drafts' folder.
>
> On exec of fts 'index'
>
> ????doveadm index -u testuser at example.com -q '*'
>
> I get a
>
> ????401 unauthorized
>
> indexing failure, only for the one dir+message in 'Drafts',
>
> ????2020-11-13 05:43:21
> indexer-worker(testuser at example.com)<8MKVNHmNrl/DJgEA+IOfAw>:
Info:
> Indexed 0 messages in Queue
> ????2020-11-13 05:43:21
> indexer-worker(testuser at example.com)<OHfzNHmNrl/DJgEA+IOfAw>:
Info:
> Indexed 0 messages in testing
> ????2020-11-13 05:43:21
> indexer-worker(testuser at example.com)<yMxJNXmNrl/DJgEA+IOfAw>:
Info:
> Indexed 0 messages in Junk
> ????2020-11-13 05:43:21
> indexer-worker(testuser at example.com)<uDCbNXmNrl/DJgEA+IOfAw>:
Info:
> Indexed 0 messages in Trash
> ????2020-11-13 05:43:21
> indexer-worker(testuser at example.com)<kA/uNXmNrl/DJgEA+IOfAw>:
Info:
> Indexed 0 messages in Sent
> ????2020-11-13 05:43:22
> indexer-worker(testuser at example.com)<cEw/NnmNrl/DJgEA+IOfAw>:
Error:
> fts_solr: Indexing failed: 401 Unauthorized
> ????2020-11-13 05:43:22
> indexer-worker(testuser at example.com)<cEw/NnmNrl/DJgEA+IOfAw>:
Error:
> fts_solr: Indexing failed: 401 Unauthorized
> ????2020-11-13 05:43:22
> indexer-worker(testuser at example.com)<cEw/NnmNrl/DJgEA+IOfAw>:
Error:
> Mailbox Drafts: Transaction commit failed: FTS transaction commit
> failed: backend deinit (attempted to index 1 messages (UIDs 3..3))
> ????2020-11-13 05:43:22
> indexer-worker(testuser at example.com)<2OUMCHqNrl/DJgEA+IOfAw>:
Info:
> Indexed 0 messages in INBOX
>
> I get NO output atm in my solr logs @ this 'index'; just, atm, in
the
> dovecot logs, above
>
> I've checked all my usual perms.
> Indexing _was_ working with a prior solr v8.6.3 ...
>
> Either I've misconfigured some perm, and am missing it, or
> new-solr-version v8.7.0 is causing me some grief.
>
> (1) Can anyone yet verify Doveoct/fts-solr working with solr 8.7.x?
> (2) What logging config in Dovecot gets me more/detailed output for
> the fts_solr fail?
> (3) Any obvious clues as to what, specifically, is the source of this
> prob?
>
(1) I can't, I'm still on an earlier version

(2) see here: https://wiki.dovecot.org/Plugins/FTS/Solr

two useful settings are debug and rawlog_dir=whatever to be added in the
same line as fts_solr

(3) strange you don't see anything on the solr server. Something is
giving a 401 response so if it's not solr what is it? Are you now using
apache in front of solr? One obvious thing to check is the url= part of
your fts_solr setting including the credentials and the solr core name.
You may also want to test your solr installation from the command line.
For example this does a manual soft commit:

curl
https://user:password at
solr.example.com:443/solr/dovecot/update?softCommit=true
<https://user:password at
solr.example.com:443/solr/dovecot/update?softCommit=true>

If your solr server is correctly configured you should get back
something like this:

<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
? <int name="status">0</int>
? <int name="QTime">3</int>
</lst>
</response>

John

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20201113/d41337ae/attachment.html>

On 11/13/20 9:46 AM, John Fawcett wrote:
>> (1) Can anyone yet verify Doveoct/fts-solr working with solr
8.7.x?>> (2) What logging config in Dovecot gets me more/detailed output
for the fts_solr fail?
>> (3) Any obvious clues as to what, specifically, is the source of this
prob?
>>
> (1) I can't, I'm still on an earlier version
I'll look for the 8.6.3, rather than 'current' (8.7.0) bins, spin up
an instance, and do some dubl-checking
> (2) see here: https://wiki.dovecot.org/Plugins/FTS/Solr
> 
> two useful settings are debug and rawlog_dir=whatever to be added in the
same line as fts_solr
ah, thx!
> (3) strange you don't see anything on the solr server. Something is
giving a 401 response so if it's not solr what is it?
head-scratcher, atm.  the logging^ should help.
> Are you now using apache in front of solr?
nope.  direct to solr.
> One obvious thing to check is the url= part of your fts_solr setting
including the credentials and the solr core name.
sure.

hasn't changed from prior working config.

admin UI is accessible/responsive.

i _can_ exec some solr commands without any error, e.g.

	sudo -u solr /srv/webapps/solr/solr/bin/solr create -c dovecot

	sudo -u solr /srv/webapps/solr/solr/bin/solr config \
	-c dovecot \
	-p 8984 \
	-action set-user-property \
	-property update.autoCreateFields \
	-value false

are fine.  it's the indexing that's not ...
> You may also want to test your solr installation from the command line. For
example this does a manual soft commit:
> > curl https://user:password at
solr.example.com:443/solr/dovecot/update?softCommit=true
<https://user:password at
solr.example.com:443/solr/dovecot/update?softCommit=true>
> If your solr server is correctly configured you should get back something
like this:
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <response>
> 
> <lst name="responseHeader">
>    <int name="status">0</int>
>    <int name="QTime">3</int>
> </lst>
> </response>

thx.  will check that in a bit ...